http://www2.hawaii.edu/~ztomasze/argax
©2011 by Z. Tomaszewski.
Based on the empirical data presented here, it appears that Marlinspike's reincorporation feature worked correctly but that it had no discernible effect on players. In addition, Demeter successfully provided a story-like experience while still offering most players a sense of agency in that story.
This hypothesis was supported by the fact that using reincorporation resulted in more concise stories with a greater number of events in the main story thread, more threads spliced together during the story, and fewer extra threads left over at the end of the story.
This hypothesis was supported by the fact that signficantly more user actions are made necessary to the main story thread when reincorpration is used.
Because user ratings of story structure did not significantly vary between the two reincorporation conditions, there was no support for this hypothesis.
Because user ratings of agency did not significantly vary between the two reincorporation conditions, there was no support for this hypothesis.
Although there were clear significant differences in the system's internal representation of the story when reincorporation was used, there was no corresponding difference on any participant response measure. There are a number for possible reasons for this.
First of all, before any scene can play, all of its preconditions must be met. This means any scene that can play at a given point in the story is already going to be fairly relevant to the story. This is especially true given the limited number of possible scenes in Demeter, all of which are related to the same relatively small space of potential stories. Thus, when a reincorporation weighting is then used to select between the two or three scenes that could currently play, any improvement in selection will be subtle.
The other effect of reincorporation is narrating all hooks. This too is a subtle effect resulting in only an extra sentence or two for most scenes. These hooks make a big difference to the system's view of the story by providing explicitly-modelled connections between events. However, as suggested by Chatman (1978), audiences will often infer causality between events even if such a connection is not explicitly narrated. So explicitly narrating certain connections between scenes may add nothing significant to player's mental model of the story.
Because most reincorporation effects would be subtle for players, perhaps a different experimental instrument would have been useful here. For example, it may have been more informative to ask users about their experience of specific instances of reincorporation within a game session or to summarize their perceived story structure afterwards. Asking about specific reincorporation instances would at least clarify whether players even noticed reincorporation effects in the story. It may also have been useful to ask users to explicitly rate how the two game experiences differed, rather on assuming any difference in experience can be measured with a rather coarse 5-point Likert scale. A larger sample size may also have helped, although effect sizes for most of the survey measures were fairly small.
In addition, while very convenient, a post-game survey as used here does have a number of inherent limitations. First of all, the users' ratings are summative of the complete experience after the fact. A survey assumes participants can reliably recall all of the salient details of their experience at the time of the survey. Since the same survey was used in both cases, there may be a priming effect at work that would lead participants to give the same responses on the second survey as they did on the first.
Different methods may have circumvented some of these limitations. Concurrent measures--such as using periodic in-game pop-up questions or having players think aloud--would measure their experience as it unfolds. However, such measures are not without their own drawbacks as they might also break players' sense of immersion in the story.
Beginning with the medium, the text-based input was the least-liked aspect of the game. Both as measured by the system metrics and by participants' responses, the command error rate was fairly high--about 1 in 5 commands. Participant responses correlated significantly with the system measure of world-level agency. A number of participants were also unclear as to what actions could be performed. Although the error-rate generally decreased while play speed increased on the second play, these two problems of input and clearly afforded deeds remained substantial obstacles to world-level agency, which is a precursor to story-level agency.
On average, story-level agency was not rated very highly. Opinions varied widely on this, though. Actions performed by players produced about 35% of the most memorable events. Some sense of agency was the most enjoyable aspect of the game for 25% of participants--more popular than any other aspect of the game. However, feelings of inadequate agency was the least enjoyable aspect of the game for about 15% of participants. The system measures also suggested that Demeter could perhaps provide more regularly used actions of high import.
According to the poetics model, players should experience agency at each narrative level when material affordance for action are balanced by formal narrative constraints (Mateas 2004). Obstacles to world-level agency obviously limit the material affordances that are then available for story-level action. This is suggested by the strong correlation found between world-level and story-level agencies.
However, it was interesting to note that a number of participants also complained that they were not provided sufficient formal constraints. That is, 4 participants noted that the lack of a provided objective--that they didn't know what they were "supposed" to do in the game--was the least enjoyable aspect of the game. While my goal in developing Demeter was to provide an open world in which any action leads to a story, this is a reminder to all interactive drama developers that they may want to still provide at least an initial objective for players. However, the data here only showed that a lack of story-supplied objectives decreased enjoyment of the interactive drama in some cases. There was insufficient data to show that this decreased enjoyment was actually due to a lessened sense of story-level agency.
Participants' objectives may also provide a different interpretation of the participant complaint that potential actions were not apparent. In Demeter, most players head fairly quickly up into the Zeppelin in order to investigate the death of the crew. Because the revenant may not appear immediately, players tend to wander back and forth along the length of the Zeppelin. If players are focused on this "find the killer" objective, then there are few obvious actions to perform in the Zeppelin to directly achieve this. In particular, none of the objects there readily afford a direct solution to this problem.
While it is a concern that agency may be limited in relation to the player's objectives (or lack thereof), it still seems that the greater problem for Demeter is that possible actions must be recalled rather than examined as a list of options. This system opacity is compounded by the fact that it is not immediately clear which objects are interactive. For example, if the player's goal is to find the killer, they could look at the corpses for clues or up into the gas bags for hiding places. Look was the first verb taught in the tutorial, yet few players try doing these things. The player will occasionally smell a strange smell or see the revenant scramble away up the girders of the Zeppelin. Yet only two players tried to smell something and none tried to climb the girders. This is probably because most players simply did not think to try these things because they were not obviously possible.
Therefore, although significantly more player actions were made necessary to the internal story structure when reincorporation was used, players did not report a corresponding difference in their sense of agency. This is may be due to the somewhat limited world-level agency achieved in Demeter. The limitations of a post-game survey experimental design, as discussed above in relation to story structure, may also be at work here too.
Reincorporation effects aside, player responses were generally positive that the game experience was story-like overall. Responses were mildly positive that the story was well-formed with few to no irrelevant events. The story's ambiguous "cliffhanger" ending caused some complaints, though. However, it was interesting to find that, beyond the events of the Action level, all other narrative levels also captured players' attention.
First of all, characters are important, although they did not make or break the game. That is, the NPCs were rarely mentioned as either the most or the least enjoyable part of the game. The autonomy of NPCs made for some memorable events, but their general two-dimensionality and lack of believability in certain contexts was also a cause for some complaints.
The Zeppelin setting and some scenery items were both memorable and enjoyable for many participants.
The descriptions of both objects and events--that is, the narration and the suspenseful mood that it produced--was well-received. This is a reminder that manner was indeed an important level to reinstate in the poetics model of narrative.
The text-based medium was something of a barrier, though. Reading was disliked by some. It also likely contributed to the difficulty some participants experienced in visualizing and orienting themselves in the virtual world.
Thus, in conclusion, reincorporation was internally useful to the system in producing the story. Overall, the result was a successful interactive drama with a decent story structure. However, agency was somewhat impaired by the text-based interface. This means that Marlinspike's success at converting world-level deeds into story-influencing actions was likely occluded by the fact that many players did not have the material affordances necessary to experience agency at either level.
|
Argax Project : Dissertation :
A Rough Draft Node http://www2.hawaii.edu/~ztomasze/argax |
|
Last Edited: 29 Apr 2011 ©2011 by Z. Tomaszewski. |