http://www2.hawaii.edu/~ztomasze/argax
©2007 by Z. Tomaszewski.
The experiment to test the effect of Marlinspike's reincorporation feature was executed in the following way. Participants were recruited by email and then participated online. Participants were randomly assigned to one of two groups. They completed a background survey and then played the Demeter game twice, once with reincorporation on and once with it off. The two groups differed in whether they played first with reincorporation on or with it off. After each play session, they completed an identical response survey.
Here are the details of this process.
First, each interested participant clicked a link provided in a recruitment email message. (See Appendix 3 for a copy of this email message.) This link directed participants to a webpage where they were assigned an anonymous participant ID number and thus assigned to one of two experimental groups. Even ID numbers were assigned to Group 0 and odd ID numbers were assigned to Group 1. Since IDs were assigned consecutively, this is effectively a random assignment to groups.
Participants were then sent to an initial online survey hosted by SurveyGizmo.com. They were required to give informed consent and provide background information, which included the participant's gender, age group, education level, previous computer experience, computer game experience, and roleplaying game experience. (See Appendix 2 for a full copy of this background survey.)
Each participant was then sent back to the game web server that initially assigned the participant IDs. There they could play the Demeter game online in their web browser. The game was executed in a modified version of the Java-based Zag interpreter (Zeppieri 2005). The first game session started with a short in-game tutorial that had to be completed before the game itself started. This was used to help introduce participants to the text-based game format and reduce learning effects between the two game sessions.
When the game window was closed, the modified version of the Zag interpreter sent back to the game web server a transcript of all input entered during the game. By comparing the recorded timestamp for when a given participant arrived at the game page and the timestamp for when the game sent back its transcript, it was possible to track how long participants spent on each game.
After completing the first game session, participants clicked a link on the game page to be taken back to SurveyGizmo.com to complete a response survey. This three-page survey asked questions concerning their experience in terms of story structure and user agency. (See Appendix 3 for a full copy of this response survey.)
At the end of the response survey, participants were directed again to the game server for a second game session. They then completed the same response survey regarding this second play session.
Those participants in Group 0 (even IDs) first played the game with reincorporation on and then with it off. Group 1 (odd IDs) played first with reincorporation off and then with it on. This was used to control for the effects of initial exposure to the system.
The author imperative option used by Marlinspike in selecting scenes was also turned off for all games in the evaluation. This is because the intent of this study was to examine only the effect of reincorporation as compared to selecting possible scenes randomly.
The experiment ran for two weeks. After all data was collected, I replayed each recorded game transcript to review the different game sessions. I also collected certain game data concerning the system's internal view of the structure of each finished story.
|
Argax Project : Dissertation :
A Rough Draft Node http://www2.hawaii.edu/~ztomasze/argax |
|
Last Edited: 15 Apr 2011 ©2007 by Z. Tomaszewski. |