Argax Project

Participant Backgrounds

The technique used to randomly assign participants to groups resulted in somewhat uneven groups. Of the 24 participants, 15 were assigned to Group 0, while the remaining 9 participants were assigned to Group 1. There also proved to be some significant differences between the two groups with regard to previous gaming experiences.

The following is a summary of the background information for the 24 participants used in this evaluation.

Demographics

Here are the distributions of gender, age, and education for all 24 participants combined, and then for each of the two separate groups.

Gender was fairly evenly divided between the groups, though men outnumbered women in all cases. On average, Group 0 was older than Group 1 and more highly educated, though not significantly so. The majority of participants had completed college or a higher level of education.

Computer Experience

The remainder of the background data is presented here using a standard format for each question. This format includes a total mean score for each response, a distribution of all response scores, and separate means for the two groups. When these means are significantly different from each other (p < .05), they are given in bold and the resulting p-value is given in the discussion below.

The questions in this section aim to measure participants' general comfort level on a computer, particularly with the sort of reading and input methods required to use the game.

Possible responses for this first question included Strongly Agree (4), Agree (3), Neutral (2), Disagree (1), or Strongly Disagree (0).

The following question uses the following scale: six or more hours (4), three to five hours (3), one to two hours (2), less than one hour (1), rarely or never (0).

The following question allowed the following responses: "I regularly read text on a computer screen without trouble" (2), "I dislike reading text on a computer screen, but I can tolerate it for short periods" (1), or "I avoid reading text on a computer screen whenever possible" (0).

The following question on command line interfaces allowed these responses: "I use a command line at least once a week for many different tasks" (4), "I use a command line at least once a week for a small number of tasks" (3), "I use a command line a few times a year, or I have regularly used a command line in the past" (2), "I rarely use a command line" (1), or "I have never used a command line (or, I do not know what a command line interface is)" (0).

From these results, we can see that all participants were comfortable using a computer and did so for multiple hours a day. There was little complaint about reading text on a screen. All but 2 participants were familiar with a command line interface, and most had some degree of experience using one. There were no significant differences between the two groups on any of these measures.

Digital Narrative Game Experience

These questions explore participants' time spent play digital games using the following scale: six or more hours (4), three to five hours (3), one to two hours (2), less than one hour (1), rarely or never (0).

It is interesting to see that those participants who play PC games generally spend more time on them than those who play console or other digital games.

Some people may not currently spend much time playing games but may still have played a great deal in the past. The following questions attempt to measure this, as well as the general breadth of participants' digital game experience. They use the following scale: Over 100 games (5), 50 to 100 (4), 20 to 50 (3), 5 to 20 (2), 1 to 5 (1), or none (0).

So every participant had played at least one digital game before. However, those in Group 0 had played significantly more computer games, Mann-Whitney U = 31.5, n1 = 15, n2 = 9, p = 0.03, and significantly more games on other digital platforms, Mann-Whitney U = 29, n1 = 15, n2 = 9, p = 0.02, than Group 1. (Mann-Whitney was used to determine signficance here because the response scale was not a continuous scale.)

These questions measure the participants' familiarity with various digital game genres using the following scale:

• I have played a number of games in this genre (4)
• I have played at least one game in this genre for many hours (3)
• I have briefly played or watched someone else play a game like this (2)
• I know what this is, but I have never played one (1)
• I do not know what this is (0)

Probably due to having played more games in general, Group 0 was noticeably more familiar with every game genre than Group 1. However, only two of these mean differences were statistically significant: interactive fiction, t(22) = 3.05, p = .01, and adventure games, t(22) = 2.93, p = .01.

Non-digital Interactive Narrative Experience

These questions measure the participants' familiarity with various traditional interactive narrative game genres using the following scale:

• I have played a number of games in this genre (4)
• I have played at least one game in this genre for many hours (3)
• I have briefly played or watched someone else play a game like this (2)
• I know what this is, but I have never played one (1)
• I do not know what this is (0)

Again, Group 1 is noticeably more familiar with these game genres. However the difference was only significant for table-top roleplaying games, t(22) = 2.43, p = .02.

Therefore, the random assignment to groups was not fully successful in producing equivalent groups. Group 0 was larger and its members had played significantly more digital games than those in Group 1, as well as being more familiar with interactive fiction, adventure computer games, and table-top roleplaying games.

Incomplete Participants

As mentioned when describing recruitment, 53 participants finished the initial background survey, although only 24 of those went on to complete all other stages of the study. On average, the backgrounds of the 29 "incomplete" participants did not significantly differ from those that completed the study, with the following exceptions.

The incompletes had less experience with interactive fiction games (1.21) than those that completed the study as a whole (2.67), t(51) = 2.28, p = .03, although it is important to note that their mean score is still higher than the Group 1 mean (1.11). The incompletes also had less experience on average with social simulation games such as The Sims (1.97) than those that completed the study (2.58), t(51)= 2.05, p = .05. However, I do not believe either of these differences are practically significant in explaining why certain participants did not complete the study.

ToDo

• Although no significant differences were found on other measures between the two significantly different groups, I could look closer at correlations between specific background ratings and all other ratings. A quick glance through a matrix of correlations does reveal some interesting high correlations here and there, though nothing obviously systematic. I think a full examination would take some sort of multiple regression analysis (with which I am not yet familiar). I'm not currently doing this analysis for two reasons. 1) This is not directly related my focus here on the effects of reincorporation. (If I did do it, I'd put in an appendix.) And 2) I don't currently have time. But I would like to come back to this later and see if there's a conference paper in this.
• More on above: Since there's no difference between reinc and play effects, I would use the mean of the two responses given for each question. (This is what I did in the above correlation matrix.) But how much do these two measures tend to vary for each question? To verify, for each question: 1) Get a distribution of score differences. 2) Another way to look at this: Is there a high correlation between play 1 and play 2 scores?

Argax Project : Dissertation : A Rough Draft Node http://www2.hawaii.edu/~ztomasze/argax

Last Edited: 01 May 2011 ©2011 by Z. Tomaszewski.