Argax Project

Node Status: COMPLETE

Evaluation

Hypotheses

I expect that, when compared to a traditional command line IF user interface, the menu-driven Skald user interface will improve the ability of users (particularly novice users) to play interactive fiction. Specifically, Skald will:

Together, these effects should increase users' sense of agency when it comes to moving through an IF game world.

This change in interface may also effect how people play the game in other ways. Examples include:

At a higher level, the Skald interface may reduce some of the engagement that comes composing commands without any prompting. Some users may find it to be a more tedious way to construct a command. Skald may also make certain puzzles easier to overcome, which would also reduce the challenge of the game, and thus possibly make the game boring. Alternatively, affording these actions may reduce frustration, thereby increasing engagement. Thus, I expect that Skald will affect users' engagement and enjoyment of IF, but not necessarily in a positive direction.

Experimental Setup

Subjects were recruited by email (Appendix A) sent to a number of student mailing lists at the University of Hawaii at Manoa. After completing an informed consent form (Appendix B) and background survey (Appendix C), participants were randomly assigned to two groups: Group CS and Group SC.

Participants were then asked to play two different online text-based IF games: Captain Fate and The Queen's Heart. The games are described in more detail below. The order of the two games was the same for all participants, but the user interface of the two games varied. Group CS played Captain Fate with a traditional command line interface (CLI) and then The Queen's Heart using the new menu-driven Skald interface. Group SC first played Captain Fate using Skald and then The Queen's Heart using the command line interface. User input for each game session was recorded, which allowed for the generation of various game metrics, such as time spent and the range of verbs and objects used.

After each game session, participants were asked to complete a short one-page response survey (Appendix D). Finally, participants completed a one-page survey asking them to compare their two game experiences (Appendix E).

The Games

Spoiler warning: This section describes the primary puzzles and possible endings of the two games. You can skip this section if you would rather not know the details of the games before you play them. Knowing the details here is not necessary for you to understand the experimental results reported in the next chapter.

Captain Fate

Captain Fate was a humorous example game used in the Inform Beginner's Guide (Firth 2004). It was then converted to a TADS game and extended by Eric Eve (2005). With Eric's permission, I returned it to its original Inform length and modified it slightly to work with Skald.

The game opens as follows:

Impersonating mild mannered John Covarth, assistant help boy at an
insignificant drugstore, you suddenly stop when your acute hearing deciphers a
stray radio call from the police.  There's some madman attacking the population
in Granary Park!  You must change into your Captain Fate costume fast...!

--------------------------------------------------------------------------------

Captain Fate
by Roger Firth and Sonja Kesserich
Converted to TADS3 by Eric Eve
Modified by Zach Tomaszewski

[Type HELP for instructions on how to play.]

--------------------------------------------------------------------------------

On the street
On one side - which your heightened sense of direction indicates is north -
there's an open cafe now serving lunch.  To the south, you can see a phone
booth.

>

As stated in the intro text, the goal of this game is to change into your superhero costume. It turns out that the inside of the phonebooth is too visible to passers-by. That leaves the cafe to the north. Benny's Cafe has a restroom you can use, but you need to be a customer to ask for the key. So you have to order a coffee first. You have no money on you, but, if you are observant, you will find a lost coin in the restroom. Once you have paid for your coffee and returned the restroom key, you can head back out of the cafe to save the world.

Captain Fate has three possible endings, identified here by the ACTION that causes that ending. First, if you fail to lock the restroom door behind you, you will be interrupted while changing into your costume, thus revealing your secret identity (CHANGE). Second, you may be knocked unconscious if you try to attack Benny once you're in your costume (ATTACK). And finally you may successfully leave the diner in your superhero costume (SOUTH).

The Queen's Heart

The Queen's Heart is a game I authored as part of the 3-day Global Game Jam 2013 (Tomaszewski 2013). I then cleaned it up a bit and modified it slightly to work with Skald.

The game opens as follows:

You stand alone in the dark, sorrow in your heart and earthworms in your hand.

You stand listening.

You can hear a faint scratching... a sort of scurrying...  No, it is a
burrowing...  Yes, it is definitely a burrowing sound... though a rather small
one.  Too small for a badger, too steady for a rabbit, which leaves only one
thing: a mole.  There is a mole burrowing through the soft earth to your right.

You look down at the earthworms in your hand.  "A bit of mole-fishing, then?"
you ask yourself.  "A fresh mole for the Queen, my love, to grant her the
strength she needs?"  You feel the warmth of a blush on your face, a rush of
blood.  You only call the queen "your love" in the most secret and lonely of
places, like down here in the tunnels.

You glance around carefully, first down the tunnel one way and then the other.
No one to see you, and no one to hear.  No one else in a long long time.

You see dimly but well enough in the dark, though you could walk and hunt
through these tunnels blindfolded if you wanted to.  You've guarded them for so
long that you know every twist and turn, every bump of moist soil.  As the last
of the Goblin Guard, this is your domain: The Warren of the Eternal Queen (may
she protect us all).

The earthworms are wriggling, and they bring you back to the task at hand.
"Right then.  Worms or mole for dinner?"

No one answers.

-------------------------------------------------------------------------------

The Queen's Heart
by Zach Tomaszewski

[Type HELP for instructions on how to play.]

-------------------------------------------------------------------------------

The Eastern Tunnels
Twisting tunnels stretch off in all directions through the raw earth.  Here and
there, stone arches and wooden support beams shore up the weight of the soil
above.  The soil is a little moister here than to the west.

A mole is poking the tip of his nose out of a small hole.

>

In this game, the player directs a lonely goblin knight who is guarding his sleeping Queen. His long solitude has left the goblin rather quirky, and he is not a very reliable narrator.

To the east is a large pool of water at the base of a well. To the west through the tunnels lies the Queen's Chamber. There, the Queen lies slumbering on her stone bier covered by a thick fur blanket. As the goblin wanders the tunnels, he has a recurring compulsion to check on the Queen to see if she is still alive. There are signs that he has been performing this same ritual over and over for years: holding his breath and listening for the Queen's soft heartbeat.

The initial mole puzzle has no effect on the final outcome, but it sets the tone of the game. The player can ignore the mole, lure the mole out of the hole and kill it, or simply let the mole have the worms. Killing the mole increases the sorrow that the goblin carries; feeding it lightens his burden.

Once the player listens to the Queen's Heart, there is a crash and a splash due to a little human girl falling through the rotted wooden cover of the well. The goblin then has the choice to push a large rock down onto the girl (KILL) or to climb up out of the well (UP). Once out of the well, the "iron taint of Man" removes the goblin's glammer so that he can be seen by the girl. The player can either choose to push a wooden plank down for the girl to climb up, or else he dislodges the plank when climbing back down for the same effect. The girl climbs out of the well and escapes. As with the mole, releasing the girl lightens the goblin sorrow. However, an observant player might notice that after a trip up to the outer world, the goblin sees things differently, as if with fresh eyes. For example, his once shiny sword is now tarnished with age.

Compelled to check on the Queen again, two possible endings are now possible. If the goblin killed the girl (KILL), he is protected from the outer world and his Queen still lives. On the other hand, if the goblin entered the outer world (UP), either his glammer is gone or his delusions were broken. When he listens to the Queen now, he fails to hold his breath first and finds that he cannot hear her heartbeat. Trying the ritual again, he realizes that perhaps the heartbeat he has been listening to all of these years is his own, beating in his ears. He is now ultimately alone and without purpose.

Works Cited

Notes

  1. This is not the same as eliminating all user error. Users may still accidentally select the wrong menu option or otherwise make poor action choices.