Argax Project:: Dissertation: A Rough Draft Node

Node Status: COMPLETE

Medium

Our experience of the medium can be described at different levels of detail. At its most basic, our experience of a medium may utilize a number of sensory channels--the visual, auditory, tactile, etc. This raw sensory experience corresponds to Laurel's (and Mateas's) definition of Enactment.

At a higher level, as Laurel suggests, we discern patterns based on this sensory experience. From various sounds, we may differentiate music or speech. From our visual experience, we may differentiate text, diagrams, photographs, animation, or live action. We might call these differentiated sensory patterns the modalities¹ of the medium. They are essentially what Robert Stam refers to as "tracks". As Stam describes it: "The novel has a single material of expression, the written word, whereas the film has at least five tracks: moving photographic image, phonetic sound, music, noises, and written materials" (Stam 2000, p. 59).

We may also want to consider that, also as Laurel describes, there are certain conventions (something like a proto-language) that develop for these different modalities. For instance, a shot-reverse-shot with a fade can signify a reminiscent flashback in film. Comic books use different "word balloon" conventions to show whether a character is speaking, whispering, or thinking.

The specific sensations, modalities, and conventions depend on the particular medium used. Each level provides the material necessary for constructing those above it, while formally constraining those below it. This is a different, broader notion of medium than Aristotle's, which would correspond only to what I am calling the modality.

Object above, Medium below (containing Conventions, Modalities, Sensory Channels), informed by material and formal causes.

Model demonstrating the formal and material causes at work within any particular medium.

This reformulation of medium opens the way for a media-specific analysis of works, as called for by N. Katherine Hayles (2002). Whether a story is conveyed as a live performance, a film, or a novel, we should be able to explore the particular details of its material embodiment in a medium and how that specific embodiment affects our conception of the work as a whole.

In an interactive medium, the medium also provides interface controls that affect the imitated objects². When experiencing a drama, the users move through the stages of material causality: from their sensory experience, they discern separate modalities, each with their various conventions for relaying the objects of drama. When providing user inputs, the drama system must make this same transition. The system may allow for various channels for input, such as haptic or auditory. Haptic input might involve different modalities, such as movement of the mouse or the pressing of keyboard keys. Mouse use has a number of conventions concerning the difference between left-clicking, right-clicking, and double-clicking.

When discussing interaction, we are mostly concerned with the user's agency within the narrative (object) context--how the user can affect the world of the story. However, it is useful to remember that both the user's understanding of the story world and her attempts to control it must pass through the medium. Thus, the medium serves as an interface to the object. It presents a "system image" to the user by which the user makes sense of the represented object (Norman 2002).

==>

Works Cited

Hayles, N. Katherine. Writing Machines. Cambridge, MA: MIT Press, 2002.
Norman, Donald A. The Design of Everyday Things. New York: Basic Books, 2002.
Stam, Robert. "Beyond Fidelity: The Dialogics of Adaptation." Film Adaptation. Ed. James Naremore. New Brunswick: Rutgers University Press, 2000. 54-76.

Notes

Although Aristotle's manner is sometimes translated as mode, that is not what I mean here. Modalities, as defined here, essentially correspond to such parts as Aristotle described as comprising the medium--spoken language, musical rhythm, color and form.
Object-affecting interface controls are only present in an interactive system that internally models the objects of the story--such as the characters and events--and then intends for the user to affect those object representations. As we will see in the complete reconstructed poetics, some systems may intend only that the user interact with the manner (discourse) of the narrative--such as changing camera angles or reading the events of a hypertext story in a user-selected order. But regardless of which narrative level is being meaningfully affected by the interactions, all such user interactions must pass through the medium (and its components levels).

ToDo

"controls that affect the imitated objects": are we affecting the objects as such, objects as represented, or our mediated experience of the objects?
I added a note (#2) regarding this. To answer the question more directly: as users, we intend to affect the objects as we assume they are represented internally by the system. That is, from our experience of the medium, we form our own internal representations of the objects, and we assume these correspond (at least in important respects) to the presenting system's own internal model. This is a bit deeper than I want to go here though: I just wanted to say our understanding of the objects comes "down" through the medium, and any interactions intended for those objects goes back "up" through the medium.