Michael Mateas

Working Notes, by Zach Tomaszewski

for ICS 699, Fall 2005, directed by Dr. Kim Binsted

Website/Overview

http://www.lcc.gatech.edu/~mateas/

Thesis

Mateas, Michael. "Interactive Drama, Art, and Artificial Intelligence." Ph.D. Thesis. Technical Report CMU-CS-02-206, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA. December 2002. <http://www.cs.cmu.edu/afs/cs/project/oz/web/papers/CMU-CS-02-206.pdf>

1. Intro

• Expressive AI - using AI as art. AI: build systems. Art: build concrete representations.
• Tools: ABL, an extension of HAP.
• Previous projects: Subjective Avatar, Office Plant #1 (responds to user's email), Terminal Time.

2. Interaction and Narrative

• The decline of straight adventure game; rise of action. Emergent narratives: MUDS, MMORPGs, Sims. New media and electronic literature--hypertext, lingering IF domain. Laurel's concept of "interactive drama"--player as first-person character immersed in the story, not simply as an observer.
• Neo-Aristotelian Theory of ID. Laurel's distinction between dramatic and literary narratives (adding how puzzles break dramatic immersion). Murray's Aesthetic categories: Immersion, Agency, Transformation. Believing that Immersion and Transformation already largely included in traditional narrative theory, attempts to add Agency to Laurel's take on Aristotle (w/ formal and material causes). "A player will experience agency when there is a balance between the material and formal constraints."--that is, when the formal dramatic/plot affordances are commensurate with material lower levels (thought, language, pattern, spectacle). Places Agency at Character level, since in I.D., player is a character.
• Transformation--an aspect of 3rd person viewing. At odds with 1st person Agency. Rules of the system become apparent over multiple replays, revealing where Agency is thwarted. Need smoothly mutable plot, not discernible branch points. Unified plot, but different each time.
• Chose to use NLP input (rather than menu options) to balance formal and material causes (on same level as other characters).
• Neo-Aristotelian interactive drama critiqued (Frasca 2003, Aarseth 1997, etc.) Fate (story) vs. Freedom (interaction). Mateas: replacing "interaction" with "agency", which depends on affordances--balance between material and formal. This does not mean complete freedom, but only freedom [and effect!] based on story goals. Both story creation by AI and user interaction restrained to the story context.
• [Track down and clarify this Arist. debate next term? 3-4 week paper?]
• Frasca (2001; Boal 1985): More focused on open-ended system where user can add rules (like new Sims behavior, etc.)
• Gaming (ludologists) vs Narrative (narrativists) positions. [Ah, academic debate! No wonder these things take 5 years--you have to theoretically prove it before you can start actually proving it!.]
• Specific Facade goals. Short story length. No role-playing required on part of interactor. [Seems like the possibility of creating a relationship with Grace or Tripp was largely cut during production.]
• Sources: [Mateas 2000b; Mateas 2001b; Mateas 2003a].

3. Facade Architecture

• Autonomous agents: decisions based on private goals, internal state, and local world state. At odds with considering global story state.
• Story vs Autonomy spectrum. Strong story breaks down sense of character; broken walls and limited data hiding. Seductive strong autonomy position: slim, unidirectional interface from story affecting character goals. Characters work fine without story, but can drop them into one. Assumes:
  • DM decision infrequent. (True at scene level, but not at beat/interaction level.)
  • internal structure of agent can be decoupled from DM (At low-bandwidth, perhaps. But w/ frequent control, need access to internal to have fine enough control--else, conflicting goal mess. May need at least "story hooks" to be called by DM.)
  • characters can coordinate themselves. (Brings it all together--how do you get two characters to argue, revealing certain information, as a plot point, if they're so autonomous?)
• (from McKee). Events that turn (change) values (property of an individual/relationship--trust, love, hopelessness, etc.). Smallest unit of value change: a beat.
• Beats in Facade include: preconditions (needed in order to apply), priority and weight (among applicable beats), effects/results (change in story value).
• Facade: a DM with knowledge of beats, calls for next beat. Micro-manager brings beat about by controlling characters and allowing user interaction, etc. Player may actually act to abort beat and return to DM.
• Thus, for Facade, characters are bundles of low-level, story-related functions--ways to accomplish specific tasks. Motivation of characters for their tasks is handled by the story manager. ABL also deals with coordination.
• NLP. Story world (including avatar). DM (includes a story memory).
• Sources: [Mateas & Stern 2000; Mateas & Stern 2002]

4. AI and Art

• Expressive AI and it's relation to traditional AI.

5. ABL

• A Behavior Language. Extends HAP (a previous Oz language), adding multi-agent coordination.

6. ABL Idioms in Facade

7. Expressive AI: Affordances and Semiotics

8. Facade Drama Manager

• Local vs. Global agency. Getting immediate reactions, vs. affecting long-term story.
• Simulation seems fruitful: local agency, and c+e produce story. However, as from Tailspin, simulation based on character goals does not lead to good stories. Universe (Lebowitz) and Minstrel (1991) have some sim, but include story knowledge/control.
• Tension: maximumum global agency (choosing a single path through a tree) requires the greatest number of authored nodes. On the other hand, limited global agency (where each local choice selects an as-yet-unselected node) means only need a handful of nodes.
• Drama manager has a concept of a good story: whether an evaluation function (ala P.W.), or implicitly as local heuristics, or a mixed set. Constructs from story pieces.
• Beat Manager. A beat language. Details of beats (precond, priority, weight, effect, and actions).
• Beat sequencer. When next beat is needed:
  • Initialie all unused beats
  • Evaluate preconditions of all, creating Satisfied set of beats
  • Evaluated priority tests of all Satisfied, generating HighestPriority set
  • Score each HighestPriority in terms of how it applies to some story arc
  • Evaluate weight tests of each ScoredHighestPriority, and multiply each by the highest weight
  • Randomly draw a beat from WeightedScoredHighestPriority according to probability distribution
• Scoring a beat. Determine a story goal (value) in a given number of beats (min, max), giving an ideal story slope. Story history produces path/graph in this direction, attempting to follow the idea slope. Choosing next beat attempt to reach story goal without major local oscillations. Requires a selection of (at least one) beats. On failure, may abort sequencing, allowing next level to try something else (Facade doesn't do this). Complications: multiple story values, complex story arc, polyvalent beats (range of possible beat results). Previous beat sequencing/results.
• Facade story design (spoiler warning).
  • 4 topics: Art/Advertising, Rocky/Manipulative marriage, Facade (rich/phoney vs. real), Trip's affairs.
  • 2 story values: Tension (revealing the problems), 1 to 4, over 30 beats. (3 way points: Will G+T stay together? Will you still be a friend? Climax.) Affinity (who's side you seem to be on): not controlled by drama manager, but local.
  • Flavor of a beat in a topic will vary depending on the tension. Beats selected based on affinity, and this is largely how the user controls the story.
  • Additional beats (extra info, intro, etc.), often with greater prereqs.
  • Global mix-ins--interaction that break beat temporarily, generating more story content, before returning to beat.
• Description of specific beats.
• Story graph (nodes and link: all author), DM (author nodes, DM links), generative drama (links and nodes generated).
• Future: Interest in low-level, scene generation based on specific needs. Hierarchical decision processes in the DM--a higher level, that informs the beat manager.

[See Also:]

Regarding system/story manager, see:

• Blumberg, B. and Galyean, T. 1995. Multi-level Direction of Autonomous Creatures for Real-Time Virtual Environments. In Proceedings of SIGGRAPH 95.
• Weyhrauch's Thesis. Moe--advesary-search drama manager.
• McKee, R. 1997. Story: Substance, Structure, Style, and the Principles of Screenwriting. New York, NY: HarperCollins. [Source of "beat" idea.]
• Assanie, M. 2002. Directable Synthetic Characters. In Forbus K. & Seif El-Nasr M. (Eds.), Working Notes of the AAAI-2002 Spring Symposium on Artificial Intelligence and Interactive Entertainment. AAAI Press, pp. 1-7.

Projects

Facade

• 5 years of development with Andrew Stern. Formed the company Procedural Arts
• Parts of the infrastructure (namely ABL, for describing agents) will be released from the Experimental Game Lab at Geogia Tech.
• Requires 1.6Ghz! 800MB download.
• [See this page's source comments for recommended reads.]
• First (tipsy) impressions:

  Powerful. The story moves along, the relationship comes to a head. I'm caught up in it, trying to help, to direct, but it overwhelms me. There's a story progression that doesn't rely on me, but I'm still affecting it to some degree, but not controlling it. A very nice combination, a equilibrium of power. Halfway between spectator and participator, and the spectator part is true because the story just keeps moving.

• Results:
  • Marriage broken--Tripp leaves (reveals his poverty upbringing).
  • Forcefully removed from the premises for not helping (coming onto Grace).
  • Marriage flaws (T's affair, G's secret artistry, phoney) revealed, yet needs work
  • Tripp leaves (again); largely as first time.
  • Marriage forced/manipulated through proposal; artistry revealed. Grace leaves.
• Periodic Yes/No questions. Characters keep track of responses. Feedback: "That helps me", "Let's focus on Tripp". Recaps at end of input. Input language processing not that great (probably because few clear verbs and nouns).

Terminal Time

• Allowing audience bias to direct a historical documentary.