Conceptual Overview of HCI and Interaction Design

A study guide, by Zach Tomaszewski

for ICS 699, Fall 2006, directed by Dr. Dan Suthers

Overview

Scope of HCI has historically progressed from ergonomics (70s), to considering human cognition and psychology (80s), to effects of/on workflow and communication (90s), to effects on culture and society. These various aspects of HCI can be broken down in the following ways:

• Relevant (theories of) human processes: Cognition, Collaboration/Communication, Affect
• How people interact with systems: Interaction
• How to best design systems that work well with people: Interaction Design (including: Needs and Requirements, Design and Prototyping, Evaluation).

This overview sketch was built from:

• Norman, Donald. (2002). The Design of Everyday Things.
• Preece, J., Rogers, Y., & Sharp, H. (2002). Interaction Design: Beyond Human-Computer Interaction.
• Carroll, J. M. (Ed.) (2003). HCI Models, Theories and Frameworks: Toward a Multidisciplinary Science.
• Suthers, Dan. Lecture slides.

Cognition

Cognition involves: attention, perception/recognition, memory, learning, communicating (listening, reading, speaking), problem-solving (planning, reasoning, problem-solving).

• Perception: gestalt processing/principles; use of color and psychophysics; cultural and semantic connection (such as to color--red is "danger" or "stop").
• Attention: evidence of pre-attentive processing.
• Memory: short-term/working (7 +/- 2) vs. long-term. Recognition vs. recall. Effects of mode/context.

Frameworks for exploring cognition

Norman (including 7 Stages of Action)

• Cognition is Experiential (effortless--perception, action, reaction) [unconscious] vs. Reflective (designing, learning, writing, problem-solving).
• Errors can be slips (automatic, subconscious) or mistakes (after conscious deliberation). Most human tasks are shallow and/or narrow, and are processed at a low-level; so slips are most common form of error. (So need to design to tolerate errors, as they will happen even when user consciously understands system.)
• 7 stages of action:
  • Execution: Goal => Intentions -> Action Sequence -> Execution
    
  • Evaluation: Perceived State -> Interpretation -> Evaluation => Goal

People form mental models (of a system)

Able to mentally simulate its operations to make predictions. [See Interaction.]

People are information processors

• Info Processing Theory: mind has perceptual, cognitive, motor, and memory components.
• (Info processing model doesn't account well for external mental aids.)

Foundation for: GOMS (Goals, Operators, Methods, Selection)

• Family of models; Predictive of performance [See Evaluation - Predictive].
• Goal (desired end state), Operators (cognitive and physical actions to move towards goal), Methods (procedures for accomplishing goal), Selection (rules for deciding between methods).
• Applies to expert-level procedural tasks.
• Includes following (sub) models:
  • Keystroke level model
    • Provides actual numerical times/predictions of user performance
    • Best for simple, repetitive tasks. Flat model: mental and physical aspects combined in the model.
  • CMN -- Expands the "M" of GOMS, more of a cognitive hierarchy
  • CPM (Critical Path Model) -- Some things can be done in parallel.

External Cognition

• People use external representations to aid cognition: reduce memory load (reminders, etc), computational off-loading (doing long division), annotating/cognitive tracing (rearranging/visualizing; example: creating this document!)
• Norman's "knowledge in the world" (verses most other cognition theories, which focus on "knowledge in the head"). Tradeoffs: retrieval time, learning time, efficiency, ease of first use, aesthetics of system.

Parallel/Neural Processing

Connectionist/neuron-based theory of cognition.

Distributed Cognition

Whole system as an Info Processor. [See Collaboration.)

Collaboration & Communication

Mechanisms

Conversation

• Rules of conduct: turn-taking, selecting next speaker, repairing breakdowns
• formal (board meeting) verses informal (chat) conversations. Affects both content and rules/coordination.
• synchronous vs. asynchronous; remote vs. collocated.
• Mediated by a wide range of systems: phone, video conferencing, chat, MUDs, etc. CMC (computer-mediated communication).
• Coherence (context, what we are talking about/referring to/responding to); tendency of conversation to diverge, but also must converge/conclude.

Coordination (of behavior)

• Through: verbal/non-verbal conversations; schedules, rules, and conventions (such as scheduling lectures and room use at a university); shared external representation
• Can also be formal or informal
• In systems: Who can use what, when. Whether people will follow rules/maintain culture.

Awareness

• Determining another's location, state, mood, progress, etc.
• Overseeing/hearing another may change behavior without explicit conversation (ex: stock traders)
• In systems: can users monitor each other (who's logged on, typing, etc.). May be explicit, allowing notification (by users) rather than monitoring (of other users).

Frameworks

Language as Action (Speech Act, Conversation for Action, etc.)

Seeing conversations/statements as: assertives, commissives, declarations, directives, expressives.

Distributed Cognition

Info propagated through system in various representational forms, through conversation and coordination.

Common Ground Theory (Contribution Theory)

Communication is not just info transfer (except perhaps at very low-level with standardized vocabulary). Need to negotiate meaning, or at least shared understanding. Proposition is common ground if all involved know it and know that all involved also know it. Start with conversational rules and other communal common ground. Rest of common ground developed through conversation in shared context. Grounding constraints/resources.

Affect and Emotion

• Affective aspects are those which produce an emotional response.
• We're good at recognizing emotions in others and responding accordingly.
• Affect includes/affects cognition, physical, behavioral aspects of individual.
• Emotion: reaction to an event based on goals, needs, or concerns of individual. (Not mood/temperament; or sentiment toward objects.)
• Older "mental" system, closely tied and influencing cognition.
• Important: trust, motivation (enjoyment/interest), confidence/control/agency.

Designs may strive to:

Convey or elicit (positive) emotions

Can use icons/animation to express emotion; pleasing aesthetics; helpful interface agents; emoticons (for expression of emotion through CMC).

Avoid (negative) emotions, particularly frustration

Often frustrated by poor usability. Also:

• gimmicks to cover deficiencies (ex: "Under construction" pages)
• redundant/obvious error messages ("The program crashed.") or incomprehensible errors ("Error 14532.")
• overburdening user (requiring too much memory load; plugin/update required)
• appearance (ugly/annoying interface; sound, music, or pop-ups; distracting animations)

In general: good/clear design, helpful feedback/useful error messages, and contextual/relevant help all help alleviate frustration.

May also use emotion through an anthropomorphic agent.

• Kinds of agent: synthetic character (ex: game NPC), animated agent (ex: Clippy or other interface tutor), emotional agent (ex: Woggles), embodied conversational agent (ex: Eliza, etc.)
• Design concerns for agents: believability, appearance, behavior, modes of interaction.

Also, affective computing--building systems to either exhibit or recognize (thru methods: predictive; real-time (physiological, gesture, facial expression, speech, etc); retrospective (user-reported)) emotion. Various levels possible (arousal/avoidance vs. specific emotion).

Interaction

Important aspects of product (for sales): aesthetics/appearance, price/cost, maintainability, number of features, prestige, usability.

Usability

efficiency, effectiveness, safety, utility (to user), learnability, memorability.

User experience

satisfying, enjoyable, fun, entertaining, helpful, motivating, aesthetically pleasing, supportive of creativity, rewarding. (Some of these may actually conflict with usability--as in making a game challenging.)

Interaction modes

instructing/commands; conversing/agent; manipulating (and navigating); exploring/browsing (information).

Interaction system paradigms

WIMP/windows, ubiquitous computing, etc. [General platform for intended system.]

Mental Model

Norman: Users form a mental model of the system. In perfect world, Gulfs of Execution and Evaluation between user and system are bridged.

• Execution (getting the system to do what you want):
  • Visibility -- what actions/controls are available?
  • Affordances -- properties that can determine how a system/control could be used
  • Constraints -- limits on possible uses of system/control (physical, cultural, semantic/logical)
  • Mappings -- relationship between controls and their results in the system/world (natural mappings: physical analogies, culture standards (up is more, etc))
  • Gulf: when intentions or user's actions don't map to system controls.
• Evaluation (seeing what the system just did):
  • Feedback -- what action was just done; system's current state
  • Gulf: when user can't determine system's state or whether expected results have occurred.

Ergonomic/Motor Details

• Fitt's Law (time to select an object on the screen): T = a + b (lg (D/(S+1))
  
• Hick-Hyman Law (time to select from n items (such as a menu)): T = a + b*lg(n)

Interaction Design

Four basic steps:

• ID needs and establish requirements
• Develop alternate designs that meet the requirements
• Prototype, and evaluate interaction with the different designs
• Build a solution, iteratively evaluating by these steps

Three focuses:

• focus on users and their needs
• specify usability and user experience goals that solution will achieve
• iterate to refine design

Life Cycle models

Simple

As laid out above: ID needs, design, build alternative prototypes, evaluate

Waterfall

linear progression from requirements to testing; from software engineering

Boehm's Spiral model

spiral to reduce risks [of design commitment]

Rapid Applications Development

time-boxing into smaller projects; high user involvement in requirements stage

Star model

from HCI, constantly return to evaluation

Usability engineering

Some Specific Design Approaches

Scenario-based Design (Rosson & Carroll)

Uses narrative scenarios so design artifacts are more approachable by non-designer stakeholders. Problem scenario (what a specific user does right now) -> activity scenario (how they could do it in abstract) -> information and interaction scenarios (what interaction and info needed to achieve activity) -> scenario machines (narrow, deep prototypes).

Usage-centered Design (Constantine and Lockwood)

Focused on the tasks/use. User roles (relations to system) and tasks they engage in -> essential use cases -> contexts (combining different use cases into info and operators with fewest different contexts) -> abstract interface.

Needs and Requirements

Determining user needs and translating them into system requirements.

Users

• Who are they? Stakeholders: all of those who will be impacted by the system.
• Characteristics, capabilities, roles, experience level.
• "Needs" may not be conscious or known.

Data gathering

• Questionnaires, interviews, focus group/workshops, naturalistic observation, study existing documentation.
• Ethnography (an experience, difficult to summarize "implications for design", lets you build a frame of reference rather than coming to the study with one).
• Qualitative vs. quantitative data
• When planning data gathering: focus on needs, involve all stakeholders, use combo of techniques, run a pilot first, use props for discussion if possible, make intelligent compromises when faced with real world constraints (time and money).
• [Similar techniques also used in Evaluation.]

Data interpretation/analysis

Different techniques to capture the discovered user needs:

• Task description: Scenarios; use cases; essential use cases
• Task analysis: Hierarchical Task Analysis, (GOMS).

Requirements

• Intended system's specifications--specific, unambiguous, clear as possible, (measurable).
• Functional: what the system will do (function, data requirements)
• Non-functional: constraints on system (context of use--physical/hardware, social, organizational, technical, environmental; kinds of users; usability goals; project resources)

Design and Prototyping

Design

User-Centered Design

• Early focus on users and their tasks; empirical measurements; iterative design.
• Study user [See Needs and Requirements] but also include them in design (either as co-designer or through evaluation of prototypes)
• Can be difficult to coordinate such high level of real user involvement (time, money, scheduling, consistency of user input throughout project).
• Participatory design: need techniques to bridge designer and user cultures (PICTIVE, CARD).

Conceptual design [System designer's model]

Frameworks/aspects to consider

• Different interaction modes (activities vs. objects) and paradigms (WIMP, etc.). [See Interaction]
• Suitable interface metaphors (Especially to support difficult features of system, but beware false mappings)

Expanding this:

• Functions the system will perform; how functions are related (task progression); data requirements/information to be displayed for each function.

Physical design [System image]

• Should support cognitive processes (ala Norman [see Interaction]) and embody the conceptual design
• May be guided by interface design guidelines/heuristics: style guides/OS look-and-feel reqs; menu and icon design; screen design (breaking tasks into screens, and laying out each screen); data display/formats.

Prototypes

Aids discussion with stakeholders, design team, and in testing during design iteration. May be evolutionary prototyping (becomes system) or throw-away prototypes.

Low Fidelity.

Cheap and quick. Doesn't look like finished product. Include: storyboards of use; sketches of interfaces; index card walk-throughs; wizard of Oz software.

High Fidelity

Expensive, more time consuming. Users can get sidetracked by superficial details and bugs; but closer to actual system, so may be easier to visualize and understand. Includes: horizontal (wide range of functions in low detail) or vertical (can complete one task all the way through)

Evaluation

Evaluation Paradigms: quick and dirty (fast, informal data); usability testing (carefully controlled); field studies (situated in natural conditions); predictive (actual users not required).

Preparation for Evaluation

Preece et al's DECIDE framework: goals for testing; operationalize as specific questions to answer; choose testing paradigm/techniques; identify practical issues (time, money, scheduling, expertise in testing); handle ethical questions (consent, user compensation, etc.); evaluate data (reliability, validity, etc.); do pilot study first.

Observing Users

• Observing: as an outsider or insider; in controlled (lab equipment, "think aloud" recordings) vs. natural environment (examining the people, places, and things involved in the task).
• Start with goals (what interested in) and recording strategy (notes, still camera, audio recording, video), but be flexible and ready to refocus.
• Verify observations/interpretations with group/user members.
• Indirect observation methods: interaction logging; user diaries/journals.
• Interpret (grouping qualitative notes, or statistical analysis of quantitative data) to find incidents of note, patterns, or groupings.

Asking Users

• Getting at facts, behavior, beliefs, attitudes [particularly good for last two, compared to other methods]
• Interviews: open/unstructured (still organized, just broader questions); structured; semi-structured (mixed); group (more than one interviewee; focus group).
• Questionnaires: specific/structured questions for large sample. (Not on hand to correct misunderstandings, so pilot testing even more important than usual.)

Testing Users

• Differences between user testing and experiments. (Experiment: searching for new knowledge; indep/dep variables; control and replicatability more important.)
• Determine goals -> questions; select users; develop tasks to complete (often after exploration of system first); run test.
• Look for such things as: time to complete task, # of tasks completed; # of pages/screen accessed; # of errors made; nature of navigation trail; negative comments made; user satisfaction (questionnaire after test).

Modeling users: Experts and Predictive models.

Inspections of design by an expert evaluator, rather than a user.

• Heuristic evaluation: 5 evaluators catch 75% of design problems, but each individually misses some and falsely labels other features as problems. Heuristics tend to be specific to the application domain.
• Cognitive walk-through: evaluator simulates a novice user's goals and mental state at each step of a particular task--likely assumptions, ability to see and map system features to their intentions, ability to evaluate system's response, etc.
• Pluralistic walk-through: users, evaluators, and designers individually documenting what actions they would take on a potential design, then discussing it together.
• Predictive models: GOMS, keystroke level model, Fitt's law.