In the previous assignment, I suggested that conceptual metaphors shape our understanding and use of the World Wide Web. This first raises the question of which conceptual metaphors--specifically--are used to understand the Web. I believe this would best be discovered by archival research of literature describing both Web use and Web construction. Examining such a corpus would reveal a number of metaphors people have used, as well as provide some basic situational context for each (for example, whether the target audience for a particular metaphor is generally novice users or experienced designers).
Yet even if we determine the metaphors people use to describe using the Web, it is not certain that they actually use these metaphors when surfing. Even if they do, this does not tell us whether using these metaphors actually affects the experience of using the Web in any way. Thus, I propose the following experimental setup to test whether the use of Web metaphors actually affects the use of the Web.
In such an experiment, the independent variable is which metaphor a person is primarily relying on to understand their Web experience. Past psycholinguistic research has suggested that it is possible to "prime" one metaphor over another by invoking an experience of the source domain in the subject. For example, when given a written scenario in which it is ambiguous if a person is moving through time or if events are moving towards a person, a subject is more likely to interpret the description as time moving towards a person if they just had an object move towards them. Assuming this to be true, it should be possible to encourage the use of either THE WEB IS A COLLECTION OF DOCUMENTS or THE WEB IS A BUILT SPACE by having a person deal with documents or a built space before beginning a Web task.
Specifically, one group of subjects would be asked to find the contents page of a popular magazine or a chapter on a certain topic in a textbook. Another group of subjects would be asked to find the restroom or a certain room number in a hallway they haven't been to before. (As a control, a third group would be asked to play a game of computer solitaire for the same length of time as the searching/navigating tasks.)
To check the extent that subjects have been successfully "primed", they would be requested to provide a running verbal description of their actions while performing a number of Web-related tasks. This description will be recorded and checked for instances of each metaphor.
The dependent variable is successful Web use, as evidenced by completion times on a number of Web-related tasks. These Web-tasks would involve different levels of the Web--finding a specific element within a single page, finding a certain page within a site, and finding a site on the Web as a whole. At the site level, task sites would include an instance of both document-like collections (such as a Web-based book) and more spatial examples (such as an online store).
This setup is a variation of static group comparison, with a "mid-test" of levels of metaphor use.
| Treatment (IV) | Mid-test Observation | Observation (DV) | |
|---|---|---|---|
| Construct | Reliance on conceptual metaphor | Experience shaped by metaphor | |
| Variable | Use of either SPACE or DOCS metaphor | Web use success | |
| Operational definition | DOCS: magazine and book search exercise SPACE: hallway navigation exercise CONTROL: non-significant priming exercise | Number of utterances relying on each metaphor during Web use | Time to successfully complete Web exercises |
I expect that, while the dominant metaphor might affect task completion, the extent of the effect will be mediated by the sort of task undertaken. For example, page-level tasks should benefit more from a document metaphor than a spatial metaphor.
There are a number of challenges to this experiment's validity, only some of which have been controlled for. The effectiveness of the priming exercises can be confirmed with the utterance count. There should be a higher percentage of utterances of a particular metaphor when the subject is primed for that metaphor. The solitaire exercise it to provide data for non-primed subjects, while still requiring some sort of pre-experiment exercise of the control group. Subjects would be randomly assigned to each group, making this an experimental (not a quasi-experimental) design. These steps should boost construct and internal validity.
Other possible threats to the internal validity of this study are those of setting, instrumentation, and maturation. While the setting may not be the user's preferred surfing environment, hopefully it will not be significantly different. Also, while most Web users don't continually narrate their own actions, the benefit of confirming the metaphor priming outweighs that of the unnatural setting. Though it has been stated that the Web sites used in the tasks may be either more document or more spatially oriented, they should be similar to each other in as many other aspects as possible--amount of text, number of pages in the site, etc. Finally, it seems rather doubtful that in the space of this short experiment, subjects' Web competency will perceptibly change of its own accord.
However, since the intended population is "Web users", this study will have very questionable external validity. Due to language, geographic, and other barriers, it would not be possible to randomly sample from all users of the Web. Since subjects need to be physically present, even sampling of all Oahu Web users would be too inconvenient. It seems we must forgo a random sampling of the target population.
In an attempt to counter potential objections to such a non-random sample of subjects, this study would need to collect data on a number of other possible, confounding influences. These include subjects' age, gender, length of time using the Web, the frequency of Web use, and whether subjects have ever designed their own Web sites.
Another important consideration is subjects' familiarity with the metaphor source domains. That is, do they work with paper documents very frequently? Do they tend to get lost in large buildings or have other difficulties when navigating spatially? A post-experiment questionnaire could collect some of this data (though this assumes people can objectively self-report their abilities.)
Finally, if one metaphor seems to be more useful to Web users, it may be a correlation that people who excel in that source domain are also good at using the Web. For example, it may be that people who use the SPACE metaphor for the Web are not better surfers because of the metaphor. Instead, they may be competent Web users because of some other attribute that also improves spatial reasoning--such as a strong background in math, science, and technology. While asking subjects for their college major, hobbies, or career field may rule out the effects of this particular alternate hypothesis, there may be others.
Boland, Richard J. and Ralph H. Greenberg. "Method and Metaphor in Organizational Analysis." Accounting, Management, and Information Technology. 2.2 (1992): 117-141.
Bergen, Ben. Cognitive Linguistics group conversation. Magoos, Honolulu, HI. 10 Oct 2003.
| ~ztomasze Index:
CIS: 703--Assignment 2 http://www2.hawaii.edu/~ztomasze |
Last Edited: 13 Oct 2003 ©2003 by Z. Tomaszewski. |