User-Centered Design for Web Developers

A Bibliography Plan by Zach Tomaszewski

for LIS 601, Fall 2000, taught by Joyce Yukawa


Table of Contents




Introduction

Discussion of Topic

Ergonomics, human factors, human engineering, and human-machine systems are all areas of study involving how humans interact the machines and environments they build for themselves. Specifically, the study involves learning how to improve the machines or environments in order to make interacting with them a more relaxed, natural experience. Though these are still relatively young fields of study, they have been in existence for decades now. Recently, the related study of human-computer interaction, or HCI as it is often abbreviated, has developed as more of our daily work involves computers. Even more recently, the study of user interfaces, especially graphical user interfaces, has evolved. Again, the purpose of all these fields of study is to discover how to make the respective system easier to use--in other words, to increase its usability.

In the past four or five years, some of the major figures in the study of usability have turned their focus to the growing World Wide Web and the many poorly designed web sites it contains. Their work has resulted in many reports, articles, and guidelines on how to improve a web site's usability. This bibliography concerns these new, largely undocumented works.

Intended Audience

This bibliography is intended primarily for web designers or developers interested in learning more about designing sites with the user in mind. While many of the principles of usability apply to a great number of different areas of design, I will try to focus here on those specific to constructing web sites. However, as many of these sources may exist only as part of larger texts on the topics of usability or human-computer interaction, a user will often forced to search these broader areas to find more material.

Existing Bibliographies

During my research for this paper, I discovered the following existing bibliographies, one concerning web usability and the other concerning HCI:

Nature of this Bibliography

The primary focus of this document is not actually to create a bibliography. Rather, it is meant only to examine the field and devise a search strategy for later bibliographic development. It will examine some of the best search terms to use, determine which databases are likely to be most helpful, and discover any problems that are likely to arise.

The convention used is that all subjects or other controlled vocabulary will be stated in CAPITAL letters. Natural language terms used in queries will be stated in italics.

The style manual used for citations, when possible, is Kate Turabian's Guide for Writers of Term Papers, Theses, and Dissertations3

Endnote numbers are all hyperlinks from the text to the corresponding citation; links from the citations lead back to the appropriate point in the text. In the spirit of Internet connectivity, links have been provided to the sources discussed whenever possible, both from within the text and from the endnotes.

Preliminary Discussion of Search Terms

When I originally chose the topic of this bibliography, I phrased it as "user-centered design in web design." Better phrased as a natural language query, it would be: user-centered design AND web.

Natural Language

Before actually searching, I did some concept analysis. First I broke down my original topic/query into pieces. Then I tried to think of as many synonyms as possible for each of those pieces or combinations of pieces.

Original Conceptuser-centered design in webpage
Possibly synonyms usercentereddesignweb page(s) / webpage(s)
humanfocusedconstructionweb site(s) / website(s)
personinteractioncreationInternet
noviceinterfacebuildinginformation systems
accessible / accessibility engineering data systems
usable / usability   information space
      WWW / web

I am sure there are many more synonyms to these concepts and many more variations of the words above. But this was a good start. Using this table, I could reorder the many pieces to easily determine multiple possible search queries. It was not even necessary to take a word from each column. For example: human focused AND information systems, or simply accessible web sites.

Controlled Vocabulary

For a taste of likely controlled vocabulary I could use, I consulted the Library of Congress Subject Headings (LCSH). 4

The one related hierarchy, from broader to narrower, seemed to be:

    HUMAN-MACHINE SYSTEMS
        USER INTERFACES (COMPUTER SYSTEMS)
            GRAPHICAL USER INTERFACES (COMPUTER SYSTEMS)

Related to USER INTERFACES (COMPUTER SYSTEMS) was the heirarchy:

    HUMAN ENGINEERING
        HUMAN-COMPUTER INTERACTION
            NATURAL LANGUAGE PROCESSING (COMPUTER SCIENCE)

A list of other subject headings that seemed likely to show up were:

    WORLD WIDE WEB (INFORMATION RETRIEVAL SYSTEM)
    WEB SITES
    WEB PUBLISHING
    HYPERTEXT SYSTEMS
    INTERNET (COMPUTER NETWORK)
    INFORMATIONAL DISPLAY SYSTEMS
    INFORMATION SERVICES -- QUALITY CONTROL
    INFORMATION STORAGE AND RETRIEVAL SYSTEMS -- ENGINEERING
    COMPUTER SOFTWARE -- HUMAN FACTORS

Foreseen Problems

Whenever a person performs a information retrieval search, there are two factors at work: precision and recall. Precision is defined as the number of relevant retrieved documents divided by the total number of retrieved documents. When nearly every returned document actually concerns what you're looking for, your search had high precision. Recall is defined as the number of relevant documents retrieved divided by the total number of relevant documents in the system. If there are 103 documents in a database concerning the topic you're searching on, and you retrieve 98 of those documents with one query, your search had excellent recall.

A second thing to realize is that neither classification schemes or natural language searches are ever perfect. If a cataloger catalogs a book only under the broader heading USER INTERFACES (COMPUTER SYSTEMS) because it deals with many types of user interfaces, and then if you search only under the narrower term of GRAPHICAL USER INTERFACES (COMPUTER SYSTEMS), you will not retrieve the book. Yet the book may have three or four chapters of interest to you concerning graphical user interfaces. In order to retrieve this book and improve your overall recall, you must search under both the USER INTERFACES (COMPUTER SYSTEMS) and the GRAPHICAL USER INTERFACES (COMPUTER SYSTEMS) subject headings.

The same problem applies to natural language. If a document only contains the singular term web site (a very good chance depending on the author's writing style), and you search only for website (two words) or web sites (plural), you will not retrieve the document. This problem can be somewhat alleviated with truncation and wildcard characters on accepting systems, but it will always remain a problem. In order to reach a high level of recall you must perform many exhaustive searches with as many applicable word combinations as possible. Perfect recall is very rarely even possible, but recall can always be improved.

Yet, in most cases, when you increase recall you decrease precision. By entering a broader search to get more relevant documents, you also generally get more documents that are not relevant. You may have pages of non-relevant results to scroll through in order to find some of the last few relevant documents in the system in order to increase your recall.

The major problem with this specific bibliographic undertaking was the lack of predetermined levels of acceptable recall. Generally when people search, they find it more enjoyable to have high precision -- of the results they do get, most are useful to them. However, most (though not all) bibliographies try to be exhaustive on their topics, i.e., have near perfect recall on all relevant systems. Since this paper is only a sort of pilot study (indeed, it was only planned to record a total of 5 to 7 annotations!), this level of recall seems a ridiculous goal. So on one hand there seems to be a natural inclination to perform only a couple very precise searches to see what materials are available and quickly move on to the next system; and on the other hand, there is the need to experiment with a number of different searches of varying form, depth, and complexity in an attempt to imitate a pursuit of high recall. This duality of mind-set was never satisfactorily resolved during the research for this paper.

A second issue, which is really only an observation rather than a problem, is the redundancy of citing web sites in the endnotes when much more convenient (and usable!) links can be placed directly in the text. Indeed, this is the entire underlying beauty of hypertext in an academic setting! Links are based on URLs, or, more technically, URIs: Uniform Resource Identifiers. Since the purpose of citing is to show the source of information or ideas and to include enough information about the source that the reader can find it again, a hypertext link, based on URIs, by definition points to a unique source! There is no actual need for further information to differentiate it from other works. It may be polite to include the author (if it is known, which is often not the case with web sites); also it is good usable web design practice to include sufficient information to inform the reader about the destination of the link. But for normal web pages, a strict citing structure is made unnecessary by the very nature of URIs and their designation of unique resources. This paper includes both forms -- links and citations -- for "backwards compatability."




Searches

Preliminary and "Browsing" Searches

First I wanted to test the waters and see what kind of terms where actually being used. Also, browsing is an excellent, though often overlooked, complement to plain searching or querying.

UHCARL 5

To begin with, I did a search on UHCARL. This reason behind choosing this as a first database was two-fold. First, for the most extensive and for good introductory material, I always prefer a good book. Secondly, it gave me a chance to explore the controlled vocabulary I examined above and learn what was actually being used in real systems.

I first stated with a keyword search for web site design AND user. This pulled up only one book: Jared Spool's Web Site Usability: A Designer's Guide 6 in the Library of Congress call number area of TK 5105 .888. The descriptors listed in the record were:

      WORLD WIDE WEB (INFORMATION RETRIEVAL SYSTEM)
      HUMAN-COMPUTER INTERACTION
      WEB SITES -- DESIGN

In an attempt to get a few more results, I tried a second keyword search for user AND web. This still resulted in only a handful records. An example record is one for Design Wise 7, found in the QA 79 .9 area of the stacks. Its descriptors were:

      HUMAN-COMPUTER INTERACTION
      USER INTERFACE (COMPUTER SYSTEM)
      USER-COMPUTER INTERACTION
      SOFTWARE DESIGN
      EVALUATION STUDIES

As you can see, the only common subject heading between these two resources is HUMAN-COMPUTER INTERACTION. An investigation of this subject heading revealed almost 100 corresponding records, most in the QA 76 .9 range. Books on usability were evident in this list, such as Human Factors and Web Development 8. This book also had the headings:

      WEB SITES -- DESIGN
      WEB SITES -- PSYCHOLOGICAL ASPECTS
      WORLD WIDE WEB (INFORMATION RETRIEVAL SYSTEM) -- PSYCHOLOGICAL ASPECTS

Relevance, as expected, seemed to follow the heirarchy laid out in LCSH:

      HUMAN-COMPUTER INTERACTION   (broadest category; least relevant results)
USER INTERFACE (COMPUTER SYSTEM)   
WEB SITES (narrowest category; most relevant results)


As each of the above three headings retrieved between 80 and 100 records, there was no correlation between specificity of the heading and the number of records retrieved with it.

Other headings of note that I encountered were:

      GRAPHICAL USER INTERFACE (COMPUTER SYSTEM)
      SYSTEM DESIGN
      SYSTEM ANALYSIS
      SYSTEMS ENGINEERING

Besides the call numbers QA 76 .9 and TK 5105 .888, I ran across NC 998 .4, which concerns graphical aspects of interfaces. Of these, TK 5105 .888 proved to be the most relevant.

Overall, there was not one good subject heading. WEB SITES -- DESIGN was helpful, but included a large number of books only concerning HTML and other aspect of coding and construction that are not aspects of usability. WEB SITES -- PSYCHOLOGICAL ASPECTS was quite helpful, but limited in its use. Interestingly enough, a keyword search for web usability retrieves little besides the two most seminal books on the subject: Jared Spool's Web Site Usability: A Designer's Guide 9 and Jakob Nielsen's Designing Web Usability 10.

Rainbow Books: used computer books section 11

I chose this "database" as a way to see what terms were currently being most used to describe this topic in the publishing industry. Rainbow Books is a used book store. Their computer book section has six shelves of approximately 40 to 50 books each, making for an estimate of about 270 books total. Most of the titles looked to between 6 months and 5 years old.

Looking over the titles, approximately 10 of the books had to do with user-centered design. Some of the titles were repeats; there were only about 7 different individual titles. Of those books, only one did not have the term web usability either in the title or in the book cover's description. (This book, Looking Good on the Web 12, did mention user's perspective and easy navigation on the cover however.)

From this experience, I feel that web usability is the most relevant and pertinent term to base a search on.

Web Surfing

Starting with a site I already knew, Yale's Design and Development 13, I followed links from site to site. When I reached a site where no further good links on the topic of web usability or user-centered design were evident, I retraced my steps a few pages and tried again in a different direction. Since most sites on a particular topic link to others on that topic, this proved to be quite a fruitful technique. It lead me to such important usability and HCI sites as Usable Web14 ACM SIGCHI Web Special Interest Area15 Nielsen Norman Group16 and many more. Further browsing starting from Usable Web and returning there to start again whenever a dead end was reached would likely be one of the most varied, extensive, and current sources of information on this topic.

Again, the most common term, by far, used to describe this topic was web usability.

Search Engines

Since the topic of usability is relatively new and since it concerns the web and skillful construction of the web, it stands to reason that most of the pertinent resources will be on the web. Though the Internet and the World Wide Web serve as the basic data set for all search engines, each has a different way of indexing resources and different way of ranking results with respect to a query. Thus, it makes sense to try a few as each will give different results.

When querying search engines, I entered the query and then viewed the first 40 results to get an idea of both the effectiveness and kind of sites being retrieved. Usually I could tell from the title and brief description whether an article was relevant or not. If there was doubt, I would go to the web page and scan the actual content. On some of the pages, I actually read the entire page.

Northern Light 17

Northern Light is a good representative search engine. It has a large index and nice search features.

A search for "web usability" retrieved a little over 10,000 items. While not all of the initial results seemed equally in-depth or useful (some were only book reviews), practically all of the first 40 results appeared quite relevant, which is very impressive for a search engine web search.

A nice feature of Northern Light is their Custom Search Folders. These are an easy way for a user to, with a single click, reduce the number of results retrieved and, hopefully, increase the relevancy of those remaining. For example, selecting the Search Folder "World Wide Web" dropped the number of search results returned to 11 -- a very steep drop indeed from 10,000! Sadly, half of those were in Special Collections, which are articles that require a fee to be read, and overall the results seemed less relevant than the initial 40 results of the original search.

Examining some of the other folders, the number of results were significantly reduced, but the results were usually not as relevant. In my experience, this is a rare occasion with Northern Light's Customer Search Folders, which are usually a fast, excellent way to refine searches.

For comparison, I tried a search for "human computer interaction". (Though this has been a subject heading in previous discussion, it is in italics here since search engines like Northern Light only allow natural language searches and do not used controlled vocabulary.) This resulted in over 68,500 hits. While some such as Human Computer Interaction 18, contained information pertaining specifically to the World Wide Web and web design, most pages were broader in their coverage. Many conferences and university course descriptions were listed too, a sign either that HCI is a broader topic than usability or that it is more established as a discipline. These results were basically as I expected.

I then tried a search for my original idea: "user-centered design" AND web. After examining the first 40 of over 4,400, I determined that, on a whole, this was a more fruitful search than the one for "human computer interaction". Yet, I feel that "web usability" retrieves documents of greater merit and in a higher concentration of usefulness.

Google 19

Google is a search engine based on link popularity: those site that have more sites linking to them are given a higher ranking among returned search results. This makes it a good engine to find popular and well-known sites.

I began with the usual search: "web usability", which returned about 23,900 results. Google did fulfill its claimed ranking design since Usable Web 20 and Jakob Nielsen's site 21 were both in the top 5. Also included was Open Directory's usability category, which will be discussed later. Besides these big-name sites, there looked to be good relevance in the top 40 results. There were quite a few book reviews, but some guideline sites, and even discussion of a new company site that specializes in checking your web site's usability.

Searching for "human computer interaction" retrieved 164,000 results with Usable Web 22 again in the top 10. A quite reputable site in the HCI field, ACM/SIGCHI Home Page 23 was number 1. Though there was high computer interaction relevancy, there was very little specific to the web or web design. The top 40 results did included an interesting article entitled "Towards Accessible Human-Computer Interaction," 24 which, besides a nice discussion on aspects of accessible design, included the term universal design, meaning a design for everyone, not just the average user or disabled users.

"User-centered design" AND web produced about 5,600 hits with quite good relevancy. I noticed that many sites that have the keywords user-centered design also contain the word usability.

Direct Hit 25

Direct Hit is often marketed as a popularity engine since the results that get more clicks are ranked higher in future results.

Usable Web 26 is ranked number 1 in response to "web usability". Other top results exhibited decent apparent relevance. There were a few strange results (they were possibly false drops) concerning topics such as blenders and documentation usability. Also, there was an interesting article on the ethical requirements of both designers and users of web sites. 27 The relevancy of results dropped quickly from there, however, and by the last 10 reviewed (hits 31 to 40), none appeared relevant.

I learned that Direct Hit does not display the number of hits. Also, they have Related Searches feature, quite similar to Northern Light's Custom Search Folders, that looks to be quite helpful. On this particular search, Related Searches included "user centered design" and other usability-related searches It was also reassuring to note that, besides the few common sites I've mentioned here, most of the sites listed on the three different search engines were different. That means that checking different search engines is indeed a worthwhile activity.

Breaking up the order a little, I searched for "user centered design" AND web next. The results were practically worthless! They were all web design companies, which implies that the quotes and Boolean operator in the query didn't work. I explored Direct Hit's Advanced Search option, but user +centered +design +web was the best search I could construct, which apparently requires that all terms at least be present. With this reformulation, only two results were returned.

A search for +human +computer +interaction returned pretty typical HCI results (which are generally not helpful to a web developer interested in applying principles of usability to her work). The non-typical thing was that there were only 10 results. Direct Hit is not the place to go if you are looking for high recall! Since most web searches are primarily concerned with precision, this may not be considered a big problem by Direct Hit.

Directory Searches

Directories are often mistaken for search engines, but there is a fundamental difference: directories are compiled by human editors into custom categories. This means that all sites in a directory have supposedly been reviewed and so are usually of higher quality and relevance that the results of a search engine query. However, directories are often quite small and rather out-of-date since they are limited in size to the number of people working on them. Directories are the closest thing to controlled vocabulary available on the web. I explored directories in much the same way I have tried to explore controlled vocabulary: find all the categories that might contain pertinent information.

Yahoo 28

Yahoo is the most well-known and often-used directory.

Of course, burrowing through the many branches starting at "Home > Computers and Internet > Internet > World Wide Web" would likely have had the highest recall. Yet none of the branches immediately appeared particularly helpful. So I entered a search for "web usability" to get an idea of relevant categories and found the following:

"Home > Arts > Design Arts > Graphic Design > Web Page Design and Layout > useit.com" includes UseIt and the humorous "Jakob Nielsen Drinking Game." 29

Up one level to "Home > Arts > Design Arts > Graphic Design > Web Page Design and Layout" revealed lots of decent resources, including Patrick Lynch's (co-author of the Yale manual) classy homepage30 Though there is some good reading in this category, most of it is not terribly relevant.

Down one level to "Home > Arts > Design Arts > Graphic Design > Web Page Design and Layout > Accessibility", I found 9 links, all very relevant to accessibility, which is an large part of usability.

There was some very mildly related information among the 6 links in "Home > Social Science > Communications > Writing > Writing for the Web."

"Home > Science > Computer Science > User Interface" was probably the most relevant category so far. Among the 16 or so links to UI design stuff were links to UseIt31 Usable Web32 and other recognized usability sites.

"Home > Science > Computer Science > Human-Computer Interaction" also has one or two usability sites. I thought I might find something helpful in "Home > Science > Engineering > Ergonomics", but I did not. Searches for user-centered and interface did not reveal any further useful categories.

Overall, Yahoo covered the major sites on the topic, though I was rather surprised there was not just a little more material available.

Open Directory 33

Open Directory is a directory maintained by volunteers. It is used by many of the large search engines to complement their search results.

I started directly with "Top: Computers: Internet: WWW: Web Usability," which I discovered in a search at Google. Other categories could then be found just be following the "see also references." The Web Usability category alone has 127 links, making it a great resources. It contains the big-name sites as well as a great many sites that I did not encounter at all through search engines or browsing.

Moving down a level in the heirarchy to "Top: Computers: Internet: WWW: Web Usability: The Experts," this category contains 27 links to articles and sites by only the well-known people in the web usability field.

A "see also" reference took me to "Top: Computers: Software: Internet: Clients: WWW: Browsers: Accessibility," which, as I've said before, is an issue that shares a few important features with usability. Open directory's accessibility category contains most of Yahoo's 9 entries plus many more for a total of 26 entries.

"Top: Computers: Human-Computer Interaction" contains the usual mix of over 100 different HCI resources, some of which included top usability sites.

Besides some companies' and consultants' pages, "Top: Computers: Software: User Interaction" has some HCI stuff listed. There were no noticeable usability resources here, however.

An interesting category topic in its own right, "Top: Reference: Knowledge Management: Information Architecture" contains almost of 40 links. A very small number--two or three only--are on accessibility or usability.

The only useful item I found in "Top: Science: Social Sciences: Psychology: Industrial and Organizational: Human Factors and Ergonomics" was a link to the Usability Professionals' Association 34, whose resources page has some interesting links to a handful usability and HCI sites.

The category "Top: Computers: Internet: WWW: Authoring: Style Guides" includes 8 style guides, one of which specifically mentions usability in its description. Yale CAIM Web Style Guide 35 is also included in the list.

I have not used Open Directory much in the past. Yet, on this search, I found it to be a great resources. The relationships between different categories in the heirarchy are well noted, making it easy to find huge amounts of related information. Each category is quite extensive in the number of resources it contains, especially when compared to the respected Yahoo directory.

Traditional Online and CD-ROM Database Searches

Expanded Academic Index 36

The Expanded Academic Index covers a number of different scholarly journals and popular magazines.

Starting out with a keyword search web usability, I found 11 resources, half of which were book reviews. There were no common descriptors! Possibly useful entries might be under WEB SITES - PLANNING or WEB SITES -- MANAGEMENT in this system.

And so, following my one lead on controlled vocabulary, I examined the over 150 subdivisions of WEB SITES. The following descriptors were of interest:

There were also many of the same subdivisions of WORLD WIDE WEB SITES. This seemed to be to be a lack of authority control since WEB SITES and WORLD WIDE WEB SITES are generally considered to be synonymous.

EAI also contains over 20 subdivisions of HUMAN-COMPUTER INTERACTION. Articles here matched past discoveries concerning HCI resources: as a whole, they were somewhat related to web usability, but only slightly. Likely related terms include NATURAL LANGUAGE INTERFACES, USER INTERFACES, and GRAPHICAL USER INTERFACE, the latter two each with around 25 subdivisions.

A search for user centered design web produced 2 articles, neither of which particularly useful. One was an interview of an author of a book on web design that mentions a user centered approach.

Overall, this is not the best place to find pertinent material, which was anticipated. What is found here on the topic is mostly articles discussing the ramifications of usability, which is not of much help to web designers trying to actually implement it.

Uncover 37

Uncover is a database of journal descriptions and tables of contents. As such, it does not include controlled vocabulary searches.

Performing a word search for "web usability" retrieved thousands of articles for web and over 300 for usability. Since there were so many articles for web, Uncover disregarded the term and only returned results containing usability. Apparently word order matters in how Uncover searches, however, because when I did a search for usability web, it retrieved only 19 items. Not all were relevant, probably because the terms were searched for as usability AND web and not together as the appropriate phrase "web usability".

Human computer interaction resulted in almost 400 articles. Simply adding the term web to the query dropped the total results to 3. One of the remaining articles mentioned an online HCI bibliography 38. I don't recall running across that before in my search, though I was not focusing on HCI as much as usability.

Searching for user centered design pulled up only about 40 items. Again, adding web dropped it to 2. Adding internet (instead of web) also dropped it to 2, one of which is different from the 2 retrieved with user centered design AND web.

As with EAI, this database contains journal articles, reviews, interviews, etc. It will likely be hard to get exactly what I'm looking for in such databases since most such articles do not focus on implementation, which is of greatest interest for the intended audience of web designers. Also, in retrospect, the searches in Uncover were much to brief. Considering this is a natural language search, as many different variations of the terms as possible should have been used. For example, all combinations of web, website, webpages, etc. should have been used.

PsychLIT 39

PsychLIT is a CD-ROM database of scholarly psychology journal articles. I used the section covering the most current articles, from 1998 to Oct. 2000.

Web AND usability retrieved 16 abstracts of high relevance, most discussing studies such as usability in older users and the effects of graphics on perceived usability. Adding the term design only dropped the total results by 2, which I think lends support to my claim that these articles are indeed useful to web designers.

HUMAN COMPUTER INTERACTION in the descriptor field pulled up over 450 results. Adding the keyword web dropped it to about 50 and then adding design led to only 20 articles, again all quite relevant.

"User centered AND design" only revealed 15 entries. Adding web dropped that to 3. Interestingly, these had nothing to do with web sites or web pages. Rather, they all concerned searching and search engines.

An interesting note: user centered pulled up 15 documents. On the other hand, usability pulled up over 300, showing once again that it is the more commonly used term. Combining the two searches led to only 5 documents. This is a good demonstration of how increasing precision reduces recall. By not searching for both of these terms, separately, a good deal of material is likely to be overlooked.

I enjoy using the SilverPlatter interface -- it's so easy to just keep adding terms to refine searches! While this is generally great for precision, any in-depth study seeking a comprehensive review of the most resources available would have to be more cautious in order to avoid missing certain topics or articles by wanton use of added keywords.

Print Searches

No search exercise is complete without checking a few print resources.

Balay's Guide to Reference Books 40

Though a nice guide, I did not expect to find much in the form of a reference book beyond what I had already found in print through UHCARL.

I found nothing in the index for the following terms: usability, accessibility, user-centered, World Wide Web, or human computer interaction.

Under Internet I did find a journal 41 that reported itself to be "a guide to journals, newsletters, texts, discussion lists, and other resources on the Internet." With its age and broad focus, it is not terribly useful.

Under Human engineering handbooks, I found a couple books on human factors design and human engineering, but they were not specific to usability on the web.

New Encyclopaedia Britannica 42

As expected, web usability is too specific to be covered in Britannica. There was brief discussion of human factors engineering, user interfaces in telecommunications systems, user orientation in systems engineering, and a brief article on the World Wide Web as an information retrieval system. Basically, this is not a source worth referencing on the topic of usability.




Conclusions

These searches were far from exhaustive. Yet many more resources could be found by simply applying the basic search principles used here: concept analysis to construct more natural language queries, using more different combinations of search terms, exploring the relationships between controlled vocabulary to increase the range of material examined, etc. Of course, total recall of all relevant materials is impossible, especially in a constantly growing system like the Internet. So some threshold of sufficient recall, though arbitrary, should be determined before the searching is started.

In the limited confines of this search, I think it has been shown that each system is different. The same queries -- web usability, HUMAN-COMPUTER INTERACTION, and user-centered design AND web -- produced different results depending on the system used. For example, web usability easily retrieved over 40 very relevant documents on Northern Light; the same search on Direct Hit produced less than 20, though both engines are working on indexing the same Internet data set. Also, the queries cannot even be consistently applied on all systems. Not all systems have controlled vocabulary, so HUMAN-COMPUTER INTERACTION necessarily becomes human computer interaction in many cases. Also, strange behavior such as with Direct Hit -- submitting user +centered +design +web rather than "user centered design" AND web and receiving very different results -- make it hard to compare the same search across all systems.

Working with the data gathered, however, I would say that a natural language keyword search for web usability is undoubtedly the best way to start off on any system. Following up with related keyword searches or exploring the related controlled vocabulary is a good follow up.

HUMAN-COMPUTER INTERACTION was quite a reliable search term for getting some information on most systems, but since it is a broader concept than web usability, there will always some sifting that needs to be done in order to find anything actually relevant. Still, HUMAN-COMPUTER INTERACTION or WEB SITES are likely the most helpful common controlled vocabulary terms.

User centered design AND web was not very helpful. It is a good example of a query that could be used to increase recall over and above what can be found with web usability.

Web usability is an increasingly important field and eventually someone will likely publish an official bibliography. But until then, refer to some of the fine resources covered in the Sample Annotations of this paper, and you'll be off to a good start.




Sample Annotations

Electronic Resources

Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

A directory of (currently) 1,134 links to online usability articles, guides, and resources. The author explains that resources must deal with both usability and web site construction in order to be included. The links are divided into categories, such as Design Tips, Methods, Events, Issues, etc. Probably the most comprehensive online webliography available on the topic of usability in web design.


Lynch, Patrick, and Sarah Horton. "Yale CAIM Web Style Guide." <http://info.med.yale.edu/CAIM/manual/> Accessed: 24 Nov. 2000.

Likely the most respected and popular web style guide. Discusses many of the same issues brought up by usability experts: page layout, site navigation, color choice, etc. Published by Yale's Information Technology Services--Medicine department, which is very active in both web design and instructional software development. Originally written in 1995 and updated.


"Nielsen Norman Group: Strategic Human Centered Design." <http://www.nngroup.com/> Accessed: 24 Nov. 2000.

Home of both Jakob Nielsen and Donald Norman, both long-time, reputable names in the usability field. They have recently focused more attention on web design. Contains information on coming conferences and services and links to the directors' respective web pages. However, for actual guidance on user-centered design, it is better to look directly to their work or to other sites.


"Alertbox: Jakob Nielsen's Column on Web Usability." <http://www.useit.com/alertbox/> Accessed: 24 Nov. 2000.

A collection of column articles by the biggest online name in web usability. Includes every article dating back to mid-1995. Was formerly monthly, but articles are now released bi-weekly. Most articles are brief and focus on a single issue of usability.

Print Resources

Nielsen, Jakob. Designing Web Usability: The Practice of Simplicity. Indianapolis, IN: New Riders, 2000. [TK 5105 .888 .N56 2000]

By a well-known usability expert, this is a perfect resource for a web designer concerned about usability. Has tips and guides to construction often with reasons or statistics of user behavior to illustrate the need for them. Covers page design, site loading speed, style sheets, frames, navigation, and more.


Spool, Jared, et al. Web Site Usability: A Designer's Guide. San Francisco: Morgan Kaufmann Publishers, 1999. [TK 5105 .888 S72 1999]

Based on research of actual users. Discusses navigation, within-site searching, graphics, etc. Diagrams, analyzes, and rates real web sites to illustrate principles. Often cited in the usability field for its factual, testing-oriented focus.




Endnotes

1. Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

2. Perlman, Gary. "HCI Bibliography: Human-Computer Interaction Publications and Resources." <http://www.hcibib.org/> Accessed: 26 Nov. 2000.

3. Turabian, Kate L. A Guide for Writers of Term Papers, Theses, and Dissertations. 6th ed. Chicago: University of Chicago Press, 1996. [Ref LB 2369 .T8 1996]

4. Library of Congress. Library of Congress Subject Headings. 21st ed. Washington, DC: Library of Congress, Cataloging Distribution Service, 1998.

5. UHCARL. (OPAC at University of Hawaii-Manoa, Honolulu, HI)

6. Spool, Jared, et al. Web Site Usability: A Designer's Guide. San Francisco: Morgan Kaufmann Publishers, 1999. [TK 5105 .888 S72 1999]

7. Head, Alison J. Design Wise: A Guide for Evaluating the Interface Design of Information Resources. Medford, NJ: CyberAge Books, 1999.

8. Forsythe, Chris, Eric Grose, and Julie Ratner, eds. Human Factors and Web Development. Mahway, NJ: Lawrence Erlbaum Associates, 1998. [TK 5105. 888 H86 1998]

9. Spool, Jared, et al. Web Site Usability: A Designer's Guide. San Francisco: Morgan Kaufmann Publishers, 1999. [TK 5105 .888 S72 1999]

10. Nielsen, Jakob. Designing Web Usability: The Practice of Simplicity. Indianapolis, IN: New Riders, 2000.

11. Rainbow Books and Records. 1010 University Ave. Honolulu, HI.

12. Gray, Daniel. Looking Good on the Web. Scottsdale, AZ: Coriolis, 1999.

13. "Yale's Web Design + Development: Resources: Links." <http://its.med.yale.edu/wdd/wdd_resources/index.html>. Accessed: 22 Nov. 2000.

14. Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

15. "ACM SIGCHI Web Special Interest Area." >http://www.acm.org/sigchi/web/> Accessed: 24 Nov. 2000.

16. "Nielsen Norman Group: Strategic Human Centered Design." <http://www.nngroup.com/> Accessed: 24 Nov. 2000.

17. Northern Light. <http://www.northernlight.com> (Search Engine.)

18. "Human computer interaction." <http://www.slais.ubc.ca/courses/libr500/2000-2001-wt1/user.htm> Accessed: 26 Nov 2000.

19. Google. <http://www.google.com> (Search Engine.)

20. Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

21. "Alertbox: Jakob Nielsen's Column on Web Usability." <http://www.useit.com/alertbox/> Accessed: 24 Nov. 2000.

22. Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

23. "ACM/SIGCHI Home Page." <http://www.acm.org/sigchi/> Accessed: 26 Nov. 2000.

24. Bergman, Eric, and Earl Johnson. "Towards Accesible Human-Computer Interaction." Advances in Human-Computer Interaction. vol 5. Ed. Jakob Nielson. Norwood, NJ: Ablex Publishing Co., 1995. Available at "Sun Microsystems Accessibility Program - Accessible Human-Computer Interaction" <http://www.sun.com/access/developers/updt.HCI.advance.html> Accessed: 26 Nov 2000.

25. Direct Hit. <http://www.directhit.com> (Search Engine.)

26. Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

27. McDaniel, W.D., and P.C. McGrew. "Ethical Considerations in Web-Site Design: A Planar Approach." <http://www.enable.evitech.fi/enable99/papers/mcdaniel1/mcdaniel1.html> 28 Feb. 1999.

28. "Yahoo!" <http://www.yahoo.com> (Web Directory.)

29. "The Jakob Nielsen Drinking Game." <http://www.rc3.org/clips/nielsen_drinking_game.html> Accessed: 26 Nov. 2000.

30. "patricklynch.net home." <http://www.patricklynch.net/> Accessed: 26 Nov. 2000.

31. "Alertbox: Jakob Nielsen's Column on Web Usability." <http://www.useit.com/alertbox/> Accessed: 24 Nov. 2000.

32. Instone, Keith. "Usable Web." <http://usableweb.com/> Accessed: 26 Nov. 2000.

33. "Open Directory Project." <http://www.dmoz.org> (Web Directory.)

34. "Usability Professionals' Association." <http://www.upassoc.org> Accessed: 26 Nov 2000.

35. Lynch, Patrick, and Sarah Horton. "Yale CAIM Web Style Guide." <http://info.med.yale.edu/CAIM/manual/> Accessed: 24 Nov. 2000.

36. Expanded Academic Index ASAP, InfoTrac, UHCARL database. 1997-

37. Uncover, InfoTrac, UHCARL database.

38. Perlman, Gary. "HCI Bibliography: Human-Computer Interaction Publications and Resources." <http://www.hcibib.org/> Accessed: 26 Nov. 2000.

39. "PsychINFO." PsychLIT. Boston: SilverPlatter, 1998 - 10/2000. CD-ROM.

40. Balay, Robert, ed. Guide to Reference Books. 11th ed. Chicago: American Library Association, 1996.

41. Internet World. Internet World's On Internet. Westport, Conn: Mecklermedia, 1994-. [Ref TK 5105 .875 I57 I585 1994]

42. New Encyclopaedia Britannica. 15th ed. Chicago : Encyclopaedia Britannica, 1997. 30 vols.




Usability features included in the construction of this document

Valid HTML 4.0!     Valid CSS!