SUBSCRIBE NOW!

Vol. 23 No. 1 — January 2003

NIST is a nonregulatory federal agency within the U.S. Department of Commerce. Its mission is to develop and promote measurements, standards, and technology to enhance productivity, facilitate trade, and improve the quality of life.

The Research Library supports the research activities of the NIST scientific and technological community. The library, which resides on the main NIST campus in Gaithersburg, Md., serves a population of 2,700 and has a collection of more than 200,000 items. As a research consultant and reference librarian, I was a member of the NVL Redesign Team, which undertook the mission of redesigning our Web site in the spring of 2001.

The NVL's Focus on Success

To publicize this major redesign of the NVL, we borrowed a playful and appropriate theme from the popular TV series Star Trek: The Next Generation. We designed a poster to showcase the "Next Generation NVL" as it was transformed from the old to the new in Star-Trek-like fashion. Ours is a research institution of physicists, chemists, and engineers who could easily relate to Star Trek, transporters, and anything having to do with quantum science.

When word went out that the NVL was to undergo a major redesign, there was concern among some NIST scientists that the NVL might lose some of its value or that its resources might become difficult to find. "Don't mess with the NVL!" was how one customer put it. To make sure that we didn't stray from what was good and worked well on the NVL, ISD staff conducted focus groups to learn what our customers liked about the NVL and what they wanted to see changed. Later on, we conducted Web usability testing on the redesigned site to ensure that we had indeed made the right changes. We knew from the beginning that, although we'd be testing users' ability to navigate and find information on our redesigned site, our users would also be comparing the new site's functionality and usability with the old one's. So it would be important for us to prepare carefully and earn our users' approval!

Although there are many elements and phases to redesigning a large-scale Web site such as the NVL, in this article I'll be focusing on Web site usability and the testing that we did to make sure our Web pages allow our customers to navigate the site easily and to find the information they want.

Listening to Our Customers

When the Web site was originally designed in 1994, frames were popular but presented some usability problems. On the left side of the page was a navigation bar made up of image buttons (see Figure 1). The main body of the page contained content that focused on what was new on the NVL. No one kept up with this What's New section, and eventually the lack of attention caused it to become dated. This section, and others like it, also lacked a well-defined system for archiving.

To redesign our Web pages, we needed to make sure that we knew what makes a site "usable." Close to the time of the NVL redesign project, Web sites throughout our parent organization, Technology Services, were also undergoing redesigns. As a part of the learning process, Web usability training was arranged for a small group of Technology Services staff members. I was one of about 30 people who attended the 2-day, on-site training session, Web Sites that Work: Designing with Your Eyes Open, in January 2000. In February 2001, my colleagues and I attended a 1-day refresher course in Web usability training.

Both workshops, conducted by Jared Spool of User Interface Engineering, were lecture/discussion sessions emphasizing audience interaction and hands-on participation. During these courses we learned how to design and conduct Web usability tests. We also learned how users follow the "scent of information" to navigate sites and how to design links that increase that scent. Spool explained that links on a Web site give off "scent," or provide visual or verbal cues that users follow. If a link's scent is strong, customers will more quickly locate the information they want.

"When they heard our site would undergo a major redesign, some of our customers were worried that it would become harder to use."

Before we began any redesign efforts, we wanted to learn what our customers liked about the NVL and what changes they wanted to see. To create a more customer-oriented Web site, we conducted two focus group sessions in April 2001. The focus group results showed that customers were generally happy with the many resources accessible through the NVL but that they wanted more. They wanted specific databases and always more e-journals—the more full-text information, the better. They also wanted a streamlined process to get to the information. In essence, they wanted one-stop shopping. The site's navigation also needed attention, and at least one or two participants said that the site was not very attractive. So, we prepared to set a new standard!

Based on our understanding of site usability from the workshops and the results of our focus group sessions, we began to design a new NVL home page in the summer of 2001. The new home page linked to most of our top-level content, to achieve the "one-stop shopping" that users had requested. We placed the navigation bar on the right-hand side of our secondary pages, the same side as the mouse for our right-handed user majority. This was an experiment, in response to some users' frustration with the old NVL's left-side navigation bar. (Research at the National Cancer Institute has shown that users click with greater efficiency on topics that are located on the right, closer to the scroll bar.) At this stage, we also carefully selected names or labels for the links. A subgroup of the NVL Redesign Team tackled this project and came up with an overall design similar to our current home page (see Figure 2).

The Usability Test: Our Users Inspect the New Site

Once we had redesigned the site and moved content into a new content management system, we were ready to test the new NVL to see how well it worked. Our Web usability training had given us the background and foundation of knowledge to create the right testing conditions for our Web site.

To further my understanding, I attended the ASIS&T 2002 Information Architecture Summit held in March in Portland, Ore. I found one particular program by Jason Withrow to be especially useful in helping design Web usability tests. His presentation, Choosing the Best Path: Techniques for Assessing and Improving Scent, focused on developing good information scent on Web sites. There, I also learned about quantitative and qualitative user test metrics, which I'll get to later in this article.

"Our Web usability testing demonstrated that, overall, the new Web site was well-organized and easy to navigate."

Our first step in doing Web usability testing was designing the list of questions to ask our subjects. Our aim was to keep the usability testing session short—no more than half an hour—so we included just nine questions. As in any research institution, our scientists are busy and we wanted to make participation as appealing and as simple as possible.

Here is a list of some of the questions that we asked our test subjects. Although the questions are unique to the information found on our Web site, we tried to focus on some of our more popular products and services.

• What journal articles have been published on quantum computing over the last 3 years?

• Does the NIST Research Library provide electronic access to the journal High Performance Polymers?

• Find the full text of the following patent: Nanoelectrode arrays, U.S. 6,325,904.

• What is the exchange rate converting 2,000 euros into U.S. dollars?

• What is the proper format for citing a book in the Journal of Research of NIST?

• What meetings are being held at NIST today?

• Whom do I contact for help with printing my NIST Special Publication?

We conducted usability test sessions with five NIST scientists. We selected the five subjects from a list of over 30 possible participants. We had learned from Jared Spool that we did not have to have a large set of test subjects to learn if our redesign efforts had been successful. Working from the longer list of possible participants, it didn't take long to find five individuals who were willing to test the new NVL. Most seemed eager to see the changes and improvements.

Although our focus group sessions had pulled from a mix of NIST staff members, including support staff and researchers, participants in our smaller Web usability test group were scientists, chosen because NIST scientists and researchers are the primary customers and users of our Web site. From the list of 30, I selected individuals whom I knew would be supportive of our efforts to improve the Web site. I knew most of these people to be open in both their praise and criticism of our site. Not all of them were frequent users, which offered a broader critique.

We gave participants the choice of holding the usability testing in their offices or in the library. Three of the five chose their offices while the other two came to the library, where they were located in a private area away from most distractions.

How Users Rated Our Site

We used quantitative and qualitative test metrics to measure the success of our redesign project. In quantitative terms, we measured path directness, path frequency, and time and completion rates. To measure path directness, we determined the "optimal path" to the right answer and the number of clicks it should take to reach that point. We then calculated the number of clicks it took our test subjects to reach the destination and compared that number to the optimal path. We also looked at what paths were chosen most frequently, how long users spent on each question, and whether they found the appropriate answers. Qualitative metrics included recording verbal user comments and looking for signs of indecision or indications of frustration or confusion. The form that I used to record test results is represented in Figure 3.

How quickly did our usability test subjects find answers to the questions? The speed of locating answers was, in most cases, quite fast. I found that either the subjects were able to quickly reach their destinations—or they were completely stumped and were never able to find the answers. Our results showed that some questions were answered in as little as 10 seconds, while in a couple of cases no answer was found and the task ended without success after 5 minutes.

One question had all but one person stumped. Subjects were asked to convert euros to dollars, which required an exchange rate converter.At the time the usability testing was being conducted, the answer to this question could be found under the topic link labeledDictionaries, ETC. We thought this category nicely described a section where customers could find a wide range ofWeb resources such as dictionaries, a thesaurus, and phone directories. However, this was not apparent to our test subjects. We quickly learned that we needed to find another label for this category of resources. Reference Shelf was a natural.

Our Web usability testing demonstrated that, overall, the new Web site was well-organized and easy to navigate. In most situations, our customers seemed to be able to move around the site quickly and find needed information easily. Even our experimental right-hand navigation bar seemed to work well. The testing did uncover a few problems, mainly with the labeling of links that didn't precisely convey content. Based on these results, we renamed some of the labels and changed the locations of some items on the site. But we were pleased that for the most part the test results indicated that our new design was a success.

Important Lessons Learned

One of the valuable lessons that we learned from the process of redesigning our Web site was the importance of customer input. It seems only logical that to design a more customer-focused Web site, you need to know your customers, what they want, and how they find information on the Web. Their direct involvement from the beginning to the end was critical. We believe that a large part of our success was due to our careful use of focus groups before we began redesigning the NVL and to the Web usability testing we performed near the end of the process, which redirected some of our work to make the final product even better. In the end, we all agreed that we'd won our customers' stamp of approval at NIST—thanks to careful planning and our customers' valuable input!

 

Further Reading Fleming, Jennifer. Web Navigation: Designing the User Experience. Sebastopol, CA: O'Reilly & Associates, Inc., 1998. Morville, Peter. Information Architecture & Organizational Strategy. ASIS&T Annual Meeting. Washington, DC, November 2001. National Cancer Institute. Research-Based Web Design and Usability Guidelines. http://usability.gov/guidelines. Rosenfeld, Louis, and Morville, Peter. Information Architecture for the World Wide Web. Cambridge; Sebastopol, CA: O'Reilly & Associates, Inc., 1998. Now available in its 2nd edition, 2002. Spool, Jared. Web Sites that Work: Designing with Your Eyes Open. User Interface Engineering, 2000. Workshop manual used during NIST on-site training. Spool, Jared, Tara Scanlon, Will Schroeder, Carolyn Snyder, and Terri DeAngelo. Web Site Usability: A Designer's Guide. San Francisco, CA: Morgan Kaufmann Publishers, Inc., 1999. Withrow, Jason. Choosing the Best Path: Techniques for Assessing and Improving Information Scent. ASIS&T 2002 Information Architecture Summit, Baltimore, MD, March 2002.