POST-TEST ANALYSIS

After gathering all of the data, the recordings and notes were analyzed for general patterns and themes. Considering this was the beginning of our research and the first round of testing, we did not expect to come to any breakthrough conclusions—and we didn’t. But we did see some consistent patterns emerge with each user. Some observations surprised us, and others just confirmed our suspicions. For example, we found that students don’t quite understand what the word “reference” means. When asked to search for an electronic encyclopedia within a guide, only one student thought to look under the Reference tab. We also observed that students assume lists of databases are ranked by importance. And if given the option to choose JSTOR to search for articles, they will—even if it isn’t the most appropriate database to choose based on their research question.

While simple observation gave us plenty of useful feedback to make positive changes to our guides, we found that some of the most meaningful data came from the direct feedback of students. The think-aloud portion of the test allowed us to capture dozens of memorable quotes, some of which directly impacted work that was being done behind the scenes, such as creating jargon-free database descriptions. One student astutely observed, “Database descriptions help a lot to decide if I’m in the right direction or not. If I click on it and it turns out I don’t need it, I might waste two or three clicks.” Written suggestions from the post-test questionnaire also gave us plenty of ideas for future usability studies.

Usability Testing: LibGuides 2.0

After we finished testing the old platform, we were ready to tackle some of the design decisions for LibGuides 2.0. A major feature of the new system is the ability to change the guide layout from tabbed to side navigation. After doing some research on general user preferences for webpage navigation, the LibGuides working group determined that either layout would be suitable for our purposes, but we wondered if our users had a preference.

To test this, we turned to a free, web-based heat-mapping tool from Optimal Workshop called Chalkmark (optimal
workshop.com/chalkmark.htm). Chalkmark uses a screen shot of the page being tested, and a task is set for the user to complete on that page. The screen shot tests are distributed to users via email and completed on their own computers. The software records the location on the page where the user first clicks (also known as the “first impression”) and displays the data visually as a heat map. (See Figure 3 below.)

Using an A/B testing approach, we created one test for each layout and had users complete identical tasks on each. The tasks were things users would typically do on a LibGuides page, such as finding a book, finding an article, and finding help. Just as in the usability tests of LibGuides 1.0, we tried to recruit students via email, but we received very little response this time around. So we decided to change gears. With an iPad and a bucket of candy in hand, we roved the library asking for participants. This method turned out to be much more successful; we gathered data from 32 participants in total—16 for each navigation layout. (See Figure 4 below.)

In the end, each navigation option had roughly the same percentage of correct clicks. Not seeing a clear winner, we opted to keep the tabbed navigation layout with plans to test this again in the future.

Although the heat-map test did not show a clear layout preference, we gained other insights. For example, we discovered that when users are looking for help on a page, they are more likely to click on the librarian’s profile box if it is located on the left side of the page rather than the right. While it may seem minor, this small change can make a big difference in user experience.

Post Migration Work

Despite our efforts to do as much prep work as possible prior to migration, there remained a great deal of work consolidating links to databases and other resources. In addition, the evolving guidelines and templates require continued work as we move through the system to make existing guides more uniform. We are currently making use of high-level student staff to systematically review guides looking for orphan database links and replacing them with links to the corresponding database assets. They are also working through each guide owner’s link assets to consolidate any remaining duplicate assets.

There is yet another transition underway at Northwestern. We are preparing to move to a new integrated library system—Alma by ExLibris. The new record structure in Alma will require us to devise a project for replacing the multitude of links to specific catalog records that are currently present in our guides. Since many of our guides contain book assets (formerly Books from the Catalog box content), all the catalog references in these must be changed, plus there is an opportunity to further eliminate redundancies by consolidating duplicate book assets at the same time.

Future Plans

In keeping with our UX department’s philosophy of continuously checking in with our userbase, we plan to continue running frequent and small-scale usability testing with LibGuides. The LibGuides working group is currently designing  a new set of guide templates and developing a substantially revised set of staff guidelines. The templates will provide a solid foundation on which to build new guides, putting into practice the basic principles of our instructional philosophy. Templates will be designed and revised as we learn through continued usability testing what works for our users and what doesn’t.

We also plan to move our guidelines away from lengthy, verbose pronouncements of what shall and shall not be done and replace that with more engaging documentation—guides for staff with examples of best practices, instructions on how to use the features of the system effectively, and a more interactive environment where staff can share ideas. This holistic approach, incorporating perspectives from both the consumers and the staff, has served us well in our LibGuides migration, and we plan to take a similar approach in the future as we upgrade other Springshare applications.