Using Participatory Design to Improve Web Sites
by Tatiana Nikolova-Houston
In a previous adventure (Computers in Libraries, April 2002), I told
readers how I saved deteriorating treasures for further study by digitizing
Slavic medieval manuscripts. A great deal of trial and error went into capturing
those photos and producing the Web sites to display them. Now, let me share
with you my recent experiences and the lessons learned as the work has continued.
These les sons include discovering the historical accident underlying much
medieval study, the resulting lack of Web sites, and how participatory design
has mitigated this problem.
The Historical Accident
"Why were the Middle Ages called the Dark Ages?" my daughter challenged me
one day. I started to lecture her, and she interrupted me impatiently: "No!
Because there were so many knights! Do you understand, mommy? Because of the
(k)nights?" Seriously though, there is a reason for the nickname. One of the
earliest and most important art historians, a 16th-century Italian named Giorgio
Vassari, decided that Byzantine art was "ugly and clumsy."1 He labeled
the Middle Ages as the Dark Ages, referring specifically to Byzantine art.2 Not
only did Vassari dislike Byzantine art, he labeled Byzan tine art history as
a mere "option." Because of Vassari, the gap between East and West widened.
For example, based upon my interviews with professors of library history, art
history, Slavic languages, and medieval hypertextuality, western European medieval
studies have distanced and marginalized southern Slavic medieval studies.
Vassari's 16th-century historical accident still affects the availability
of resources and collection development in libraries today, as Slavic resources
wither in the margins of library collections while French and English literary
genres flourish and occupy entire floors of libraries. One professor of Slavic
languages stated: "Demoralizing! One little quarter on the fifth floor was
dedicated to Russian, and within this corner, a tiny bit of corner to other
The Situation Today
Five years ago, the Internet contained no Web sites with images of Slavic
manuscripts.3 Today, there are only three. Bulgarian and other Slavic
scholars grieve about this "clear gap" in Slavic cultural heritage resources
on the Web.4 They blame obsolete data formats and tools, linguistic
difficulties, lack (with minor exceptions) of manuscripts that already have
been digitized, and lack of financial support for digitizing manuscripts and
resolving copyright issues. Other factors include the wide variety of computer
hardware and software that prevent standardization.5
I asked myself several questions: Can any librarian confront the academic
community? Could I refute Vassari? Could I fill this gap in Slavic resources
with online materials? If I did, would scholars accept them? How do you measure
acceptance? The answers lie in participatory design (PD), a method in which
the researchers and scholars who use a Web site help to design it.
But would they help me re-create my sites?
Participatory design (PD) originated in Scandinavia in the 1970s as a way
to empower workers by involving them in the design of tools and artifacts.6 The
method drew on the workers' "tacit knowledge"—i.e., their implicit or
un ar ticulated knowledge learned and transmitted through experience and apprenticeship.
At first glance, PD sounds like user-centered design, but there are differences.
PD is design by users, while user-centered design is design for users.7
In today's library school curriculum, we have developed several disciplines
that design user-friendly computer interfaces, such as user studies, information
architecture, and human-computer interaction. However, Web designers rarely
involve users in the actual design process.8 Whether it's a cause
or an effect of not involving users, contemporary interface design focuses
on usability, not functionality.9
Back in 2002, I had digitized manuscripts, expecting to create a Web presence
for Slavic medieval manuscript treasures and to fill the existing gap in Slavic
resources. The site I created received a warm welcome, but I felt it was incomplete.
Then, in spring 2005, I discovered PD. It seemed to be the answer, since it
involves the users in creating the intellectual content, design, and navigation
of a Web site. I resolved to use PD to improve my three Slavic manuscript sites:
• Slavic Medieval Treasures from Bulgaria (2002), http://www.ischool.utexas.edu/~slavman
• Byzantine Medieval Hypertexts (2003), http://www.ischool.utexas.edu/~slavman/hypertexts
• Slavic Medieval Treasures (2004), http://www.ischool.utexas.edu/~slavman/manuscripts
The Three Stages of Participatory Design
Traditionally, PD occurs in three stages. During the first "discovery" stage,
the researcher-designer gains the trust of the participants, explores their
working practices, and studies their goals, values, and needs. She gets additional
context from examining the visual and textual sources used in practice.
In the second "evaluative" stage, the participants explore and evaluate the
artifacts (e.g., the three Web sites), focusing on their strengths and weaknesses;
then they encourage each other to tell about their positive experiences with
The third "prototyping" stage involves brainstorming with the participants
as they suggest ideas, sketch concepts, and envision future use and developments
in the field. Finally, the participants evaluate the new design and approve
its final version.
Stage 1: Discovery
I began the PD process by asking faculty and students from the library history
and the Slavic, Byzantine, and medieval studies departments this question: "Would
you use my Web site in teaching and research if you and I, creator, patron,
and professor, work hand-in-hand to create it?" Other questions followed. "What
information sources do you need and use? Would online Slavic manuscripts change
current teaching practices? Can primary sources be presented online?"
I collected answers to these and other questions in semi-structured interviews,
written surveys, classroom observation, and by examining documents such as
class syllabi. Some of the interviews occurred through e-mail exchanges, which
greatly facilitated the transcription process.
Good research requires verification. I verified my data through the use of
different methods of collection, a pro cess known as triangulation. In this
case, the face-to-face interviews and the classroom observations produced the
same information as the data collected from e-mails, documents, and surveys.
To further authenticate the data, I asked each informant to confirm the accuracy
of my version of their statements, a process called "member checking."
Here's what I found out: Here are the answers I received to my initial
• Would you use my Web site in teaching and research if
you and I, creator, patron, and professor, worked hand-in hand-to create
it? All participants expressed skepticism about the credibility and quality
of Internet sources. For example, one person said: "The Internet? I don't
trust the Internet except for quick information." Most professors and many
students echoed this statement: "Those are intellectual toys!" However, many
of the participants used online databases and bibliographies. Could I expand
that Internet usage with their help?
Virtually every person I asked agreed to participate in the redesign of the
sites. Will they continue to use the sites in the future? They promised to
do so, and I believe they will.
• What information sources do you need and use? Although
the faculty members instructed students in the importance of research with
primary sources, almost everyone relied on secondary sources. Their reasons
varied: Some faculty members had developed excellent private libraries and
felt that these secondary sources were sufficient for their teaching needs.
Since students had 70 to 120 readings during a semester, they tended to follow
Zipf's Principle of Least Effort and to search for online sources. Even with
this student preference for the Internet, none of the participants trusted
online sources at the commencement of the project.
• Would online Slavic manuscripts change current teaching
practices? Teaching practice varied by discipline and professor. Library
history lent itself to lectures illustrated with slides and video. Slavic
language and literature are studied by reading and discussing transliterated
texts, with cultural and historical context added by the professor. In one
innovative course on the hypertextuality of medieval manuscripts, students
interacted with man uscripts and re-created them as electronic hypertexts.
This approach empha-
sized the primary sources—the manuscripts. Byzantine art history required
a discussion mode of teaching with high-quality visual images. Byzantine
art history faculty members observed that students needed to experience the
touch and feel of real manuscripts in a library environment. Yet, how many
libraries house medieval manuscripts? Direct contact with primary sources
saved as online manuscript images could foster development of independent
research skills in topics ranging from history and literacy to everyday life.10
• What is the value of online primary sources for teaching
or research? In the U.S., most Slavic and Byzantine manuscripts remain hidden
in East European special collections. The Dumbarton Oaks Research Library
and Collection in Washington, D.C., and the Hilandar Research Library at
Ohio State University are notable exceptions. They house microfilm, color
slides, and facsimiles of medieval manuscripts that might provide the heuristic
thrill of discovery—that pre-digested second ary sources cannot. The
participants in this study expressed the opinion that a strong and instructive
presence of Slavic and Byzantine manuscripts could be established on the
Internet. But would they use it?
So You Want to Try PD on a Web Site?
Keep in mind the three commandments of Web sites: Know thy users!
Know their needs! Know their sources! Also remember the three criteria
of site layout: content, design, and navigation. And don't forget—the
intellectual content must drive the design.
1. Know your audience, their needs, and their sources.
2. Explicitly state your intended audience and purpose.
3. Situate the information in its spatial and temporal context.
4. Write simply or supply a glossary.
5. Write well in a level appropriate for your audience.
6. Provide authoritative and reliable information along with a bibliography.
1. Achieve a simple and consistent page layout.
2. Support images with text, but use plenty of images.
3. Use "thumbnail" images that lead to high-quality enlargements.
1. Make navigation simple and intuitive.
2. Provide a search function for the site's content.
3. Enhance the site (and please your audience) with a little interactivity.
4. Decrease the use of external links, or be prepared to update them
5. List your site on other authoritative sites.
6. Avoid the need for a scroll bar whenever possible.
In this study, several professors noted that "A projected image in the classroom
is indeed worth a thousand words, and certainly sticks longer [than words]." Therefore,
useful online sources must emphasize the visual, and herein rests their power.
By their very nature, manuscripts are "hypermediated" (accessible through several
reader-controlled paths) and visually richer than tradition al printed culture.
Manuscript images are vital in teaching about manuscripts.
Images can captivate the user, but they must conform to Web constraints. One
participant reminded us: "Images are the most important, but Web sites must
follow best practices of Web design. They must find the right balance between
the number of images: right size for better downloading and high quality." He
suggested that Web sites should offer a gallery of searchable "thumbnail" images
that lead to larger files.
Images might initially captivate and motivate a broader audience than traditional
printed text. Without context, however, images impart an obscured and skewed
understanding. The Byzantine and Slavic life, culture, and traditions portrayed
in manuscripts require knowledge of the wider political and cultural context
of the Middle Ages. One participant in this study emphasized the importance
of comparing Eastern and Western traditions chronologically and thematically
because, in the Middle Ages, people from both traditions inter acted with each
other. A Slavic manuscript Internet presence, therefore, must include copious
illustrations and text to provide context of both the Slavic and the contemporaneous
Stage 2: Site Evaluation
Slavic Medieval Treasures from Bulgaria. All participants commented
positively on the navigation and content of my first site (http://www.ischool.utexas.edu/~slavman) but suggested improvements. Specific recommendations included adding an explicit
introduction stating the purpose and intended audience of the site; a map of
Bulgaria; a comparison between Slavic and Western traditions; information on
codex, parchment, and paper media; and facts about the number and types of
manuscripts in the collection. General recommendations included additional
and improved images, the use of appropriate fonts, and an improved writing
style (because poor writing undermines the authority of the intellectual content).
Slavic Medieval Treasures. I constructed this site (http://www.ischool.utexas.edu/
~slavman/manuscripts) to consolidate my research on various Slavic topics and to augment them with
multimedia features such as video. I chose the Slavic manuscript tradition
and its relationship to the Byzantine manuscript tradition as the theme. The
participants found that this site had corrected many of the problems of the
first site. They appreciated the uniqueness of the material, its air of authenticity,
its aesthetic qualities, its multimediality, and its organization. They grasped
its purpose and intended (i.e., academic) audience. They recommend ed the title
page as a model for the other sites.
Problems included the lack of a glossary to clarify the academic jargon, inconsistent
design, lack of images, and lack of thumbnail images linked to larger, higher
quality images. One participant felt that the spiritual aspect of the site
would undermine its authority among scholars who might not be comfortable with
manuscripts portrayed as an expression of a living religion, rather than as
Byzantine Medieval Hypertexts. This site (http://www.ischool.utexas.edu/~slavman/hypertexts) received the few est negative comments and the most praise. Participants found
the intellectual content to be academic and acceptable for teaching and research. "A
truly impressive collection of images, commentary and translation," said one.
They also appreciated the balance between text and images and the integration
of images into the scholarly text. As with the other sites, this site lacked
consistency in its layout. Most participants preferred a single column of text
illustrated by images.
Stage 3: Prototyping a Site
The third stage of PD involves participants in the design of the artifact
they might use. Thus, the sites become a product and reflection of what the
participants considered important for their teaching and research.
My participants evaluated the navigation and content of the three sites. They
followed the Web site evaluation criteria developed and recommended by Alastair
Smith.11 Smith's navigation criteria is rather comprehensive: back
buttons, usability, scrolling, "bells and whistles," working links, functionality,
loading time, and interactivity. The content criteria included scope, accuracy,
authority, authenticity, a balance between text and images, colors and aesthetics,
currency, graphics and multimedia, links to other sources, organization, purpose
and audience, quality of writing, uniqueness, user friendliness, integrity
and reliability, and reference to other sources.
My survey of participant opinions showed that the three sites required improvement
in navigation (back buttons), usability, working links, and interactivity.
Common problems with intellectual content for all three sites included quality
of writing, lack of uniqueness, and lack of reference to other sources.
To Flip or Not to Flip?
I had to decide the best way to move from one image to another. The British
Library, for example, presents its online Bible with pages that appear to flip
the same way a physical manuscript would. My participants widely diverged when
faced with this question. The library historian preferred "flipping" to scrolling;
the Byzantine art historian preferred flipping to slides and felt the flipping
of virtual manuscript pages had the potential "to present the entire manuscript,
to experience movement through the pages, to have a sense of the material and
texture of the pages." For one faculty participant, however, forcing a manuscript
design onto a Web site manifested a "nostalgia" complex and would create an
accessibility problem for visually impaired users. His students echoed him;
they felt animated flipping pages were a showy and distracting gimmick.
The Success of the PD Model
Perhaps it was the refreshments, or perhaps it was the feeling of empowerment.
Participatory design created a more intimate social atmosphere between the
researcher and the participants—it was an atmosphere of trust. The participants
shared their feelings, values, needs, and knowledge. After 11 interviews, two
observations, one joint prototyping session, five participant surveys, and
numerous e-mail conversations, I felt more confident that I understood the
tacit knowledge of the users.
I wasn't able to use some of the recommended techniques for PD—e.g.,
participants did not permit videotaping. I had to follow their inclinations,
such as their willingness to share their thoughts in writing. E-mail helped
our communication tremendously by allowing the opportunity to think about responses,
then to respond at leisure.
Participatory design does take time, but it is worth the effort. If you want
to try it, keep in mind the value of time for the participants, especially
the faculty. Prepare yourself and your project ahead of time through homework
and research. Choose extra participants, because some will drop out along the
PD Can Be Beneficial
Like it or not, research and pedagogical technology change. If we use PD to
create online resources, we can maintain our position in the academic community.
Although many still do not trust online sources, the PD "marketing strategy" convinces
users that the online sources they design can be used for academic teaching
and research. I invite the readers of Computers in Libraries to adopt
the PD method. It will prove a most rewarding experience.
1. Gombrich, E. (1987). "The Values of the Byzantine Tradition:
A Documentary History of Goethe's Response to the Boisseree Collection." In The
Documented Image: Visions in Art History. Syracuse University Press: New
2. Trachtenberg, M. (2001). "Desedimenting Time: Gothic Column/Paradigm
Shifter." RES 40: 5–24.
3. Nikolova-Houston, T. (2001). "The Internet and the Virtual
Scriptorium of Slavic Medieval Manuscripts." Libraries in the Age of the Internet,
Sofia, Bulgaria, Union of Librarians and Information Services Officers (ULISO):
4. Dobreva, M. (2005). "Medieval Slavonic Written Cultural Heritage
in the E-World: The Bulgarian Experience." In Review of the National Center
for Digitization. Yugoslavia.
5. Miltenova, A. (2002). "The Sofia Corpus of Data on Slavic
Manuscripts." In Slavic and East European Information Resources, 3 (2/3),
6. Greebaum, J. & Kyng, M. (1991). Design at Work: Cooperative
Design of Computer Systems. Hillsdale, N.J.: Lawrence Erlbaum Associates.
Also Spinuzzi, C. (2005). "The Methodology of Participatory Design." Technical
Communication, 52 (2), 163–174.
7. Iivari, N. (2004). "Enculturation of User Involvement in
Software Development Organizations—An Interpretive Case Study in the
Product Development Context." In Proceedings of the Third Nordic Conference
on Human-Computer Interaction. ACM Press, 287–296.
8. Dillon, A., Sweeney, M., & Maguire, M. (1993). "A Survey
of Usability Evaluation Practices and Requirements in the European IT Industry." In
J. Alty, S. Guest, and D. Diaper (Eds.), HCI ‘93. People and Computers
VII. Cambridge: Cambridge University Press. Also Taxen, G. (2004). Introducing
Participatory Design in Museums. Paper presented at
the Proceedings of the 8th Biennial Participatory Design Conference. Toronto,
Canada, July 27–31.
9. Dillon, A. (2004). Designing Usable Electronic Text.
New York: Taylor & Francis.
10. Winkler, A. "Digitized Medieval Manuscripts in the Classroom:
A Project in Progress." The History Teacher, 35 (2), 1–22.
11. Smith, A. Testing the Surf: "Criteria for Evaluating Internet
Information Resources." The Public-Access Computer System Review, 8
Tatiana Nikolova-Houston is a doctoral candidate from the School of Information
at the University of Texas–Austin. This article evolved from her attempt
to redesign her existing Web sites and to supply online resources for Slavic
and Byzantine studies. She has received the 2002 IREX travel grant, The Manuscript
Society Maass Research Grant for 2004–2005, and the 2005 ATLA Bibliography
Grant. Her e-mail address is email@example.com.