|"Integrating Information Seeking and IR" and "Information Services—Practice
and Research" were the themes of the fourth Libraries in the Digital Age
(LIDA) conference, held May 2124 at the Interuniversity Centre in
Dubrovnik, Croatia. Speakers contrasted information seeking and information
retrieval (IR) and called for new models of relevance for the evaluation
of information seeking systems. They also explored the concept of "information
literacy" and presented examples of successful programs.
The conference drew 98 participants (70 from Croatia) from 10 countries
and included 25 students. The format featured eight workshops, two tutorials,
37 papers, and a poster session. The European Chapter of the American Society
for Information Science and Technology awarded cash prizes for the three
best student posters. Two tutorials were also held. Tefko Saracevic, from
the School of Communication, Information and Library Studies at Rutgers
University in New Jersey, led "Perspectives of Information Science in a
Digital Age" and Peter Ingwersen, from the Department of Information Studies
at the Royal School of Library and Information Science in Copenhagen, Denmark,
led "Perspectives of Webometrics: With a Special Emphasis on Small World
The event's co-directors were Saracevic; Ingwersen; Tatjana Aparac,
from the Department of Information Sciences' Faculty of Philosophy at the
University of Zagreb in Croatia; and Pertii Vakkari, from the Department
of Information Studies at the University of Tampere in Finland.
The opening remarks were provided by Ingwersen, who developed the clear
distinction between "information seeking and information retrieval" (IS&R)
and "information retrieval" itself. In his paper "Integrating IS&R,"
Ingwersen explained this difference by noting that (in Denmark) the concepts
represent two different research communities that do not communicate with
each other and that have few cross-citations. He emphasized, however, that
both groups cite the same body of research literature from cognitive researchers.
Ingwersen characterized "hard-core IR" by static experimentation, lack
of knowledge about user behavior, and only rare cases of interactive information
retrieval (IIR) utilization. He believes that members of the information
seeking community are infrequent users of "systems" who have no concern
for best-match algorithms. He said that the "old hands want to expand the
field of information retrieval, while younger researchers are integrating
information seeking and information retrieval.... This does not mean abandoning
mainstream IR values, but [it] means interaction...."
Ingwersen listed the three revolutions in IR (citing researchers Stephen
E. Robertson and Micheline Hancock-Beaulieu): the Interactive Revolution,
in which IR systems are becoming increasingly interactive; the Cognitive
Revolution, in which cognitive behavior places pressure on algorithmic
IR; and the Relevance Revolution, which is expanding the classical concepts
of relevance to "higher orders." (New models of relevance were featured
within the information seeking theme.)
Ingwersen closed by saying that the "new generation" has a mutual interest
in both groups, including dynamic information needs, information behavior
applied to algorithmic relevance feedback, and simulated work tasks to
keep control of natural experiments.
Following these opening remarks, Saracevic noted that information seeking
is not part of the mainstream research in either Europe or the U.S., as
"most money from the National Science Foundation (NSF) is going into computer
science and less into information seeking research." Christine Borgman,
a professor at UCLA's Department of Information Studies, disagreed, saying
that in the U.S., six digital library projects recently received $25 million
from the NSF for information seeking research and that another 25 to 30
projects are underway.
Carol C. Kuhlthau, a professor at Rutgers University's Library and Information
Science Department, presented the "Longitudinal Study Methods in Information
Seeking and Retrieval" workshop. Longitudinal studies, she explained, provide
data over time and are particularly useful for documenting transition and
change. She talked about several implications for the methodology in information
retrieval, such as moving beyond the single incident/one-point-in-time
approach to the consideration and accommodation of the information search
process. She noted, in particular, Rutgers professor Nicholas Belkin's
interactive approach to information retrieval.
In his workshop, titled "Evaluation Methods for Interactive Information
Retrieval," Belkin offered the reasons that evaluation in this area is
problematic and then addressed the issues. The problems stem fromthree
sources. First, past measures are based on criteria for the evaluation
of non-interactive "batch mode" systems. Second, the nature of interactive
systems contradicts general concepts of evaluation (such as replicability).
Third, people who are engaged in interactive information retrieval will
behave differently. A searcher's prior knowledge will affect his or her
use of the system. Other problems include evaluation criteria, replicability,
knowledge and salience, and realism. As a model, he discussed these issues
as they relate to TREC Interactive IR tracks. Usability, satisfaction,
and efficiency criteria are at least as important as performance; user
judgments of relevance can be utilized in place of external judges; and
salience and realism can be simulated.
Kalervo Jarvelin, from the University of Tampere, presented the results
of his paper "Perspectives on Laboratory-Based IR Research." The paper
developed objections against traditional IR research and provided responses
to them. These objections included uncertainty of needs, the lack of real
users and real work tasks, the variety of collections, the insufficiency
of recall and precision as measures, heavy averaging, and IR experiments
in isolation. In defense of these criticisms, he said that users and tasks
are not needed to test realism and that averaging is not a limitation of
the model. He agreed that the variety of collections and the uncertainty
of needs shouldbe looked at within the models. The concept of relevance
was expanded to include algorithmic, topical, cognitive, and situational
levels (which were the subject of separate presentations).
Jarvelin concluded by noting that the laboratory model is not dead.
Rather, it's limited but is still needed for algorithm development. He
said that broader frameworks are necessary; integration with information
seeking is required; and that humans, work tasks, relevances, cognition,
and novel criteria must be included and understood.
In a session titled "Performance Indicators for IS&R," Ingwersen
and Jarvelin introduced the use of new performance measures in information
seeking and retrieval. They said that evaluations must concern process
characteristics of work and that seeking and searching tasks and relevance
must be multilayered, situational, and dynamic. These relevance levels
should be algorithmic, topical, cognitive, situational, and socio-cognitive
and should have degrees of relevance. They noted that in the reassessment
of 38 TREC topics using degrees of relevancy, it was found that of the
prior (binarily relevant) documents, 50 percent were marginally relevant,
33 percent fairly relevant, and 16 percent highly relevant. Thus, the comparison
of binary and graded relevance may affect evaluation results.
Ingwersen then introduced the new relevance types of Intellectual Topicality,
which is the aboutness relation of a query; Pertinence (cognitive), the
perceived correspondence of information need; Situational, the relation
as perceived between the task and the situation/problem; and Socio-cognitive,
the task as perceived by a group in context.
In conclusion, Ingwersen stated that the laboratory IR model is needed
for comfortable systems evaluation, but it may be extended toward users
through the application of their proposed performance indicators. He said
that thorough user-oriented evaluation requires broader scenarios and different
types of indicators and that "standard metrics may remain a dream in this
Discussions of relevance were continued by Pertii Vakkari in a workshop
titled "Relevance—Which Criteria?" He further defined system or algorithmic
relevance as the ranked output of information objects that are commonly
judged against an expert's binary relevance assessment. Vakkari then introduced
the idea of "topicality," which asks how well (the topic of) the user's
conceptual construct matches (the topic of) conceptual constructs in retrieved
documents. He explored the impact of prior knowledge on these utilities,
citing studies by Daniel Isenberg, Vimla L. Patel and Marco Ramoni, and
Amanda Spink. He also cited longitudinal studies by Kuhlthau that differentiated
the types of information needed in task-performance stages. Situational
relevance, he concluded, is "a relationship between a person's information
problem and the information that contributes to the resolution of that
problem. Relevance depends on the contribution."
Vakkari also led a session titled "Task-Based Information Searching:
Tactics, Terms and Search Results." He said that ideas from information
seeking and information retrieval must be integrated and that this is not
an issue for a single study but for a series of studies. He noted that
"IR is a broader process of information seeking" and from information retrieval
"human interaction with systems must be taken into account."
Vakkari next contrasted the search styles of graduate students in psychology
and information science. For example, the number of search terms and tactics
increased for information science students, but remained constant for psychology
students. One conclusion of this longitudinal research is that the size
and specificity of the user's search vocabulary grows during the stages
of the task performance and that the use of operators and tactics depends
on the searcher's knowledge of the systems.
Katriina Bystrom, from the Swedish School of Library and Information
Studies,also discussed the evaluation of a system based on task performance
in her paper "Information and Information Sources in a Task Context: Methods
and Results." She reported on research that concerns the use of different
information types by municipal administrators relative to task complexity.
One conclusion based on detailed task diaries was that increasing task
complexity fosters the use of people as information sources.
Peiling Wang, an associate professor at the University of Tennessee,
presented the results of her paper "Research on Web Searching: User Modeling
& System Design." The research was based on both log data and experiments
with searchers, which included their verbalization (concurrent withtheir
searching and retrospectively). The log data showed that the most-searched
categories included entertainment and recreations, career, academe, sports
and housing, people, places, and things. Most query structures were simple,
did not use advanced features, were short (an average of two to three words),
and used natural language. Iterations were rare in the case of failed searches.
Wang's conclusions concerning system design were that a one-size-fits-all
interface does not work and that instruction on systems use should be embedded
in the interaction process. She said, "Users will not come for help."
The first track concluded with a panel discussion, titled "Research
Questions and Methodology for IS&R." It was chaired by Jarvelin and
featured Belkin, Borgman, Ingwersen, Kuhlthau, Vakkari, and Wang.
Belkin noted: "We are still far from achieving an understanding of information
seeking behavior and how IS&R systems should be designed." Performance
evaluation should be based on the task of the systems and people. Criteria
other than relevance should be developed. Borgman stated that it's time
for a "new paradigm for information seeking and information retrieval."
She said that information seeking is not yet well-enough understood for
laboratory experiments. Ingwersen stated that behavioral aspects should
impact on design and called for more longitudinal studies. Kuhlthau said
that such studies should concern the process of cognitive development and
understanding tasks over time. Wang raised the issue of imbedding instruction
as an aid for information seeking, especially in Web-based systems.
The second theme, "Information Services—Practice and Research," was
briefly introduced by Tefko Saracevic. Borgman's keynote speech, "Whither,
or Wither, Libraries?" was based on her book From Gutenberg to the Global
Information Infrastructure: Access to Information in the Networked World.
Borgman asked what the roles of libraries are in the information infrastructure
and then listed and discussed four challenges: the invisible infrastructure,
content and collections, preservation and access, and institutional boundaries.
She explored the impact of digital libraries and emphasized that digitization
does not equal preservation. Digitization could be a big time bomb, she
stated, given the migration of systems and formats.
Borgman also discussed ways that librarians can maintain visibility
in the digital environment and encouraged them to become proactive in all
aspects of the parent organization. Despite the movement toward digital
libraries, she noted that new libraries and facilities are continuing to
be built to hold conventional collections, more books are being published,
book fairs are flourishing, e-books are not yet doing well, anddual (paper
and electronic) journals are doing better than their electronic counterparts.
Marija Dalbello, from Rutgers' School of Communications, Information
and Library Studies, presented the results of her paper "Selection Criteria
Digital Collections: Is There an Emerging Model?" She identified and
examined over 400 digital projects that concern cultural analysis in the
U.S. and Europe and classified each project into various categories for
analysis. She used the ARL Digital Initiatives Database to obtain descriptions
of 412 sites, most of which reflect digital conversion of historical material.
Eighty percent of these could be browsed, but not searched.Despite their
growth, Dalbello concluded that "emerging digital collections are not presenting
sustained historical narratives in their presentation of material.... Digital
libraries springing up in various institutions are as yet unconnected masses
of fragments of data and visual information."
David Bawden, Lyn Robinson, and Polona Vilar, from City University in
London, led the session "Literacies for the Digital Library." The concept
of "information literacy" was discussed, with its many synonymous terms,
including digital literacy, computer/IT/network literacy, library literacy,
media literacy/mediacy, and information fluency/information mastery/informacy.
They noted the requirement for employment as one of the major motivators
in becoming information literate. In discussion, Kuhlthau said: "We do
not have the information literacy problem solved. More work is needed."
"Information Literacy in the Wired University" was the subject of a
workshop by Marta Deyrup and Beth Bloom, from Seton Hall University Libraries
in New Jersey. They described the creation of an information literacy program
(which included online and multimedia tools) that's targeted to the undergraduate
The results of a paper titled "Accessing Electronic Information in U.K.
Further and Higher Education. Findings from JUBILEE, a JISC Funded Research
Project" were presented by Kathryn Ray, Graham Coulson, Debbie Proud, and
Linda Banwell, from the School of Information Studies at the University
of Northumbria in the U.K. The paper reviewed JUBILEE (Joint Information
Systems Committee User Behavior in Information Seeking Longitudinal Evaluation
of Electronic Information Services), a current 3-year research project.
The study found that the strategy of imbedding electronic information services
(EIS) into the university curriculum, tailoring user training, raising
user awareness, and other factors has increased users' access and their
ability to effectively use EIS.
Oliver Obst, from the Central Medical Library at Westphalian Wilhelms
University in Germany, led a session titled "LOTSE is Coming Aboard." He
described the Library Online Tour & Self-Paced Education (LOTSE), a
tutorial designed to teach students library and information literacy using
Koraljka Golub, from the Department of Information Sciences' Faculty
of Philosophy at the University of Zagreb, who last year discussed "Digital
Collection for Blind and Visually Impaired People (in Croatia)," presented
new research about information accessibility for the disabled in Croatia.
Using W3C guidelines, Golub and her co-author, Nikolaj Lazic, found that
none of the 16 Croatian public libraries included in the research conform
to these guidelines. She then listed a number of libraries on the Web that
"Legal Deposit of Digital Documents—Practices and Trends" was led by
Nenad Prelog, assistant minister at Croatia's Ministry of Foreign Affairs,
and his co-author, Hrvoje Stancic, from the University of Zagreb's Department
of Information Sciences. Prelog discussed approaches in othercountries,
as well as issues of accessibility, which are complicated by copyright
and physical access. He noted other problems, including authenticity and
the migration of storage media. There is "no simple all-in-one solution,"
The linguistic challenge of the Web and the new Europe was discussed
by Maja Bratanic and Vesna Turcin, from HIDRA, The Croatian Information
and Documentation Agency, in their workshop "Multilingual Searching Using
Thesaurus: An Example of Croatian Versions of the European Union Thesaurus
EUROVOC." They talked about their creation and use of the Croatian version
Robert Hayes, a professor emeritus at UCLA, was recognized at the conference
dinner for his 25 years of consecutive service to the Croatia Library and
its information science community. Hayes also presented a workshop titled
"Models for Library Management, Decision-Making and Planning." This session
was based on his recent book, which includes programs on a CD-ROM for modeling
The next LIDA conference will be held in Dubrovnik in May 2003. Further
details about this year's event are available at http://www.ffzg.hr/infoz/lida.
Emil Levine is a library and information consultant living in Vienna,
Austria. He recently designed the digital libraries for the United Nations
Industrial Development Organization (UNIDO) and the United Nations Comprehensive
Test Ban Treaty Organization (CTBTO). His e-mail address is email@example.com.