FEATURE
Search Successes and Failures
by Marydee Ojala
Information professionals view online searching somewhat differently from other people. We enjoy the process, revel in trying search strategy variations, experiment with different search tools, and question why our searches succeeded or failed. Yes, we still want answers to our research endeavors, but we bring a healthy dose of skepticism and curiosity to the entire search experience.
Many generative AI (gen AI) search tools now describe themselves as “answer engines.” The premise (and promise) is that they skip intermediate steps and move directly from the query to the response. This is fine for ready reference and factual questions, but it can miss the nuances of a more complex research topic. There’s a big difference between which team won the World Series in 1960 and how composite materials used in baseball bat manufacturing could change the game.
Information professionals want to know what goes on under the hood when they’re searching. Answer engines are notorious for their “black box” technology. What sources informed their training sets (more likely to be TikTok than Online Searcher)? How do they determine relevance? Have changes been made in fine-tuning a model to reflect political stances on sensitive topics? Why am I seeing this response to my prompt? We have no good way of knowing answers to these questions, even when we ask the answer engines.
Gen AI technology is based on predictive analytics. Based on past usage, it predicts the next word or phrase to provide an answer; it doesn’t actually “know” anything, nor does it understand the meaning of words and phrases. Thus, searching a large language model requires a very different approach from the many traditional search engines used by librarians.
One important attribute of library subscription databases is their structured nature. Designated fields allow for very precise, targeted search statements. Want to search someone with the first name of Carl? There’s a field for that. But wait, it’s actually spelled Carle. For some databases, you might need a Boolean OR in your search.
On the web, whether it’s a gen AI tool or web search engine such as Google, it’s harder to restrict your search to first name only. In this example, Eric Carle pops up more frequently than any other name, and, obviously, Carle is the surname not the first name, so it is an incorrect answer. Humans recognize that Carl Lewis, Carle Vernet, and Eric Carle are not the same individual. I felt lucky I didn’t see Cardi B in search results. But it took four iterations of my Gemini prompt (my last one had a scolding tone to it) for the chatbot to give a (somewhat) decent response.
Speaking of search failures, in light of the ruling that Anthropic had used copyrighted books to train Claude and owed book authors compensation, The Atlantic introduced a search engine for authors to see if their books were included in the LibGen database (theatlantic.com/technology/archive/2025/03/search-libgen-data-set/682094). The search is rudimentary: It’s one search box, and it only searches the author field. Creativity, however, is important. Authors might be entered in various forms (m ojala, ojala m, or marydee ojala), and, yes, different spellings require separate searches. But it goes off the rails with its similarity, sound-alike, algorithm, which delivers Ajala, Ocalla, Mojola, and Opala as possible matches to Ojala, all of which ignore the first name entered in the search box. Can you fix this by using the double quote syntax (“marydee ojala”)? Yes, most of the time that works, even though experienced online searchers know it doesn’t always work properly on Google.
We should celebrate our search successes, particularly if they involve invoking librarian magic (such as those double quotes), and look at our failures as learning exercises. Did we use the wrong tool, or was the tool not up to the job? Sometimes, search failures are unavoidable, but we need to distinguish between those in which the search technique was inappropriate for the tool and when an answer doesn’t exist (photograph of a medieval monk, for example). Pivoting from gen AI to web search to subscription database is key to search success. That’s part of the process we enjoy.
|
