The Future of Search Is Intelligent
by Phil Bradley
As an industry and as a profession, we have seen our fair share of dramatic changes, from bibliographic databases going online and the development of CD-ROM-based products to the arrival of the internet, the rise of Google, and the role that social media plays in our day-to-day and professional lives. Every single one of those changes has brought with it an opportunity for us, as professionals, to change the way we work and to aid and assist our patrons in different and exciting ways. Now, we have another such example in the form of AI tools, specifically those that will affect and change internet search but that also impact a wider spectrum outside of search.
You cannot have failed to notice the wave of news and interest in AI in the last few months and the affect that it’s had, not just in areas such as search and art, but in almost anything that requires the use of the written or, indeed, the spoken word. I want to talk in particular about ChatGPT in its various incarnations and give you an overview of how our information world has changed—literally overnight.
I’m not a great fan of clickbait headlines, which is why I have buried this one—I didn’t put “seismic shift” in there, even though that’s what I think it is. In the 30-plus years that I’ve been working with the internet, we are now at the top of a wave that it going to completely change and revolutionize everything we know about internet search and indeed many, if not all, aspects of how we do our jobs as librarians and information professionals. You can be forgiven for being skeptical at this point, but bear with me while I explain this in more detail.
LARGE LANGUAGE MODEL
ChatGPT (chat.openai.com/chat) is a large language model trained by OpenAI, its developer. The “GPT” part of the name stands for “Generative Pre-training Transformer” and refers to a type of machine learning technique that involves training a model on a large dataset to generate text. It’s capable of generating responses to a wide range of prompts or questions. The current version was trained on content found on the internet with a cutoff date of 2021. Thus, it’s not able to give current world information, but as we will see, that’s not really an issue at this stage. It’s obviously not “human” and doesn’t have personal experiences or feelings. It exists solely to assist users by generating text-based responses.
So far, so uninteresting. Let’s take a look at what it can do, how you can use it (specifically to search the internet, but more generally as well), and at some of the questions and issues the development of this tool brings with it. I’ll start with the obvious. Because it has been trained on content it’s found on the internet, it can answer questions for you. Think of it, if you will, as Google on steroids. I can ask it any factual questions, and it will give me an immediate answer. This answer will be couched in written conversational English, rather than giving me a list of webpages that I then have to scroll through to find the relevant link and search the page to get the exact information I need.
This is important because people don’t want to search the internet, they want to find information. Google and all other traditional search engines help, but there’s still an obstacle in place, which ChatGPT overcomes. It’s capable of giving me a quick summary of a subject. I can ask it to explain the causes of the American Civil War. It can do this based on the countless articles on which it has been trained. But surely, Google can do the same thing? Not really—Google can give you quick summaries or snippets. Here’s a key difference: I can ask ChatGPT to expand on any of the things it has “said.” I can ask it to rephrase its answer into bullet points. I can ask it to rewrite the information so that a young teenage student can understand it. All of this happens in the chat interface; I don’t need to go anywhere else, visit other websites, or sift through pages to find the information that I need.
I can go much further than this, however. I can ask it to give me a list of five leading recruitment agencies in a particular country, region, or state. I can then ask it to provide me with a list of their web addresses. Next, I can ask it to put that information into a tabular format for me. Then, I can ask it to include a list of the names of the CEOs and the company contact details and add it to the table. All of this is achieved in seconds—I can’t stress enough how quickly this happens. What would have taken a skilled information professional perhaps 30 minutes or longer to do is done by ChatGPT inside a minute.
Does this make ChatGPT a search engine? No, it does not. However, my car is not a truck, but I can still use it to transport bricks from one place to another. So no, ChatGPT is not a search engine in the traditional sense, but it can be used as a search engine or, more importantly, in conjunction with a search engine.
ASKING THE RIGHT QUESTIONS
A big problem with research is finding the right question to ask. Traditional search engines are not good at this. Google searches for what you ask for, with the addition of adding in a few synonyms here and there. Search is at best clunky and unwieldy. However, I can ask ChatGPT, “Which is the best search engine to use to find out information on child equality in Ireland, and can you provide me with a search I can run to obtain that information, with an emphasis on legislation?” I don’t have to worry about any arcane functionality such as site: or inurl:. ChatGPT understands what I want and suggests a search engine and a search string to use to get that information.
Alternatively, I can simply say, “Provide me with a list of searches that I can run on Google that will give me results that I can read to get a good understanding of the rise of COVID in the United States.” It will do that—again, in an instant. I don’t need to understand the subject in order to get a good list of searches to run, and I have saved myself a huge amount of time.
I can make a more nuanced request to ChatGPT. If I ask, “How can I construct a search to find information about elephants which is suitable for young children?” it provides me with a perfectly competent search strategy that includes phrase searching, Boolean operators, site: functionality, and other search considerations. Alternatively, I could ask it for a list of terms and phrases that I could use to search within a particular subject area. Traditional search engines can give you some suggestions of what other people have searched for, but the list of suggestions that I get from ChatGPT is far more in-depth and nuanced.
I can use this tool to suggest resources for specific groups. I can ask it to suggest a good search engine or academic resource to find medical information. It comes back with good, high-quality suggestions. Moreover, I can ask it for a list of journals in a particular subject area. If necessary, I can ask it to produce a complex query that I could submit to a specific database such as PubMed.
NO OVERT ADVICE GIVEN
While ChatGPT cannot provide me with advice as such (that’s outside of its parameters), it can give me suggestions on how to come to decisions. For example, it can’t respond to a query about the accuracy or trustworthiness of a website, but it can give me a few tips for evaluation. It suggested I check domain names, evidence of bias, dates, and sensational headlines. I was then able to use that list and ask it to check against the BBC’s website. ChatGPT came back with a summary indicating that, yes, the BBC site could be considered a reputable and reliable source of news and information. However, with this answer as with all of the others, it gave me various caveats.
This is a point worth stressing: Unlike traditional internet searches that simply give you a list of results and leave you to it, ChatGPT goes the extra mile. For example, if I ask Google, “Is milk good for me?” Google provides answers that generally say that yes, it’s good for me to drink milk. However, if I ask the opposition question, Google provides answers that prove the opposite. Google is far less interested in providing independent and impartial information and far more interested in keeping users happy, giving confirmation bias in the hope that they will return and perhaps click on a link and earn Google some money.
In order to get a useful question, searchers need to put some thought into it, perhaps by writing a query about the “advantages and disadvantages of drinking milk.” ChatGPT seems to do this by default. When I ask the question about milk, it provides both positive and negative reasons for consuming the liquid. I have found it to be a far more balanced and informative way of getting the information. Of course, if I’m in any doubt, I can also ask it to give me some suggested websites that I can go to in order to research further, or for a list of suggested research that I can look at which is appropriate for my level of understanding. This is far, far beyond the capabilities of any existing search engine.
ASKING FOR SUGGESTIONS
Let’s look further at some of the other tasks that ChatGPT can perform. I can ask it for book recommendations based on books that I have previously enjoyed or authors that I like. I can tell it that I like a particular genre of fiction and get suggestions. There are other tools that can help do this, but none of them are as precise as ChatGPT. For example, I can say that I like detective fiction based in Norfolk in the U.K. with an emphasis on history or archaeology with a strong female character and which has been published since 2015. Within seconds, I have half a dozen, and I can then go back and ask for a synopsis as well. I can ask for literary criticism of a book or a poem.
I can ask it to act as an entomologist or even as a guidance counselor. For example, I asked it to act as a career counselor, gave it a list of things that I enjoyed doing within a work environment, and asked it if I would be a good fit as a librarian. (Thankfully, it said yes!) I can ask it to provide a list of interview questions that I would probably be asked if applying for a job in a particular company or role. I can even ask it to act as a technical support assistant: I imagined a situation in which my speakers were not working, and “Have you plugged them into the computer?” was the third suggestion it came up with, which was the one I was thinking about.
All these suggestions have not even scratched the surface of what AI chat can do. Many educators are up in arms because students can simply ask for a 1,500-word essay on the fall of Rome, and that’s exactly what they get. ChatGPT writes it fresh every time, and it is plagiarism-free (although some dispute that). However, it’s fair to say it can be run through another AI tool, which can identify that it’s been written by AI. So the clever (or sneaky) students ask ChatGPT to write an essay on the fall of Rome in the style of a 15-year-old student. They can then run it through another tool to further rewrite it in order to pass the AI test.
As of now, traditional homework is broken. We can take this to ludicrous extremes since teachers and other educators can use AI tools to create lesson plans. I’ve used these myself: I asked for a lesson plan for a 1-hour session to include timings on the role of AI in the world of librarianship. I was then able to get it to expand each section in turn. I could then take that information, pass it to another AI program to paste it to an AI avatar to appear on the screen to read the content aloud, or I could use it to create a PowerPoint-like presentation. (I could have used it, but there’s no fun in that. I have to wonder how many people would have realized what I’d actually done.)
I could continue. I could tell you about how ChatGPT can be used to create code that can be instantly copied and pasted by programmers. I could wax lyrical about how doctors are giving it symptoms of pretend patients and getting accurate diagnoses. Or how marketing executives and creators are using it to write advertising copy, blog posts, and entire children’s novels. But there’s only so much space available. If you are still skeptical, I’m afraid that I have to tell you that none of this is science fiction. It is happening in the world right now, and you have to start using these tools and understand them to be in a position to exploit their full value and avoid misuse.
CHATGPT HITS MAINSTREAM SEARCH
Microsoft has invested heavily into OpenAI, and it’s interesting to see what it’s doing with ChatGPT. As of early February, the company rolled out a new version of Bing and an adaptation of Edge to incorporate the AI tool. As previously described, I can now have a conversation with the search engine, facilitated by ChatGPT, to answer questions. If I ask about the Apollo moon missions, I get a useful and accurate answer. I could then ask for an expansion of information on an aspect of the information I’ve been given. This is a major shift in the way that search works. We are moving from an interrogatory approach to a conversational approach, one that will appeal to people who just want an answer to a question, not the challenge of sifting through links to webpages.
Google is, as usual, late to the game. It introduced Bard in a disastrous launch following which Alphabet dropped more than $100 billion in share value. If nothing else, this should demonstrate how important AI is becoming to the business world. Yandex has YaLM 2.0 in Russian in the works, with plans for its appearance before the end of 2023. The Chinese search engine Baidu plans to launch, in March 2023, a ChatGPT-style service called Wenxin Yiyan in Chinese, Ernie Bot in English. Naver, the South Korean search engine, plans to launch a ChatGPT-style service called SearchGPT in Korean in the first half of 2023.
I am happy to confidently assert that by the end of this year, internet search will have evolved into something completely different. AI isn’t going to be a “Google killer,” but it is going to be a “Google evolver.” Perhaps we won’t “google it” anymore. Will we “ChatGPT it” instead? In 5-years’ time, the idea of returning a list of webpages to answer a query is going to seem laughably inept. Internet search isn’t about to change, it has changed. As of now.
There are many challenges ahead, of course, not the least of which is funding. There are plenty of ways funding will occur—by subscription, the insertion of ads between answers, by freemium models (OpenAI now has a $20/month premium subscription option for ChatGPT), and investment. Microsoft has given billions of dollars to OpenAI and clearly expects something in return.
At the moment, the internet is financed in large part by advertisements. If I can ask ChatGPT to list five air fryers that are rated for size, cost, color, and efficiency, why would I waste time looking at ads? If an inquirer can get all of the information/content that they need pulled off of the website, why would they even want to go to the website?
Indeed, what exactly is the future of the traditional website, SEO services, and content creators? At the moment, no one knows. If users are going to simply accept the responses that they get from a conversational AI model, how do we deal with misinformation and fake news? If we rely on the moral and ethical concerns of commercial companies, we are in big trouble. Moreover, people will start to create their own models, resulting in chatbots that give users information from the viewpoints of various political parties or religious groups.
ChatGPT currently doesn’t give out any information on individuals, nor does it search through social media networks. But it’s only a matter of time before either it or another tool does. If we match that with the ability to search for faces with tools such as PimEyes (pimeyes.com) or people finders, the existing concerns that we have about privacy seem inconsequential. We haven’t even scratched the surface in the ways in which everyone’s lives are going to be affected by this sudden leap forward in AI.
In summary, I hope that you’re concerned. Indeed, I rather hope that you may be slightly scared, because you should be. That seismic shift has already taken place, and we can’t go back. All that we can do is understand where it’s taking us and harness it. I like to think that, as information professionals, we are perfectly poised to do this. AI search in particular and AI tools in general are going to change the way the world deals with data and information and how it’s manipulated. That’s something we are already experts in, so, by all means, be concerned and scared, but also excited. Thing are changing very quickly, and it’s going to be a great ride!