ONLINE Magazine
Table of Contents Previous Issues Subscribe Now!
Internet Search Engine Update 
by Greg R. Notess 
Reference Librarian, Montana State University
Internet Search Engine Update goes up on the Web at as soon as it is written, approximately one month before the print issue mails to subscribers. 

AllTheWeb has had a redesign and new features added. Most significantly, AllTheWeb now includes fully indexed PDF files in its database. The PDF files are usually identified with a .pdf designator after the title. While no direct limit is available at this time, you can add url.all:pdf to a search (or use the advanced search with pdf in the "must include" word filter with "in the URL" selected) to see some examples. This is an especially significant addition since Google’s PDFs are not fully indexed, with roughly the first 120K of the document indexed while AllTheWeb indexes all the text within the PDF file no matter how long it is. AllTheWeb Advanced Search now has an IP address range limit, and it is the only search engine with this feature. The new design has made it easier to access the tabs to move between databases (now at top instead of bottom), and has added Overture ad results at the top under a Sponsored Search Listings heading.

AltaVista has begun a freshness initiative to bring its entire database more up-to-date, and it has new techniques that aim to discover new, non-commercial sites. AltaVista is testing a paraphrase tool that will provide more robust suggested searches. It has expanded its multimedia databases by 73%, which includes images, audio, and video databases. According to an AltaVista representative, this expansion was driven in part by a 31% increase in multimedia search request since the beginning of this year. The combined multimedia databases now include more than 140 million URLs for multimedia content, and AltaVista adds over 800 new news-related images per day. 

Answer Services have been popular, with Google introducing in beta form a fee-based service called Google Answers [] and Yahoo! changing its Yahoo! Experts site from its own software to one powered by LiveAdvice. Yahoo Experts remains a fee service, but continues to have the free Ask Yahoo! service that only answers selected questions. Google Answers usually cost from $4-$50 and makes the questions and answers available for all to see on the site.

AOL has announced that its AOL Search will switch from Inktomi to Google this summer. Paid ads (sponsored links) have already moved from Overture to Google’s AdWords database. Similar switches will occur at AOL’s CompuServe and Netscape search sites.

Direct Hit is no longer available as a stand-alone search engine and redirects to Ask Jeeves' new search engine, Teoma. However, Direct Hit results do still show up at several partner sites including HotBot, MSN, and others.

Excite, now owned by InfoSpace, has become a meta search engine. It had been just showing Overture results but it now combines results from several ad databases (Overture, FindWhat, Sprinks) and from others including AltaVista, Inktomi, LookSmart, and FAST. Unfortunately, it does not identify the source of each listing. 

Fast Search & Transfer, the Norwegian company that builds the database and search interface for AllTheWeb has announced a net profit for first quarter of 2002. With all the pressure on search engines to show a realistic business model, it is pleasing to see another search engine reporting profitability.

Google has been busy again the past few months. In addition to the answer service mentioned above, it has now automated some English spelling corrections. When certain terms find no hits, Google will automatically guess the correct spelling and give results for that. Google clearly states that the original search found no results and that its guess gave the listed results. 

Google is opening access to its APIs for developers and programmers to experiment with. Some interesting examples of its use are the API Proximity Search [] for doing up to 3 term proximity and the Fagan Finder Google Advanced Search [] that has the ability to choose specific dates. In another experimental mode, Google now shows some of its ideas under development at the new Google Labs [] site that presently showcases a definition finder, telephone access to Google, and keyboard shortcuts.

InfoSpace has announced an agreement with Fast Search & Transfer to include FAST results on InfoSpace meta search engines. These include Dogpile, MetaCrawler, and Excite.

Teoma has relaunched its Web site with new graphics, layout, and design. It has added some paid links from Overture at the top under the "Sponsored Results" heading. Suggestions for narrowing a search, formerly in folders, are now on the top right under a "Refine" heading. The metasites formerly called Expert's Links are now called Resources and are designated as "Link Collections from Experts and Enthusiasts." Teoma still lacks advanced search capabilities and has a smaller database than Google and AllTheWeb, but plans to increase its size in the next few months. Also note that its phrase search can find pages that only "closely match the given phrase."

Greg NotessGreg R. Notess (; is a reference librarian at Montana State University and founder of

Comments? Email the editor at

[Contents] [ONLINE Home] [Subscribe] [Top] [Information Today, Inc.]