ONLINE Magazine
Table of Contents Previous Issues Subscribe Now!
Internet Search Engine Update 
by Greg R. Notess 
Reference Librarian, Montana State University
Internet Search Engine Update goes up on the Web at as soon as it is written,

approximately one month before the print issue mails to subscribers. 

AlltheWeb has expanded its file type coverage and advanced search capabilities. AlltheWeb is now the first and only search engine to provide searchable access to Web-accessible Macromedia Flash files. Its Advanced Search page now includes a new a Flash file format limit. It also has a limit to just-included pages that have specific "Embedded Content" such as images, audio, video, Flash, Java applets, JavaScript, and VBScript. AllTheWeb has added an "in the host name" field search and a regional domain limit based on top-level country domain groupings. The date limit can now specify any date range, and there is a document directory depth limit and a personal home page limit.

AOL Search is now using Google.results under the "Matching Sites" heading along with the Google AdWords results under its "Sponsored Links" heading. The Google databases on AOL replace Inktomi, the former Web search results provider, and Overture, the former Sponsored Links provider. AOL-owned CompuServe and Netscape also are using the two Google databases. AOL says that its search is "Enhanced by Google," while CompuServe is "Powered by Google," and Netscape Search uses "Features Results from Google." All three actually combine some material from the AOL network properties with the Google results.

Ask Jeeves is now displaying "sponsored links" advertisements from the Google AdWords program. It has also followed Google's lead by offering an Ask Jeeves toolbar. The toolbar can be installed within Internet Explorer for users of Windows 98 or higher. It has quick button access to a variety of Ask Jeeves searches, including News, Dictionary, Market, Weather, and saved searches.

DayPop, the blog and news search engine, went down in early September. The person who runs the site went to Italy, and it crashed while he was gone. Once he returned in the middle of the month he discovered that he was out of disk space. And so the Internet community is deprived of this useful resource featured in my September "On the Net"column. On the other hand, it also demonstrates that a useful search engine still can be built and run by an individual. Editor's note: At press time, DayPop is gradually being restored.

Google announced that it has increased the number of pages that it crawls and re-indexes as frequently as every day. Previously, this had been about 3 million pages, but there's no word on what the new number is. Most of these are news or blog pages that are updated daily, but some are pages that have not changed for over a year. Google has also increased its database size claim. Now the small text at the bottom of the main search page boasts 2.4 billion Web pages searched as compared to AlltheWeb with 2.1 billion. Whatever Google's claims, be sure to search both for more comprehensive coverage.

InfoSpace, which operates several meta-search engines including Dogpile, Excite, WebCrawler, MetaCrawler, and InfoSpace itself, has reached an agreement with Google so that the meta search engines can include both Google results and the Google AdWords advertisements. In April, InfoSpace reached a similar agreement with FAST. Since both Google and FAST often prevent other meta-search engines from including their results, this gives InfoSpace search engines an important advantage.

Inktomi has been losing partners, but is still competing. With both FAST and Google claiming over 2 billion pages, Inktomi has made an increase as well, and company representatives also now report a database with more than 2 billion Web pages.

iWon, the browse-for-sweepstakes-entry portal, has also dropped Inktomi in favor of Google. iWon is continuing with Overture for its "Sponsored Listings," but its "Web Sites" section is now powered by Google. In addition to the Web database, iWon also provides the "Images" tab for access to Google's image database. The "Shopping" tab looks like another Google database but is actually an iWon and Dealtime combination.

MSN Search now lets users see beyond the first 200 results. The limit is now at 1,000. And for Internet Explorer users, there is a new feature for those that click the search button for the IE Search Assistant. They can now see a "Search Preview" in the main pane which shows mini snapshots of the matching Web page results. However, there is not a snapshot for every result listed in the left pane.

SpyOnIt, a free alert service, shut down, "on an indefinite hiatus," but hope to bring at least part of the service back online. That may happen by sometime late 2002, so check back.

The Wayback Machine has had very few 2002 archived pages for most of this year. But by October, data from the first 6 months of 2002 should be online. While it is too soon to tell if this will become the standard update schedule, do not count on using it for a recent archive solution.

WiseNut has finally refreshed its database. Since Fall 2001, WiseNut's database stagnated, with the bulk of it from a July 2001 crawl. At some point this September WiseNut finally updated its database with a new one. Even so, it does appear to be primarily a crawl from May 2002. It is certainly much fresher than before, but it is still not very up-to-date.

Greg NotessGreg R. Notess (; is a reference librarian at Montana State University and founder of

Comments? Email the editor at

[Contents] [ONLINE Home] [Subscribe] [Top] [Information Today, Inc.]