[ONLINE]
web search feature

Avoid the Void: Quick and Easy Site Submission Strategies

Danny Sullivan

ONLINE, May 2000
Copyright © 2000 Information Today, Inc.

Subscribe

"If I build it, they will come."

This is one of the worst assumptions a Webmaster can make. No one will come unless you also promote your Web site, and search engines are one of the most important places to do this.

Am I trying to turn information professionals into search engine spammers? Not at all. There are simply some essential rules that even library and research-oriented Web sites must follow in order to make them more "findable" in search engines. This is especially so in a Web that becomes increasingly dominated by commercial material.

RISE OF THE HUMANS

Yahoo! is no longer the only major search service that uses human power to generate its main results. Now, the majority of them depend on human editors to compile their listings. In turn, those editors largely depend on submissions from site owners--that means you! By taking an active role and submitting your Web site, you greatly increase the odds that it will be listed in a way that can bring you visitors.

Before submitting, do some basic prep work. Have ready the title of your Web site, the URL, a contact name and email address, and a 20-word description. Ensure that the description contains any important terms that will help locate your site.

Next, submitting at a human-powered search engine, also called a directory, means navigating to the right category for your Web site. One method is to drill down from top-level listings on the home page. A better option is to think of the top ways your visitors may be looking for you. Search for those words, then submit to the most relevant category that appears in the results. That way, people who search for those words will be more likely to find you, because they'll discover your category.

Once you've found the right category, look for the "Add Site" link (or something similar) near the top or bottom of the page. Follow the link, and then fill out the simple forms with the information you have prepared.

Submitting to Yahoo!

Yahoo! (www.yahoo.com) is the most important search engine to be listed with, bar none. Most sites see a significant traffic jump, once they get in. But getting in can be the hard part. That's why Yahoo! offers a Business Express service for some categories. For $199, Yahoo! will notify you within seven days whether or not your site will be listed. The fee doesn't guarantee that a site will get in, but most sites that use the service are added.

By all means, get money budgeted to pay for this service if you are not already listed in Yahoo!. The expense is small compared to the time you will save and the visitors you'll bring in.

Unfortunately, most non-commercial categories only offer Yahoo!'s Standard Submission option. This service is free, but you may end up submitting several times without success.

If your site is not listed within two weeks after using Standard Submission, resubmit exactly as before. Wait another two weeks. If it still fails to appear, submit a third time and also follow up with a polite email to url-support@yahoo-inc. com to check on your submission. Include your URL and mention that you've just done a third submission attempt.

The Open Directory's Reach

The difficulty sites have getting listed with Yahoo! led directly to the rise of the Open Directory (www.dmoz.org). Launched in 1998, the Open Directory uses more than 20,000 volunteer editors to compile its listings. While owned by AOL, the listings are open to any Web site to use, hence the "Open" in its name.

Once you are listed here, your site will eventually be available in the main results of MSN Search (search.msn.com), Netscape Search (search.netscape.com), AOL Search (search.aol.com), and Lycos (www.lycos.com). Many other smaller search engines also use Open Directory information. Overall, getting listed in just this one place improves your findability immensely.

Unlike Yahoo!, there's no Express service at the Open Directory. Just submit normally, and if you don't get in after three weeks, try resubmitting. Editor links are also listed at the bottom each category. Click on a link, and you can send email to the editor asking about your submission.

Other Human-Powered Directories

LookSmart (www.looksmart.com) is an important service on its own, but it also provides the main results used by the even more popular MSN Search service (search.msn.com). That makes it important to be listed here. Like Yahoo!, there's a $199 Express option. Unlike Yahoo!, this is available for any category, and I recommend using it. Otherwise, you can submit normally for free. If you don't get in after six weeks, send a message using the "Check Status" form, http://www.looksmart.com/aboutus/contact/checkstatus.html.

Two other directories are Snap (www.snap.com) and Go Guides (guides.go.com). At Snap, your submission will appear within minutes in its secondary LiveDirectory listings. Eventually, you may be promoted into the main Snap listings. Go Guides provides the human-powered categories that appear in Go.com's crawler-based search results. To be listed, sign up as a guide, locate a category for your site, and submit. When another guide approves your listing, you'll appear within your category.

PROPER TAGGING

Despite the rise of human-powered services, crawler-based search engines such as AltaVista continue to thrive and survive. Because they use automation, even if you do nothing, your site may still appear in these type of services. But by prepping your pages with proper title and meta tags, you can better ensure that they may rise to the top of search results.

A key step is to give each page in your Web site a descriptive HTML title tag. When you make or revise a Web page, take a moment to review its contents, then summarize what it is about, just as if you were writing a headline for the page. Then put this summary into the title tag. The format is shown as follows, and your summary replaces the "title goes here" text.

<title>title goes here</title>

Next, every page should have a meta description tag on it. This controls how your page is described in the listings of crawler-based services that support the tag. It's an opportunity to tell users exactly what your pages are about, in your own words, so make use of the tag.

An easy step is to simply use the opening paragraph of the page for the description. The format of the tag follows--you replace the part that says "description goes here" with your description.

<meta name="description" content="description goes here">

Most search engines will read the first 200 characters of your description. Don't worry if you go longer than this, the extra characters will be ignored and don't affect the process.

There's also a meta keywords tag, which allows you to define keywords that you think your page is relevant for. Just putting the words here does not guarantee your page will automatically rank well for those terms, but it can help, especially if those words appear within the HTML copy of the page. The tag's format follows. Just replace the "keywords go here" section with your keywords:

<meta name="keywords" content="keywords go here">

Most search engines that support the tag will read up to 1,000 characters of text. Typically, page authors will separate the terms within the tag by commas. Feel free to do the same, but commas are not required, nor should you use them to separate important multiple word phrases.

Also, if you find the keywords tag confusing, then don't use it. It's not worth spending a lot of time worrying about it, in relation to the relatively minor impact it will have on your rankings.

Other Tags

What about Dublin Core tags? None of the major search engines support them, so don't use these tags for their benefit. However, if you have other reasons to use them, certainly carry on doing so. Just understand that you'll still have to have the additional tags I've mentioned in the sidebar.

Also, you may occasionally see a "robots" meta tag or a "revisit" tag. The first should only be used if you need to block pages and can't use a robots.txt file to do so (see http://info.webcrawler.com/mak/projects/robots/robots.html for more about this file). It is not needed to get crawlers to index your site. The second has no effect on the major search engines and should not be used.

CATERING TO CRAWLERS

Tagging your pages is all well and good, but it has no impact if the pages themselves never get listed. Search engines do not list every page on the Web, a fact that has become well publicized over the past two years. But by making some architectural changes, you can increase the odds that more of your pages will get listed.

If you use frames, get rid of them. Frames put up a serious roadblock to many search engine crawlers. If you did nothing else, your search engine-related traffic will increase because more of your pages would get listed.

Large sites face another problem. In general, search engines gather a sample of pages from each site they visit. They don't do a deep crawl and list every page. This means that if you have multiple Web sites, you'll probably get more pages represented than if they are all within one single site.

For instance, imagine a typical academic site, where each department is given its own section:

http://www.university.edu/science/

http://www.university.edu/literature/

http://www.university.edu/library/


Although these are distinct sections, they still reside within the main university.edu Web site, and a search engine will typically sample only a certain number of pages from within that entire site.

In contrast, it's fairly easy to configure many Web servers so that each department can have its own independent Web site, using subdomains, such as:

http://science.university.edu/

http://literature.university.edu/

http://library.university.edu/

Search engines will treat each of these sites independently, which ultimately means more pages indexed.

SUBMITTING PAGES

In the past, there used to be some advantage to doing a "deep" submit of Web pages to crawler-based search engines. This is no longer the case. In general, submitting only your home page is sufficient. I also recommend submitting one page from "inside" your Web site, just in case your home page is inaccessible or has some other problem. By submitting the inside page, you'll give search engines an alternate entrance to begin their crawl.

WHERE DO I SUBMIT?

Go to any of the major crawler-based search engines and look for "Add URL," "Submit Site," or similar links that usually appear at the bottom of their home pages or via their help sections. The main crawler-based services to cover are AltaVista (www.altavista.com), Excite (www.excite.com), FAST (www.alltheweb.com), Go/Infoseek (www.go.com), Google (www.google.com), HotBot (www.hotbot.com), Lycos (www.lycos.com), and Northern Light (www.nlsearch.com). At Lycos, be sure to use only the link on its home page to submit to its crawler database.

How often to resubmit is another question that is sometimes asked. In general, you only need to do this if you notice an adverse change. For instance, if your Web site statistics software shows that you no longer get much traffic from a particular search engine, then resubmitting may help.

CONCLUSION

There's much more that can be done to promote a Web site through search engines, but the preceding steps are the most essential. If you've done nothing, concentrate on these, and you should see some improvements. By taking just a little time away from your searching activities, you can help ensure that others can find you!


Tagging In Practice

<head>
<title>Welcome to ALA</title>
</head>
Now let's put these components together, with a real life example based on the American Library Association's home page. On the top is a clipping from the "head" section's source code as it currently appears on the Web. On the bottom is the same section with some highly recommended improvements.

The original title was "Welcome to ALA." I added the full name after this to help ensure that anyone searching for the association by name, rather by acronym, would find it.

<head>
<title>Welcome to ALA, The American Library Association</title>
<meta name="description" content="Official Web site of the American Library Association, which provides leadership for the development, promotion, and improvement of library and information services and the profession of librarianship.">
<meta name="keywords" content="ALA home page, official Web site of the American Library Association, libraries, awards, grants, scholarships, censorship, copyright, intellectual property, continuing education, accreditation, intellectual freedom, international relations, legislation, preservation of library materials">
</head>
The site had no description tag, so to make one, I simply copied the text from an introductory paragraph on the home page. This small step took less than a minute-yet it would make the ALA's listing more attractive and help it rank better for some terms. One key modification I made was to add "Official Web site" to the description. This is important. Users do search for "official" sites, so make use of this term to help reassure them that your site is the correct one.

I drew the keyword terms from looking at ALA's Interest and Activities page, and you can see that commas have not broken up phrases. It would be just as valid not to have the commas at all. This information is not placed into some type of relational database that requires terms to be delimited. Instead, the keywords tag just provides more ordinary text for the search engines to index, but in an area that the user doesn't see.

Also, notice that the tags are shown within the "head" area of the Web page. That's important for them to be read properly by search engines.

Danny Sullivan is editor of Search EngineWatch.com, http://searchenginewatch.com. He can be reached through that site, which also provides more information about search engines and submission issues.

Comments? Email letters to the Editor at editor@infotoday.com.

[infotoday.com] [ONLINE] [Current Issue] [Subscriptions] [Top]

Copyright © 2000, Information Today, Inc. All rights reserved.
Comments