On The Net
The UCE (Spam) War and Its Impact
By Greg R. Notess
Reference Librarian Montana
The Internet's killer application is not Web browsers. It is e-mail. Communication
capabilities have become so interwoven into most of our workand homelives
that it is sometimes difficult to remember how we managed to function without
it. And younger generations never knew.
This essential foundation of the Internet is under attack. Sending
e-mail is so easy. Millions of e-mail addresses are readily available.
Combine these two concepts, and e-mail becomes an obvious direct marketing
campaign. Unsolicited e-mail messageshawking everything from
mortgages to online casinos to unknown stocks to Nigerian banking scamshave
proliferated. These messages fall under the heading of Unsolicited
Commercial E-mail (UCE), often known, unaffectionately, as spam. And
UCE is growing at an increasingly annoying pace.
More than just an annoyance, the flood of UCE is also changing e-mail
behavior. Aggressive filtering will occasionally cause the loss of
real messages. Attempts to avoid spam make people try new ways of communicating
their e-mail addresses. So even if you are among the lucky few who
have so far avoided the avalanche of unwanted e-mail, it is important
for the information professional to be aware, at least, of the impact
and changed behaviors that have resulted.
IMPACT OF UCE
E-mail usage continues to grow. According to a recent Computerworld article
(Gretel Johnston, "We've All Got Mail," Computerworld Online.
Sept. 27, 2002 [www.computerworld.com/printthis/2002/0,4814,74682,00.html]),
the total number of e-mail messages is expected to jump to 60 billion
a day by 2006 from the current 31 billion a day. And the majority of
the jump will be in spam, notifications, and alerts. Mike France, in "Needed
Now: Laws to Can Spam" (Business Week :100, Oct. 7, 2002) [www.businessweek.com/smallbiz/content/sep2002/sb20020926_5958.htm],
notes, "Spam now accounts for 38 percent of all e-mail traffic, up
from 8 percent last year," according to filtering company Brightmail.
How does it happen that one out of every three messagesor moreis
an unrequested stock pitch, banking scam, or ink jet cartridge ad?
E-mail address harvesters are to blame, at least in part. These unpleasant
little bots crawl the Web looking for e-mail addresses. Ever post a
message on an online forum or include your e-mail address on a Web
page? Then it has probably been harvested and sold to the spammers.
Almost immediately, e-mail messages with subjects such as "Mortgage
Rate Alert," "Re: where to get Viagra cheap," or even "Stop Adult & Spam
E-mail" start showing up in your mailbox. Some will have your e-mail
address in the subject or try to derive your name from the part before
the @ sign. Other subjects start with "adv" or end with a random character
string like "blfwkyj" that can be used to track the abysmally low response
rate. An increasing number come as HTML e-mail, full of pictures and
links. And then there are those that arrive with attachments, some
quite large or containing a virus.
What defenses does the end-user have against UCE? There are many,
none of which will be 100 percent effective. Private addresses, host
filters, personal filters, whitelists, add-on programs, and other solutions
can all make a significant dent. The best solution for individuals
varies greatly, depending on their e-mail program, e-mail host, address
availability, comfort with extra software, and many other factors.
These days, the first step is to keep your e-mail address unpublished.
Not getting any spam yet? Do your best to keep your e-mail address
from getting published on a Web page. If you want to post a message
in an online guest book, Usenet group, or Web forum, beware. Anytime
that an online form asks for your e-mail address, look at the privacy
policy to see if it states whether your account will be kept completely
confidential or not.
As a second step, establish several e-mail addresses. I have one
just for UCE. Anytime I need to enter an e-mail address on a form where
I do not trust the site to keep it private and I have no need to receive
e-mail from them, I use the spam address that has no connection with
my regular e-mail address. Another approach is to establish one e-mail
address just for friends and family and keep that one separate from
any work e-mail address, especially if the work address is posted on
your organization's Web site or otherwise published somewhere. But
even a private, unpublished address may get picked up.
One common approach used by Usenet and public Web forum aficionados
is to visually mask their real e-mail address. Instead of firstname.lastname@example.org,
the address will appear in both the header and the message as something
like "Spam prevention remove numbers for address <email@example.com>.
One Web site uses this defense:
"To prevent bots from yoinking my e-mail address I display it somewhat
cryptographically. Just remove all of the x's to get the actual e-mail
Others suggest more technical approaches. For contact information
on a Web site, a form can be used to mask the actual e-mail address.
But this does not display the e-mail address to those who would like
display and create a mailto: link that should not get picked up by
an e-mail gatherer. The following example shows how to create a mailto:
link on my last name that points to my e-mail address of firstname.lastname@example.org:
var contact = "Notess"
var e-mail = "greg"
var e-mailHost = "notess.com"
document.write("<a href=" + "mail" + "to:" + e-mail
+ "@" + e-mailHost+ ">"+ contact + "</a>" + ".")
Most of us eventually have our e-mail address found. Unless we want
to switch to a new address every time we start getting UCE, and then
change all our business cards and notify all contacts, it is time to
look into filters.
There are many levels of filters that can be applied. Filters can
be set up in your e-mail program, on your e-mail account, and even
at the server level for your e-mail domain. At the simplest level,
most e-mail programs have some kind of filtering options which I explored
more fully in my November/December 1998 ONLINE column "Filtering
the E-Mail Storm." Look for the most common words in the spam subject
lines, and filter any message with those words or phrases directly
to the trash folder. Terms like "mortgage," "viagra," "ink jet," "get
rich quick," "hgh," and "porn" are some to consider. Just be sure not
to use terms that colleagues or business associates might use in their
subject lines. The spammers are so familiar with this technique that
they use many spelling permutations (like "viag," "pr0n," and "hg h")
to get around the filters.
Your Internet access provider can put many filters on at the server
level, and more and more companies, consumer Internet providers, and
other organizations are doing the same. Some of these server filters
will just flag UCE. For example, flagged e-mail may have a subject
starting with " POTENTIAL SPAM " before the regular subject
heading. These server filters use a variety of sophisticated techniques
to try to identify the spam beyond just looking for specific keywords.
An Internet access provider may also have individualized e-mail account
server filter options. With this kind of set-up, the user can control
what kinds of e-mail to filter and what to let through. A simple selection
may ban all e-mail with executable attachments, a great way to avoid
virus-carrying messages. If there is an option to block file extensions,
input the following to block most executable files: .BAT, .CHM, .COM,
.EXE, .HLP, .HTA, .LNK, .PIF, .REG, .SCR, .SHS, .VBE, .VBS, .WSF, .WSH.
WHITE AND BLACK LISTS
E-mail program filters and e-mail provider filters may also offer
whitelist and blacklist options. The blacklist lets the user designate
specific e-mail addresses or domains to completely exclude. All messages
from the blacklisted addresses are disposed of, no matter what the
content is. These are best reserved for the pernicious unwanted e-mail
that always comes from the same address.
A whitelist lets you specify addresses from which e-mail should always
be delivered. Ideally, the whitelisted addresses bypass all other filters.
That way, if a boss or colleague sends an e-mail message that happens
to contain a filtered word in the subject line, the message will still
get through. Whitelist your organization's domain, .gov, .mil, and
even .edu along with all your most frequent correspondents.
DISPOSITION OF UCE
So what happens to a filtered or blacklisted message? Several dispositions
are often available. A filter can segregate the messages into a separate
folder or special location so that the messages can be reviewed occasionally
to make sure nothing that should have come through got filtered inappropriately.
Or the filter can move all the messages straight to the trash folder.
Some of the server-side solutions give disposal options such as a
black hole or a bounce. Messages sent to the black hole are just tossed
and are not recoverable. Bounced messages go back to the sender. Some
systems let the user specify a message to go along with the bounced
message. For example, an e-mail rejected because it contains an executable
file could be bounced and include a message such as, "This address
does not accept executable attachments."
And then there are the add-on software products like MailWasher and
Cloudmark SpamNet. MailWasher, available at www.mailwasher.net,
works for users whose Internet access provider is also their e-mail
host. It checks e-mail before you download it and both deletes obvious
spam and bounces it back to the sender. The clever part about it is
that it bounces in a way that makes it appear that your e-mail address
is not valid in the hopes that the spammer will remove the address
from their database.
SpamNet, available at www.cloudmark.com,
works on Outlook 2000 and XP and should be available soon for Outlook
Express. Beyond just identifying UCE, it moves it to a special Spam
mailbox as well as adding that message to a database of UCE so that
other SpamNet users will have that message automatically filtered as
UCE AND OPEN RELAY DATABASES
There are several other Internet groups that have begun putting together
databases of known spammers and sites that act as open relays for spammers.
Databases such as www.dsbl.org, www.ordb.org,
and www.spamcop.org can
be used in connection with filtering software at the server level or
in conjunction with an anti-spam software program. Open relays are
mail servers that will process a mail message even though neither the
sender nor the recipient is a local user. These are heavily used by
spammers, and blocking e-mail from such a site is an efficient way
to cut down on UCE.
But sometimes sites are acting as an open relay unintentionally.
A site might get added to one of these databases for some other reason.
If e-mail from your organization is suddenly getting blocked elsewhere,
check to see if it has gotten added to such a database. All have a
process for removal.
The problem with any defensive moves on the spam front is that they
all can block or discourage legitimate e-mail. I have had opt-in newsletters
to which I have specifically subscribed flagged as "potential spam." A
press query from a company that I had not thought to include in my
whitelist got bounced by another filter because that domain name had
been mistakenly included in one of the open relay databases.
Senders of e-mail also need to be especially aware of these kinds
of UCE prevention behavior. Ever send an e-mail to someone who never
responded? It could be because your e-mail was filtered straight into
a black hole or the trash folder. Alternatively, the recipient's e-mail
host may have flagged or deleted the message without even a notification
to the recipient. While most individual-to-individual e-mail should
get through, for any important message, you can no longer assume that
it has been received.
Do you send out an online newsletter or run an e-mail discussion
group? Don't be surprised if some subscriptions bounce back or a new
member accuses you of sending spam. Even with the industry-accepted
practice of a confirmed, opt-in subscription process, people forget
that they have subscribed. This is one advantage to a confirmed, opt-in
process in which the subscriber first puts their e-mail address into
a Web form and then gets an e-mail that requires a response to confirm
that they do want to be included. Then you have a record of
the subscription request in case anyone subsequently decides the newsletter
is spam and reports it to one of the databases.
Regulatory or legislative solutions may be needed to help stop the
inundation of UCE. But with strong opposition from marketing organizations,
these solutions may be a long time coming, in spite of the Direct Marketing
Association's recent turnabout, when it announced on October 20, 2002
it now supports spam legislation. The press release quotes H. Robert
Wientzen, president and CEO of The DMA, as saying, "Without a solution
that includes legislation, legitimate marketers who use e-mail to
communicate with consumers will continue to suffer at the hands of
spammers.... Spam must be stopped, and we will take every step necessary
to ensure that e-mail is not lost as a marketing channel to the likes
of Nigerian widows and unseemly and illegitimate come-ons."
In the meantime, there is no single solution that can be guaranteed
to prevent all UCE. And most people will have very different options
depending on their e-mail hosts and software. Use whatever combination
of tools works best for you, but remember to be aware of their potential
impact on legitimate e-mail sent to you and on e-mail you send to others.
R. Notess (email@example.com; www.notess.com)
is a reference librarian at Montana State University and founder of SearchEngineShowdown.com.
Comments? Email the editor at firstname.lastname@example.org.