Back in the days of Windows
3.11 for Workgroups we had a neat little network. Strung together with
several miles of state-of-the-art, category-five wire into brand-new 10-megabit
Ethernet hubs, the whole thing sailed out onto the Internet on 56K frame
relay circuits. Our network was only the second to be wired in the city
after Bank of America put together its ATM center and the telephone company
was looking for tenants for its very empty frame relay switch.
3 p.m. — do you know how much bandwidth your network is using?
With Word and Lotus sitting
on Moby Fred, our reliable Netware 3.1 server, and our centrally controlled
Saber Menu making it impossible for anyone to change anything, it was a
sweet world. Before long Netscape replaced Mosaic and WinWeb, but nothing
else much changed.
At the LAN level, Ethernet
has pretty well taken over the world. There is a smattering of other topologies,
but percentage-wise they add up to single digits. Down deep in the bits
and bytes, Ethernet is kind of a strange animal. In fact, when you first
study how it works you come away with puzzlement. How can it possibly work?
It is utter chaos down there—absolute utter chaos.
The acronyms most often
associated with Ethernet are "CSMA/CD." The first two-thirds of this stand
for "Carrier Sense Multiple Access." What does this mean in English? Well,
"carrier sense" is like listening for the dial tone on a telephone. When
you pick up your phone and hear the dial tone sound, you are "sensing the
carrier." Your ear listens for a dial tone. That little network interface
card in your computer, the "NIC," senses the carrier. It's the same thing.
"Multiple Access" is like
a party line. In a basic Ethernet system, every computer that is turned
on and hooked up is listening for a dial tone on the party line all the
time. That's what "Carrier Sense Multiple Access" does. Each computer is
always listening for words from another computer that are destined for
itself. Just like a letter in the mail, those words are wrapped in an envelope
with an address written on the outside. The post office calls this a letter;
Ethernet calls this a "packet."
While a letter has information
printed on paper and folded up inside the envelope, a packet is more like
a freight train traveling on a single track (a monorail), in this case
a single wire, where the contents are strung out on this monorail one after
the other, bit by bit. The address information is in the locomotive part
of the train along with some other stuff identifying what kind of packet
it is, how long it is, etc. The locomotive is the "header." The caboose
is the "footer" whose basic job is to say, "OK, I'm done. Anything after
this does not belong to this train."
Now, you have a single monorail
track on your local area network. Just one. How many trains can be on that
monorail at one time?
One, of course. This is
real life. It's very physical. Only one train (packet) at one time. The
interesting thing about these packets (let's leave the train alone for
awhile) is that they are seen by every single computer on the LAN. It's
as if the postman were delivering the same letter to every household on
the block and depending on the homeowner to take only the letters belonging
to him while rejecting the letters belonging to others. The only packets
accepted by your computer are the ones with your computer's address on
them; all others are seen, but ignored.
Like a Speeding Train
What happens if you want
to send a packet out onto the Ethernet wire? Your computer just goes ahead
and does it whenever it wants. It shoves a packet onto the wire without
even waiting for the green signal light. Most of the time that is successful.
Ethernet speeds begin at 10 megabits, or 10 million bits per second. A
modest-sized packet of a few hundred bytes is on and off the wire in nanoseconds,
literally. There is room for thousands of packets on the Ethernet wire
in a single second. On a slow network, your packets basically have the
wire to themselves at any given time. In fact, they must
have the wire to themselves. If they don't, a collision occurs.
That's what the "CD" part
of CSMA/CD stands for. "CD" is "Collision Detection." Ethernet has to know
when this happens. If you happen to be looking at most hubs these days
when it does, you see a little yellow or orange light blink. Most of the
time the light blinks green, but if it blinks orange, a collision has occurred.
Ethernet is smart enough to know when this happens, and when it does happen
the network's response is to wait a random length of time, measured in
hundredths of a second, and then retransmit the same packet again. The
second time through it usually goes fine, and you never know the difference.
At these speeds it's transparent to you.
At the physical level this
is all very straightforward and, once you take the jargon out of it, very
easy to understand. There's no magic here at all, though it's very fast;
that's true. You certainly don't want to delve into the contents of the
headers and footers if you can possibly avoid it. CSMA/CD is just a quirky
kind of party line transmitting data instead of voices. No big deal.
Hunka Hunka Burnin' Hub
The Ethernet architecture
works very well when there is a small amount of traffic on the network.
What is considered small? Less than 30-percent utilization. Whoa! What
happened to our 10 megabits? Well, they went away. You get to use a maximum
of three. That's still fast. We had 100 computers on a single LAN like
this for years with no ill effect. We ran about 10-percent utilization
and peaked to 30 percent occasionally, usually when I downloaded some monster
file from the Internet.
Once you reach 30-percent
utilization, collisions happen more and more often. A collision means retransmitting
the packet. When you do this a lot, performance suffers and the network
begins to slow down. You could solve the issue by changing the type of
system you have. For example, Token Ring uses a much different approach.
It comes in 4-megabit and 16-megabit flavors, but the key difference here
is that you can't send a packet unless you have the token. It's like the
stories of Native American tribal councils where you couldn't talk unless
you had the "talking stick." When you stop and think about it, it's an
orderly way to organize a meeting. Likewise, with Token Ring you can utilize
your bandwidth much more effectively.
The problem with that approach
is that you have to change out all your network interface cards in all
your computers, throw away your Ethernet hubs and get Token Ring hubs,
and bring your entire system down when you do this. As we all know, a single
malfunctioning computer is not good; a single malfunctioning network is
Hubba Hubba Hubba
We saw this issue heating
up a couple of years ago by virtue of using a cute little product called
"Lanalyzer," by Novell. I'm not going to give you addresses and all that
because it is no longer sold, but it's relevant to the story. You'll see
why below. Anyway, Lanalyzer ran on a Windows 3.11 computer that I had
at my desk. It had three yellow dials that looked like speedometers where
utilization percentages and a couple of other factors were shown as they
happened (in real time). You could see the number of packets sent and received
by each computer on the network, capture those packets, and look at them
(because each computer sees 100 percent of all the packets), and generate
alarms if traffic exceeded a certain threshold, say 30 percent.
With Lanalyzer we saw our
LAN traffic get busier and busier. There were more and more 30-percent
peaks, probably because the Big-I Internet was weaseling its way more and
more into our lives. Graphical stuff was taking over from text-based stuff.
We had to do something. The cheapest quick fix we could muster was to install
a single "switched hub" in place of the "Mother of All Hubs," which was
at the center of our star pattern. Mother Hub fed all other hubs from a
A switched hub divides your
network into segments so that traffic from the other segments is not allowed
through to your segment. Each port of the switched hub is
effectively its own Ethernet segment, with nothing getting through unless
it is destined for a computer on your segment. Each segment's percentage
of utilization went down instantly to the 1- to 2-percent range. Collisions
were nonexistent. Our network was saved, for about $2,000.
the little guy that had warned us of the impending bandwidth catastrophe,
wouldn't work anymore. You can guess why. It was measuring traffic only
on my segment, no one else's, so now I could see traffic to technical services,
but not to administration or, most importantly, to the computer room where
all our outside lines converged. Besides, it wouldn't work on Windows 9X
either, so Lanalyzer effectively worked its way out of a job.
The other nice thing about
these switched hubs was that they had a couple of 100-megabit ports on
them as well, so we stuck another hub just like it in the computer room,
thereby exponentially increasing our bandwidth to most of the servers and
the Internet gateway. We managed to very cheaply extricate ourselves from
what amounted to a huge capacity issue, but in so doing we went blind because
we couldn't see our network anymore. Recently we found some new tools that
have rectified the situation, which is what I want to talk about next.
Regulating Traffic Bursts
One suite of programs that
can replace Lanalyzer is from SolarWinds (http://www.solarwinds.net;
$995). The new Engineer's Edition (v. 3.2) includes a Network Performance
Monitor, and it circumvents the segmentation problem by allowing you to
measure traffic at a router port. Since the router is usually pretty close
to all your servers, if you can see the router you can see the majority
of traffic on your LAN. In our case the main router has interface cards
for every single branch, therefore not only can we measure traffic on the
LAN, but we can also get to traffic at every branch. Figure
1 shows traffic on the Qwest port, which is our Internet provider,
over several days. You can see that traffic jumps up to about 1.25 Mbps
(million bits per second) during midday. Since this port is hooked to a
T-1 line that has a maximum speed of 1.544 Mbps, you can see that we're
pretty close to capacity on that line.
Figure 1: Our Network Performance Monitor shows daily traffic bursts.
This is a frame relay T-1.
Without getting into too many specifics, frame has a "Committed Information
Rate" (CIR) which is guaranteed by the phone company, and a maximum line
speed, which is the capacity of the T-1 (1.544 mbps). Our CIR is 768 Kbps
(kilobits—not megabits; so thousands instead of millions), about half the
maximum line speed. Anything above 768 (about .75 on the graph) is "burst
traffic." That traffic is not guaranteed and it can be lost on the line.
If this happens, the packets are re-sent.
Now you have all the information
to come to a conclusion. That conclusion is: We're probably in trouble.
Every day we regularly burst far above our guaranteed rate. When everyone
starts pounding on those PCs in the afternoon to get out onto the Big-I
Internet, we go to near-maximum capacity. We're not only well above our
CIR, we're almost to the maximum capable line speed. Increasing our CIR
isn't the real solution here, since bursting is allowed anyway. The real
solution is getting another T-1 line.
We've decided to go with
AT&T for our second T-1 line, albeit this time a straight T-1 where
we enjoy all the capacity of the line without the need to "burst." This
will allow some redundancy to our Internet service. If both Qwest and AT&T
ever go down, it will very likely be due to a problem that will affect
the entire city, not just the library.
This Network Performance
Monitor is but one of many different SolarWinds tools that you get for
the thousand-dollar price that I mentioned. There are several dozen tools
included (see Figure 2) that measure bandwidth,
discover just what is on your network (with 300 devices it can get a little
complicated), measure the CPU load of your routers, and do all sorts of
network management tasks. This Engineer's Edition is the most expensive
of four editions, but it is the only one that offers this Performance Monitor
as part of the suite. At this stage I couldn't imagine being without a
monitor of this sort. You can pay a lot more for more sophisticated monitors.
We've seen them available in hardware form for many tens of thousands of
dollars. From that perspective, $1,000 is cheap insurance.
Figure 2: Solar
Winds' Engineer's Edition contains all these network tools.
MAC Address Discovery
IP Network Browser
Router CPU Load
Adv. Subnet Calculator
IP Address Management
DHCP Scope Monitor
DNS/Who Is Resolver
SNMP Brute Force Attack
SNMP Dictionary Attack
Update System MIBs
Michael Schuyler is the
systems librarian for the Kitsap Regional Library System in Bremerton,
Washington. His e-mail address is firstname.lastname@example.org.