Searcher
Vol. 10 No. 3 — March 2002
• FEATURE • 
Dialog Pricing Redux: Déjà Vu All Over Again
by Mary Ellen Bates • Principal, Bates Information Services
Table of Contents Previous Issues Subscribe Now! ITI Home

Sidebars: Improving Your Profile Pricing Ch-Ch-Changes Supersearcher Tips Search Tables Search Tables [PDF]

There are many things I'm thankful for — my health, my family, my new dog, my job ... the list could go on and on. And when I think about the online world, I thank whatever deities, fates, or karmic influences spared me from having to figure out how to set prices for anything other than my own time. I have talked with a number of information industry people about pricing and have concluded that the basic problem is trying to figure out the value of something that is neither strictly a product nor a service.

Sure, there are a few surviving Web sites that provide a stripped-down version of document retrieval services at no cost or for a minimal fee: ft.com, FindArticles.com, Electric Library, and MagPortal come to mind. What most serious online researchers want, though, is a combination of a high-quality product — a wide range of full-text information sources with a deep archive — and high-end services — tools for building sophisticated searches, relevance ranking of results, flexibility in output options, enterprise pricing, etc. So they go to aggregators such as Dialog, Factiva, and LexisNexis, all of which offer a wide range of content and a full-featured array of search-and-retrieval functions.

When searchers sign up for a particular online service, they assume that they'll get fast responses from their searches, virtually no down time, telephone customer service 24/7, and free training. However, they don't necessarily assign a monetary value to these services — "of course you're going to be constantly upgrading your servers and improving your search tools, but why should I pay extra for what I take for granted?"

Add to this the fact that any pricing plan by definition trains users in how to search. If you're charged for connect time, you learn to think offline and type fast. If you're charged only for output, you use that system for searches in which zero hits tells you something valuable. If you're charged for processing time, you shy away from using resource-intensive tools. This isn't rocket science ... any animal trainer knows that you reward desired behavior with treats and you punish unwanted behavior with negatives. Like Pavlov's dogs, we searchers learn to use an online service a particular way when rewarded with lower online bills. Over the course of the past few years, it has been very interesting to watch Dialog's experimenting with its pricing strategies and the resulting behavior of its subscribers.
 

Dialog's Pricing History in a Nutshell
Those of us long in the tooth remember searching Dialog in the old days. It helped to be a touch typist and upgrade as quickly as possible to faster modems, since most of the cost of a Dialog search was the connect-time charge. The faster you ran your search and logged off, the cheaper the search. Over the years, Dialog tweaked the relative cost of connect time and output, moving from an emphasis on the time spent online to the number of documents displayed. In large part, this switch responded to the improvements in modem speed; it's not too tough to figure out that your revenue is going to drop significantly if your customers shift from 300-baud (30 characters per second) acoustic couplers to 2400-baud (240 characters per second) modems.

Then things got weird. In June 1998, with little warning and no prior training for its customer support staff, Dialog rolled out DialUnits — essentially a charge for system processing cycles — to replace the familiar connect-time charge. This change, however poorly executed, was driven by the need to accommodate the shift from searching via telnet and packet-switched networks (remember Tymnet and Telenet?) to searching on the Web. In an HTTP environment, you can't reliably calculate connect time, so Dialog decided to base the price on something it could count — how hard its computers worked. The problem, of course, is that searchers had no idea how DialUnits were calculated and it's impossible to audit DialUnits.

Unfortunately, due to the hasty implementation of DUs, the result was that virtually all searchers found that the cost to search Dialog had risen dramatically and unpredictably. I compared the pre- and post-DU costs for a number of searches and concluded that, well, we were getting shafted. A month later in July 1998, Dialog scrambled to correct some of the more egregious problems in DU pricing, although the end result was still a 20-50 percent increase in cost. For the blow-by-blow description, see my article "Dialog's DialUnits: A Price Increase in Sheep's Clothing" (Searcher, September 1998, http://www.infotoday.com/searcher/sep98/bates.htm).

Two months later, Dialog announced another change in DialUnit pricing, this time eliminating the policy of rounding up DialUnit charges to the next full unit and eliminating DU charges for administrative commands such as Display Sets and HELP. The end result was that costs dropped down to pre-DialUnit levels. For an analysis of the impact of the September 1998 changes, see "Dialog's DialUnits Revisited: Lassie Come Home" (Searcher, November/December 1998, http://www.infotoday.com/searcher/nov98/bates.htm).

In May 1999 I decided to run all my test searches again. Imagine my surprise when I discovered that searches were consistently incurring more DialUnits than they had the prior year. After much hemming and hawing, Dialog executives admitted that, yes, they had "revised the algorithm for calculating DialUnits." Can you say "unannounced price increase"? I thought so. My third article on Dialog's DialUnits, "Dialog's DialUnits: There Is a Great Disturbance in the Force," appeared in Searcher, July/August 1999 [http://www.infotoday.com/searcher/jul99/bates.htm].

The only good news in all of this was that in May 2000, Dialog was sold to The Thomson Corp. The online search community grew cautiously optimistic to find Dialog once again in the hands of management who knew and understood Dialog's core market.
 

Re-Connecting with Connect Time
Fast-forward to 2001. In June, Dialog announced that it was revising DialUnit pricing and would introduce (or, more properly, re-introduce) a connect-time pricing option. A sentence in the press release piqued my curiosity: "Currently, when users execute the same search on different Dialog platforms (DialogClassic, DialogClassic Web, and DialogWeb), results can vary, depending on how each platform processes some of the commands." I went back and re-executed all of my test searches and, sure enough, there were striking disparities in cost. A chart showing the prices for identical searches in DialogWeb, DialogClassic and DialogClassic Web appeared in an Information Today Inc. Newsbreak [http://www.infotoday.com/newsbreaks/nb010626-1.htm].

The revision of the DU algorithm was completed in October 2001 and it did indeed smooth out the differences in cost among the various platforms and virtually eliminated the penalty of searching on DialogClassic. Connect-time pricing finally rolled out in January 2002. In the meantime, Dialog also repackaged its flagship content in a number of end-user-friendly products, including Dialog1, Dialog Company Profiles, DialogSelect, and Dialog Open Access. All but one of these are only accessible to existing Dialog customers. Dialog Open Access is available to anyone with a credit card — no subscription to Dialog is required. The connect time vs. DialUnit pricing discussion does not come into play here, since all of these products are entirely output-based.
 

Connect-Time Basics
So, how does connect-time pricing work, now that it's finally rolled out and operational? Answer? Pretty much like you'd expect. The meter starts running as soon as you issue a Begin command. There are a few free files (although I use the word "free" advisedly, since Dialog still charges $13 an hour for "Telecommunications/Internet charges") — the ONTAP training files, and most of the Dialog Finder files (410, 413, 414, 415, 416). Beilstein Online (390) is also free of connect-time or DialUnit costs, but output costs range from $10 to $300 per record. Note that DIALINDEX (411) is not a free file in either pricing plan. It costs $40/hour in the connect-time pricing and $1.75 per DialUnit.

In general, connect-time pricing parallels DialUnit costs. With the exception of Beilstein Online, which bases 100 percent of its charges on output, files that have higher DU costs also have higher connect-time charges. A quick comparison of prices shows the following rough correlation:
 
A file with DialUnits of: has connect-time charges of around:
$20  $150/hour
$6 $80-$100/hour
$3 $40-$60/hour
$1 $35/hour

Note that both DialUnit and connect-time pricing still have separate charges for output. The per-record charges are the same for both plans. And both plans impose that silly $13/hour Telecommunications/Internet charge, so even DialUnit pricing has a connect-time component.

What happens if you search several files simultaneously with connect-time pricing? Say you wanted to search INSPEC ($85/hour), NTIS ($75/hour) and Ei Compendex ($90/hour) — will you be charged a total of $250/hour for the search? Thank goodness, the answer is no. Instead, just as with DialUnit searching, the connect time is split among the three files in proportion to the amount of time spent in each file. The following is the logoff notice for a combined search of files 2, 6, and 8. The total time spent running the search was 0.066 hours, but I was charged for 0.022 hours in file 2, 0.018 hours in file 6 and 0.025 hours in file 8.


09jan02 11:05:38 User000000 Session D8471.2

$1.87 0.022 Hrs File2

$1.87 Estimated cost File2

$1.35 0.018 Hrs File6

$1.35 Estimated cost File6

$2.25 0.025 Hrs File8

$2.25 Estimated cost File8

OneSearch, 3 files, 0.066 Hrs FileOS

$0.79 TELNET

$6.26 Estimated cost this search



I was curious to see if the breakout of time really did correlate to how much time was spent in each file, so I ran the following OneSearch in an abstracts-only file (ERIC) and a full-text file (Trade & Industry):



SYSTEM:OS - DIALOG OneSearch

File 1:ERIC 1966-2001/Dec 05

(c) format only 2001 The Dialog Corporation

File 148:Gale Group Trade & Industry DB 1976-2002/Jan 08

(c)2002 The Gale Group

Set Items Description

—- ——- —————-

?s book/tx

S1 317406 BOOK/TX

?cost

09jan02 10:57:10 User000000 Session D8470.4

$0.00 0.000 Hrs File1

$0.00 Estimated cost File1

$1.20 0.015 Hrs File148

$1.20 Estimated cost File148

OneSearch, 2 files, 0.016 Hrs FileOS

$0.19 TELNET

$1.39 Estimated cost this search


By limiting my search to the TX field, which doesn't exist in ERIC, I ensured that the search would only execute in Trade & Industry. Then I checked the cost and, sure enough, all the connect time was allocated to the full-text file and no time charged to ERIC. It warms my heart to see things work the way they should.

What sets Dialog's pricing plans apart is that you can toggle between the two plans almost at will. Though you can't switch the pricing during a search, you can change from one to the other at the end of one search session, log off and log back on again to have the switch take effect. You determine which pricing plan you want by adding or editing a line in your Profile — SET CONNECT ON for connect-time pricing and SET CONNECT OFF for DialUnit pricing. Note that the default is DialUnit pricing, so if you haven't done anything, that's what your pricing plan is now. [See the sidebar "Improving Your Profile" for step-by-step instructions on creating and editing your Profile.]

OK, so you can't change the pricing plan within a search session; you have to log off and log back on for the change to take effect. Getting in touch with my inner hacker, I thought I could get clever and do a complex search using connect-time pricing (which would be cheaper than DialUnit pricing), KEEP the records to Set 0, change my Profile to put me in DialUnit pricing, issue a Logoff Hold command (which would retain my Kept records in Set 0), log back on with DialUnit pricing, and then print out the results (at virtually no cost, since displaying records incurs connect-time charges but almost no DialUnit costs). Alas, the Dialog programmers thought of this hack first. When I did the Logoff Hold and then reconnected, I got the following message:


SET CONNECT NOT EXECUTED

PRICING METHOD MUST BE THE SAME ON RECONNECT


Drat! Foiled again! Each search has to be done using either connect-time or DialUnit pricing; you can't mix and match.

For more information on connect-time and DialUnit pricing, go to http://www.dialog.com/pricing. This page includes links to a Connect Time Quick Reference report, a cheat-sheet on how to edit your PROFILE and change from DialUnit to connect-time pricing, price lists for connect time and DialUnits, and a FAQ on DialUnit pricing.
 

The Bates Pricing Tests
I first developed a set of test searches back in May 1998, when Dialog announced the switch from connect-time pricing to DialUnits. I tried to test as many different aspects of Dialog searching as I could, including simple look-ups, resource-intensive searches, multiple-file searches, document-delivery searches, sit-there-and-read-results searches, and a few searches that I knew would give unpredictable results (heh, heh, heh).

A number of people at Dialog have told me that they have used these sample searches as they modified the DU algorithm and developed the connect-time pricing. Too bad I can't figure out a way to charge them BatesUnits for each use of the searches.

A couple of notes before we dive into the nitty-gritty of the results. I didn't include very many output-heavy searches, since the cost of output has remained fairly steady throughout Dialog's pricing changes, and since output costs should be virtually the same under DialUnit and connect-time pricing. As noted above, neither DialUnit nor connect-time pricing includes output costs, and the per-record charges are the same regardless of which pricing plan you use. I suppose that if you have a 2400-baud modem and you wanted to capture to file a large number of records, you would see a slight increase in cost under the connect-time plan, but the solution in this case is to send the results to e-mail (type HELP EMAIL for instructions).

I conducted the searches using a DSL line; again, if you use a slow modem or if you type very slowly, you may notice slightly higher connect-time charges, but I don't believe that the difference will be significant. According to Dialog, a majority of their usage is now on the Web, so I assume that most people have a fast connection. (And if you still use a 2400-baud modem to search Dialog on the Web rather than via telnet, you're a glutton for punishment.)

Finally, those of you who have followed the Dialog pricing saga since 1998 will notice that in order to better test how DialUnit pricing works in the real world and to eliminate multiple searches that tested virtually the same feature, I have changed a couple of the searches slightly over the years. Since connect-time pricing is time-sensitive, I included some browse-time in some of the searches, to calculate the impact of reading time as well as searching. Table 1 shows the 20 searches and how much each search costs using six different scenarios — the search on DialogWeb, DialogClassic (via telnet), and DialogClassic Web with DialUnit pricing and then a repeat on all the same platforms with connect-time pricing. These searches were done in early January 2002. According to a letter sent to Dialog subscribers, Dialog will be imposing a 10 percent surcharge on DialUnit pricing for DialogClassic (via telnet) searches beginning February 2002. To factor that in, I have calculated the cost with the 10 percent DU surcharge for the DialogClassic DialUnit pricing column. (See the sidebar "Pricing Ch-Ch-Changes" for the details of all the changes in fees.)

So, let's go through the results, a search at a time.

Looking at Table 2 the first search strategy is straightforward: no heavy-duty CPU search, no time-consuming output. As expected, the cost under DialUnits was pretty consistent across platforms. However, the cost under connect-time pricing varied more than I would have expected. Why the noticeably higher cost for DialogClassic? Behind the scenes, Dialog executes a FileHold command with DialogWeb under connect-time pricing. FileHold is similar to PAUSE, that wonderful command that turns the connect-time meter down to $30/hour while you sit and think. Given the vagaries of the Web, Dialog decided to insert a FileHold each time it sits and waits for another command during a DialogWeb search. Note that this FileHold isn't applied during DialogClassic or DialogClassic Web, because of the way in which users connect to Dialog. However, you can type a PAUSE command if you're in connect-time pricing, thus turning down the connect-time rate while you read output, check with a patron, answer the phone, and so on.

Bottom line: Not much of a difference in cost between the two pricing plans for a simple search.

Search 2 (see Table 3) is a multi-file search and includes some CPU-heavy commands. The limit s1/1997:2001 command is a killer; regardless of the size of the LIMITed set, Dialog still looks for all records from years 1997 through 2001 and then compares that set to the LIMITed set. DialUnit and connect-time costs were similar across platforms, and connect-time pricing was consistently less than DialUnit pricing. The different here was that a complex search, executed in several files, tends to use more processing power than a simple search. Combine that with the ability to turn off the connect-time meter with the PAUSE command, and this is a search tailor-made for connect-time pricing.

Bottom line: Resource-intensive searches are cheaper on connect-time pricing, particularly if you use PAUSE or DialogWeb's FileHold to limit the connect-time charges while reading search results.

Search 3 (see Table 4) is a quick look-up search, which turned up no records. As any searcher knows, sometimes "0 hits" tells you exactly what you need to know. The cost was about a third higher under the connect-time pricing, probably because the search required minimal CPU processing.

Bottom line: With quick simple searches, you may not save anything with connect-time pricing.

Search 4 (see Table 5) is designed to see how much you're penalized for typos and to see the relative cost for a simple company look-up. The cost is less with connect-time pricing, although not significantly. Since format 9 in file 516 costs $3.89, most of the expense comes from output charges rather than either DialUnits or connect time. In this type of search, I would look at the relative cost of DialUnits vs. connect time. As noted in my rough comparison above, if the connect-time charge is $100/hour, then DialUnits usually run about $6 each. For file 516, though, the DialUnit cost is relatively high compared to connect time — $9.25/DialUnit and $100/hour connect time. As a result, a DU search will likely be more expensive than a connect-time search, particularly for straightforward searches.

Bottom line: For simple searches, compare DialUnit and connect-time costs. If a particular file weights one pricing plan higher than the other, switch accordingly.

In Search 5 (see Table 6), connect time wins hands down. Because of behind-the-scenes processing, using the Report/Titles command in any of the market research files is a very CPU-heavy search. (Side note to Dialog: Don't make it expensive to search for market research reports. Searchers know that they can go to other sources such as MarketResearch.com and search for free.) Anyway, if you use any market research files, definitely choose connect-time rather than DialUnit pricing.

Bottom line: Search market research files using connect-time pricing.

Search 6 (see Table 7) looks at the cost for using a couple of CPU-heavy commands — SORT and REPORT. As in search #4, most of the cost was in output; in this search, the Report output costs $9. If you eliminate the output costs from the price, you see that the search cost is significantly higher with DialUnit pricing (roughly $4) than with connect time (roughly $1.50).

Bottom line: Searches that require SORT and REPORT should be done using connect-time pricing.

Search 7 (see Table 8) also makes use of Dialog's power tools — LIMIT and Remove Duplicates. It also uses the PAUSE command which, as in Search #2, mitigates the connect-time charges. (Without the PAUSE command, the cost for DialogClassic with connect time would have been $5.99 — big difference!)

Bottom line: Power-tool searches should be done in connect time; remember to use the PAUSE command if you spend time reviewing output.

Search 8 (see Table 9) is of one of the highest-priced files, the Derwent World Patents Index file. Connect time is $291.50/hour; DialUnits cost $25.89 each. (Note to self: Make sure you don't BEGIN a search in 351 by mistake!) This search is expensive any way you run it, but a lot less expensive if you know what you're doing and can go in and out quickly. The difference in price between DialUnit and connect-time pricing is enormous; this is one of those times when you'd just be handing free money to Dialog if you run the search under the DialUnit plan. Note too that the penalty for searching DialogClassic with DialUnits is quite pronounced in this search — $53.78 on DialogClassic as opposed to $38.22 on DialogWeb. Ouch!

Bottom line: The more expensive the file, the better off you are going with connect-time pricing, particularly if you can search efficiently.

Search 9 (see Table 10) tests one of the lesser-known search functions of Dialog — the Target command. It's a kind of fuzzy-logic search that provides relevance-ranked output. [Type HELP TARGET to get more information on how this command works.] Note the enormous difference in cost between DialUnit and connect-time pricing. It doesn't take a rocket scientist to see that one should never use the Target command under DialUnit pricing. Note too the difference in connect-time pricing between DialogWeb and DialogClassic. Apparently the large price difference is a function of the automatic application of the FileHold command in DialogWeb.

Bottom line: For Target searches, always use connect-time pricing.

I developed Search 10 (see Table 11) during the early days of DialUnit pricing, when ADDing files to a search caused an automatic rounding up of DialUnits and, hence, a big cost increase. The ADD command still carries some DialUnit "overhead" that you don't have under connect-time pricing, hence, the significant cost savings by running this search under connect-time pricing rather than DialUnits.

Bottom line: If you think you might need to ADD files to a search, use connect-time pricing.

I'll admit that MAP is not a command I use every day. In fact, if I think I will need to use it, I'll call the Dialog Knowledge Center and ask them to walk me through the steps. Searches 11 and 12 (see Table 12) use some expensive files — CHEMNAME (file 301) is $275/hour or $31.25/DialUnit, CA SEARCH (file 399) is $180/hour or $12.55/DialUnit, and IMSWorld R&D Focus (file 445) is $150/hour or $15.25/DialUnit — along with the resource-intensive command MAP. These searches won't be cheap, regardless of the pricing plan. But, again, focused searches in which I knew what commands to type turned out noticeably cheaper under the connect-time plan. Search #11 shows relative costs that include no output costs, so demonstrate the "pure" processing costs. Search #12 includes output, but the cost difference is still notable.

Bottom line: With expensive files and resource-intensive searches, plan ahead and the search will be cheaper with connect-time pricing.

In Search 13 (see Table 13) I'm showing the results based on faith. When I ran the search in January 2002, file 414 was free of connect-time costs but incurred DialUnits of $1.25 each. The DialUnit versions of this search cost $0.62, $0.71, and $0.63 respectively. However, Dialog announced that, effective February 1, 2002, there would be no DialUnit charges for file 414, so the only cost — using either DialUnit or connect-time pricing — is the silly "telecommunications/Internet charge" of $13/hour or, in this case, 19¢ across the board.

Bottom line: Free files are uniformly not quite free.

Searches 14, 15, and 16 (see Table 14) test whether administrative commands that are supposed to be free of DialUnit charges are truly free. According to the DialUnit FAQ [http://products.dialog.com/products/dialog/price_list/pricing_faqs.html]:

The following administrative commands — those used for basic maintenance functions — do not generate DialUnits:


COST

DISPLAY SETS

EDIT Includes commands used within the online editor: CHANGE, COPY, DELETE, INSERT, LIST, MOVE, QUERY, QUIT, RENUM, and SAVE. Also includes other EDIT commands, such as EDIT EMAIL and EDIT ADDRESS.

HELP All HELP commands, including HELP RATES and HELP FORMATS

KEEP

LOGOFF Also includes the LOGOFF HOLD command.

RECALL All RECALL commands, such as RECALL ALERT, RECALL ADDRESS, RECALL.

RELEASE All RELEASE commands, such as releasing SearchSaves or Alerts.

SAVE All SAVE commands, such as SAVE ALERT, SAVE and SAVE TEMP.

SET All SET commands, such as SET ALIAS, SET NOTIFY, and SET SUBACCT.

SHOW All SHOW commands, such as SHOW FILES. All other commands generate DialUnits based on the amount of system resources used to execute the command.


My experience has shown that all of these commands still do incur DialUnits in DialogWeb, contrary to Dialog's statements that they don't. Brian Holland, Dialog's Senior Vice President, Strategic Pricing, has assured me that this is simply a programming error that will be corrected forthwith. I'm noting it here because this has been a known and uncorrected problem for years, so it's one correction I'll believe when I see it.

The examples given here show the discrepancy between DialogWeb DialUnit pricing, in which all commands, even those that are supposed to be free, incur DialUnits, and DialogClassic and DialogClassic Web DialUnit pricing, in which the administrative commands really are free.

Note that Dialog doesn't claim that the BEGIN command is free; in fact, it does incur a DialUnit charge of 0.065 DialUnits. That's not much in most files, but you would notice the "hit" if you accidentally Began an expensive file. For example, if I intended to type b 30 and instead typed b 130, it would cost me $1.75.



?b 130

14jan02 11:40:37 User000000 Session D8504.2

$0.23 0.064 DialUnits File1

$0.23 Estimated cost File1

$0.02 TELNET

$0.25 Estimated cost this search

$0.25 Estimated total session cost 0.272 DialUnits

File 130:PHIND(Daily & Current) 2002/Jan 14

(c) 2002 PJB Publications, Ltd.

Set Items Description

—- ——- —————-

?b 30

14jan02 11:40:39 User000000 Session D8504.3

$1.75 0.064 DialUnits File130

$1.75 Estimated cost File130

$1.75 Estimated cost this search

$2.00 Estimated total session cost 0.336 DialUnits

File 30:AsiaPacific 1985-2002/Jan 10

(c) 2002 Aristarchus Knowledge Indus.

Set Items Description

—- ——- —————-

?


For this reason, each of the sample searches has a small "overhead" DialUnit charge, since I had to issue the Begin command before doing anything else. However, there should never be more than the 0.065 DialUnit cost if the only commands you type are the administrative commands listed above.

Bottom line: You'll always incur some DialUnit cost when you move from file to file; I trust that by the time you read this, we won't be charged for other administrative commands while on DialogWeb.

Search 17 (see Table 15) is a plain old document delivery search — log on, find the article, print it, logoff. A document in format 9 in file 484 costs $3.20. Do the math and you'll notice that you pay quite a bit more for the delivery of the document with DialUnits than with connect-time pricing, depending on which platform you use. Your cheapest option is DialogWeb with connect-time pricing (overhead cost of only 51¢); your most expensive option is DialogClassic with DialUnit pricing (overhead cost of $1.79). This reflects a general pattern that DialogWeb is the least-expensive platform and DialogClassic the most expensive.

Bottom line: When you do a straight document delivery search, use DialogWeb and connect-time pricing.

Search 18 (see Table 16) attempts to replicate an info pro doing a relatively simple search for a client, providing a list of titles to the client, and then logging back on to retrieve the full records of the documents selected by the client. In the early days of DialUnits, when DUs were rounded up to the next full unit, this was absurdly expensive. Now that we have fractional DUs, the cost is just about the same under DialUnit and connect-time pricing.

Bottom line: For intermediary searches in which the searcher downloads titles and then logs back on to retrieve individual records, there's no real difference in cost.

I added these last two searches (see Table 17) to test the difference in cost between DialUnit and connect-time pricing in an output-heavy search. For some searchers, this is what most of our research looks like — you know what you're looking for, you go in and find it, you print out a bunch of records, you're done.

Search 19 is executed in the OneSearch collection of newspapers and is pretty straightforward. Search 20 was written to incur some CPU processing time with those LIMITs. Note that I used the PAUSE command in the DialogClassic connect-time searches to cut down on the total amount of connect-time charges. Since format 9 records in file 88 cost $3.25 each (or a total of $81.25 in this search), if you just look at the search charge, the difference in cost is significant. Again, the most expensive search is DialogClassic on DialUnits ($12.76 in "overhead"), the cheapest is DialogWeb on connect-time ($2.08 in "overhead").

Bottom line: With a straightforward search when you expect to print a large number of records, the cost will be pretty similar whether you use DialUnits or connect time, but you may be penalized for using DialUnits on DialogClassic.
 

Which Is Better?
Let me talk about taxicabs for a moment. Here in Washington, D.C., cabs don't have meters. Instead, they use a geographic zone system to determine the fare — if you catch a cab in one zone and your destination lies within that zone, you pay $5; ride from one zone to an adjoining zone and it's $6.90; move through three zones and it's $8.60, and so on. One of the arguments for this system is that it prevents unscrupulous cab drivers from taking unsuspecting tourists on the "scenic" route to their destination in order to increase the metered fare. (Just ask me about my $15 cab ride from the Philadelphia train station to a hotel that turned out to be only eight blocks away....) It also, conveniently, ensures that congressional staff can catch a cab from Capitol Hill to downtown for $5. On the other hand, if you happen to catch a cab at the edge of one zone, you'll pay $1.90 more than if you'd caught the cab a block away. DC natives like the zone system because they know how it works and know how much a trip will cost, but they also know that short-haul trips cost more than they would under a meter system. The bottom line, of course, is that there probably wouldn't be any net change in the total cost of taking cabs if we switched from zones to meters.

In some perverse way, Dialog pricing reminds me of DC cabs (although at least Dialog doesn't have bone-jarring shock absorbers and we get to choose whether we want zones or meters). For Dialog natives, connect-time pricing amounts to a price decrease, and a fairly significant one depending on your search habits. In my tests, I found no example of when a search would have been significantly less expensive under the DialUnit pricing, whereas a number of searches were dramatically cheaper with connect-time pricing. (See, for example, searches 2, 5, 7, 8, 9 and 10.)

If Dialog isn't a system you use much, and you are often stumped when doing a search — stopping to dig up a Bluesheet, pondering why you just got an error message, or mulling over the idea of ADDing a file to your search — then DialUnits might save you a little money. However, I still think that judicious use of the PAUSE command (or searching on DialogWeb, which initiates a FileHold in the background between each command) would keep connect-time costs reasonable.

The other important factor for determining whether you should switch to connect time is how you relate to the ticking clock. Those of us who cut our teeth on Dialog searches back when most of the cost of a search came from the connect-time charge learned to plan our searches ahead of time, look up tricky commands, figure out what prefixes to use in advance, and so on. Then we logged on, ran our search, and logged off quickly. Some folks, though, never get over worrying about how long they're on and that every minute is COSTING THEM MONEY, ACK! If you're one of those people, DialUnits are for you. You'll pay a bit more, but your searches will probably be better because you're not spending the entire time stressed out about the meter running. However, you're still much better off switching to connect-time pricing for those searches that are dramatically less expensive under the connect-time option, such as market research file searches and searches that you know ahead of time will require a lot of CPU processing. In those situations, plan ahead, unplug your phone, take a few deep breaths, and then just do the search using connect-time pricing.

Likewise, if you're the kind of person who can get distracted easily — or you're in an environment with a lot of unavoidable distractions — you'll have to either remember to PAUSE each time you're interrupted or switch to DialUnit pricing, which doesn't penalize you for sitting online doing nothing.

I was somewhat intrigued by my own reaction to the difference in "feel" of a DialUnit search vs. a connect-time search. Having just completed the test searches for this article and concluding that most experienced searchers should switch to connect-time pricing, I settled down to do some real-live research for a client. It was a project that had five or six different concepts that would require some mix-and-matching. I also wanted to try limiting each concept to just the title and descriptor fields, and so on. I decided to try the searches out in an ONTAP version of one of the files I planned to search; this would not only let me see the relative number of hits with various strategies, but also let me type in my search strategy for free, since I was using connect-time pricing. Once I moved the search over to the regular files, I did have that momentary stomach-twinge... . "Quick, think fast, don't just sit there, do something!" I got over it in a minute, once I realized that my entire search wouldn't take more than 10 or 15 minutes, and so the total connect-time charges wouldn't amount to more than $10 or $15 — a small fraction of the total cost, once I included the cost for all the downloaded articles.

I'm glad I switched to connect-time pricing as my default. I no longer hesitate before using power search tools such as Remove Duplicates, SORT, and REPORT because I know that these tools will only cost me pennies rather than dollars. And I like knowing that I can calculate the total cost by just watching the clock rather than trying to read the tea leaves or chicken entrails or whatever it is that happens behind the scenes in calculating DialUnits. I suspect that most Searcher readers will conclude the same.



Improving Your Profile
Every time you log onto Dialog, it checks to see if you have a Profile set up; this saved file lets you set a number of options for your search. It's a nifty tool for personalizing your Dialog experience — you can create customized output formats, change the way search terms are highlighted in your output, require that users provide subaccount information before searching, and so on. For loads of information on what you can do with Profile, download the 80-page misnamed Pocket Guide from http://library.dialog.com/pocketguide (misnamed because there's no way it'll fit in anyone's pocket) and go to the section on SET commands, which explains all the commands available for your Profile.

If you just want to change your pricing from DialUnits to connect time, though, here's the abbreviated version.

Start in one of the free files (any ONTAP file, or file 410, 413, 414, 415 or 416).

First, check to see if you already have a Profile:

?recall profile

If Dialog responds: >>>No PROFILE exists, then do the following steps:

?edit

Editor entered

Name: *NEW*

Total lines: 0

Line increment: 10

Last line: 0

INPUT: 10

?set connect on

INPUT: 20

? [hit the enter key]

Returning to EDIT mode

EDIT:

?save profile

PROFILE stored

Exit from editor

?

If, when you type recall profile, Dialog responds with a numbered list of commands such as:

Line Command

—— ———-

1. SET NOTICE $1000

2. SET HILIGHT ON

?

then perform the following steps to add a line to set your preference from DialUnits (the default) to connect-time pricing:

?edit profile

Editor entered

Name: PROFILE

Total lines: 2

Line increment: 10

Last line: 20

EDIT:

?insert

EDIT:

INPUT: 30

?set connect on[if you want to have the default to DialUnits, you'd type SET CONNECT OFF instead]

INPUT: 40

? [hit the enter key]

Returning to EDIT mode

EDIT:

?save

—>Replace "PROFILE"? ( Y = Yes N = No )

?y

PROFILE stored

Exit from editor

?

As you can probably guess, SET CONNECT ON changes your pricing to connect-time and SET CONNECT OFF changes your pricing plan to DialUnits. Remember that you have to log off after editing the Profile and log back on for the change to take effect.

Pricing Ch-Ch-Changes
Dialog sent out a letter to all subscribers in January, alerting them to changes in its fees for Dialog, DataStar, and Profound. Here's the good news and the bad news:

Good news:

  • Elimination of the $75/month minimum fee for Dialog
  • Elimination of DialUnit charges for most "Finder" files on Dialog
  • New upgrade for DialogLink software some time in 2002
  • New training manuals some time in 2002
Bad news:
  • Increase in Dialog and DataStar annual services fee (doubling in the case of DataStar)
  • Increase in DialUnit cost for DialIndex
  • Increase of telecommunications charge from $12 to $13/hour on Dialog
  • Across the board 6.5 percent increase in output charges (for Dialog, DataStar, and Profound)
  • Increase in most connect-time charges on DataStar
  • Ten percent surcharge for DialogClassic DialUnit searches
The last of these items gets me steamed; why is DialogClassic — the telnet version that we old codgers know and love — targeted for a 10 percent surcharge? Dialog claims that it's because DialogClassic costs significantly more to maintain, so this is a way to allocate those costs directly to users and that the price differential will increase over time. My response is that there are lots of variables in Dialog costs, and you deal with them by factoring them all into your pricing rather than imposing surcharges on one pricing plan on one platform. I mean really — if I follow this logic, users like me who never call the Knowledge Center should get a discount because we don't require much telephone help, right?

Supersearcher Tips for the Brave New World
  • Remember that you have to log off and log back on after changing your profile between connect-time and DialUnit pricing. You can't switch between the two pricing plans during a single session.

  •  
  • In general, DialogWeb is the least-expensive platform and DialogClassic the most expensive. This pricing disparity is bound to increase, so get used to searching on DialogWeb.

  •  
  • Since pricing in OneSearches is allocated across files (in both DialUnit and connect-time pricing), open the least expensive file first. This doesn't make an enormous difference, but it can cut the cost by a bit.

  •  
  • Think before you search. If you're going to do a simple search with a patron sitting at your side reading everything, you may want to switch to DialUnits. Conversely, if you are going to do a complex search, making use of Dialog's power tools, set your pricing to connect time.

  •  
  • PAUSE is your friend! If you're on connect-time pricing, remember to use the PAUSE command whenever you review output or stop to think. Note, though, that PAUSE only works for 10 minutes. After that, Dialog assumes that you've abandoned the search and will disconnect you.

  •  
  • The more expensive the file, the more likely that you'll save money with connect-time pricing.

  •  
  • Make a list of the files you search regularly and compare DialUnit and connect-time prices to make sure the prices are comparable. If it turns out that your "regular suspects" have anomalous pricing, go with whatever is cheaper. A very rough rule of thumb is if DUs for a particular file cost about $6, then connect time should be around $80; files that charge $4 per DU usually cost $40 for connect time.

  •  
  • Get in the habit of noticing what pricing plan you're on each time you log on. DialogWeb doesn't tell you, so click the [Cost] icon to see whether you're being charged DialUnits or connect time. On DialogClassic and DialogClassic Web, you'll see the notice, "Cost is in DialUnits" or, "Cost is in Connect Hours" when you first log on.

  •  
  • Know your habits. If the phone rings and you're likely to forget that you're logged on, stick to DialUnit pricing.

  •  
  • If you're going to build a complex search with a lot of search terms, set your account to connect-time pricing, go into a free file, type in your search strategy, SAVETEMP it, then execute it in the expensive files.

  •  
  • Don't forget to logoff when you're finished your search, particularly if you use connect-time pricing. You'll incur 10 minutes of connect-time before Dialog finally kicks you off, and that can add up if you wander off in the middle of an expensive file. 

 
Table 1. DialUnit vs. Connect-Time Pricing • [View tables in PDF format]
DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic Web(4) DialogWeb DialogClassic (telnet) DialogClassic Web
1 s marathon? ? and galloway; t 1/7/1-6
[read them for 2 minutes]
48 $12.29 $12.48 $12.49 $12.10 $14.90 $13.51
2 s cheney(15n)oil; 
limit s1/1997:2001; 
s (dick or vice()president)(3n)cheney;
limit s2/tx; c 3 and 4; rd; 
t 6/3/1-7 from each;
PAUSE [read results for 4 minutes]; k 6/1-4 from each
148, 16, 15, 9 $45.88 $49.65 $47.59 $36.69 $39.40 $38.05
3 e na=molle, j 234 $0.90 $0.99 $0.92 $1.17 $1.31 $1.31
4 s waremart? and cy-eureka and st=ca;
s waremart? and cy=eureka and st=ca; 
t 2/9/1
516 $6.53 $7.48 $6.54 $4.98 $5.68 $5.68
5 s nutrac??tical? ?; 
report s1/titles; 
[display one complete table of contents]
770 $7.52 $8.91 $7.78 $1.39 $1.79 $1.79
6 s cx=humboldt and sc=5411; sort
s1/all/sa,d; report s2/co,cy,sa/1-10
516 $12.83 $13.35 $12.86 $10.19 $10.79 $10.79
7 s wenckebach and (pacemaker? ? or
pace()maker? ?); 
limit s1/human; rd ; 
t 3/8/1-20 from each; 
PAUSE [review for 4
minutes]
155, 72, 55 $4.29 $4.93 $4.71 $2.10 $2.41 $2.70
8 s (photo()voltaic? or photovoltaic?) and
(vehicle? or camper? or car or cars); 
rank s1/all/ic; s ic=h01l?; 
c 1 and 2; t 3/8,ti/1-5
351 $38.22 $53.78 $40.95 $15.11 $18.03 $18.03
9 target ‘genetic engineering’ *strawberr? 47, 484 $0.91 $0.99 $0.91 $0.05 $0.20 $0.28
10 s raychem/ti; s fib??optic? or fib??()optic? ?;
c 1 and 2; add 636 696 148; repeat; 
t 3/8,k/1-3 from each
papersca,
636, 696
$1.91 $2.42 $2.19 $1.24 $1.56 $1.57
11 s sy=aluminian acmite; map rn; b 399; exs 301, 399 $8.33 $9.21 $8.32 $6.11 $7.66 $7.66
12 e na=lisinopril; s e3; 
map rn temp; b 445; exs; 
s s1/profile; t 2/9/1
301, 445 $52.82 $55.78 $52.15 $44.71 $46.68 $46.68
13 s jn=searcher; report s1/journal 414 $0.19 $0.19 $0.19 $0.19 $0.19 $0.19
14 ?rates 629 629 $0.54 $0.32 $0.28 $0.01 $0.05 $0.05
15 b 522 ; cost 522 $1.17 $0.68 $0.61 $0.01 $0.00 $0.00
16 b 522 ; b 122 522, 122 $1.08 $1.01 $0.95 $0.01 $0.22 $0.22
17 s jn=(nation or the nation); s s1 and pd=931025 and tossing/ti; 
t 2/9/1
484 $4.68 $4.99 $4.69 $3.71 $4.67 $4.67
18 s lucent()technologies/ti; 
s s1 and definity/tx; 
t 2/8/1-6; logoff; 
[log back on]; b 148; 
t 09095023/3,tx; 
b 16; t 06470226/3,tx
148, 16
current2
$10.67 $10.91 $10.30 $9.95 $10.91 $10.91
19 s bruce()hornsby and grateful()dead ; limit
s1/1998 ; t 2/9/1-20
papersca $52.62 $53.00 $52.65 $52.35 $52.63 $52.64
20 s road()rage or (aggressive or stress? or anger or angry or violen? or
danger?)(2n)(drive? ? or driving) ; 
limit s1/1999 ; 
limit s2/fulltext ; t 3/8/all ;
PAUSE; read for 5 minutes ; t 3/9/1-25
88 $86.37 $94.01 $87.44 $83.33 $86.24 $85.81


[View tables in PDF format]
Search #1 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic Web(4) DialogWeb DialogClassic (telnet) DialogClassic Web
1 s marathon? ? and galloway; t 1/7/1-6
[read them for 2 minutes]
48 $12.29 $12.48 $12.49 $12.10 $14.90 $13.51


[View tables in PDF format]
Search #2 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic Web(4) DialogWeb DialogClassic (telnet) DialogClassic Web
2 s cheney(15n)oil; 
limit s1/1997:2001; 
s (dick or vice()president)(3n)cheney;
limit s2/tx; c 3 and 4; rd; 
t 6/3/1-7 from each;
PAUSE [read results for 4 minutes]; k 6/1-4 from each
148, 16, 15, 9 $45.88 $49.65 $47.59 $36.69 $39.40 $38.05


[View tables in PDF format]
Search #3 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic Web(4) DialogWeb DialogClassic (telnet) DialogClassic Web
3 e na=molle, j 234 $0.90 $0.99 $0.92 $1.17 $1.31 $1.31


[View tables in PDF format]
Search #4 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic Web(4) DialogWeb DialogClassic (telnet) DialogClassic Web
4 s waremart? and cy-eureka and st=ca; 
s waremart? and cy=eureka and st=ca; 
t 2/9/1
516 $6.53 $7.48 $6.54 $4.98 $5.68 $5.68


[View tables in PDF format]
Search #5 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
5 s nutrac??tical? ?; report s1/titles; [display one complete table of contents] 770 $7.52 $8.91 $7.78 $1.39 $1.79 $1.79


[View tables in PDF format]
Search #6 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
6 s cx=humboldt and sc=5411; sort
s1/all/sa,d; report s2/co,cy,sa/1-10
516 $12.83 $13.35 $12.86 $10.19 $10.79 $10.79


[View tables in PDF format]
Search #7 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
7 s wenckebach and (pacemaker? ? or pace()maker? ?); limit s1/human; rd ; 
t 3/8/1-20 from each; 
PAUSE [review for 4 minutes]
155, 72, 55 $4.29 $4.93 $4.71 $2.10 $2.41 $2.70


[View tables in PDF format]
Search #8 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
8 s (photo()voltaic? or photovoltaic?) and
(vehicle? or camper? or car or cars); rank
s1/all/ic; s ic=h01l?; c 1 and 2; t 3/8,ti/1-5
351 $38.22 $53.78 $40.95 $15.11 $18.03 $18.03


[View tables in PDF format]
Search #9 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
9 target ‘genetic engineering’ *strawberr? 47, 484 $0.91 $0.99 $0.91 $0.05 $0.20 $0.28


[View tables in PDF format]
Search #10 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
10 s raychem/ti; s fib??optic? or fib??()optic? ?;
c 1 and 2; add 636 696 148; repeat; t
3/8,k/1-3 from each
papersca,
636, 696
$1.91 $2.42 $2.19 $1.24 $1.56 $1.57


[View tables in PDF format]
Search #11 & 12 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
11 s sy=aluminian acmite; map rn; b 399; exs 301, 399 $8.33 $9.21 $8.32 $6.11 $7.66 $7.66
12 e na=lisinopril; s e3; map rn temp; 
b 445; exs; s s1/profile; t 2/9/1
301, 445 $52.82 $55.78 $52.15 $44.71 $46.68 $46.68


[View tables in PDF format]
Search #13 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
13 s jn=searcher; report s1/journal 414 $0.19 $0.19 $0.19 $0.19 $0.19 $0.19


[View tables in PDF format]
Search #14, 15 & 16 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
14 ?rates 629 629 $0.54 $0.32 $0.28 $0.01 $0.05 $0.05
15 b 522 ; cost 522 $1.17 $0.68 $0.61 $0.01 $0.00 $0.00
16 b 522 ; b 122 522, 122 $1.08 $1.01 $0.95 $0.01 $0.22 $0.22


[View tables in PDF format]
Search #17 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
17 s jn=(nation or the nation); s s1 and
pd=931025 and tossing/ti; t 2/9/1
484 $4.68 $4.99 $4.69 $3.71 $4.67 $4.67


[View tables in PDF format]
Search #18 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
18 s lucent()technologies/ti; 
s s1 and definity/tx; 
t 2/8/1-6; logoff; [log back on]; 
b 148; t 09095023/3,tx; 
b 16; t 06470226/3,tx
148, 16
current2
$10.67 $10.91 $10.30 $9.95 $10.91 $10.91


[View tables in PDF format]
Search #19 & 20 DialUnit pricing Connect- time pricing
Commands File(s) DialogWeb(1) DialogClassic(2)
(telnet)(3)
DialogClassic (Web)(4) DialogWeb DialogClassic (telnet) DialogClassic Web
19 s bruce()hornsby and grateful()dead ; limit s1/1998 ; t 2/9/1-20 papersca $52.62 $53.00 $52.65 $52.35 $52.63 $52.64
20 s road()rage or (aggressive or stress? 
or anger or angry or violen? or
danger?)(2n)(drive? ? or driving) ; limit
s1/1999 ; limit s2/fulltext ; t 3/8/all ;
PAUSE; read for 5 minutes ; t 3/9/1-25
88 $86.37 $94.01 $87.44 $83.33 $86.24 $85.81

Footnotes
(1) http://www.DialogWeb.com
(2) telnet://dialog.com
(3) Price includes estimate of cost as of February 1, 2002, when DialUnits are surcharged 10% on DialogClassic.
(4) http://www.dialogclassic.com
 

Mary Ellen Bates can be contacted at mbates@BatesInfo.com or http://www.BatesInfo.com.
Table of Contents Previous Issues Subscribe Now! ITI Home
© 2002