WEEKLY WHINE

Focal Plane: Empty searches

This is the second instalment of the GoobNet Focal Plane, an occasional series wherein we highlight an unimportant social problem, trying to make you care about it. Previous edition: Coffee Immunity.

David Broznak is reviewing an event log from THU 01 NOV 2001, created by the main web server at Internet search engine LookSilly.com. To the untrained eye it looks like a long list of IP addresses and URL requests. But Broznak's eye is far from untrained.

"Look at this right here," he says. We do. "That's an empty request. Here's another one about fifteen minutes later. The next one was only three minutes after that. There are far too many."

All of the things he's pointing to have an URL that includes &str=&. "str is the string for which the user is searching. The ampersands separate each variable. Take the URL http://www.looksilly.com/look.pl?sec=all&type=2&str=&trans=n&x=38&y=12. It tells look.pl to look in all sections and do a search of type 2 for the string '', without translating. x and y are not used."

So what happens in this case? "The server's response is 'Sorry, you can't search for nothing.'" He demonstrates, and this is exactly what the server responds. Of the 30,000 requests that LookSilly.com receives every day, fully .5% of them - 150 - are empty requests.

Drain on resources

This shouldn't be a problem, right? The server can just respond, "Ha ha, you're dumb. Type something next time," or something of the sort.

Not so, says Alana Rayder of Moore-Evans Consulting in San Diego, CA. "Most search engine scripts start with a test of whether the search string is empty. That may be a relatively quick test, but multiply it by the number of searches and the cumulative effect is astounding."

The Search Engine Research Department at Moore-Evans, which Rayder chairs, estimates that the processing time of a request scales as the square root of the fraction of empty requests that the server receives. Explains Rayder, "Say we've got a server that gets 1% empties. If they can cut that to .25%, that lets them process each request twice as fast. That's an enormous boost to productivity, and more importantly, it improves customer satisfaction. Nobody wants a search engine that they have to wait on."

We took to the streets to find out whether this scaling relation really holds. LookSilly.com, where the empty request fraction is .5%, has an average response time of 2.5 seconds. Most of the 18 sites that we surveyed did in fact fall near a square root curve; for example, portHOLE[tm] by GoobNet's average response is 15 seconds, and 3% of all responses are empty. [GoobNet Enterprises, Inc is the parent company of portHOLE[tm] by GoobNet.]

Cutting back on empties

Is it really practical to institute empty search reduction programmes? The lowest empty request fraction we found was at AnalogueFox.Net, which receives empties only .08% of the time. Its response time averages .92 seconds.

Doris McKenna is the lead developer for AnalogueFox.Net's search engine. "We were really taking a performance hit from empties for our first few years," she recounts. "Our empty rate used to be several percent. That wasn't going to do at all. On the development team, we tried to optimise the code so that it would process empties faster, but there's only so much you can do on the code level.

"So we started to think about how we can prevent users from typing nothing to begin with. That's the first place to go after the problem, clearly. We came to the conclusion that we needed to educate users about the importance of submitting a nonempty search request."

They started by adding a message beneath the search textbox on their site: "Please do not click 'Go Find Out' until you have typed something in." Later, when their ad campaign kicked into high gear, they made sure to promote the presence of text. Their slogan, "Finds Whatever You Type", was particularly suited for this issue. One print advert was headlined "Type Nothing. Get Nothing" and explained that submitting an empty request is guaranteed not to find what you're looking for. Many users also reported the helpful presence of a large billboard near Chicago, IL that read "Think before you click. Use the Search For field."

Widespread application

Is AnalogueFox.Net's success story going to spur a trend in the search engine industry? Rayder hopes so. "We're currently working with the AnalogueFox team to develop an empty reduction programme that any number of search engines can use," she says. "We figured we'd have trouble trying to convince them to share some of their trade secrets, but they were quick to realise that a widespread empty reduction programme benefits the entire search engine sector."

Many search engines have already taken the first step by offering explanations of why there are no results when an empty search is conducted. Such sites include:

• BBC News
• Yahoo
• Search.com
• Lycos Philippines

Broznak says that he has been trying to alert LookSilly.com management to the importance of the empty request problem for some time but only began to do something about it at the request of Moore-Evans, which was originally hired for other consulting purposes. "It seems as though they're not listening to their own people any more. They give me lip service for so long, and suddenly when these outsiders show up with their ideas, someone's listening."

Whether from within or without, nearly every search engine is being pressured to come up with a resolution to the empty request problem. For users, that leaves one certainty: The responsibility is as much theirs as that of the engines themselves. Prepare yourself to be pressured not to submit until you've typed something.