toto article listarticle list
Research Commentary on the Spire Project

Start with more than a search engine.
Approach the Internet with a fist full of search tools.
By David Novak

Altavista, a top global search engine, spawned a second search engine called Raging ( The website describes itself as, "the most advanced technology ... only the most relevant Web pages for your query ... the biggest and freshest index of Web pages ... the only search site you will ever need."

What to make of this?

Raging is an alternative interface to the same database of webpages used by Altavista. It is neither new nor brilliant. It is just marketing. It does, however, highlight the difficulty in deciding what we need. Where do we go to search the Internet?

We are of course asking the wrong question. My state library allows me to search for books by title, author and subject. Their books are also shelved by Dewey decimal number.

How do I find a book? I usually invoke several techniques. I may search by title, then expand to see all the books in that subject category... then go browse the shelves. True, not the most efficient system in the world but I get really good results.

The Internet is a vast library of information. We are enticed by the freedom of instant free information to think of it as a radical change to the world of research. It is not. The rules are the same, just blown large to accommodate the large quantity of information.

Yes, we must still use a combination of search tools to find information. We need to recognize and select each search tool and technique for what it does best and what is does not.

Back to our search engine Raging. It is a good search tool with a large database of 350 million webpages, and allowance for the complex search terms and fields that technically proficient searchers like me prefer. A good similar tool, with little overlap, would be All-the-Web, indexing 350 million webpages again with complex search terms and fields.

For those less proficient, or for searches where you already know the name you are looking for, I'd select Google or a meta-search engine like Debriefing. They have a broad reach, are quick, and list your sites at the top. No, these engines falter rapidly in the face of more difficult search needs.

Of course, I'd also consider the directories, seeking topic specific guides like those indexed by Argus and AlphaSearch, or searching sites with numerous site reviews, like the Scout Report and BUBL.

And since we mentioned Dewey earlier, both BUBL and the Dublin Core categorise websites by Dewey number.

Improvements from here require we either use search techniques like Boolean and field searching, or focus in on one of the many formats the information comes in: books, discussion, zines, databases, news: each of which have a collection of fine search tools.

For books, there are free books online, three free government publication databases, bookstore catalogues, library catalogues and several authoritative commercial databases.

A search engine is only one of many search tools - usually the wrong one. Oh, sure we will get some kind of answer but more often than not, searching for the top 20 sites which contain the words 'patent and/or database' is not the most brilliant way to trawl the net.

For far too many of us, searching the net means Yahoo and a search engine. We need more flexibility. We need to think more about what tools and techniques are good for answering which questions.

There are so many tools and techniques to help you find information on the Internet. Should you find searching the Internet unrewarding, it is more likely a result of your confusion, not the net being disorganised.
* * *
David Novak manages The Spire Project, an Internet research resource and thinktank.

to article listThe Spire Project - better ways to find information.
Like this? You should attend our public seminar and receive our bi-monthly update notice.
 | | | Project Background | Feedback. Copyright©David Novak 2002.