Searching the web is often just wishful thinking. I truly question the sense in sending a single word to a large search engine, then looking at the top 20 webpages from a list of 50,000. Just how silly to then assume you have found what the Internet has to offer?

This is computing at its most optimistic!

And on this point, may I also express my amusement when a friend exclaims, "I searched the Internet and didn't find a thing." Huh? You searched the Internet? Please! Even searching four of the top search engines scarcely qualifies as a search of the web. And there is so much more to the Internet than the web. Besides, you probably just typed in the wrong words.

To do a proper search, you need to understand a little bit about how search engines work. Search engines reach into a database filled with the text of a great many webpages. Search engines like Altavista (altavista.com) and All-the-Web (alltheweb.com) have databases of over 350 million webpages.

During the building of these databases, certain parts of each webpage are recorded separately, like the title and web address. The technical term for this is a 'field'. It works much the same as books in the library are indexed by title, author and subject. The US Library of Congress catalogue (catalog.loc.gov) has more than forty fields but search engines tend to have just a few: web address, title, maybe language and link.

Which leads me to ask, when was the last time you searched for a webpage by title?

Search engines also record the order words appear in each webpage, so we can craft a search request for words appearing next to each other on the webpage. The term for this is 'proximity' and it allows us to search for a phrase like "patent research". Commercial databases, like the ones on CD-ROM in your local library, let you dictate precise distances between desired words. On the web, we usually just have quotes "" to keep words together.

Field searching and proximity are the keys to a more valuable search. Search engines have made great strides in using fields, proximity, +/- and something called link analysis to automatically rank the more useful sites in the first twenty they show you. Don't trust this ranking system. You can do so much better simply crafting a more precise search request.

Let's now look at the use of the plus + symbol. If you search for patent research, you are asking for all the webpages with the words patent or research. This is why the number of matches are so very large. If you want just the webpages with both words, you need to type: +patent +research.