toto article listarticle list
Research Commentary on the Spire Project

This White Paper was an internal document describing experience with promotion as of December 2003. Updated material on this topic appears in David's book, Internet Informed.

White Paper on Internet Search Engine Optimization.

By David Novak

Once upon a time, in the early Internet, promotion could be overlooked. Quality information would attract its own attention; build its own fame. Publish and the audience found you.

The Internet has changed. Actually it has changed several times, each new mutation gradually replacing the last. It changes still; the next change already upon us.

Thus, promotion on the Internet is a mixed and moving target. Not only does the rather complex technology hinder our way but achieving the objective of the moment may not help us in the near future once the goalposts move. At first glance, Internet promotion appears entirely about achieving the highest ranked position on the search engines. This is short-sighted. Under most circumstances, promotion on the Internet should unfold over time in line with a strategic vision of your potential destinations.

My name is David Novak. I am Australia's leading expert on Internet research. I personally discovered and adapted much of the field and have taught this guidance to many of Australia's librarians through my seminars and the Spire Project ( The Internet environment is home to me. My perspective, however, differs at times from others who will provide you guidance on Internet promotion. I have a longer view, and a view of one who lives on the research side of the transaction.

This white paper explains a slightly simplified view of Internet promotion general enough to cover most circumstances. If at any time you feel you need further explanation, please contact me to arrange for a discussion.

David Novak
The Spire Project - better ways to find information

Prominence and Relevance. There are other ways to use the Internet in promotion but none currently rival the importance of working with the search engines. Search engines - particularly the current favourite: Google - use their own definitions of these two concepts to decide where information should be placed on the results page. To harness the search engines to our promotional aims, we focus on these two concepts.

A webpage with excellent information, many links and a central position in the many roads that weave through the Internet is a prominent webpage. Such pages are famous; the most recognized sites among their peer websites. Such fame is rewarded with placement towards the top of search engine results pages - with all the increased attention and traffic this bestows. Prominence is good.

The early search engines did not rank sites based on prominence - one of the reasons for their demise. Today's search engines do. This change, led by Google some three years back, resurrected the failing search engines at a time when an important Yahoo directory listing was the key to Internet promotion. Today, complex computer algorithms analyze link frequency and the prominence of the linking pages to calculate the prominence of a given webpage. This judgement of prominence then becomes an important factor in how information is ranked. In this way, prominent webpages are judged as prominent, displayed as prominent, and a lovely reinforcing circle is formed.

If we make a webpage more prominent, we improve its position on the search engine results pages.

The exact computer algorithms are not public knowledge. Besides changing with time, these algorithms also differ between search engines. Consequently, we are a little in the dark when trying to improve webpage prominence. We know the original Google algorithm since it was published as part of a university thesis. We can guess much more just in the way ranking behaves. Absolute clarity is lacking.

Google makes a measure of prominence available through a tool on their toolbar. Install the Google toolbar on your Explorer web browser then visit the site you're interested in. Its a simple number from zero to nine that tells you how prominent Google considers this page.

Prominence, as measured by the search engines is primarily about the number and quality of links. We can achieve prominence in four ways:

1_ We can ask for additional links, particularly from other prominent sites,
2_ We can buy added prominence, paying for links from very prominent sites,
3_ We can design in ways that do not waste prominence, and
4_ We can assist visitors to link to us.
Let us look at each in turn.

Links are the key to building prominence. Seek out and request links and we create a more prominent webpage. Thanks to the link: field search we can readily ask search engines to share with us the places that already link to a given site. Similarly, with a small list of direct competitors or comparable sites, we can see which sites link to competing or comparable sites. This is our first list of webpages to ask for links. Sharpen a pencil, write a persuasive reason for linking, then ask politely.

Many sites will ignore our request. Many will turn us down. Again, using link companions we can always find more websites to ask. Link companions are simply sites listed beside your own on Internet directories.

Our focus is simply links and the steps work just as easily in reverse. I am asking one of the editors of the Open Directory Project to drop a link to a competitive site on the grounds of plagiarism. The request simple needs to be made. I am not certain how long they will take before they drop the link or how many times I will need to ask. They may never drop the link. But if they do, the plagiarized site will lose it main claim to prominence and it will sink back into relative obscurity. Problem solved. The request is simply worth my time.

[follow-up: request worked - see art23.htm for more.]

Links have value. Ask, convince, sell, perhaps even petition for a link and we build the prominence of your website.

The task of requesting links is very important. Do not overlook this step. Many links are available just for the asking while other links may have broken and can easily be fixed. There are limits to its value, however. If we are not careful, we will ask links from unlikely sites, we will ask links from webpages with little prominence and we will find ourselves chasing links for hours with little to show for it. Link chasing is neither glamorous nor rewarding. Taken too far, it simply wastes time. Request links from the sites most likely to provide them, then wait and repeat if helpful.

In July 2002, a court case in the U.S. pushed search engines out of the habit of selling improvements in search engine ranking. Before July 2002 it was routine on many search engines. Since then, you can not pay for a higher ranked position. Pay for advertising on a search engine; you buy advertising. Your website does not rank higher as a result.

That said, we can still buy prominence. Just not from the search engines. Not normally. What we can do is purchase a listing on a prominent website, wait for the search engines to notice our newly purchased prominent link, then see them judge our page as more prominent because of the link.

Yes, we can buy a link not for the traffic it will lead our way but for the prominence bestowed on listed pages.

The Yahoo directory is prominent. The Online Directory Project (ODP or Dmoz) is prominent. Looksmart is a little prominent. We can purchase a listing on the Yahoo and Looksmart directories.

A note of potential confusion here: a simple search of Yahoo is actually a search of Google as of December 2003. Currently, while Yahoo assembles its recent purchase of the Inktomi search engine database, Yahoo is simply repackaging the Google database, adding their own advertising. The Yahoo directory is something else - something you must request specifically from the Yahoo website.

The Online Directory Project (ODP) also appears as the Google directory. The ODP is actually found on about a dozen prominent sites around the world. Thus a link on the ODP slowly becomes about a dozen links.

A second note of confusion: Google finds information quite effectively and has probably already indexed your website. Indexing a webpage is not the same as ranking a webpage prominently. In the early days, part of promotion was requesting search engines to index webpages. This is usually no longer required. The search engines find new pages without help. What is required is help being judged as an important, prominent webpage.

Back to buying prominence. We can't buy prominence from Google, Altavista or AlltheWeb (Lycos/Fast). These global search engines are actually legally not permitted to sell a higher ranked position. To do so would make them a paid inclusion search engine - there are some but they are not popular.

[Note: oooh, this may have changed. Look into this further since both Yahoo and MSsearch may have slipped into becoming paid inclusion search engines...]

And this is why we buy placement in the Yahoo directory.

Realistically, the Yahoo directory is a passe tool. It was once synonymous with searching the Internet - about three or four years ago. Today it is largely overlooked. Even a search of Yahoo defaults to a search of a search engine not the directory.

Ah, but the global search engines do recognize a Yahoo listing as a statement of prominence. And now you may detect the back door by which a search engine can ask you for money.

There is a great deal of movement in how search engines rank and obliquely ask for money. Yahoo, Google and Microsoft are likely to be the big players at the end of 2004. Each is working hard to uncover that magic mix of a way to ask for money but not.

Until such purchasing becomes clear, we can but purchase an initial boost by buying placement on the Yahoo directory. This is expensive by Internet standards at US$299 per site as an annual fee. Consider this almost as an advertisement in the Yellow pages except that we are most unlikely to repeat this purchase. I simply doubt the directory will be as important next year. Besides, we are buying a listing to give us a leg-up in gaining the appearance of prominence.

The Open Directory Project (ODP) is another prominent directory. Try to secure a listing with them. But since they don't accept money, make a clear argument for the importance of a link and try, try again. Looksmart, a third directory site, is not nearly as important as Yahoo in building prominence according to the literature I am reading.

Keep in mind, the purchase of listings in Yahoo is not for the audience who read the Yahoo directory, which in any case is rather small. We buy listing in Yahoo for the prominence that the search engines assign to sites that appear in Yahoo. Crazy as this seems, this method of promotion is clearly supported and demonstrated.

In a very similar manner, as we hunt for useful sites to request links, we need not shy away from paid listings in public directories. Should a specialist directory ask for money, why not? A link from a prominent webpage has value. Check the prominence of the site, consider if useful traffic could follow, then go ahead. Any link, is a link: and adds to your statement of prominence.

The known portion of the Google ranking algorithm suggests that prominence is shared between webpages when links are forged. This suggests it is possible to make certain pages more prominent than others by linking in a cluster pattern. There are further suspicions that jumps across several directories may diminish prominence, that unwise or unwarranted links may diminish prominence and other unusual patterns may similarly diminish prominence. Search engines also check for suggestive abuse of their ranking system and penalize such sites. Sites with a simple architecture, grouped by concepts and linked logically should fare well. Sites linked in different ways will simply need to check if comparable sites are improved or penalized for such behavior. We should also redirect any changed or hanging links. Link checker's can be important in this regard.

The fourth approach to building prominence is in helping others link to you. A link made by someone else is as good as a link painstakingly requested or purchased by us. We want to help people link to our pages. In fact, the visitors can just as easily be internal users within our organization. We want all visitors to link to our pages.

Many websites suffer technical faults that throw up barriers to linking. Overly complex web addresses, slow web servers, and overly heavy webpages full of graphics all hurt a sites prominence because they make linking less likely. An exciting user-experience helps. Achieve this simply by designing a more enjoyable viewing experience - graphics, imagery, rollovers and other techniques to make static pages more interactive.

An often overlooked opportunity lies in simplifying a web address. Domain names are not case sensitive and the leading www should be redundant. I always refer to my website as:
instead of:
The first style is clearly more memorable.

Sites may also suffer from policy barriers that limit publishing or linking to a website. Legally gray publishing guidelines may dissuade staff from publishing online. Just as likely, strategic guidelines may limit publishing. While perhaps totally advisable, such limitations do have a cost in terms of potential site prominence. A large site is judged as having additional prominence (since internal links are still links). An interesting site entices more people to link.

What on your website entices visitors to link to you?

Long-term strategy often directs us to a future where more is published on the Internet for promotion sake. Understanding that a degree of prominence is at stake may lubricate the internal decisions necessary to see more published online. Most sites start with brochure-type material and suffer from a brochure-like feel. Better sites deliver prominent documents they generate normally. Gradually, more is released through the Internet. This means we can initially look to existing print brochures and ask should any be online. Next we ask why they are not online. At a later stage we fill in the information left off the brochures: the pictures, descriptions and detail you want to share but must leave off print documents for want of space. The web excels in delivering such information.

Much later, websites often fit tightly into the strategic marketing plan. Instead of just presenting information, a website may demonstrate your qualities. A school may demonstrate its strong learning environment with a very open publishing policy showing much learning-centered discussion and extensive proof of a professor's knowledge. A bank may foster recognition by publishing a collection of highly respected economic reports, delivering these freely online in full. A speaker like myself may demonstrate their leading edge of innovation and understanding with articles on Internet research and links to articles in prominent trade press. In each case, marketing plans direct us to publish more, knowing full well such publishing increases our prominence.

In summary, prominence is an ideal that search engines approximate from the number and quality of links. Important information gets more links but so does manually asking for links and even purchasing links. We can also help people link by simplifying our web address, improving the viewing experience and putting better information on the web.

There is another side to the ranking equation. Prominence is the first criteria: Relevance the second. Relevance comes from having the words the searcher is seeking displayed prominently on the webpage. Relevance starts with having the words requested.

Suppose I search for the Johnathon Smithy website in Australia. These are my search words: Johnathon Smithy Australia

If Johnathon's homepage does not include the words Australia then that webpage will not appear in the results. The purpose of the page fails simply because it does not include the requested words. From a designer's perspective, we need to guess the words people will search with. In the case of Mr Smithy, we should probably include geographical markers, the words Australia, on the home page in case a prospective client searches for these words.

What other words should we include?

Say we search for the words: Tennis Coach. Many of the pages at the top of the results list will have Tennis Coach in the title. These pages will appear before pages that mention tennis coach within the page or discuss a coach for tennis (rather than tennis coach).

Essentially, the title is considered of primary importance when judging the relevance of a web page. And search engines rank websites not only by prominence but also by relevance.

Relevance is judged by the title, meta-tags and perhaps important headlines within the text. Relevance may also extend to linking text - the words associated with the incoming links to your pages. Relevance considers if words appear next to each other on the page (proximity) and the number of times they appear on the page (frequency). Simple repetition of the words may help but can equally be abused - which may penalize you (either now or in the future). A little finesse is called for.

What words do you want to be found under? These words should be placed on a prominent position within the webpage. Craft your webpage to match the words used by your clients.

There is a nice twist to this. Many words we want to be found under will already be 'owned', for want of a better work, by many other more prominent webpages. Realistically, if we can not appear in the first thirty or fifty results for a specific search, it is not worth any effort trying to reach the audience who search with those words. Is the Spire Project about internet research or internet search theory? I certainly own Internet search theory. I am unlikely to be found if people search for internet research.

If we are clever, there may be specific words you can aim to 'own'. These will be words that don't feature on comparable sites with higher prominence.

Search engines rank pages based on a combination of prominence and relevance. First come the prominent pages that feature the search words in prominent positions. Next come prominent pages that feature the search words. Next, less prominent pages with the words, stepping down progressively. Neither prominence nor relevance is enough to be ranked highly but a prominent and relevant page will certainly rank highly.

I have been careful to repeatedly refer to time in this white paper. Our interest in prominence, relevance and search engines are all transient. These are stationary structures seen from a moving car. For very complex reasons I can explain at another time, the Internet is growing cloudy. The prominence ranking technologies of the current batch of search engines are failing. They will not be replaced as easily as Yahoo style directories were replaced two years ago by today's prominence ranked search engines. Consequently, we are entering a phase of growing confusion on the Internet.

In such an environment, Internet promotion should focus on three concepts: not just prominence and relevance but also awareness. Awareness comes with links, endorsements, recognition and mention in directories and discussion. It is a wider concept than prominence. During the era of the directories, awareness was often best achieved with a prominent directory listing. Today, awareness is often best achieved with a prominent ranking on Google. Tomorrow, awareness will be achieved with a range of efforts.

There are two measures of awareness - firstly your own log file. Every website records some basic information about its visitors. This includes some factual information about how visitors currently find you. Not everyone finds you with a search engine. Prominent directories, topic-specific sites and the like also play a role. A focus just on prominence would miss this.

The second measure is a list of all sites that link or refer to you by name or web address. This list can easily be created with the help of the search engines. Often references - mention of your name or web address - are twice as numerous as the number of links.

If we focus solely on improving search engine prominence, we will discount the value of a link from a specialist directory. Our log file may well indicate we have many visitors who find you through such a directory. Similarly, awareness of your website, perhaps just a mention of your name or your website (but not a link) is of no value in improving search engine calculated prominence. But such references certainly help build awareness. With a forecast for a cloudy Internet ahead, we want to lay down as wide a footpath as possible. Search engines will not always hold such an important role in Internet promotion.

Awareness is not an add-on service to improving search engine ranking. It must not be. If anything, the reverse holds true. An interest in improving prominence and relevance is but one element in Internet promotion - an important element - currently an important element - currently the most important element - but by no means the only element. Publishing policy, strategic objective and the way the public actually finds you also bears consideration.

Prominence, Relevance, and Awareness are the concepts that guide us as we transpose our normal promotional work to the Internet. There is no magic unknown trick that elevates hidden webpages into view. There is only the normal process of promotion, adapted to the Internet environment. There are, however, some very specific recommendations that arise from an understanding of how search engines rank their results.

Many of these recommendations do depend to a degree on what is being attempted and what opportunities exist. We have sketched a general picture of how Internet promotion works. The next step is to gather information about your particular project, your existing links, your current traffic and the strategic vision of what you hope to achieve with your website. Only then will we know what steps to take. The real work begins now.

My pleasure to advise you,

David Novak of the Spire Project

to article listThe Spire Project - better ways to find information.
Like this? Attend our public seminar and receive our bi-monthly update notice.
 | | | Project Background | Feedback. Copyright©David Novak 2003.