How Search Engines Work: What You Need To Understand

Since we’re almost done with Mark Joyner’s “Learn to Blog for Profit & Fun“, I thought we should take a look at how search engines work in order to help us really maximize our blog and website traffic.

“Why can’t my site be found on Google?”
Excerpted from Word Tracker’s “Keyword Basics: How Search Engines Work”

I’ve been asked that question many times before.  In fact, I’ve asked it myself when I took my first babysteps into the online business and internet marketing world.  After all, it’s important to appear.  But most people wonder “Why isn’t it happening to me?”

When I first started online, I looked upon search engine optimization as a black art - something magical and mysterious that could only be solved by enrolling in an expensive course to master its intricacies, or hiring some over-priced practioner to do it for me.  Since money was scarce in the beginning days of my online business, I admit that for a while, I was at a loss as to what to do to compete with businesses that were fully optimized and could afford to have experts working with them and sharing their search engine optimization secrets. 

The big secret is there is no big secret

It’s true: the ‘big secret’ of search engine optimization is that there is no big secret.  All it takes is an understanding of the inner workings of a search engine - just what makes them tick (or crawl as the case may be)

Inside the guts of a search engine

There are essentially three pieces of software that together make up a search engine - the Spider software, the Index software and the Query software.  Understanding these is the key to unlocking search engine optimization and getting top rankings in Google, MSN, Yahoo or any other search engine.

Let’s take a closer look:

The Spider software ‘crawls the web looking for new pages to collect and add to the search engine indices’. This is a metaphor. In reality, the spider doesn’t do any ‘crawling’ and doesn’t ‘visit’ any web pages. It requests pages from a website in the same way as Microsoft Explorer, or Firefox or whatever browser you use requests pages to display on your screen.

The difference is that the spider doesn’t collect images or fancy designs - it is only interested in text and links AND the URL, (for example, http://www.Unique-Resource-Locator.html) from which they come: it doesn’t display anything and it gets as much information as it can in the shortest time possible.

A spider loves links because they lead it to other web pages that have the things that it loves, guess what? more text, links and URLs!

The Index software catches everything the Spider can throw at it (yes, that’s another metaphor). The index makes sense of the mass of text, links and URLs using what is called an algorithm - a complex mathematical formula that indexes the words, the pairs of words and so on

Essentially, an algorithm analyses the pages and links for word combinations and assigns scores that allow the search engine to judge how important the page (and URL) might be to the person that is searching. And of course it stores all of this information and makes it available to people who are searching.

The Query software is what you see when you use a search engine - it is the front end that everybody thinks of as a search engine. It may look simple but it presents the results of all the remarkable search engine software that works away invisibly on our behalf.

The main feature of the query software is the box into which people type their search terms.

Type in your words, hit search and the search engine will try to match your words with the best web pages in can find through searching the web.

But this too is a metaphor and perhaps the most important one.

The query software doesn’t search the web - it checks the records that have been created by its own index software. And those records have been made possible by the raw material the spider software collects.

What you need to understand about search engines

What you need to understand is that the search engine has done all the hard work of collecting and analysing web pages, BUT it only makes that information available when someone does a search by entering words in the search box and hitting return.

The words people use when they search therefore determine the results the search engine presents. We call them keywords - that might sound fancy but keywords are only ‘the words people use when they search’.

And keywords are what Wordtracker provides - many millions of them. Use keywords in your website copy and you will prosper: ignore them and your online business will surely perish.