How Do Search Engines Work?

 

One of the biggest reasons why many organizations are not doing even simple things to help people find their websites in search engines is they simply don’t understand how search engines work. This article will explain the basics of how search engines work and shatter some search engine myths and help you understand what you can do to help people find your website in search engines.

One of the biggest reasons why many organizations are not doing even simple things to help people find their websites in search engines is they simply don’t understand how search engines work. This article is intended to explain the basics of how search engines work and in the course of doing so shatter some search engine myths and help you understand what you can do to help free twitch bots people find your website in search engines.

What is a search engine?
The first to step to understanding how search engines work is to understand what a search engine is. The simplest explanation is that search engine is a tool for finding things online. There are many different types of search engines but they can be put into two major categories…

Human-Powered Directories
The first “search engines” were not actually search engines as we think of them today,Guest Posting but searchable directories of websites organized by hierarchical categories. A site is added to a directory when the website owner fills out a submission form on the directory’s website requesting their site be included in the directory and include their website’s title, description, URL (web address), and category. A moderator later reviews the site and if it meets the directory’s criteria a listing for that site is added or activated. The original Yahoo was originally a human-powered directory, and directories like Open Directory Project continue on today.

The downside of the human-powered search engine is that it only includes websites that have been submitted to it, which means you may not find what you’re looking for especially if it’s a new web page. The other downside from the directories’ point of view is that reviewing every site submitted is very labor-intensive and costly. I know because my company’s “search engine”, OurChurch.Com’s Directory of Christian Websites, like almost all church/Christian “search engines”, is a human-powered directory.

Crawler/Spider-Powered Search Engines
The next generation of search engines has programs which actively seek out new sites and read them into their indexes. These programs are called crawlers, spiders, robots, or bots. All of largest and most popular search engines today are of this type, including Google, Yahoo, MSN, and Ask.

The rest of this article is focused on how these crawler/spider-powered search engines work because more than 99% of searches are done on this type of search engine.

The Crawler
As mentioned above, the first part of a search engine is the crawler (AKA spider, robot, and bot). The crawler reads pages into the search engine raw database of every page the search engine crawler has read. Some important things to know about crawlers…

1) Search bots periodically reread every web page in their database. Why does this matter?

• You don’t have to do anything when you change your site. If your site is already listed in a search engine and you make changes to your website, search engines will eventually update their information.
• You do have to be patient. The frequency of re-crawling varies depending on the search engine, the importance of your site (as determined by the search engines), and how often you update it. It could take a day or it could take more than a week.