SEO

Search Engine

A search engine is a system that finds, organizes, and ranks web content so it can answer queries with the most relevant results. Google is the dominant one, and understanding how a search engine works is the foundation of all SEO.

A search engine is a system that crawls the web, stores what it finds, and ranks it all so that when you type a question, it can hand back the most relevant answers in a fraction of a second. Google is the one nearly everyone means when they say search engine, but Bing, DuckDuckGo, and others work on the same fundamental principles. I put this term in the glossary because everything else in SEO descends from understanding what a search engine is actually trying to do. Strip away two decades of jargon and the job of a search engine has never changed: take a messy, enormous web and a vague human question, and return the handful of pages most likely to satisfy that person. Your job in SEO is to make your page one of those.

bolt

A search engine is a matchmaker between questions and answers. SEO is everything you do to make sure it confidently picks your page as one of the best answers.

The three jobs every search engine performs

No matter how complex the technology gets, every search engine runs the same three-step process under the hood. Learn these three jobs and the entire field of SEO suddenly has a clear shape, because nearly every technique you will ever use exists to help with one of these steps.

  • Crawling: automated programs called crawlers or bots follow links across the web, discovering pages and reading their content. If a crawler cannot reach your page, it might as well not exist.
  • Indexing: the engine processes and stores what it crawled in a massive database called the index. A page that is crawled but not indexed still will not show up in results.
  • Ranking: when someone searches, the engine sorts through its index and orders the relevant pages by how well each answers the query, using hundreds of signals. This is the part SEO obsesses over.

targetWhy all three steps must succeed

These three jobs form a chain, and a break anywhere stops the whole thing. A brilliant page that crawlers cannot reach never gets indexed. A page that gets indexed but ranks on page nine never gets seen. This is why SEO is never just about writing good content. You have to make sure the engine can find your page, store it, and then judge it worthy of a top spot. Each step is a distinct discipline, and a serious SEO program addresses all three.

Example

Picture publishing a fantastic new guide. First, a crawler has to find it, usually by following a link to it from another page on your site or your sitemap. Next, the engine reads it and decides to store it in the index. Finally, when someone searches the topic, the engine weighs your page against every competitor and assigns it a rank. If you block crawlers by accident, or your page loads so slowly the bot gives up, you fail at step one and nothing downstream matters. Making sure those mechanics work cleanly is the heart of technical SEO.

Crawl, index, rank

Memorize these three words. Crawling finds your page, indexing stores it, ranking positions it. Every SEO problem you ever face will turn out to live inside one of these three steps, which makes diagnosis far simpler.

One more shift worth naming, because it defines this era. Search engines are no longer only returning ten blue links. Google now assembles AI-generated overviews, and standalone AI engines like ChatGPT and Perplexity answer questions by pulling from web content directly. The underlying logic still rhymes with crawl, index, and rank, because these systems still have to find your content, understand it, and decide it is trustworthy enough to use. The surface changed, the foundation did not, and the pages that win are still the clearest, most credible answers a machine can find.

lightbulbPRO TIP

Before you spend a day optimizing content, confirm the search engine can actually crawl and index your most important pages. A single misconfigured setting can hide your whole site, and no amount of great writing fixes a page the engine cannot see.

Want this handled by someone who has measured search for 20 years?

Work with me