The Story of Search | The history of search, and how modern search engines work.

Chapter 1

Humans & Books

Giant piles of books

There's a lot of text in the world.

Penelope pondering what to search for... Renaissance printing?

And when we need to find things —

Penelope with stack of books, History of Print Volume One through Four, wondering to herself, where's The Gutenberg Press?

like specific topics, phrases, and even individual words —

Young Penelope with fresh cup of coffee scanning the volumes of text...

searching for them can be a time-consuming process.

Middle-aged Heroine with cold cup of coffee... still scanning through the volumes of text...

Page by page we read each word, hoping to find what we're looking for.

Old Penelope with moldy cup of coffee... still scanning through the text...

This could go on for a long time.

But for many books, there's a tool to make the process more efficient:

Bathed in the light of the gods, the Volume is opened to the Index page

the Index!

Penelope transcribing pages from the Volume into an Index page

Creating an index takes a lot of work up front: words must be listed alphabetically, with a reference to their page location.

Close-up of Penelope's index finger scanning the page, finding all the pages on which 'Gutenberg Press' is mentiond

But now we can find the topic we're looking for —

Young Penelope, with fresh of coffee, quickly finding what she's looking for

and bingo, jump to the pages that the index references!

Penelope glancing over at a glowing laptop

We've solved the search problem for books, but what about computers and big data?

Let's investigate what this looks like!

Chapter 2

Computers & Databases

Rows of data center racks that extend to the horizon line

There's a lot of data in the world

Magnifying glass over database showing rows and rows of zeros and ones

And databases store a lot of text.

Penelope searching 'Gutenberg Press'

If we need to find something, the solution is slow.

Computer calling the database...

Row by row, the computer scans the text for the terms we're looking for.

Spinning beach ball of death slowly returning results

This could go on for a long time.

But there's a tool to make this process more efficient.

You guessed it —

Hallelujah Praise Sweet Baby Jesus! The index file!

the Index!

The databse transforming into an index file

Row by row, the text in the database is analyzed and inserted into the index.

This process is called indexing.

Neatly organized pages sorted alphabetically

Like in the back of a book, the index is organized alphabetically for fast lookup.

The index file pointing to the exact location of the database

For every term that's inserted into the index,a reference to the original row in the database is attached for later use.
(We'll explain why later!)

The search engine finds the exact match

Now, when the computer searches for a term, it skips the database and heads straight to the exact match that's present in the index!

The result? Getting the exact answer you need, fast!

Penelope in lab coat and giant rubber gloves, ready to dive in and get her hands dirty!

Let’s dig deeper and use an interactive example with real data!

Chapter 3

Create an Index

Click on the button below to see how we create indices.

Indexing transforms raw text into a set of "tokens". Tokens are added to the index with a reference to each document containing the token.

Fantastic! Now we have an index. On to the search!

The search engine quickly finds records for matching tokens.

Chapter 4

Coming soon!

We've just scratched the surface of search technology,
and will continue to add chapters to The Story of Search.

New chapters will covers things like relevancy,
analysis, and getting started with open source engines.

Until then...

Learn how search works, one interactive illustration at a time with storyofsearch.com

&

Send Feedback
(we'd love to hear from you!)