I'm sure that unless you are a keen webmaster/search engine enthusiast, you will have never heard of searchhippo.com. This is basically a one man project (like gigablast) who made a search engine as a hobby and did pretty well.

If you go to http://www.searchhippo.com/tech-overview.php you can see a really well explained overview of the different sections of a search engine. Also, if you are interested there's a way to participate with it on the site with a 2 mb datafile to play with if you are interested in alternative mechanisms or building your own search engine.

Rob