To wil's question in 503113: After talking with Laura, members of the internet, and my own gut - I'm going to start down this road using Sphinx.
Over the next week, this bug will be split up into multiple dependencies.
My general approach is going to be as follows:
* Research Sphinx and integrate it with mysql to build our search indexes - verify this via some commandline tools - that this is searchable. Get some sense of timing on the index building too.
* If the index build takes a while (I imagine it will), we'll make a delta-indexer as well - and run that as often as possible to simulate real-time indexing.
* Cron-ify and package this up nicely so it's easy to install in development and production environments.
* Write an API for search this'll be part of the regular AMO API.
* Use this API to perform search on AMO.
* Collect current production metrics: mysql QPS, lag, cpu load, anything useful that we think might get better.
* Collect new statistics when we launch and blog about how much better our lives are.
I'm adding ozten and laura to the CC since they probably have input to give me. Or rather Laura has input and Ozten might be using sphinx for something else.
Also - Nick's GDoc needs to be split into separate bugs as well as Sphinx won't do everything and a lot of those are UX.

Moving to 5.0.9 - the only thing that could block me is Fennec work - which is up in the air, but I expect to eat a week out of me.
Per 503354 we *DO NOT* need to do delta indexing.
Here's what happens next:
* Write a new Search API in Cake - versioned as 1.3 as not to disrupt Firefox
* Use API in Frontend to get results
* Package up and write crontab's for indexer as well as fabric tasks for indexing
* Collect production metrics, mysql QPS, search qps, lag, cpu load, anything useful
* Load test sphinx SE based on existing logs