Our search engine desperately needs retooling. If there's no objection from those in the know, I'd like to migrate us to MySQL 4. The fulltext search in 4 has boolean capabilities built right in, meaning we could remove our hackish and buggy parser, and wouldn't need to stack so many MATCHes together in a query when some poor sap types in "chemical composition of the earth's atmosphere oxygen nitrogen" or something.
(Our search queries are also frequently *dog slow*. This is exacerbated because, being a myisam table, it locks when someone tries to write it and another read is pending. I don't _think_ this lock virulently spreads to other tables joined with it, but it's annoying anyway.)
Other things to think about:
* Stopwords. Can we just get rid of the damn stopwords and search anything?
* "Title results" vs "Text results" - this two-prong approach is, I think, rather confusing. We could have a single search index field with the title text weighted more heavily (by repetition?), and just give a single set of results.
* Text extracts: these show the raw wikicode, and often include language links, HTML code, etc. Yuck! If we can strip these, that might be good.
* Character entities: should be folded to their raw equivalents in the search index, so searching a page containing "Schrödinger" and one containing "Schrödinger" gives identical results.
* 'Power search' is perhaps a little confusing, and there's currently no way to get to it short of doing two searches.
* 'Search' and 'go' buttons are not clearly demarcated; several people have noted confusion. Better labelling or better arrangement is needed.
* Redirects. We generally want to filter out redirects that seem duplicative of other things already listed, but *must* show them for alternate names. Clearer labeling of redirects would help as well.
-- brion vibber (brion @ pobox.com)