Jimbo, the tricks that you mention (weighing titles more than text,
weighing rare words more than common ones, stopwords etc.) are already
used by mysql's fulltext index; I don't think it would make sense to
reinvent the wheel, especially since we now have real-time searching
and boolean searches, both of which are kind of nice. I am also pretty
sure that the mysql index is reasonably fast, being written in C.
They explain a bit about it at
http://www.mysql.com/doc/F/u/Fulltext_Search.html
Regarding the three letter limit: since we right now already parse
the search string for AND, OR and NOT anyway, it should be pretty easy
to remove short words from the search string, start the query without them,
and then later report the results with a warning like
The short word "the" was ignored.
Axel