Lars Aronsson wrote:
more efficient can a new parser be? Twice as fast? Is it possible to add more hardware (multiprocessor) instead?
I'm looking at adding hardware -- this machine is a single processor Pentium III/866, but it's a dual processor motherboard. I don't know how much this will help, though.
However, I will say this: based on my extensive experience running busy websites (Bomis), it seems very unlikely that the problem is just _hardware_. I think we have a long way to go in terms of optimizing first.
Just switching from mysql_connect to mysql_pconnect has resulted in a speedup from 1/2 page per second to over 2 pages per second. So that's 400% right there. Adding a second processor may speed things up by what, well 100% at _best_, but realistically probably 30%.
Anyhow, I agree with you 100% -- I don't think that regex parsing is the primary bottleneck (although I think we should profile to be sure!). I think it's excessive or inefficient SQL queries.
Is there an easy way in mysql to turn on some logging to see which queries are taking the longest? If we had some metric telling us what our bottleneck is, i.e. things that get called a lot AND that take a long time, then I bet we could end up very pleased with the results.
--Jimbo