On 7/7/06, Leon Weber leon.weber@leonweber.de wrote:
Yeah. I wanted to create these stats for dewiki, so just like duesentrieb said, let the js just call the page with a probability of somethin maybe less than 1/1000. But NullC gave me a great idea now: I'll just let the js call an empty text file and collect the stats from the apachelogs. There's no more efficient way.
To clarify, IFF you disabled logging and you wrote an apache module to maintain a counter in shared memory using an efficient datastructure (perhaps a Judy array on page titles) and worked out the locking issues... it would be faster.
However, apache logging is async and append only. It's the simplest form of writing that could happen, and if we moved the logs into tmpfs it would likely be darn close to optimal.
Although toolserver is disk bound, what is killing us is random seeks (see iostat, we are constantly pegged at 350-400 TPS but moving less than 6MB/sec)... so I would not expect much problems from an async and append only writer because all of it's activity will be mostly sequential.