On Mon, 2003-04-28 at 16:46, Erik Moeller wrote:
Brion, you're missing my point. I agree with you entirely that things need to "get done". My suggestion to have a public discussion was to find out which things we can get done reasonably quickly (because, realistically, we all have other things to do) with substantial impact; to figure out the server situation, which features should be disabled, who might contribute which piece of code etc. If we can sort these things out in the next few days via mail, fine. I'm no IRC junkie. But we need to implement at least some reasonable emergency fixes, and think about a mid term strategy.
Well maybe, but my experience with using online chats like this is: * Everyone sits around for several hours babbling, waiting for the other folks to show up and complaining about the problems they're having logging in. * By the end, someone has scribbled up a page with a work plan, which everyone ignores in the future. * During all this time, they _could_ have been doing something productive instead...
As for code, this is one thing I'd like to talk about: If we have the Nupedia Foundation set up...
The status of the non-profit is indeed another thing Jimbo could shed some light on...
- Page viewing is still kinda inefficient. Rendering everything on every
view is not so good...
Why? It's just PHP stuff.
Obviously not, since that PHP stuff needs data from the database to work. :) Buying milk at the grocery store is more convenient than keeping and milking cows at home not because the milking process is more time consuming than grabbing a bottle from the fridge, but because maintaining the cow is a huge effort and milk is only available from the cow under certain conditions. Or, um, something like that.
Our bottleneck is the database server. Fetching stuff from CUR and converting it into HTML is not an issue. 20 pass parser? Add another zero. Until I see evidence that this has any impact on performance, I don't care. Turn off link checking and all pages are rendered lightning fast.
And that would be a pretty piss-poor wiki, wouldn't it? :)
What would be useful is to maintain a persistent (over several sessions) index of all existing and non existing pages in memory for the link checking. A file on a ramdisk maybe? I think it would be worth giving it a try at least, and not a lot of work.
Sure, it _might_ help. Code it up and see!
- The page saving code is rather inefficient, particularly with how it
deals with the link tables...
I doubt that a *relatively* rare activity like that makes much of an impact, but I'll be happy to be proven wrong.
Slow saving impacts everyone who tries to edit articles; four edits per minute may be _relatively_ rare compared to page views, but we're still running thousands of edits per day and it's a fundamental part of what a wiki is. It's absolutely vital that editing be both swift and bug-free, and if we can reduce the opportunities for saving to get hung up, so much the better.
Caching special pages seems like a reasonable approach.
Unfortunately that doesn't really solve the problem any more than replacing the search with a link to Google solves the search problem.
If updating these cached pages is so slow and db-intensive that it takes the 'pedia offline for fifteen-twenty minutes (which it does), then nobody's going to want to update the caches (last updated April 9...) and they become outdated and useless.
It works as a temporary crutch in place of "blank page -- this feature has been disabled, please find a less popular web site to play on", but it's not a solution.
Watchlists could definitely be improved, haven't seen a good way to do this yet, though. It could be done on page save, but with a much-watched page, this again would add severe drain, with possibly no overall benefit. Improve the SQL and indexes? Maybe, but I'm no SQL guru.
Which Himalayan mountain do we have to climb to find one? :)
-- brion vibber (brion @ pobox.com)