On Mon, 2003-04-28 at 14:14, Erik Moeller wrote: Reporting a serious problem is all well and good, but isn't the same as _fixing_ it. We don't need to sit and chat. We need *code* and we need a second server to divide the "must-do-fast" web work the and "chug-chug-chug" database labor. Here are some things you can work on if you've got time to spend on Wikipedia coding: * Page viewing is still kinda inefficient. Rendering everything on every view is not so good... Caching can save both processing time in conversion to HTML, and in various database accesses (checking link tables, etc) with its associated potential locking overhead. We need to either be able to cache the HTML of entire pages (followed by insertion of user-specific data/links or simple options through style sheet selection or string replacement) or to cache just the generated HTML of the wiki pages for insertion into the page structure (plus associated data, like interlanuage links, need to be accessible without parsing the page). We need to tell which pages are or aren't cacheable (not a diff, not a special page, not a history revision, not a user with really weird display options -- or on the other hand, maybe we _could_ cache those, if only we can distinguish them), we need to be able to generate and save the cached material appropriately, we need to make sure it's invalidated properly, and we need to be able to do mass invalidation when, for instance, the software is upgraded. Cached pages may be kept in files, rather than the database. I should point out that while there are several possible choices here, any of them is better than what we're running now. We need living, running _code_, which can then be improved upon later. * The page saving code is rather inefficient, particularly with how it deals with the link tables (and potentially buggy -- sometimes pages end up with their link table entries missing, possibly due to the system timing out between the main save chunk and the link table update). If someone would like to work on this, it would be very welcome. Nothing that needs to be _discussed_, it just needs to be _done_ and changes checked in. * Various special pages are so slow they've been disabled. Most of them could be made much more efficient with better queries and/or by maintaining summary tables. Some remaining ones are also pretty inefficient, like the Watchlist. Someone needs to look into these and make the necessary adjustments to the code. Nothing to _chat_ about; if you know how to make them more efficient, please rewrite them and check in the _code_. * Can MySQL 4 handle fulltext searches better under load? Is boolean mode faster or slower? Someone needs to test this (Lee has a test rig with mysql4 already, but as far as I know hasn't tested the fulltext search with boolean mode yet), and if it's good news, we need to make an upgrade a high priority. Not much to _chat_ about, it just needs to get _done_. * Alternately, would a completely separate search system (not using MySQL) be more efficient? Or even just running searches on a dedicated box with a replicated database to keep it from bogging down the main db? Which leads us back to hardware... For the server; I don't know what's going on here. What I do know is that Jimbo posted this to wikitech-l in February: -----Forwarded Message----- From: Jimmy Wales <jwales(a)bomis.com> To: wikitech-l(a)wikipedia.org Subject: [Wikitech-l] Hardware inventory Date: 07 Feb 2003 02:56:57 -0800 Jason and I are taking stock of our hardware, and I'm going to find a secondary machine to devote exclusively to doing apache for wikipedia, i.e. with no other websites on it or anything. I'll loan the machine to the Wikipedia Foundation until the Foundation has money to buy a new machine later on this year. We'll keep the MYSQL where it is, on the powerful machine. The new machine will be no slouch, either. Today is Friday, and I think we'll have to wait for Jason to take a trip to San Diego next week sometime (or the week following) to get this all setup. (The machine I have in mind is actually in need of minor repair right now.) By having this new machine be exclusively wikipedia, I can give the developers access to it, which is a good thing. This will *not* involve a "failover to read-only" mechanism, I guess, but then, it's still going to be a major improvement -- such a mechanism is really a band-aid on a fundamental problem, anyway. ------ Lots of people think it's a good thing to set up mirror servers all over the Internet. It's really not that simple. There are issues of organizational trust with user data, issues with network latency, etc. Some things should be decentralized, some things should be centralized. --- end forwarded message --- and this to wikipedia-l in March: -----Forwarded Message----- From: Jimmy Wales <jwales(a)bomis.com> To: wikipedia-l(a)wikipedia.org, wikien-l(a)wikipedia.org Subject: [Wikipedia-l] Off today Date: 19 Mar 2003 04:47:52 -0800 My wife and little girl are feeling ill today with a cold, so I'm going to be taking off work to help out. I'm already a little behind in wikipedia email, so I'll probably be slow for a few days as I dig out. We're getting a new (second) machine for wikipedia -- the parts have been ordered and are being shipped to Jason, and then at some point soon, he'll drive down to San Diego to install everything. --Jimbo --- end forwarded message --- I e-mailed Jimbo and Jason the other day about this; I haven't heard back from Jimbo, and Jason still doesn't know anything concrete about the new server. Jimbo, we really need some news on this front. If parts and/or a whole machine really *is* on order and can be set up in the near future, we need to know that. If it's *not*, then it may be time to pass around the plate and have interested parties make sure one does get ordered, as had begun to be discussed prior to the March 19 announcement. -- brion vibber (brion @ pobox.com)

On Mon, Apr 28, 2003 at 05:45:18PM -0500, Lee Daniel Crocker wrote: I'm certainly willing to help out here. I'm not in SoCal, but I should be able to help out with most administrivial tasks. I'm going to be able to help out a much more with tuning at a file system/OS/Apache level than I will be at a PHP/SQL level. -- Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN

Brion, you're missing my point. I agree with you entirely that things need to "get done". My suggestion to have a public discussion was to find out which things we can get done reasonably quickly (because, realistically, we all have other things to do) with substantial impact; to figure out the server situation, which features should be disabled, who might contribute which piece of code etc. If we can sort these things out in the next few days via mail, fine. I'm no IRC junkie. But we need to implement at least some reasonable emergency fixes, and think about a mid term strategy. As for code, this is one thing I'd like to talk about: If we have the Nupedia Foundation set up, we can collect donations. It would be stupid not to use some of that money for funding development. I don't care who is funded, but I think this could greatly speed things up. If we can't get the NF set up reasonably quickly, we should collect donations regardless, tax-deductible or not. Why? It's just PHP stuff. Our bottleneck is the database server. Fetching stuff from CUR and converting it into HTML is not an issue. 20 pass parser? Add another zero. Until I see evidence that this has any impact on performance, I don't care. Turn off link checking and all pages are rendered lightning fast. What would be useful is to maintain a persistent (over several sessions) index of all existing and non existing pages in memory for the link checking. A file on a ramdisk maybe? I think it would be worth giving it a try at least, and not a lot of work. Wasted effort, IMHO. Cache improvements have added little measurable performance benefits, and there are many, many different situations to test here (different browsers, different browser cache settings etc.). Meanwhile, our real bottlenecks (search, special pages, out of control queries) remain in place. I doubt that a *relatively* rare activity like that makes much of an impact, but I'll be happy to be proven wrong. Bugs are annoying, but I'm writing this for one reason: we need to make Wikipedia usable again on a regular basis. There are countless small problems that need to be fixed. This is not the issue here. Caching special pages seems like a reasonable approach. Watchlists could definitely be improved, haven't seen a good way to do this yet, though. It could be done on page save, but with a much-watched page, this again would add severe drain, with possibly no overall benefit. Improve the SQL and indexes? Maybe, but I'm no SQL guru. Sounds good to me. If safe enough, we should update in any case; it is my understanding that MySQL4 has support for subqueries which could, if we know what we're doing, potentially be used to write significantly more effective queries. Regards, Erik

Depends on the moderator. No moderation=unpredictable, bad moderator=bad result, good moderator=possibly good result. Just like in real life meetings. Duh. But if our database is so slow that it can't even answer simple SELECTs, we can't do anything useful, cache or no cache. And if it isn't, then we should concentrate on the queries which aren't simple. The linkcache might still be one of those bottlenecks (simply because of the sheer number of queries involved), I haven't checked your latest changes to that code. Yes, but this is really one of the more expensive wiki features that also limits all caching options severely. Impossible to work without it, but apparently hard to implement in a scalable fashion. I might. I'll have to see if it makes any difference on the relatively small de database which I'm currently using locally. It would have to be optional -- setting up the software is already difficult enough. Yeah yeah yeah. I still think we should care more about the real showstoppers. But hey, you can always _code it_. (Finally an opportunity to strike back ;-) If this downtime is unacceptable, we might indeed have to think about a query only server with somewhat delayed data availability. This could be a replacement for the sysops, too. Mirroring the Wikipedia database files (raw) should be no issue with a SCSI system, or a low priority copy process. Maybe we should stop looking in the Himalayan mountains and start searching the lowlands .. In other words: Don't search those who will do it for society or for the glory. Just hand over the cash and be done with it. Regards, Erik

Hi, The "Pages with misspellings" don't care the accent common misspellings we add in "Wikipédia:List of common misspellings". "e", "é", "è" and "ê" are interpret all as same. It's useful in search page, but very bad for misspellings detection. Do "search" & "misspellings" using the same algorithm ? Can we solve it easly ? Aoineko

Brion Vibber schrieb: Wasn't there someone on this list a while ago who has written a book about MySQL? Or am I fantasising about this? Kurt

On Mon, 2003-04-28 at 18:04, Kurt Jansson wrote: Well, that wouldn't help for performance as the data goes through a temporary table. Basically the DB's grabbing your *entire* watchlist, then only sending the most recent X items to the wiki for formatting in a list. How long will the internet be able to deal with all those new users? Won't we run out of IP addresses if IPv6 never rolls out? When will the sun burn out, leaving the earth a lifeless ball of coal?? :) Hopefully, we'll be able to keep up. Do feel free to ask other free projects and universities if they'd be interested in supporting the project... -- brion vibber (brion @ pobox.com)