On mar, 2003-02-04 at 12:35, Erik Moeller wrote: The problem was basically that for very active users, loading up a user contribs list required sorting *thousands* of rows to pick just the last 50 and show them in timestamp order. Right now I'm cheating by checking the recentchanges table first -- it lists only the last few days' worth of edits, and even an active user generally doesn't have too many edits in that time. However a very active user who becomes inactive, and falls out of recentchanges, would still be slow to query. Jan and Lars have suggested using compound indexes which include the timestamp fields _and_ the user fields so it can both select and sort using the index, and that sounds reasonable in a sick sort of database-logic way. :) -- brion vibber (brion @ pobox.com)

On mer, 2003-02-05 at 00:42, Guillaume Blanchard wrote: We do have a table that caches just the last few days' worth of recentchanges, so it's much quicker to query than the full 'cur' and 'old' revision tables. Storing the HTMLized recentchanges would be of less help, as it changes about as often as it is loaded... -- brion vibber (brion @ pobox.com)

On mer, 2003-02-05 at 00:51, Andre Engels wrote: I'm not sure that'd be reliable. Rows that are chronologically earlier may be added to the database later -- undeletes, recombining histories of broken articles, the great UseMod-Era History Restoration a few months back. So the auto-increment per-row ID number isn't a reliable indicator of chrono order. (Check some page histories on the English wiki that date from 2001 for an example, the current hackish code does just this and pays for it by sometimes being incorrect.) Timestamp can't be the primary key alone, as we only have 1-second resolution. Duplicate timestamps can, at least theoretically, happen. If an index that both selects and sorts on all the criterea we need is possible, that should do us well enough. -- brion vibber (brion @ pobox.com)

I suggest an "empty redirection". When you look for a word in the dictionary,the wiktionary ( I suggest change the name to wikionary, more easy for all languages ) (i.e. domobot ) and appears the message No matches ( No article title matches and No article text matches), the wiktionary redirects automatically to the wikipedia article (i.e. www.wikipedia.org/wiki/domobot ). This is the "empty redirection". In any case, the "see more in wikipedia" redirection can be add to, although the wikionary word it´s not empty ( to get more information from the encyclopedia about the word searched and found in the dictionary ). This is, strong and embedded interwiki links. Regards.

Try "[[en:dombots]]". Pedro M.V. wrote:

Pedro M.V. wrote: I suspect that "wikionary" would be more of a problem for people than "wiktionary" if their language is one that has a "t" in dictionary. Nevertheless, I can see the problem for a Spanish speaking person who uses a "diccionario". The Spanish language version will not need to have exactly the same name as the English Wiktionary. There is already prcedent for this in Wkikpedia. The links to Wikipedia for more information are surely a good idea if Wikipedia already has an article, but empty redirection would create an entirely different set of new problems. Using your example of "domobot" it would make access to Wiktionary's "domobot" more difficult for the person who wants to write a dictionary entry for ways. There are ways around it, but I wouldn't expect a newcomer to know that. Eclecticology

On mer, 2003-02-05 at 02:40, Philipp W. wrote: The phase 2 software (used january-july 2002) used a TIMESTAMP column. However, the damn things get rewritten every time you update the row; a lot of operations had to have "cur_timestamp=cur_timestamp" stuck in them, but we had a LOT of bugs caused by forgetting to do this. (Renaming a page caused its previous revisions' timestamps to be reset to the current time, and apparently vanish from the history list; once or twice table restructurings resulted in *every* article's last modified time being reset.) When Lee Crocker rewrote the software, he switched the type to VARCHAR and just imported the values as they're formatted for a TIMESTAMP (YYYYMMDDHHMMSS). A DATETIME column might or might not be more efficient. We can't rely on Mysql's date->text formatting functions though as we serve multiple languages out of a single server, with timezone selectable per user and formatting per language. When we _have_ the right indices, that'll be the time to hint them. :) -- brion vibber (brion @ pobox.com)

Hi I'm fairly new to wikipedia and think it's an amazing project. Congrats to all involved! I've been hit with the speed problems though, so I've been following the optimization discussions from afar and may be able to contribute somewhat. I'm hoping to have some more time to help out with optimizing as I've got quite a bit of experience in the field (at least I've just written a MySQL book, so people think I do). I was quite surprised to read that the web and database server are on the same machine. Simply separating these two (even for 2 equivalent slower machines) which you llook like you're going to do, has usually made a noticeable difference, allowing the machines to do both jobs optimally rather than hopping about trying to do both. Definitely put the mysql server on the better machine. Another fairly easy thing you can do to cut out the slow queries is to activate slow query logging. This only logs 'slow' queries and you can quickly tell quite a lot from this, and work on these queries as a priority. MySQL 4 would make sense as an upgrade. It's more than stable enough on any *nix platforms, and has some substantial advantages as some of you have pointed out already. I haven't really had a look at the code, so I'm not sure how relevant all this is, but mirroring pages makes a lot of sense, and takes unecessary load off the database. The front page, and all articles, should be available at high speed (this at least gives newcomers.something to see even if the db is churning in the background). If the front page needs to do any database queries, then mirror it as a static page every x minutes, much more efficient than doing the query hundreds of times a minute. You could mirror individual articles, but I'd need to look at the code and usage to see if this will help. Hopefully I'll be able to help out more, but these are just some thoughts that may be useful to think about for now. Apologies if they're not relevant because I haven't taken a close enough look at the details. I'll hopefully be able to dive in soon. I'd also like to take a look at the my.cnf file as well, if that's possible, as well as know the hardware of the db server. That's something that can be optimized quite quickly, and may help a bit, if it's not been done already. regards, ian gilfillan

Brion Vibber wrote: Brion, go to your room right now! I, too, will be offline for a few hours. I have to clean up my office before my mom comes to visit. You'd all be horrified to see how I live. --Jimbo