But I can ssh there.
First line in top:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 25451 mysql 17 0 989M 948M 670M R 99.9 47.1 110:55 mysqld
--- Tomasz Wegrzanowski taw@users.sourceforge.net wrote:
But I can ssh there.
First line in top:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 25451 mysql 17 0 989M 948M 670M R 99.9 47.1 110:55 mysqld _______________________________________________ Wikitech-l mailing list Wikitech-l@wikipedia.org http://www.wikipedia.org/mailman/listinfo/wikitech-l
I don't know what ssh means...
Slow Server Help ? Silence ! Sleeping Happily ? System Severily Hurt ?
Anyway, if the server is down, nothing special to say. If the server is just slow, I would just say that no connexions have been possible for the past two hours.
__________________________________________________ Do you Yahoo!? Yahoo! Platinum - Watch CBS' NCAA March Madness, live on your desktop! http://platinum.yahoo.com
On Wed, Mar 26, 2003 at 06:17:13AM -0800, Anthere wrote:
--- Tomasz Wegrzanowski taw@users.sourceforge.net wrote:
But I can ssh there.
First line in top:
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND 25451 mysql 17 0 989M 948M 670M R 99.9 47.1 110:55 mysqld _______________________________________________ Wikitech-l mailing list Wikitech-l@wikipedia.org http://www.wikipedia.org/mailman/listinfo/wikitech-l
I don't know what ssh means...
Slow Server Help ? Silence ! Sleeping Happily ? System Severily Hurt ?
Anyway, if the server is down, nothing special to say. If the server is just slow, I would just say that no connexions have been possible for the past two hours.
Somebody is going to have to give mysqld a kill (maybe a kill -9). Anybody got root on the box?
On Wed, Mar 26, 2003 at 08:46:32AM -0600, Nick Reinking wrote:
On Wed, Mar 26, 2003 at 06:17:13AM -0800, Anthere wrote:
I don't know what ssh means...
Slow Server Help ? Silence ! Sleeping Happily ? System Severily Hurt ?
Secure SHell
Anyway, if the server is down, nothing special to say. If the server is just slow, I would just say that no connexions have been possible for the past two hours.
Server is not down, only HTTP doesn't respond.
Somebody is going to have to give mysqld a kill (maybe a kill -9). Anybody got root on the box?
$ uname -a Linux www.wikipedia.org 2.4.7-10smp #1 SMP Thu Sep 6 16:16:16 EDT 2001 i686 unknown $
Should I try this ptrace() trick now ;-) ? Just kidding ...
(Tomasz Wegrzanowski taw@users.sourceforge.net):
Somebody is going to have to give mysqld a kill (maybe a kill -9). Anybody got root on the box?
I just killed and restarted MySQL and Apache. Looks like we're still getting slammed, and the logs don't show anything like an out-of-control bot, so it looks like just ordinary heavy traffic.
On Wed, 26 Mar 2003, Lee Daniel Crocker wrote:
I just killed and restarted MySQL and Apache. Looks like we're still getting slammed, and the logs don't show anything like an out-of-control bot, so it looks like just ordinary heavy traffic.
Lee, in the future when mysqld is doing its thang all too well; could you try connecting (mysql -u root -p) and running a 'SHOW FULL PROCESSLIST'? This should show the life-sucking query (somewhere among the big huge list), and it is often possible to kill just that query using mysql's KILL command.
(Mysql supposedly always reserves one connection for root for emergency maintenance.)
If you already tried that and it didn't work, please ignore. :)
-- brion vibber (brion @ pobox.com)
(Brion Vibber vibber@sal-sun062.usc.edu): On Wed, 26 Mar 2003, Lee Daniel Crocker wrote:
I just killed and restarted MySQL and Apache. Looks like we're still getting slammed, and the logs don't show anything like an out-of-control bot, so it looks like just ordinary heavy traffic.
Lee, in the future when mysqld is doing its thang all too well; could you try connecting (mysql -u root -p) and running a 'SHOW FULL PROCESSLIST'? This should show the life-sucking query (somewhere among the big huge list), and it is often possible to kill just that query using mysql's KILL command.
(Mysql supposedly always reserves one connection for root for emergency maintenance.)
If you already tried that and it didn't work, please ignore. :)
Nope, didn't know about that one. Thanks, I'll file that one away for future use.
It looks like it was a miswritten query by one of our new sysops on fr.wikipedia.org. He has learned respect for the database the hard way. :)
(Slow query log lists it as taking 31141 seconds, which is presumably where it got cut off by a kill/restart. Ouch!)
I'll have to see if there's a way to have an automatic timeout/kill query for a certain mysql user; that would reduce the impact of these occasional accidents. (Sysop sql queries are sandboxed in a 'wikisql' user account which has limited privledges. In a super-ideal world, they'd probably also run on their own replicated server where they will never interfere with the live system. But, that means more computers. :)
-- brion vibber (brion @ pobox.com)
(Anthere anthere6@yahoo.com):
I don't know what ssh means...
"Secure Shell": the standard way to log onto a machine remotely these days. At any rate, the server is up, it's just totally slammed with traffic.
--- Lee Daniel Crocker lee@piclab.com wrote:
(Anthere anthere6@yahoo.com):
I don't know what ssh means...
"Secure Shell": the standard way to log onto a machine remotely these days. At any rate, the server is up, it's just totally slammed with traffic.
hum
Say Lee, the last connexion to the french wiki was done at 11:21 am It was down from 11:21 to about a couple of minutes ago (that is 20:23) Between these two times (so ... 9 hours...), absolutely NO connexions were possible I just checked ...was the same on the en apparently
so....which traffic ? from where ? from who ? any idea ?
__________________________________________________ Do you Yahoo!? Yahoo! Platinum - Watch CBS' NCAA March Madness, live on your desktop! http://platinum.yahoo.com
(Anthere anthere6@yahoo.com):
Say Lee, the last connexion to the french wiki was done at 11:21 am It was down from 11:21 to about a couple of minutes ago (that is 20:23) Between these two times (so ... 9 hours...), absolutely NO connexions were possible I just checked ...was the same on the en apparently
so....which traffic ? from where ? from who ? any idea
Well, that's the first time I'd heard that. And there's only one server: just because there's no traffic on one of the many wikis doesn't mean the server isn't busy as hell.
The only message I got this morning was "the wiki is down". That didn't tell me anything, so I logged on and saw that the CPU load was maxed out, and that there were hundreds of active connections. Looked like regular traffic to me.
It seems to be responding better now after the kick, but I'm reluctant to restart the server (which involves breaking all active connections) unless I get a more specific report than "the wiki is down".
On Wed, Mar 26, 2003 at 01:53:29PM -0600, Lee Daniel Crocker wrote:
(Anthere anthere6@yahoo.com):
Say Lee, the last connexion to the french wiki was done at 11:21 am It was down from 11:21 to about a couple of minutes ago (that is 20:23) Between these two times (so ... 9 hours...), absolutely NO connexions were possible I just checked ...was the same on the en apparently
so....which traffic ? from where ? from who ? any idea
Well, that's the first time I'd heard that. And there's only one server: just because there's no traffic on one of the many wikis doesn't mean the server isn't busy as hell.
The only message I got this morning was "the wiki is down". That didn't tell me anything, so I logged on and saw that the CPU load was maxed out, and that there were hundreds of active connections. Looked like regular traffic to me.
It seems to be responding better now after the kick, but I'm reluctant to restart the server (which involves breaking all active connections) unless I get a more specific report than "the wiki is down".
Well, hard to expect much more than that from a user who doesn't have a login to the server itself. By the way, my guess is that some query, or perhaps some quirk, had caused MySQL to lock up. All of the apache processes were likely in a blocked state, waiting for MySQL to respond. Just because there are lots of processes running, doesn't mean they're actually responding.
Nick Reinking wrote:
Well, hard to expect much more than that from a user who doesn't have a login to the server itself.
Well, that's right.
By the way, my guess is that some query, or perhaps some quirk, had caused MySQL to lock up. All of the apache processes were likely in a blocked state, waiting for MySQL to respond. Just because there are lots of processes running, doesn't mean they're actually responding.
One reason I like DB's I can attach an strace at each query. No better way to get these buggers ;)
Nobody^H^H^H^H^H^HSmurf
--- Nick Reinking nick@twoevils.org wrote:
On Wed, Mar 26, 2003 at 01:53:29PM -0600, Lee Daniel Crocker wrote:
(Anthere anthere6@yahoo.com):
Say Lee, the last connexion to the french wiki
was
done at 11:21 am It was down from 11:21 to about a couple of
minutes
ago (that is 20:23) Between these two times (so ... 9 hours...), absolutely NO connexions were possible I just checked ...was the same on the en
apparently
so....which traffic ? from where ? from who ?
any idea
Well, that's the first time I'd heard that. And
there's
only one server: just because there's no traffic
on one of
the many wikis doesn't mean the server isn't busy
as hell.
The only message I got this morning was "the wiki
is down".
That didn't tell me anything, so I logged on and
saw that
the CPU load was maxed out, and that there were
hundreds
of active connections. Looked like regular traffic
to me.
I dunno Lee The wiki has been horribly slow for the past 2 weeks, and many editors have already gave up editing because of this Today, it was just as if wiki didnot existed at all during 9 hours, which were full day hours And not only was there no response from the server at all, but no response on the mailing list either. That is a bit disturbing I would say. In short, if 13/14 days it takes about 1 mn to respond for each page during day time, and the 14th day wiki doesnot exist at all, well, it won't be long for anybody to just give up
Besides, I checked on the english wiki, and I saw there were no edits in the recent changes log for many hours, so you can hardly say it is no response from "one of the numerous wiki". This was a general matter. So, we wonder.
It seems to be responding better now after the
kick, but
I'm reluctant to restart the server (which
involves breaking
all active connections) unless I get a more
specific report
than "the wiki is down".
Well, hard to expect much more than that from a user who doesn't have a login to the server itself. By the way, my guess is that some query, or perhaps some quirk, had caused MySQL to lock up. All of the apache processes were likely in a blocked state, waiting for MySQL to respond. Just because there are lots of processes running, doesn't mean they're actually responding.
-- Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN
Nick, did you read that message I forwarded several hours ago from Ryo ? I was on my old email address when I did the forward, and the wikitech list refused it (the weird thing is that I just checked pending request, and there are none, so I don't know where that mail actually is:-)). But it got accepted on the main list.
In short, Ryo reported he made a query around 11 am. Just after that query, there were no answer from the server for about 5 mn. Shortly after the server finally answered, he made another one, and immediately after he made that query, the server stopped answering...for about 9 hours
these are the queries he made
select l.cur_title, r.cur_title from cur l inner join links lnk on binary l.cur_title = binary lnk.l_from inner join cur r on binary r.cur_title = binary lnk.l_to where r.cur_text like '%#redirect%'
then simply
select count( * ) from links
could this be an explanation ? or not ?
__________________________________________________ Do you Yahoo!? Yahoo! Platinum - Watch CBS' NCAA March Madness, live on your desktop! http://platinum.yahoo.com
(Anthere anthere6@yahoo.com):
select l.cur_title, r.cur_title from cur l inner join links lnk on binary l.cur_title = binary lnk.l_from inner join cur r on binary r.cur_title = binary lnk.l_to where r.cur_text like '%#redirect%'
"where ... like '%..." is a no-no, especially on a join, as it will cause a full search of every record.
On Wed, Mar 26, 2003 at 03:00:43PM -0600, Lee Daniel Crocker wrote:
(Anthere anthere6@yahoo.com):
select l.cur_title, r.cur_title from cur l inner join links lnk on binary l.cur_title = binary lnk.l_from inner join cur r on binary r.cur_title = binary lnk.l_to where r.cur_text like '%#redirect%'
"where ... like '%..." is a no-no, especially on a join, as it will cause a full search of every record.
Note that if we were running PostgreSQL, we could set timeout limits on a per user basis. :P
On Wed, 26 Mar 2003, Nick Reinking wrote:
Note that if we were running PostgreSQL, we could set timeout limits on a per user basis. :P
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
-- brion vibber (brion @ pobox.com)
On Wed, Mar 26, 2003 at 01:18:08PM -0800, Brion Vibber wrote:
On Wed, 26 Mar 2003, Nick Reinking wrote:
Note that if we were running PostgreSQL, we could set timeout limits on a per user basis. :P
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
Well, if I had a machine beefy enough to run a copy of the en Wikipedia, I'd be happy to do so. Has the second box come in? Maybe we can throw it on both, and have a little bakeoff?
(Nick Reinking nick@twoevils.org): On Wed, Mar 26, 2003 at 01:18:08PM -0800, Brion Vibber wrote:
On Wed, 26 Mar 2003, Nick Reinking wrote:
Note that if we were running PostgreSQL, we could set timeout limits on a per user basis. :P
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
Well, if I had a machine beefy enough to run a copy of the en Wikipedia, I'd be happy to do so. Has the second box come in? Maybe we can throw it on both, and have a little bakeoff?
It doesn't take much. My test setup at home is old Compaq with a 400Mhz K6 and an 8Gb drive.
On Wed, 26 Mar 2003, Nick Reinking wrote:
On Wed, Mar 26, 2003 at 01:18:08PM -0800, Brion Vibber wrote:
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
Well, if I had a machine beefy enough to run a copy of the en Wikipedia, I'd be happy to do so. Has the second box come in? Maybe we can throw it on both, and have a little bakeoff?
Feel free to experiment with one of the smaller databases to start with.
We've a wide distribution of shapes and sizes. :) You can start small 'n' easy with Danish, for instance, or try French for a reasonable intermediate size.
-- brion vibber (brion @ pobox.com)
Brion Vibber wrote:
On Wed, 26 Mar 2003, Nick Reinking wrote:
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
Adapting the software IMHO is not the main problem, what about all those sysop's depending on e.g. mysql syntax? And I think there are more open questions. But soon after getting the latex conversion at work, I start testing if I can gamble with wikipedia at home, and this requires wikipedia to run on postgresql - no way using mysql on my home system ;)
Smurf
On Wed, Mar 26, 2003 at 10:40:01PM +0100, Thomas Corell wrote:
Brion Vibber wrote:
On Wed, 26 Mar 2003, Nick Reinking wrote:
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
Adapting the software IMHO is not the main problem, what about all those sysop's depending on e.g. mysql syntax? And I think there are more open questions. But soon after getting the latex conversion at work, I start testing if I can gamble with wikipedia at home, and this requires wikipedia to run on postgresql - no way using mysql on my home system ;)
I don't think the syntax is that radically different. Most of what people do are simple SELECTS which should be pretty much the same across every database, ever.
On Wed, 26 Mar 2003, Thomas Corell wrote:
Brion Vibber wrote:
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
Adapting the software IMHO is not the main problem, what about all those sysop's depending on e.g. mysql syntax? And I think there are more open questions.
Well, if we were to try to run the software on postgresql it would obviously be very important to adapt the software to work on postgresql. :)
However it's less clear that it would be particularly advantageous to use postgresql.
That's why I keep hoping that the people who pop up and say "hey, this would be a lot better with postgresql" can quantify the claim by making the software work with postgresql and actually comparing performance with the same operations on the same dataset on the same machine -- things we can't do until the software is able to run with postgresql.
-- brion vibber (brion @ pobox.com)
Brion Vibber wrote:
That's why I keep hoping that the people who pop up and say "hey, this would be a lot better with postgresql" can quantify the claim by making the software work with postgresql and actually comparing performance with the same operations on the same dataset on the same machine -- things we can't do until the software is able to run with postgresql.
Well, there are 2 ways, as usual. The fast way is to use the scripts coming with pg to convert tables. There are even tools to convert queries. But this will give a result you don't want to test performence with. The second one is to set up tables and views and indices as requiered by wikipedia, to get the best performance.
I think there have to be one step in the middle, the step Lee mentioned: all sql - commands in the wiki php code should replaced by functions, included from sql.inc or so. Then a change of database will affect one file, and not nearly all. I looked a short look at the sql-tables last weekend, just for fun, and at the different types mysql and pg using. Unfortunaly it is very hard to get the structure of how tables are used, if the queries are spreaded so much.
But, I will start porting the tables as soon I found some spare time. Lee, is there a loctation where I can get these mysql statements you isolated?
Smurf
(Thomas Corell T.Corell@t-online.de): But, I will start porting the tables as soon I found some spare time. Lee, is there a loctation where I can get these mysql statements you isolated?
All database access in the system should go through the functions in DatabaseFunctions.php. The only one that might be tricky to implement is wfInsertID(), which currently depends on a MySQL extension for auto-increment fields. But most databases have some similar feature.
Lee Daniel Crocker wrote:
All database access in the system should go through the functions in DatabaseFunctions.php. The only one that might be tricky to implement is wfInsertID(), which currently depends on a MySQL extension for auto-increment fields. But most databases have some similar feature.
Ok, I will try to understand what is the extention they made, first look at the function in the mysql docs did not give me a hint. Possibly I miss at midnight the reason why I need to know this number and what extention is handled with it. But currval(seq-name) must be similar.
Smurf
(Thomas Corell T.Corell@t-online.de):
Ok, I will try to understand what is the extention they made, first look at the function in the mysql docs did not give me a hint. Possibly I miss at midnight the reason why I need to know this number and what extention is handled with it. But currval(seq-name) must be similar.
The MySQL extension is the availability of auto-increment integer fields. They're handy when you need an integer key value that's guaranteed to be unique, but you don't care what it is. Without the extension, you can do that by keeping a "last used ID" filed value in a separate table and updating it every time you insert. But with MySQL, you just go ahead and do the insert, and then ask it "Oh, by the way, what integer value did you just assign to the auto- increment field of the record I just inserted?"
-- Lee Daniel Crocker lee@piclab.com http://www.piclab.com/lee/ "All inventions or works of authorship original to me, herein and past, are placed irrevocably in the public domain, and may be used or modified for any purpose, without permission, attribution, or notification."--LDC
Lee Daniel Crocker wrote:
(Thomas Corell T.Corell@t-online.de):
Ok, I will try to understand what is the extention they made, first look at the function in the mysql docs did not give me a hint. Possibly I miss at midnight the reason why I need to know this number and what extention is handled with it. But currval(seq-name) must be similar.
The MySQL extension is the availability of auto-increment integer fields. They're handy when you need an integer key value that's guaranteed to be unique, but you don't care what it is. Without the extension, you can do that by keeping a "last used ID" filed value in a separate table and updating it every time you insert. But with MySQL, you just go ahead and do the insert, and then ask it "Oh, by the way, what integer value did you just assign to the auto- increment field of the record I just inserted?"
Oh, they call that an extention. Well. There's a serial/serial8 type in Postgres, which implicitly creates a sequence and you can use the functions nextval() and currval(). So a definition of e.g. "cur_id serial8 PRIMARY KEY" should handle this.
Smurf
On Thu, Mar 27, 2003 at 06:36:29AM +0100, Thomas Corell wrote:
Lee Daniel Crocker wrote:
(Thomas Corell T.Corell@t-online.de):
Ok, I will try to understand what is the extention they made, first look at the function in the mysql docs did not give me a hint. Possibly I miss at midnight the reason why I need to know this number and what extention is handled with it. But currval(seq-name) must be similar.
The MySQL extension is the availability of auto-increment integer fields. They're handy when you need an integer key value that's guaranteed to be unique, but you don't care what it is. Without the extension, you can do that by keeping a "last used ID" filed value in a separate table and updating it every time you insert. But with MySQL, you just go ahead and do the insert, and then ask it "Oh, by the way, what integer value did you just assign to the auto- increment field of the record I just inserted?"
Oh, they call that an extention. Well. There's a serial/serial8 type in Postgres, which implicitly creates a sequence and you can use the functions nextval() and currval(). So a definition of e.g. "cur_id serial8 PRIMARY KEY" should handle this.
Or just serial, which would be considerably faster than serial8, unless you think that you will end up with more than 2147483648 rows (2^31).
Nick Reinking wrote:
On Thu, Mar 27, 2003 at 06:36:29AM +0100, Thomas Corell wrote:
Oh, they call that an extention. Well. There's a serial/serial8 type in Postgres, which implicitly creates a sequence and you can use the functions nextval() and currval(). So a definition of e.g. "cur_id serial8 PRIMARY KEY" should handle this.
Or just serial, which would be considerably faster than serial8, unless you think that you will end up with more than 2147483648 rows (2^31).
Only to meassure my expectations for wikipedia ;) But keep in mind that reusing serials is not part of the pg concept, but as far as I know mysql does. E.g. deleting articles may spend a serial.
Smurf
On Thu, Mar 27, 2003 at 06:07:04PM +0100, Thomas Corell wrote:
Nick Reinking wrote:
On Thu, Mar 27, 2003 at 06:36:29AM +0100, Thomas Corell wrote:
Oh, they call that an extention. Well. There's a serial/serial8 type in Postgres, which implicitly creates a sequence and you can use the functions nextval() and currval(). So a definition of e.g. "cur_id serial8 PRIMARY KEY" should handle this.
Or just serial, which would be considerably faster than serial8, unless you think that you will end up with more than 2147483648 rows (2^31).
Only to meassure my expectations for wikipedia ;) But keep in mind that reusing serials is not part of the pg concept, but as far as I know mysql does. E.g. deleting articles may spend a serial.
Fair enough, I do believe you're right. Still, deleted articles are not terribly common, and I think we should design this with maximum performance in mind. If we hit some odd 1.8B articles in the future, we can always figure out a way to change it to serial8. :)
Speaking of maximum performance... I have a question concerning our implementation. What will be more important in the future, clean and efficient code, or backwards compatibility? Especially when you consider the reverse_timestamp hacks everywhere that won't be needed in MySQL4 (or PostgreSQL), I would think that we should just drop MySQL3 support (especially considering that it is easy to upgrade, and nobody will be using it in a year or two). But, maybe I'm crazy - anybody else have any comments?
(Nick Reinking nick@twoevils.org):
Speaking of maximum performance... I have a question concerning our implementation. What will be more important in the future, clean and efficient code, or backwards compatibility? Especially when you consider the reverse_timestamp hacks everywhere that won't be needed in MySQL4 (or PostgreSQL), I would think that we should just drop MySQL3 support (especially considering that it is easy to upgrade, and nobody will be using it in a year or two). But, maybe I'm crazy - anybody else have any comments?
Depends on what you want compatibility with. The internal workings of the code are completely fair game as far as I'm concerned. MySQL 4 will get rid of the reverse-timestamp hack, and if we want to use Postgres or something else, we can. The "public interface" to the wiki is the URLs of various pages. Those should change as little as possible. But how they are implemented under the scenes is not a "compatibility" issue at all--hell, the whole thing might be replaced with Java servlets and an object database for all I care.
That's the whole point of client-server separation after all--how the server does its job is none of the client's business, only the fact that it does, in fact, meet its obligations. That's what I'm writing the test suite for.
On Thu, Mar 27, 2003 at 01:07:12PM -0600, Lee Daniel Crocker wrote:
(Nick Reinking nick@twoevils.org):
Speaking of maximum performance... I have a question concerning our implementation. What will be more important in the future, clean and efficient code, or backwards compatibility? Especially when you consider the reverse_timestamp hacks everywhere that won't be needed in MySQL4 (or PostgreSQL), I would think that we should just drop MySQL3 support (especially considering that it is easy to upgrade, and nobody will be using it in a year or two). But, maybe I'm crazy - anybody else have any comments?
Depends on what you want compatibility with. The internal workings of the code are completely fair game as far as I'm concerned. MySQL 4 will get rid of the reverse-timestamp hack, and if we want to use Postgres or something else, we can. The "public interface" to the wiki is the URLs of various pages. Those should change as little as possible. But how they are implemented under the scenes is not a "compatibility" issue at all--hell, the whole thing might be replaced with Java servlets and an object database for all I care.
That's the whole point of client-server separation after all--how the server does its job is none of the client's business, only the fact that it does, in fact, meet its obligations. That's what I'm writing the test suite for.
Well, I was thinking about people running the Wikipedia code (other than Wikipedia itself). They will have MySQL3 databases that require the reverse_timestamp hack - will we continue to keep this cruft in the Wikipedia code to support those people who don't want to migrate their databases, or will just clean it up and tell them to migrate their database, or stay with the old code (which is perfectly functionable, mostly).
Well, I was thinking about people running the Wikipedia code (other than Wikipedia itself). They will have MySQL3 databases that require the reverse_timestamp hack - will we continue to keep this cruft in the Wikipedia code to support those people who don't want to migrate their databases, or will just clean it up and tell them to migrate their database, or stay with the old code (which is perfectly functionable, mostly).
I, personally, am not the least bit interested in making the wikipedia code easy to use for others. It's GPL, and open, so others can do with it what they like, but my efforts will be 100% toward making Wikipedia work, and nothing else. If that inconveniences others who might have chosen to use the code, well, that's was their choice.
I do, on the other hand, concede that the ability for others to install and use the code does help the code evolve; the more eyeballs on it, the better. So I am willing to make a few concessions in that direction, but only insofar as they serve the primary goal, which is Wikipedia.
On Thu, Mar 27, 2003 at 02:35:55PM -0600, Lee Daniel Crocker wrote:
Well, I was thinking about people running the Wikipedia code (other than Wikipedia itself). They will have MySQL3 databases that require the reverse_timestamp hack - will we continue to keep this cruft in the Wikipedia code to support those people who don't want to migrate their databases, or will just clean it up and tell them to migrate their database, or stay with the old code (which is perfectly functionable, mostly).
I, personally, am not the least bit interested in making the wikipedia code easy to use for others. It's GPL, and open, so others can do with it what they like, but my efforts will be 100% toward making Wikipedia work, and nothing else. If that inconveniences others who might have chosen to use the code, well, that's was their choice.
I do, on the other hand, concede that the ability for others to install and use the code does help the code evolve; the more eyeballs on it, the better. So I am willing to make a few concessions in that direction, but only insofar as they serve the primary goal, which is Wikipedia.
Fair enough. Big smileys from me. :)
Oh, they call that an extention. Well. There's a serial/serial8 type in Postgres, which implicitly creates a sequence and you can use the functions nextval() and currval(). So a definition of e.g. "cur_id serial8 PRIMARY KEY" should handle this.
Or just serial, which would be considerably faster than serial8, unless you think that you will end up with more than 2147483648 rows (2^31).
But the issue is not just the database--it's how to implement the wfInsertID() function in the wiki code. PHP's Postgres functions don't have anything like it, so you'll have to do something like get_last_oid() and then retrieve the serial field value by OID.
Lee Daniel Crocker wrote:
But the issue is not just the database--it's how to implement the wfInsertID() function in the wiki code. PHP's Postgres functions don't have anything like it, so you'll have to do something like get_last_oid() and then retrieve the serial field value by OID.
Why I think I misunderstood something? Please tell me what's wrong.
The wfInsertID() funktion is used to get a unique autoincremented number in mySQL. And PHP has a function to return this number to you. I would now do the pg implementation by adding a 'serial' type to the table. Usually I don't do anything with this value, because pg will handle the autoincrement. Normaly mysql should do the same. If you for some reasons will know which value currently is used, you can get this in pg by the function currval(sequence_name) and the next value you can get by nextval(sequence_name).
Current PHP: function wfInsertId() {return mysql_insert_id( wfGetDB() );}
with pg: function wfInsertId() { return pq_query( wfGetDB(), "select currval('sequence_name');"); }
But usually if you e.g. fetching the data to display an articel you will get the serial as well.
By the way: using OID's is bad, very bad. Thats an internal value, noboby should use.
Smurf
(Brion Vibber vibber@aludra.usc.edu): On Wed, 26 Mar 2003, Nick Reinking wrote:
Note that if we were running PostgreSQL, we could set timeout limits on a per user basis. :P
Note that if someone wants us to run PostgreSQL, they should put their code where their mouth is and adapt the software to run with it. :)
As I said earlier, MySQL 4.0.12 is the first thing I want to experiment with; Postgres might well be the second, and I already isolated all the MySQL-isms of the code into one place so it shouldn't be too hard to adapt.
On Wed, Mar 26, 2003 at 12:54:03PM -0800, Anthere wrote:
--- Nick Reinking nick@twoevils.org wrote:
On Wed, Mar 26, 2003 at 01:53:29PM -0600, Lee Daniel Crocker wrote:
(Anthere anthere6@yahoo.com):
Say Lee, the last connexion to the french wiki
was
done at 11:21 am It was down from 11:21 to about a couple of
minutes
ago (that is 20:23) Between these two times (so ... 9 hours...), absolutely NO connexions were possible I just checked ...was the same on the en
apparently
so....which traffic ? from where ? from who ?
any idea
Well, that's the first time I'd heard that. And
there's
only one server: just because there's no traffic
on one of
the many wikis doesn't mean the server isn't busy
as hell.
The only message I got this morning was "the wiki
is down".
That didn't tell me anything, so I logged on and
saw that
the CPU load was maxed out, and that there were
hundreds
of active connections. Looked like regular traffic
to me.
I dunno Lee The wiki has been horribly slow for the past 2 weeks, and many editors have already gave up editing because of this Today, it was just as if wiki didnot existed at all during 9 hours, which were full day hours And not only was there no response from the server at all, but no response on the mailing list either. That is a bit disturbing I would say. In short, if 13/14 days it takes about 1 mn to respond for each page during day time, and the 14th day wiki doesnot exist at all, well, it won't be long for anybody to just give up
Besides, I checked on the english wiki, and I saw there were no edits in the recent changes log for many hours, so you can hardly say it is no response from "one of the numerous wiki". This was a general matter. So, we wonder.
It seems to be responding better now after the
kick, but
I'm reluctant to restart the server (which
involves breaking
all active connections) unless I get a more
specific report
than "the wiki is down".
Well, hard to expect much more than that from a user who doesn't have a login to the server itself. By the way, my guess is that some query, or perhaps some quirk, had caused MySQL to lock up. All of the apache processes were likely in a blocked state, waiting for MySQL to respond. Just because there are lots of processes running, doesn't mean they're actually responding.
-- Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN
Nick, did you read that message I forwarded several hours ago from Ryo ? I was on my old email address when I did the forward, and the wikitech list refused it (the weird thing is that I just checked pending request, and there are none, so I don't know where that mail actually is:-)). But it got accepted on the main list.
In short, Ryo reported he made a query around 11 am. Just after that query, there were no answer from the server for about 5 mn. Shortly after the server finally answered, he made another one, and immediately after he made that query, the server stopped answering...for about 9 hours
these are the queries he made
select l.cur_title, r.cur_title from cur l inner join links lnk on binary l.cur_title = binary lnk.l_from inner join cur r on binary r.cur_title = binary lnk.l_to where r.cur_text like '%#redirect%'
then simply
select count( * ) from links
could this be an explanation ? or not ?
No, I didn't get the email. As far as I know, it didn't make it to the list. I'm guessing that maybe he pushed the machine beyond what it could handle, and perhaps MySQL just shat all over itself. To be fair, it is hard to expect any software to run perfectly under low resource situations. Perhaps an upgrade to a later kernel might help, but really the problem is somebody using too much of a finite resource.
wikitech-l@lists.wikimedia.org