Pliny seems to be the cause of the current slowness, seems we're running low on memory again, although I can't say for sure why this is the case... (heavy swapping, low CPU util...)
procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 2 0 899764 12580 28732 803580 12 0 2000 640 695 1139 23 3 74 0 6 1 899772 10528 28464 803968 168 0 3916 48 881 1034 17 6 77 2 2 0 899772 10496 28492 804196 68 0 2856 140 896 1038 25 4 71 0 3 0 899772 12604 28500 801784 32 112 1652 504 968 1196 30 7 64 0 8 1 899772 10556 28524 802788 72 0 2028 408 704 1141 23 4 73 0 2 0 899772 10476 28532 802856 284 0 2552 72 821 1282 25 5 70 1 3 0 899772 12560 28512 800812 392 0 2512 420 891 1175 10 3 87 0 4 0 899772 10504 28532 802376 516 0 2852 160 789 1335 17 5 78 0 7 1 899772 10512 28560 803440 464 0 1880 440 592 869 15 6 79 0 15 3 899772 12100 28576 803244 0 0 52 1423 474 165275 1 11 88 0 6 1 899772 11700 28592 803620 20 0 396 293 549 399667 8 61 31 0 6 1 899772 12032 28624 805284 28 0 1704 792 621 1196 33 6 61 2 5 0 899764 10652 28652 806576 68 0 1372 116 659 700 7 4 89 1 5 0 899764 10552 28660 806728 148 0 1288 590 591 607 8 2 90 0 5 1 899764 10468 28700 806896 68 0 1232 472 710 856 5 5 89 0 1 0 899764 10536 28708 802908 128 4 2120 104 564 894 23 3 73 4 6 2 899764 10488 28704 799480 144 136 1164 979 552 723 15 4 80 0 3 0 899764 15916 28696 798008 308 0 1704 733 386 760 31 4 65
On Tue, Jul 29, 2003 at 02:27:30PM -0500, Nick Reinking wrote:
Pliny seems to be the cause of the current slowness, seems we're running low on memory again, although I can't say for sure why this is the case... (heavy swapping, low CPU util...)
procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 2 0 899764 12580 28732 803580 12 0 2000 640 695 1139 23 3 74 0 6 1 899772 10528 28464 803968 168 0 3916 48 881 1034 17 6 77 2 2 0 899772 10496 28492 804196 68 0 2856 140 896 1038 25 4 71 0 3 0 899772 12604 28500 801784 32 112 1652 504 968 1196 30 7 64 0 8 1 899772 10556 28524 802788 72 0 2028 408 704 1141 23 4 73 0 2 0 899772 10476 28532 802856 284 0 2552 72 821 1282 25 5 70 1 3 0 899772 12560 28512 800812 392 0 2512 420 891 1175 10 3 87 0 4 0 899772 10504 28532 802376 516 0 2852 160 789 1335 17 5 78 0 7 1 899772 10512 28560 803440 464 0 1880 440 592 869 15 6 79 0 15 3 899772 12100 28576 803244 0 0 52 1423 474 165275 1 11 88 0 6 1 899772 11700 28592 803620 20 0 396 293 549 399667 8 61 31 0 6 1 899772 12032 28624 805284 28 0 1704 792 621 1196 33 6 61 2 5 0 899764 10652 28652 806576 68 0 1372 116 659 700 7 4 89 1 5 0 899764 10552 28660 806728 148 0 1288 590 591 607 8 2 90 0 5 1 899764 10468 28700 806896 68 0 1232 472 710 856 5 5 89 0 1 0 899764 10536 28708 802908 128 4 2120 104 564 894 23 3 73 4 6 2 899764 10488 28704 799480 144 136 1164 979 552 723 15 4 80 0 3 0 899764 15916 28696 798008 308 0 1704 733 386 760 31 4 65
Interesting enough, it's only si that is high, so pliny has to read data in since he can't cache it any longer, but doesn't have to swap it out.
bi is also much higher than bo, the database can't keep all the necessary and requested data in memory.
With which intervall are the above figures gathered?
Regards,
JeLuF
On Tue, Jul 29, 2003 at 10:28:49PM +0200, Jens Frank wrote:
On Tue, Jul 29, 2003 at 02:27:30PM -0500, Nick Reinking wrote:
Pliny seems to be the cause of the current slowness, seems we're running low on memory again, although I can't say for sure why this is the case... (heavy swapping, low CPU util...)
procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 2 0 899764 12580 28732 803580 12 0 2000 640 695 1139 23 3 74 0 6 1 899772 10528 28464 803968 168 0 3916 48 881 1034 17 6 77 2 2 0 899772 10496 28492 804196 68 0 2856 140 896 1038 25 4 71 0 3 0 899772 12604 28500 801784 32 112 1652 504 968 1196 30 7 64 0 8 1 899772 10556 28524 802788 72 0 2028 408 704 1141 23 4 73 0 2 0 899772 10476 28532 802856 284 0 2552 72 821 1282 25 5 70 1 3 0 899772 12560 28512 800812 392 0 2512 420 891 1175 10 3 87 0 4 0 899772 10504 28532 802376 516 0 2852 160 789 1335 17 5 78 0 7 1 899772 10512 28560 803440 464 0 1880 440 592 869 15 6 79 0 15 3 899772 12100 28576 803244 0 0 52 1423 474 165275 1 11 88 0 6 1 899772 11700 28592 803620 20 0 396 293 549 399667 8 61 31 0 6 1 899772 12032 28624 805284 28 0 1704 792 621 1196 33 6 61 2 5 0 899764 10652 28652 806576 68 0 1372 116 659 700 7 4 89 1 5 0 899764 10552 28660 806728 148 0 1288 590 591 607 8 2 90 0 5 1 899764 10468 28700 806896 68 0 1232 472 710 856 5 5 89 0 1 0 899764 10536 28708 802908 128 4 2120 104 564 894 23 3 73 4 6 2 899764 10488 28704 799480 144 136 1164 979 552 723 15 4 80 0 3 0 899764 15916 28696 798008 308 0 1704 733 386 760 31 4 65
Interesting enough, it's only si that is high, so pliny has to read data in since he can't cache it any longer, but doesn't have to swap it out.
bi is also much higher than bo, the database can't keep all the necessary and requested data in memory.
With which intervall are the above figures gathered?
That is at a one second interval.
In your opinion, would an infusion of ram help?
If there's a programming problem that's chewing up the memory, then of course, adding more won't solve the problem. But if the size of the db, traffic factors, etc., make it seem like this is just the latest bottleneck, I will solve it very quickly.
We can probably order the ram and have Jason down there before the week is out.
Nick Reinking wrote:
Pliny seems to be the cause of the current slowness, seems we're running low on memory again, although I can't say for sure why this is the case... (heavy swapping, low CPU util...)
procs memory swap io system cpu r b w swpd free buff cache si so bi bo in cs us sy id 1 2 0 899764 12580 28732 803580 12 0 2000 640 695 1139 23 3 74 0 6 1 899772 10528 28464 803968 168 0 3916 48 881 1034 17 6 77 2 2 0 899772 10496 28492 804196 68 0 2856 140 896 1038 25 4 71 0 3 0 899772 12604 28500 801784 32 112 1652 504 968 1196 30 7 64 0 8 1 899772 10556 28524 802788 72 0 2028 408 704 1141 23 4 73 0 2 0 899772 10476 28532 802856 284 0 2552 72 821 1282 25 5 70 1 3 0 899772 12560 28512 800812 392 0 2512 420 891 1175 10 3 87 0 4 0 899772 10504 28532 802376 516 0 2852 160 789 1335 17 5 78 0 7 1 899772 10512 28560 803440 464 0 1880 440 592 869 15 6 79 0 15 3 899772 12100 28576 803244 0 0 52 1423 474 165275 1 11 88 0 6 1 899772 11700 28592 803620 20 0 396 293 549 399667 8 61 31 0 6 1 899772 12032 28624 805284 28 0 1704 792 621 1196 33 6 61 2 5 0 899764 10652 28652 806576 68 0 1372 116 659 700 7 4 89 1 5 0 899764 10552 28660 806728 148 0 1288 590 591 607 8 2 90 0 5 1 899764 10468 28700 806896 68 0 1232 472 710 856 5 5 89 0 1 0 899764 10536 28708 802908 128 4 2120 104 564 894 23 3 73 4 6 2 899764 10488 28704 799480 144 136 1164 979 552 723 15 4 80 0 3 0 899764 15916 28696 798008 308 0 1704 733 386 760 31 4 65
-- Nick Reinking -- eschewing obfuscation since 1981 -- Minneapolis, MN _______________________________________________ Wikitech-l mailing list Wikitech-l@wikipedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
On Tue, Jul 29, 2003 at 05:28:13PM -0700, Jimmy Wales wrote:
In your opinion, would an infusion of ram help?
If there's a programming problem that's chewing up the memory, then of course, adding more won't solve the problem. But if the size of the db, traffic factors, etc., make it seem like this is just the latest bottleneck, I will solve it very quickly.
We can probably order the ram and have Jason down there before the week is out.
That's what I would try first, it is probably the lowest hanging fruit. Also, moving the webservers on pliny into larousse would probably also help some (provided that larousse isn't terrible overloaded - it didn't seem like it was...)
Nick-
Pliny seems to be the cause of the current slowness, seems we're running low on memory again, although I can't say for sure why this is the case... (heavy swapping, low CPU util...)
What I notice is that whenever the Wikipedia is slow, I see several threads like this in the PROCESSLIST:
SELECT cur_id,cur_namespace,cur_title,cur_text FROM cur,searchindex WHERE cur_id=si_page AND ( (MATCH (si_title) AGAINST ('tautology')) ) AND cur_namespace IN (0) LIMIT 0, 20
usually with running times of several hundred seconds. This is the title search. The main title search is still disabled, but the Go button still runs one. There are no similar threads for the non-English wiki, which suggests (to me, at least) that something is seriously wrong with one of the concerned tables after the crash. Either that, or we're running into big scaling problems of the above query. I've already done a REPAIR TABLE SEARCHINDEX to no avail.
Regards,
Erik
Erik Moeller wrote:
usually with running times of several hundred seconds. This is the title search. The main title search is still disabled, but the Go button still runs one.
Why should the Go button do that? It seems to me that all the Go button needs to do is change spaces to underlines and redirect.
--Jimbo
Jimmy-
Erik Moeller wrote:
usually with running times of several hundred seconds. This is the title search. The main title search is still disabled, but the Go button still runs one.
Why should the Go button do that? It seems to me that all the Go button needs to do is change spaces to underlines and redirect.
The Go button is much smarter, and should be much smarter, than that. See http://www.wikipedia.org/wiki/Wikipedia:Go_button
Regards,
Erik
Oh, duh, I didn't realize all of that.
Where's the 'search' button? Gone?
What if I want to search for articles containing 'Thomas Jefferson', but more than just the [[Thomas Jefferson]] article?
Maybe I just overlooked that the main search has been removed for performance reasons right now?
Erik Moeller wrote:
Jimmy-
Erik Moeller wrote:
usually with running times of several hundred seconds. This is the title search. The main title search is still disabled, but the Go button still runs one.
Why should the Go button do that? It seems to me that all the Go button needs to do is change spaces to underlines and redirect.
The Go button is much smarter, and should be much smarter, than that. See http://www.wikipedia.org/wiki/Wikipedia:Go_button
Regards,
Erik _______________________________________________ Wikitech-l mailing list Wikitech-l@wikipedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l
Jimmy-
Oh, duh, I didn't realize all of that.
Where's the 'search' button? Gone?
What if I want to search for articles containing 'Thomas Jefferson', but more than just the [[Thomas Jefferson]] article?
Maybe I just overlooked that the main search has been removed for performance reasons right now?
It has been removed because ever since the crash, even minimal searches bog down the database server for reasons that I still don't understand. I suspect table corruption, but a REPAIR TABLE had no effect.
Regards,
Erik
wikitech-l@lists.wikimedia.org