Toolserver-l

toolserver-l@lists.wikimedia.org

1612 discussions

Reboot of nightshade and turnera tomorrow
by DaB. 27 Nov '12

27 Nov '12

Hello all, the HA-node turnera needs a reboot quite badly (it is very low on memory). It should be no problem to reboot it (because the other node (damiana) should overtake everything), but you can never be to sure enough. Therefor I hereby announce a downtime-window of 15 minutes for tomorrow Tuesday, 27th, 21:15 UTC. Normally you should notice nothing, but in worst case the toolserver will be down/unavailable until turnera rebooted completely. When the reboot of turnera is done, I will also reboot nightshade which has a very high load since we moved the stuff from thyme to rosemary. This also should not take longer than 15 minutes. Please make sure to have no open files on this box around the reboot. Sincerely, DaB. -- Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

1 1

Cannot log in
by Hazard-SJ 26 Nov '12

26 Nov '12

My access is being denied when I try to log into Willow. Does anyone else have this problem? Hazard-SJ

5 5

SGE queue waiting forever?
by Dr. Trigon 26 Nov '12

26 Nov '12

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello I have an issue with my jobs not executed or more precise queued since midnight: job-ID prior name user state submit/start at queue slots ja-task-ID - ----------------------------------------------------------------------------------------------------------------- 786288 0.50539 ircbot drtrigon r 11/17/2012 00:18:04 longrun-sol(a)willow.toolserver. 1 825178 0.50042 subster_me drtrigon qw 11/24/2012 00:06:04 1 825207 0.50039 subster_en drtrigon qw 11/24/2012 01:06:03 1 825212 0.50038 subster_nl drtrigon qw 11/24/2012 01:36:03 1 825228 0.50037 mainbot drtrigon qw 11/24/2012 02:36:03 1 825106 0.50035 subster_ar drtrigon qw 11/23/2012 21:06:03 1 825177 0.50000 maintenanc drtrigon qw 11/24/2012 00:06:04 1 825191 0.00000 subster_fr drtrigon qw 11/24/2012 00:36:04 1 ...what could be the issue here? Thanks and greetings DrTrigon -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://www.enigmail.net/ iEYEARECAAYFAlCw5PUACgkQAXWvBxzBrDA6IQCeNbMv7m11Pan5gJrrILATo3q6 m4EAnRX9gZR9uDi7nFSlywJlLzWOEhin =gLQA -----END PGP SIGNATURE-----

5 9

Anyone else having TS trouble?
by Magnus Manske 25 Nov '12

25 Nov '12

Started a few minutes ago. getting alternating errors on reloading a page; 500 (on a static HTML page, not CGI, mind you!), "This webpage is not available: The connection to toolserver.org was interrupted.", 404.

5 5

Public DB status down
by DeltaQuad Wikipedia 18 Nov '12

18 Nov '12

It appears that at least S1 and S4 are having issues getting to the database from the http side from projects like ACC ( http://toolserver.org/~acc/) and UTRS (http://toolserver.org/~unblock). I have two users from ACC verify that they have the same issues. Though through SSH I am able to clearly access both databases (using "mysql p_(database name)"). Anyone having this issue, or am I chasing ghosts (or worse our two projects are chasing ghosts). DeltaQuad English Wikipedia Administrator and Checkuser

3 2

thyme – postmortem
by DaB. 13 Nov '12

13 Nov '12

Hello all, just a little story of what happened today: As you know I planed to dump the user-databases of rosemary today to import them on thyme later. Around 12 o'clock CET I looked at the replag of thyme during a break and everything was fine. After my dinner I looked in my mails seeing an email from the OSM-guys complaining that their title-dir was away. As a background information: thyme carries the nfs-server of the user-store, title and munin – these are normally on hemlock, but because hemlock's SAN-card is broken we had to move them to another server. Short time later I spoke with Nosy at IRC about thyme. She told me that thyme is inaccessible by SSH. Few days ago we had discovered that thyme's serial- console was not working (we have put that on the datacenter-to-do-list). But without SSH and serial-console you can not even reboot a server neither access. Nosy had started to move the nfs-server from thyme to rosemary and we completed that together. Because of the missing user-store the script that checks your quota at login failed and login to linux-servers was hardly possible. I deleted the script on these boxes and added a quick&dirty-fix to puppet. These fix failed later making the login at the linux-boxes impossible for some time (even for roots). The switching of the user-store from thyme to rosemary made some problems on the userland-servers (because user-store was busy), but I think we fixed this. Maybe we have to reboot some boxes in the next days – I will send a mail if needed. Thyme also carried my wikidata-replication-program which failed too (so the replag of wikidata everywhere increased). I moved it to another server now. A strange thing is that the mysql-process on thyme is still running; even replication is working so the replag will not increase there. The next step is to reach Mark or someone from the datacenter to reboot thyme and then look where the problem was. Munin shows nothing abnormal. Just to let you know. Good night. Sincerely, DaB. -- Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

2 2

mdbtools
by Andre Koopal 13 Nov '12

13 Nov '12

Hi, Today I am working on a project where I need to convert an access database dump to something else (mysql likely). Most things I found on the web are not really suitable to script this process (it will need to run every month), but mdbtools looks to be promising. However I tried to compile that on toolserver all I got out of it are coredumps :-(. (Tried both on willow and nightshade). The ubuntu pacakge does work on my home server. Did anybody ever look into mdbtools and have it working? Is it perhaps possible to have it globally installed on toolserver, as it probably is usefull to some of the glam-projects as well? If people want an mdb file to test with, the file I need to convert is at /mnt/user-store/rce-nl-data. Regards, Andre

3 5

sql-s1-rr will be switched this evening
by DaB. 12 Nov '12

12 Nov '12

Hello all, I will switch sql-s1-rr from thyme to rosemary and back after 19:00 UTC this evening. You should notice no difference, but everything at sql-s1 will be a little bit slower while sql-s1-rr is on rosemary (and all queries running on thyme will be killed). The background of the switches is that I finally extracted the mysql-files from the wmf-binary dump and so I can re-setup thyme now with an unbroken version of s1. If everything works as planed I will handle rosemary during the next week too, so the hole s1-cluster will be unbroken again. Just to let you know. Sincerely, DaB. -- Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

2 6

502
by Magnus Manske 12 Nov '12

12 Nov '12

I suddenly see a lot of 502 error on the toolserver from the UK. Anyone else? (can't find a IRC client for OSX 10.6 quickly...)

6 6

S3 replication halted?
by Danny B. 11 Nov '12

11 Nov '12

It seems that s3 replication is halted. Did I miss any announcement or is that unplanned issue? In any case, I would be interested in ETA of catching-up to be up-to-date. Thank you. Danny B.

2 2

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Toolserver-l