Hello all,
few days ago my mom got a telephone-call by an English speaking person
babbling something about "servers" (she was not able to understand more and
terminated the line). I'm not saying that it was one of you, but the incident
inspired me for the following declaration: *NEVER* call me by phone! Even if
you are able to get a phone-number of me and even if the toolserver is
melting: There is no reason ever to call me – if you call me and I am able to
identify you I am going to delete your account. The only exception is Nosy who
has my cell-phone-number.
If you like to contact me try IRC. If I am not online try to contact another
root so he/she can solve the problem. If you can't find another root or it has
to be me and it is important, write my an eMail. If I am not able to check my
mails the chance is VERY high that I'm can not ssh to the toolserver anyway.
Sincerely,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hello all,
I successfully converted the commons-binary-dump from the WMF to the
traditional dump-format today. I will now re-import commons to everywhere we
have it already, and I will start with cassia (s2+s5)
Tomorrow, Friday, after 13:00 UTC.
During the re-import the commons-copy of cassia will be inaccessible. The
import will take some hours (depending on the load of the server) and to be
safe I say hereby that commons on cassia will be away for 24h.
Another thing: My personal eMail-Server will receive an hardware-update
starting Friday 17:00 UTC: So it could be hard to reach me by mail for a few
hours. In urgent cases, try to reach Nosy.
Sincerely,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hello all,
good news: We got a s4 and a s7-dump from the WMF, I transfer them to the
toolserver at the moment. Because the integration of Wikidata in hu/hewiki
will start soon AFAIK, I will begin with s7. Because the dumps are in the new
binary-format the actual "import" will be quite fast, the downtime of s7
should not be too long (at few hours at maximum, but I can't promise
anything). So I hereby declare a read-only-time and a downtime of s7
Tomorrow, starting 16:00 UTC.
The handling of s4 will take more time, because I have to convert the dump in
the traditional format first. I will notice you when downtimes are needed.
Sincerely,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hello,
tomorrow will be hardware maintenance for toolserver.
It will be between 11 am - 20 pm UTC.
Several hosts may be offline for a little while during this time - especially daphne (s4) and hyacinth (s3, s6 and s7).
They get some hardware replacements.
The OSM host ptolemy might go offline for new hardware, too.
Cheers
Marlen/nosy
Hello all,
Dispenser messaged me because the query
explain select * from enwiki_p.revision limit 1;
isn't working anymore. As far as I see that's caused by the recent mysql-
update. We need to patch this but it may take a few days and another mysql-
restart (will be announced separately).
You can follow the progress at [1].
Just to let you known.
Sincerely,
DaB.
[1] https://jira.toolserver.org/browse/TS-1585
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hello,
we will update the Solaris systems between 8pm and 10pm UTC on Sunday.
The servers (ortelius, willow, clematis, hawthorn, wolfsbane, damiana, turnera, ptolemy) may also be rebooted if recommended by the update.
Kind regards
Marlen / nosy
Hello,
MySQL needs a security update and we had none anyway for a while.
This means we will update MySQL on Friday 8pm - 10 pm UTC.
During this time the databases will restart at least once.
Kind regards
Marlen/nosy
Hello all,
I just got back from the general member meeting of Wikimedia Deutschland. As
you know I requested a decision about the future of the toolserver there. To
make it short: It doesn't went as well as I hoped. While the request itself
was accepted, it was changed in some important parts.
The main fear was that WMF could stop to provide us with fresh dumps and/or
replication in near future, making the toolserver more or less useless.
Although I learned from a participating WMF-board-member that no such board-
decision exists.
My request was changed in the following way: The WMF has to tell WMDE within 6
months how Wikilabs can replace the toolserver in the promised complete way.
If the answer is not satisfying, WMDE will develop a "Governance-Model" to
ensure the continuation of the toolserver. Different groups are invited into
this "Governance-Model" and it should be done until the end of 2013.
That sounds good on the first view, but there are 2 loop-holes: Nobody defined
what "complete" or "satisfying" is. In my eyes Wikilabs can not replace the
toolserver complete (in the way that all tools can move to there) and so the
answer can only be unsatisfying, but that's just a question of definition I
guess.
A second change was that the investment for the toolserver will be restricted
to the "necessary". While that is of course a matter of definition again I'm
sure that means "no new hardware if it is possible in any way".
To summarize this: In the best case we have to wait for 6 months until WMDE
officially learns that Wikilabs can not replace us, than wait for another 6
months until they will create their "Governance-Model" and in 2014 we get new
hardware.
In worst case we wait for 6 months and than WMDE and WMF agree that everything
is ok and we will never get any new hardware and somewhen the TS will shut
down (of course with the remaining tools that can not migrated to Wikilabs).
I can not imagine ways between both cases, but I'm sure they exists. In any
way we will get no (or nearly no) new hardware in 2013 – so we have to life
with that.
A good news is that the toolserver will get 3 new database-servers soon.
I have not decided yet if I will remain as root under this circumstances for
2013 – I will tell you my decision until next Sunday.
For now I will head to bed because I'm exhausted and disappointed. See you
tomorrow.
Sincerely,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hello all,
the HA-node turnera needs a reboot quite badly (it is very low on memory). It
should be no problem to reboot it (because the other node (damiana) should
overtake everything), but you can never be to sure enough. Therefor I hereby
announce a downtime-window of 15 minutes for tomorrow
Tuesday, 27th, 21:15 UTC.
Normally you should notice nothing, but in worst case the toolserver will be
down/unavailable until turnera rebooted completely.
When the reboot of turnera is done, I will also reboot nightshade which has a
very high load since we moved the stuff from thyme to rosemary. This also
should not take longer than 15 minutes. Please make sure to have no open files
on this box around the reboot.
Sincerely,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885
Hello all,
I will switch sql-s1-rr from thyme to rosemary and back after
19:00 UTC this evening.
You should notice no difference, but everything at sql-s1 will be a little bit
slower while sql-s1-rr is on rosemary (and all queries running on thyme will
be killed).
The background of the switches is that I finally extracted the mysql-files from
the wmf-binary dump and so I can re-setup thyme now with an unbroken version
of s1. If everything works as planed I will handle rosemary during the next
week too, so the hole s1-cluster will be unbroken again.
Just to let you know.
Sincerely,
DaB.
--
Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885