Toolserver-l February 2013

toolserver-l@lists.wikimedia.org

36 participants
34 discussions

Short downtime for s2 today
by DaB. 22 Feb '13

22 Feb '13

Hello all, to update a package with a security-problem, I have to restart z-dat-s2-b today. This will cause sql-s2-user and sql-s2-rr to be down. The downtime wll start at TODAY, 19:20 UTC and should not more than a few minutes. Sincerely, DaB. -- Userpage: [[:w:de:User:DaB.]] — PGP: 0x2d3ee2d42b255885

3 6

Restart of mysql on cassia (s5)
by DaB. 21 Feb '13

21 Feb '13

Hello all, to re-enable replication on cassia (sql-s5) I have to filter out wikikdata- data. This requires a restart of the mysql-process. This restart will happen TODAY, 20:00 UTC. The downtime of s5 will be ~30 minutes. Sincerely, DaB. -- Userpage: [[:w:de:User:DaB.]] — PGP: 0x2d3ee2d42b255885

1 1

sql-s5 (dewiki) will be read-only tomorrow
by DaB. 21 Feb '13

21 Feb '13

Hello all, s5 (dewiki) is the next cluster for re-setup. I already got a fresh dump from the WMF and imported it too on a server. The next step is to move the user- databases over. For this I have to switch s5 into read-only. The read-only- phase will start at Tomorrow, Thursday after 13:00 UTC. s5 carries a few very big user-databases, so I think the movement will take more an 1 day. Below is a list of the biggest user-databases, maybe their owners can take a look if some clean-up is possible? Unfortunately s5 can not stay at the new place because the server belongs to OSM. So a second read-only-phase will happen in near future (that one should be a bit faster because I will just rsync files around instead of importing). Sincerely, DaB. 1.2G u_lvova_golem_s2_zh 1.3G u_hroest 1.3G u_rdf_bots 1.6G u_multichill_commons_categories_p 1.7G u_lvova_golem_s5_de 1.8G u_mauro742 2.7G u_dschwen 20G p_merlbot_dewiki_lists 51G p_globalusage_weekly_p -- Userpage: [[:w:de:User:DaB.]] — PGP: 0x2d3ee2d42b255885

3 4

Wikidata-moving
by DaB. 20 Feb '13

20 Feb '13

Hello all, it looks like the WMF-techs are moving wikidata to cluster s5 at the moment (without announcement of course). I'm quite sure that this will break our wikidata-replication; so after my sleep I will look how the situation will be and than probably re-import wikidata everywhere (wikidata is still quite small at the moment so that's still possible). So expect the wikidata replag to rise during the next hours. Sincerely, DaB. -- Userpage: [[:w:de:User:DaB.]] — PGP: 0x2d3ee2d42b255885

2 1

Re: [Toolserver-l] Visibility of tools
by Tim Landscheidt 20 Feb '13

20 Feb '13

Silke Meyer <silke.meyer(a)wikimedia.de> wrote: > I was thinking about the list of tools a bit more... In my opinion, this > collection of tools should be more visible to the world! There is so > much work and knowledge in there and partially you really have to dig to > find them... > There is this plan to create/improve Labs/Tool Labs so that tools can be > shared with others more easily. So one very important thing is to *know* > about existing tools. > How about dragging these tools into the spotlight? (The ones on the > toolserver as well as the ones already living in Labs.) The other day I > heard the proposal from a community member to have featured tools in a > visible place, e.g. a "tool of the month" or even of the week. I like > the idea! It could shortly present a tool, what it does, where to find > it and the person(s) behind. > What do you think of it? The first place that came to my mind where The > Signpost and/or German language Kurier where we could suggest this > topic. I'll be afk for the next few days and I'd be glad to read some > opinions when back. :) I've suggested a while back (https://jira.toolserver.org/browse/TS-1271) to import the old Toolserver blog/comments to blog.wikimedia.de and then post status and ongoing project reports there. It has a different audience, probably more (potential) donors :-), but showing them what "some server in Amsterdam" is used for isn't a bad idea IMHO. amaranth needs to be fixed before that, though :-). Tim

4 3

New Rule: SGE-constraint for bots
by DaB. 20 Feb '13

20 Feb '13

Hello all, today I discovered (thanks to the mailing-list) that a few users have run several bot-instances in parallel on willow. I'm sure that these people did it by mistake, but it is annoying nevertheless and it is easy to fix: Use SGE. The problem is that I wrote several of eMails about "use SGE!" already and somehow it did not work as good as it should (if you converted you stuff already: thank you and you can stop to read here ;-)). I understand that we all are busy with our lives and Wikipedia and that we all love to "do it right…later", but as you know that resources of the toolserver are limited. So I hereby declare the following new rule: All bots have to run by SGE. A bot is every program or script that makes changes at a Wikimedia project. It does not matter if the bot runs periodically or continuous. The only exclusions are a.) interactive bots, b.) bots that can't run by SGE yet and c.) if you start a bot by hand for testing (no screen, no cron, no while). The rule will become active at Sunday, 10. February 2013. Exception b is almost NEVER the case, if it runs on a shell it is VERY likely that it can run by SGE. Some time ago I wrote a simple SGE-how-to at [1]. Maybe you all can take a look and correct things and make things more clear. In very most cases the using of SGE IS easy. Sincerely, DaB. [1] https://wiki.toolserver.org/view/SGE_for_beginners -- Userpage: [[:w:de:User:DaB.]] — PGP: 2B255885

5 10

List of tools on the toolserver - Help needed
by Silke Meyer 19 Feb '13

19 Feb '13

Hey all! I am impressed! Wondering what tools are actually running on the toolserver I started a list. It is long! You can admire it here: http://www.mediawiki.org/wiki/Toolserver/List_of_Tools For sure, it is incomplete as I used the toolserver wiki and jira to find information. In the light of building Tool Labs to become a platform the tools can run on, we need more information about the tools. So here is what I ask all tool developers for: * Please check if your tools are on the list (correctly). * Please fill in your software dependencies, data dependencies, use patterns (are they running continously? or webservices? Batch runs? etc.), the license. * If you have not given your software an explicit license, please note that only free software can migrate to Labs. Consider putting your stuff under a free license. * If you have already migrated your bot or tool to Labs and it is in the list, please say so in the last column "status". The same if a mentioned tool is abandoned and shall not move. Thanks for your help! Cheers, Silke -- Silke Meyer Systemadministratorin und Projektassistenz Wikidata Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin Tel. (030) 219 158 260 http://wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

6 8

Block mail from dysgo.org?
by Christopher David Howie 17 Feb '13

17 Feb '13

We've been receiving messages from this domain at unblock(a)toolserver.org and they appear to be related to this: <http://www.reddit.com/r/WTF/comments/l44r1/i_just_got_this_email_at_work_i_…>. Viral advertising for some film. In reality, it's a message with a crapload of images attached serving no purpose for us. Can we just block this whole domain from sending mail to toolserver accounts? It's a nuisance, and the messages are quite large. -- Chris Howie http://www.chrishowie.com http://en.wikipedia.org/wiki/User:Crazycomputers If you correspond with me on a regular basis, please read this document: http://www.chrishowie.com/email-preferences/ PGP fingerprint: 2B7A B280 8B12 21CC 260A DF65 6FCE 505A CF83 38F5 ------------------------------------------------------------------------ IMPORTANT INFORMATION/DISCLAIMER This document should be read only by those persons to whom it is addressed. If you have received this message it was obviously addressed to you and therefore you can read it. Additionally, by sending an email to ANY of my addresses or to ANY mailing lists to which I am subscribed, whether intentionally or accidentally, you are agreeing that I am "the intended recipient," and that I may do whatever I wish with the contents of any message received from you, unless a pre-existing agreement prohibits me from so doing. This overrides any disclaimer or statement of confidentiality that may be included on your message.

3 3

Coren joining WMF Ops department as Tools Labs contractor
by Sumana Harihareswara 16 Feb '13

16 Feb '13

Welcome, Coren (Marc-Andre Pelletier); I look forward to working with you to help build Tools Labs and help those who'd like to move bots and webtools to the Wikimedia Labs infrastructure. You can look at his TODO list here: https://www.mediawiki.org/wiki/Wikimedia_Labs/TODO and his profile here: https://meta.wikimedia.org/wiki/User:MPelletier_%28WMF%29 Coren, thank you for agreeing to join us and help work on this. best, Sumana Harihareswara Engineering Community Manager Wikimedia Foundation -------- Original Message -------- Subject: [Engineering] Engineering Announcement: Marc-Andre Pelletier joins TechOps Date: Fri, 15 Feb 2013 23:25:09 -0800 From: Ct Woo <ctwoo(a)wikimedia.org> To: Wikimedia developers <wikitech-l(a)lists.wikimedia.org>, Development and Operations Engineers <engineering(a)lists.wikimedia.org> All, I am pleased to announce Marc-Andre will be joining us as aTechnical Operations Engineer (contractor) starting this February 25, 2013. His primary focus will be to build up the Wikimedia Labs infrastructure and to assist the community developers to migrate their tools to that infrastructure, especially those residing on the Toolserver today. Marc-Andre is an active wikipedian and he is better known as 'Coren' on English Wikipedia and has been a volunteer editor since 2006 where he has served as administrator and arbitrator. He also always kept himself involved with the technical and procedural aspects of automated editing (so-called bots), having written and operated a copyright-violation detection bot for several years. Marc has been a Unix system administrator and occasional computer science instructor for 20+ years, in fields ranging from telecommunication to game development. He studied IT at École de Technologie Supérieure (Canada). Please join me to welcome him and you can find him on IRC (freenode.net) using the nick 'Coren'. Thanks. CT Woo

1 0

Cron errors at hawthorn for qcronsub
by Krinkle 15 Feb '13

15 Feb '13

I get a bunch of these every few weeks. Got them again today/yesterday. What's up? -- Krinkle Begin forwarded message: > From: root(a)toolserver.org (Cron Daemon) > Subject: Cron <krinkle@hawthorn> qcronsub -N dbbot_wm -m n -j y -b y -l arch='*' -l h_rt=INFINITY -l virtual_free=90M "$HOME/bots/dbbot-wm-start.sh" > Date: February 15, 2013 8:55:08 AM GMT+01:00 > To: krinkle(a)toolserver.org > > error: JSV stderr: Traceback (most recent call last): > error: JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> > error: JSV stderr: dom = minidom.parse(child_stdout) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse > error: JSV stderr: return expatbuilder.parse(file) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse > error: JSV stderr: result = builder.parseFile(file) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile > error: JSV stderr: parser.Parse(buffer, 0) > error: JSV stderr: xml.parsers.expat.ExpatError: syntax error: line 1, column 0 > Unable to run job: JSV stderr: Traceback (most recent call last): > JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> > JSV stderr: dom = minidom.parse(child_stdout) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse > JSV stderr: return expatbuilder.parse(file) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse > JSV stderr: result = builder.parseFile(file) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile > JSV stderr: parser.Parse(buffer, 0) > JSV stderr: xml.parsers.expat.ExpatError: syntax error: line 1, column 0 > JSV stderr is - xml.parsers.expat.ExpatError: syntax error: line 1, column 0. > Exiting. Begin forwarded message: > From: root(a)toolserver.org (Cron Daemon) > Subject: Cron <krinkle@hawthorn> qcronsub -N dbbot_wm -m n -j y -b y -l arch='*' -l h_rt=INFINITY -l virtual_free=90M "$HOME/bots/dbbot-wm-start.sh" > Date: February 15, 2013 1:45:09 AM GMT+01:00 > To: krinkle(a)toolserver.org > > error: JSV stderr: Traceback (most recent call last): > error: JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> > error: JSV stderr: dom = minidom.parse(child_stdout) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse > error: JSV stderr: return expatbuilder.parse(file) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse > error: JSV stderr: result = builder.parseFile(file) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile > error: JSV stderr: parser.Parse(buffer, 0) > error: JSV stderr: xml.parsers.expat.ExpatError: syntax error: line 1, column 0 > Unable to run job: JSV stderr: Traceback (most recent call last): > JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> > JSV stderr: dom = minidom.parse(child_stdout) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse > JSV stderr: return expatbuilder.parse(file) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse > JSV stderr: result = builder.parseFile(file) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile > JSV stderr: parser.Parse(buffer, 0) > JSV stderr: xml.parsers.expat.ExpatError: syntax error: line 1, column 0 > JSV stderr is - xml.parsers.expat.ExpatError: syntax error: line 1, column 0. > Exiting. Begin forwarded message: > From: root(a)toolserver.org (Cron Daemon) > Subject: Cron <krinkle@hawthorn> qcronsub -N dbbot_wm -m n -j y -b y -l arch='*' -l h_rt=INFINITY -l virtual_free=90M "$HOME/bots/dbbot-wm-start.sh" > Date: February 14, 2013 8:30:06 PM GMT+01:00 > To: krinkle(a)toolserver.org > > error: JSV stderr: Traceback (most recent call last): > error: JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> > error: JSV stderr: dom = minidom.parse(child_stdout) > error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse > Unable to run job: JSV stderr: Traceback (most recent call last): > JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> > JSV stderr: dom = minidom.parse(child_stdout) > JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse > JSV stderr is - File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse. > Exiting.

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Toolserver-l February 2013