Toolserver-l May 2013

toolserver-l@lists.wikimedia.org

46 participants
33 discussions

Encoding issue using SGE
by Hercule Hercule 23 May '13

23 May '13

Hello, I deleted one of my script, and try to recreate it. But I now have an encoding issue when I run it with SGE. I searched a solution during all this evening and can't find any idea to solve it. I hope one of you will be able to help me. I created the file /home/hercule/batch/LienFr.sh with this content : #-*- coding: utf-8 -*- #!bin/bash #! /usr/bin/python #$ -l h_rt=1:30:00 #$ -l virtual_free=500M #$ -l arch='*' #$ -m as #$ -o /home/hercule/logs/LienFr.log #$ -j y #$ -wd /home/hercule/ python /home/hercule/template.py "Lien" "Lien/Conversion automatique" -cat:"Page utilisant Lien pour un article existant" -summary:"[Bot] : transformation de liens avec le modèle {{Lien}} en lien interne, suite à la création de l'article correspondant" -assubst -pt:0 -always mutt -s "Fin de LienFr" hercule < /home/hercule/logs/LienFr.log When I simply run /home/hercule/batch/LienFr.sh in the terminal the result is correct : http://fr.wikipedia.org/w/index.php?title=Uttar_Pradesh&diff=93370887&oldid… But when I use the command line qcronsub -N LienFr /home/hercule/batch/LienFr.sh I have a problem with the summary : http://fr.wikipedia.org/w/index.php?title=Uttar_Pradesh&diff=93370948&oldid… Do you know how to solve this ? Regards Hercule

5 10

Patricia Pintilie <pintilieempire@gmail.com>
by Jeremy Baron 23 May '13

23 May '13

On May 23, 2013 7:30 AM, "Patricia Pintilie" <pintilieempire(a)gmail.com> wrote: > General chat • Re: Navigating Multiboot GRUB2 menu entries successfully. http://forum.porteus.org/viewtopic.php?t=2195&p=15042#p15042 Hi Patricia, Are you human? Please send me the square root of 16. Thanks! -Jeremy

2 1

SGE fail to run any of my tasks, they error out as "cgroup change of group failed"
by phil.el＠free.fr 23 May '13

23 May '13

[Asking here, as JIRA doesn't allow me to fill bug report (I'm delogged when submiting a bug)] Since the switch to a linux implementation of SGE none of my task can start sucecessfully A typical entry qcronsub -l arch=lx -l h_rt=INFINITY -l virtual_free=64M -notify $HOME/wsbot/wshocr end up with the creation of a file $HOME/wshocr.e* containing cgroup change of group failed then the task is killed. regards -- Phe

3 3

Introducing the new toolserver admin
by Silke Meyer 23 May '13

23 May '13

Hi all! Let me introduce Alexander Mette, the new toolserver admin who came to support DaB. and Nosy! Welcome! As soon as he is a bit more in touch with the toolserver, he would like to meet you in an office hour. (Will be announced separately.) You can reach him via e-mail as alexander.mette(a)wikimedia.de and on IRC as amette. Cheers, Silke -- Silke Meyer Internes IT-Management und Projektmanagement Toolserver Wikimedia Deutschland e.V. | Obentrautstr. 72 | 10963 Berlin Tel. (030) 219 158 260 http://wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/681/51985.

2 1

Two questions about pywikipedia
by Alex Brollo 18 May '13

18 May '13

As I told, I'm driving a pywikipedia bot (Alebot) into willow; I installed pywikipedia at $HOME, I set PYTHONPATH=/home/alebot/pywikipedia [,.....] into .environment file, and I am reviewing at my best python scripts to run them by qsub/qcronsub. Most of my scripts obviously need a statement: import wikipedia [,.....] My questions: 1. Do I really need to install pywikipedia into $HOME? 2. My scripts run happily outside SGE, but when I try to run them under SGE I get an error: ImportError: No module named wikipedia I didn't find doc for this ImportError (I suppose, PYTHONPATH is wrong). Where am I going wrong? Alex

3 3

SGE thinks hosts are overloaded while the latter are idling
by Tim Landscheidt 17 May '13

17 May '13

Hi, a "qstat -j" of a simple job yields inter alia: | scheduling info: queue instance "longrun-sol(a)willow.toolserver.org" dropped because it is temporarily not available | queue instance "short-sol(a)willow.toolserver.org" dropped because it is temporarily not available | queue instance "medium-lx(a)mayapple.toolserver.org" dropped because it is temporarily not available | queue instance "longrun3-sol(a)willow.toolserver.org" dropped because it is temporarily not available | queue instance "longrun2-sol(a)clematis.toolserver.org" dropped because it is disabled | queue instance "longrun2-sol(a)hawthorn.toolserver.org" dropped because it is disabled | queue instance "medium-sol(a)ortelius.toolserver.org" dropped because it is overloaded: np_load_short=0.791601 (= 0.391601 + 0.8 * 2.000000 with nproc=4) >= 0.75 | queue instance "medium-lx(a)yarrow.toolserver.org" dropped because it is overloaded: np_load_short=1.215000 (= 0.015000 + 0.8 * 6.000000 with nproc=4) >= 1.2 | queue instance "medium-lx(a)nightshade.toolserver.org" dropped because it is overloaded: np_load_short=1.227500 (= 0.127500 + 0.8 * 11.000000 with nproc=8) >= 1.2 | queue instance "medium-sol(a)wolfsbane.toolserver.org" dropped because it is overloaded: np_load_short=0.778613 (= 0.078613 + 0.8 * 7.000000 with nproc=8) >= 0.75 | queue instance "short-sol(a)wolfsbane.toolserver.org" dropped because it is overloaded: np_load_short=1.278613 (= 0.078613 + 0.8 * 12.000000 with nproc=8) >= 1.2 | queue instance "short-sol(a)ortelius.toolserver.org" dropped because it is overloaded: np_load_short=1.391601 (= 0.391601 + 0.8 * 5.000000 with nproc=4) >= 1.2 | queue instance "longrun-lx(a)yarrow.toolserver.org" dropped because it is overloaded: np_load_short=3.215000 (= 0.015000 + 0.8 * 16.000000 with nproc=4) >= 3.1 | queue instance "longrun-lx(a)nightshade.toolserver.org" dropped because it is overloaded: mem_free=-420765696.524288 (= 14098.726562M - 500M * 29.000000) <= 500 At the moment, we have /no/ jobs scheduled by SGE running. Meanwhile, the hosts are idling: | queuename qtype resv/used/tot. load_avg arch states | --------------------------------------------------------------------------------- | short-sol(a)ortelius.toolserver. B 0/0/8 1.52 sol-amd64 | --------------------------------------------------------------------------------- | short-sol(a)willow.toolserver.or B 0/0/8 -NA- sol-amd64 au | --------------------------------------------------------------------------------- | short-sol(a)wolfsbane.toolserver B 0/0/12 0.64 sol-amd64 | --------------------------------------------------------------------------------- | medium-lx(a)mayapple.toolserver. B 0/0/32 -NA- linux-x64 adu | --------------------------------------------------------------------------------- | medium-lx(a)nightshade.toolserve B 0/0/8 1.05 linux-x64 | --------------------------------------------------------------------------------- | medium-lx(a)yarrow.toolserver.or B 0/0/8 0.02 linux-x64 | --------------------------------------------------------------------------------- | longrun-lx(a)nightshade.toolserv BI 0/0/64 1.05 linux-x64 | --------------------------------------------------------------------------------- | longrun-lx(a)yarrow.toolserver.o BI 0/0/64 0.02 linux-x64 | --------------------------------------------------------------------------------- | longrun-sol(a)willow.toolserver. BI 0/0/64 -NA- sol-amd64 au | --------------------------------------------------------------------------------- | medium-sol(a)ortelius.toolserver B 0/0/4 1.52 sol-amd64 | --------------------------------------------------------------------------------- | medium-sol(a)wolfsbane.toolserve B 0/0/4 0.64 sol-amd64 | --------------------------------------------------------------------------------- | longrun2-sol(a)clematis.toolserv B 0/0/8 0.03 sol-amd64 d | --------------------------------------------------------------------------------- | longrun2-sol(a)hawthorn.toolserv B 0/0/8 0.23 sol-amd64 d | --------------------------------------------------------------------------------- | longrun3-sol(a)willow.toolserver B 0/0/4 -NA- sol-amd64 aduE I filed https://jira.toolserver.org/browse/TS-1650 on Monday to no avail so far. Tim

3 3

Running Pywikipedia (rewrite) bots on Tools Labs
by Russell Blau 17 May '13

17 May '13

For any users interested in migrating Pywikipedia-based bots from the Toolserver to the new Labs platform, or just interested in using Labs to run bots, I've written a page[1] describing the steps I took to get my bots running under the rewrite branch. If someone who uses trunk has gotten their bot(s) running on Labs, feel free to add instructions for that process, too. [1] https://wikitech.wikimedia.org/wiki/User:Russell_Blau/Using_pywikibot_on_La… -- Russell Blau russblau(a)imapmail.org

2 1

weird qcronsub errors (was: Output from "cron" command)
by Peter Körner 14 May '13

14 May '13

Hi Since a few days I'm getting weird errors when submitting tasks. My Cronjob calls "/home/mazder/public_html/replicate-sequences/update-submit.sh" which conains the following command: qcronsub -l h_rt=0:05:00 -l virtual_free=100M -l arch=* -l sql-user-m=1 -N mazder-replicate-sequences -m as -o '/home/mazder/public_html/replicate-sequences/sge' '/home/mazder/public_html/replicate-sequences/update-run.sh' Most of these calls produce the error below, which seems not to be an error in my code as I neither use xml nor python. Do you have any Idea what's going wrong? Peter -------- Original-Nachricht -------- Betreff: Output from "cron" command Datum: Tue, 14 May 2013 08:40:00 +0000 (UTC) Von: mazder(a)toolserver.org (mazder) An: mazder(a)toolserver.org Your "cron" job on clematis /home/mazder/public_html/replicate-sequences/update-submit.sh produced the following output: error: JSV stderr: Traceback (most recent call last): error: JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> error: JSV stderr: dom = minidom.parse(child_stdout) error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse error: JSV stderr: return expatbuilder.parse(file) error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse error: JSV stderr: result = builder.parseFile(file) error: JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile error: JSV stderr: parser.Parse(buffer, 0) error: JSV stderr: xml.parsers.expat.ExpatError: syntax error: line 1, column 0 Unable to run job: JSV stderr: Traceback (most recent call last): JSV stderr: File "/sge/GE/bin/sol-amd64/qjobtest", line 108, in <module> JSV stderr: dom = minidom.parse(child_stdout) JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/minidom.py", line 1915, in parse JSV stderr: return expatbuilder.parse(file) JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 930, in parse JSV stderr: result = builder.parseFile(file) JSV stderr: File "/opt/ts/python/2.7/lib/python2.7/site-packages/_xmlplus/dom/expatbuilder.py", line 207, in parseFile JSV stderr: parser.Parse(buffer, 0) JSV stderr: xml.parsers.expat.ExpatError: syntax error: line 1, column 0 JSV stderr is - xml.parsers.expat.ExpatError: syntax error: line 1, column 0. Exiting.

2 1

Cronie error
by Lars Aronsson 13 May '13

13 May '13

I have a cronietab job that now gives this error message: error: commlib error: can't connect to service (Connection refused) Unable to run job: unable to send message to qmaster using port 444 on host "damiana": got send error. Exiting. What does that mean, and how can I fix it? I have no idea what commlib or damiana are, but perhaps they are related to qsub? The command I tried to run (from cronie) was: qsub -N links-s7 -l sql-s7-user-readonly=1 -l h_rt=11:00:00 -l virtual_free=250M -l user_slot=5 linkstats.sh 7 Where linkstats.sh is a shell script in ~la2/ -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

2 3

Upgrade to Debian 7 next Sunday
by DaB. 12 May '13

12 May '13

Hello all. Today Debian 7 was released as the new stable version. I now that some of you are eager to upgrade so you can get new version of programs. We will not disappoint you and so for I announce a maintenance for SUNDAY, 12. May 12:00 UTC for the userland-servers. You can find the complete list of updates at [1], and a short summary below. If you notice a problem please leave a comment in the bug or at the mailing-list so the admins can look for a solution; please notice that most of the removals are just renamings – make sure to check the "New"-section also. Sincerely, DaB. Updates: bash 4.1 → 4.2 emacs 23 → 45 g++ 4.4 → 4.7 gcj 4.4 → 4.7 libboost 1.42 → 1.49 libc6 2.11 → 2.13 mono 2.6 → 2.10 perl 5.10 → 5.14 php5 5.3 → 5.4 vim 7.2 → 7.3 New: python2.7 ruby1.9 [1] https://jira.toolserver.org/browse/MNT-1322 -- Userpage: [[:w:de:User:DaB.]] — PGP: 0x2d3ee2d42b255885

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Toolserver-l May 2013