Toolserver-l January 2011

toolserver-l@lists.wikimedia.org

25 participants
24 discussions

Beginner's questions :-(
by Alex Brollo 15 Jan '11

15 Jan '11

I see alarming messages into my toolserver terminal. 1. You are already logged in....then some items like: pts:/33:80 (pts/35) 2 introducing screen -list I see another list of four items, like: 21068.pts-24.willow or like 12976.alex I too got a "There are stopped jobs" after exit, but such a message disappeared al last try. What have I to do? Such logins and screens will be deleted since inactive? have I to worry about? Please, if you'll suggest something that I'd do, use a "idiot-proof" language. :-( :-( Alex

3 9

MySQL problem
by Kolossos 15 Jan '11

15 Jan '11

Hello, If I want to load a file into MYSQL with the command[1] I get an error 1148 [2]. I did this many times before without problems. So I want to ask if there was a change in toolserver configuration and what the solution can be for me. Greetings Kolossos [1] LOAD /* SLOW_OK */ DATA LOCAL INFILE '/home/sk/data/templatetiger/dewiki/dewiki_templatetiger.txt' IGNORE INTO TABLE `dewiki_neu` FIELDS TERMINATED BY '\t' ESCAPED BY '\\' LINES TERMINATED BY '\n' ; [2] ERROR 1148 (42000): The used command is not allowed with this MySQL version

2 2

New job submission servers
by River Tarnell 12 Jan '11

12 Jan '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, We have set up a pair of redundant hosts to act as SGE job submission servers. These work by sharing each user's cronietab between both hosts, and executing jobs on whichever server is working. This avoids that problem where jobs run from cronie on one login server (such as willow) will fail to run if that host is down, even when other login servers are available. To use the new hosts, log into submit.toolserver.org and set up a cronietab (*not* a crontab) as normal. Note that these hosts are *only* for submitting SGE jobs, not for running tools on. For now this service could be considered experimental, but I don't foresee any problems with it. - river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0tewgACgkQIXd7fCuc5vJKpQCeOwDC6Y2/0rBkzGFj6A2WT1DS aaQAn0yhdkmZikrbXPdPL26EsQ5A+r+1 =PDMJ -----END PGP SIGNATURE-----

1 0

output buffer flushing broken
by Ryan Kaldari 11 Jan '11

11 Jan '11

Ever since the servers were moved to Solaris I haven't been able to get output buffer flushing in PHP to work at all. I've tried all the usual solutions: using implicit flushing, filling the buffer with 100000 chars before the flush, using ob_flush() and flush() together, setting output_buffering to 0, cleaning all the active buffers, making sure output compression is off, etc. but nothing will work. I had no problems with this a few months ago. Unfortunately, I need this to work for a toolserver tool that is going to be heavy used during the Wikipedia 10th Anniversary celebration this weekend. Here are a couple of basic test-case scripts that won't work for me on the toolserver: http://pastebin.com/sAwnvJrZ // manual flushing http://pastebin.com/TuKM14av // implicit flushing If anyone can get either of these test scripts working on the toolserver, please let me know ASAP. Thanks! Ryan Kaldari

1 1

Toolserver projects list
by Erik Moeller 11 Jan '11

11 Jan '11

Hi, is there still an up-to-date, canonical list of toolserver projects? Thanks, Erik -- Erik Möller Deputy Director, Wikimedia Foundation Support Free Knowledge: http://wikimediafoundation.org/wiki/Donate

2 1

WM database dumps
by Mihajlo Andjelkovic 09 Jan '11

09 Jan '11

Hello, everyone Did I miss announcement when WM database dumps are to start being actualized again? M

2 1

cronsub
by River Tarnell 08 Jan '11

08 Jan '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I've made a couple of changes to cronsub in response to some issues reported by users. Specifically: * cronsub now requires that the script file be executable, and will raise an error if it's not. The previous behaviour was that non-executable scripts would be executed by /bin/sh. If this affects you, the fix is to make the script executable (chmod +x). * Previously, SGE would copy the job script to a shared directory and execute it there. For example, if you submitted $HOME/test.py, it would be copied to a file such as /sge62/default/spool/wolfsbane/job_scripts/117333 before starting. This was unfortunate for Python users who depended on Python's behaviour of treating the script's directory as part of the "import" search path. This behaviour has been changed, so that the script file will be executed in its original location. I appreciate that the first item may be a breaking change for some users. However, on balance this seems like the lesser evil. - river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0ny8sACgkQIXd7fCuc5vK1UwCfaIt9gOSWym8sDOKR8bekuUaK Qy4AoLLJIB1xaUjeKdpUHLZ6npg/0FiS =VV97 -----END PGP SIGNATURE-----

4 6

Maintenance
by River Tarnell 08 Jan '11

08 Jan '11

Hi, The previous announced maintenance has been completed successfully. In case of problems, open an issue in the TS project in JIRA. For problems relating to the conversion of nightshade to Solaris (e.g. missing software), open an issue in the TS project and include "[solaris]" at the start of the subject line. Due to the disruptive nature of this maintenance, such requests will be given priority. (However, I'll be away for at least the next 12 hours, so I won't be able to resolve them immediately.) - river.

4 8

Job scheduling a python bot script
by Dr. Trigon 08 Jan '11

08 Jan '11

Hello all I'm currently trying to get 'Job scheduling' working form my modified pywikipedia bot python script. According to https://wiki.toolserver.org/view/Job_scheduling (Converting an existing cron job to use the scheduler) I tried to convert ================= 0 2 * * * nice -n +15 python /home/drtrigon/pywikipedia/runbotrun.py -cron 0 0 1,16 * * nice -n +15 python /home/drtrigon/pywikipedia/runbotrun.py -skip_clean_user_sandbox -compress_history:[] -cron 0 6 * * * nice -n +15 python /home/drtrigon/pywikipedia/subster_beta.py 2>> /home/drtrigon/public_html/DrTrigonBot/subster.html ================= to ================= 0 2 * * * cronsub -s mainbot $HOME/pywikipedia/runbotrun.py -cron 0 0 1,16 * * cronsub -s compbot $HOME/pywikipedia/runbotrun.py -skip_clean_user_sandbox -compress_history:[] -cron 0 6 * * * cronsub -s substerbot $HOME/pywikipedia/subster_beta.py 2>> $HOME/public_html/DrTrigonBot/subster.html ================= AND I tried to run to first of those jobs with 'qsub', by: qsub $HOME/pywikipedia/runbotrun.py -cron BUT this leads to the follwing output in my $HOME: ================= /sge62/default/spool/wolfsbane/job_scripts/117222: ^JThis bot is the general DrTrigonBot caller. It runs all the different sub tasks,^Jthat DrTrigonBot does.^J^J...^J: not found import: unable to open X server `' @ error/import.c/ImportImageCommand/364. import: unable to open X server `' @ error/import.c/ImportImageCommand/364. import: unable to open X server `' @ error/import.c/ImportImageCommand/364. import: unable to open X server `' @ error/import.c/ImportImageCommand/364. import: unable to open X server `' @ error/import.c/ImportImageCommand/364. /sge62/default/spool/wolfsbane/job_scripts/117222: from: not found logname: extra operand `=' Try `logname --help' for more information. /sge62/default/spool/wolfsbane/job_scripts/117222: logger_tmsp: not found /sge62/default/spool/wolfsbane/job_scripts/117222: syntax error at line 65: `(' unexpected ================= ... and this is the point were I am getting lost... this message looks VERY strange to me, since I'm not using an X server (explicitely) and I have no clue what the 'error/import.c' means there..... Please can somebody give me a hint? Any ideas are welcome!! Thanks a lot and greetings Dr. Trigon ps: by the way; thanks for the good server maintenance and all that stuff!!

2 1

.environment file
by River Tarnell 07 Jan '11

07 Jan '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, It's now possible to set environment variables in your $HOME/.environment file, as a series of NAME=value pairs, for example: PYTHONPATH=$HOME/pylib This will affect interactive logins, cron jobs and SGE jobs, but currently not web server CGI scripts. - river. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0mH1EACgkQIXd7fCuc5vJrWwCgt+nTHauHPBIJ2+jhw3J2tkPt COcAnAurVJt1Wncm6WDM8eX/h0RkTz7+ =KJ0E -----END PGP SIGNATURE-----

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

Toolserver-l January 2011