[Labs-l] Scripts which adds template to articles created by ContentTranslation tool do not work on the grid

Fri Jun 16 16:15:42 UTC 2017

Hello,

I have a script which should add a template to articles which are created
by the ContentTranslation tool (the template has parameters which depends
on language and revision which were used as the source one; this is the
reason why I use separate script). It may be found at
https://github.com/urbanecm/addPrekladCT/blob/master/addmissing.py. The
script work perfectly on my local PC and on bastion host but I can't get it
work on the grid.

The script itself is run by *python3 addmissing.py -always -file:pages.txt
-search:'-insource:/\{\{[Pp]řeklad/'* and require pages.txt file and
preklads.txt file at https://tools.wmflabs.org/urbanecmbot/test/preklads.txt.
The first contains pages that should be processed and act as the generator,
the second one is something like a database with exact templates which
should be inserted. Both files are as an example in the attachments.

When I try to run it at toollabs bastion, all works as it should. When I
send the script to grid, it do not work (see sample output below). Why? Can
somebody help me with it?

Thank you in advance,
Martin Urbanec / Urbanecm

; Output

urbanecm at tools-bastion-02 ~/Documents/cswiki/addPrekladCT
$ cat test.sh
python3 addmissing.py -always -file:pages.txt
-search:'-insource:/\{\{[Pp]řeklad/'
urbanecm at tools-bastion-02 ~/Documents/cswiki/addPrekladCT
$ jsub bash test.sh
Your job 6201363 ("bash") has been submitted
urbanecm at tools-bastion-02 ~/Documents/cswiki/addPrekladCT
$ qstat
job-ID  prior   name       user         state submit/start at     queue
                     slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
6201363 0.30000 bash       urbanecm     r     06/16/2017 18:14:42
task at tools-exec-1404.eqiad.wmf     1
urbanecm at tools-bastion-02 ~/Documents/cswiki/addPrekladCT
$ ls ~/bash.*
/home/urbanecm/bash.err  /home/urbanecm/bash.out
urbanecm at tools-bastion-02 ~/Documents/cswiki/addPrekladCT
$ cat ~/bash.*
Traceback (most recent call last):
  File "addmissing.py", line 223, in <module>
    main()
  File "addmissing.py", line 183, in main
    local_args = pywikibot.handle_args(args)
  File "/shared/pywikipedia/core/pywikibot/bot.py", line 954, in handle_args
    writeToCommandLogFile()
  File "/shared/pywikipedia/core/pywikibot/bot.py", line 1128, in
writeToCommandLogFile
    command_log_file.write(s + os.linesep)
  File "/usr/lib/python3.4/codecs.py", line 711, in write
    return self.writer.write(data)
  File "/usr/lib/python3.4/codecs.py", line 368, in write
    data, consumed = self.encode(object, self.errors)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc5' in
position 67: surrogates not allowed
CRITICAL: Closing network session.
<class 'UnicodeEncodeError'>
urbanecm at tools-bastion-02 ~/Documents/cswiki/addPrekladCT
$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.wikimedia.org/pipermail/labs-l/attachments/20170616/c25f3fc9/attachment.html>
-------------- next part --------------
Adriano_Banchieri
American_Gods_(seri��l)
Battle_of_San_Patricio
Black_Panther_(film)
Budj_Bim
��esk��_Bud��jovice_ji��n��_zast��vka
��esko-n��meck��_obchodn��_a_pr��myslov��_komora
Cockatoo_Island_(New_South_Wales)
D��rov��n��_(s����ov��n��)
Euroregion_Kru��noho����
Fotbalov��_federace_Austr��lie
Gabriel_Harrison
Heritage_Foundation
Hnut��_BDS
Hodinov��_takt
Jan_Christian_Smuts
Joseph_Ray_Watkins
Kingston_a_historick��_oblast_Arthurova_��dol��
Ku��um_Ch��n
Leti��t��_J��sira_Arafata
Litovel_m��sto
Litovel_p��edm��st��
Litovel_(��elezni��n��_stanice)
Logan_Paul
Makronky
Manfred_Weber
Mark_Billingham
Marla_Maplesov��
Meizu
Mezin��rodn��_leti��t��_El-Ar����
Mil����ek_(kniha)
N��rodn��_fotbalov��_asociace_Bruneje
N��rodn��_park_Prince_Regent
Oblast_Modr��ch_hor
Operace_Tractable
Oscar_Hertwig
Oxandrolon
Programovac��_konvence
Randomizace
Riverland
Ruggero_Pasquarelli
RYB
Seznam_d��l��_seri��lu_Zm��na_hry
Softwarov��_kompatibilita
Soy_Luna_en_Concierto
Spider-Man:_Homecoming
Sputnik_1_testovac��_EMC/EMI_model_(Lab_model)
Start_Linuxov��ho_syst��mu
Statek_Brickendon
Stephen_Marley
Such��_utonut��
Swish_Swish
Theodor_Boveri
Velk��_p��se��n��_biosf��rick��_rezervace_(Queensland)
Velk��_severn��_silnice
Vue.js
Wiring_(programovac��_jazyk)
Witness_(album,_Katy_Perry)
Yeol_Eum_Son
Zas��l��n��_zpr��v
-------------- next part --------------
Battle_of_San_Patricio	{{P��eklad|en|Battle of San Patricio|681532466}}
Makronky	{{P��eklad|en|Macaron|702131015}}
Stephen_Marley	{{P��eklad|en|Stephen Marley (musician)|705380410}}
Jan_Christian_Smuts	{{P��eklad|en|Jan Smuts|711517191}}
Heritage_Foundation	{{P��eklad|en|The Heritage Foundation|723858589}}
Start_Linuxov��ho_syst��mu	{{P��eklad|en|Linux startup process|725173832}}
Wiring_(programovac��_jazyk)	{{P��eklad|en|Wiring (development platform)|725553256}}
Softwarov��_kompatibilita	{{P��eklad|en|Computer compatibility|757991444}}
Oxandrolon	{{P��eklad|en|Oxandrolone|770272159}}
Oblast_Modr��ch_hor	{{P��eklad|en|Greater Blue Mountains Area|777907499}}
Spider-Man:_Homecoming	{{P��eklad|en|Spider-Man: Homecoming|779025675}}
N��rodn��_park_Prince_Regent	{{P��eklad|en|Prince Regent National Park|747941313}}
Black_Panther_(film)	{{P��eklad|en|Black Panther (film)|778690277}}
Riverland	{{P��eklad|en|Riverland|778280932}}
Euroregion_Kru��noho����	{{P��eklad|de|Euroregion Erzgebirge/Kru��noho����|160011488}}
Ruggero_Pasquarelli	{{P��eklad|en|Ruggero Pasquarelli|776641372}}
Fotbalov��_federace_Austr��lie	{{P��eklad|en|Football Federation Australia|778471740}}
Manfred_Weber	{{P��eklad|en|Manfred Weber|773823243}}
Seznam_d��l��_seri��lu_Zm��na_hry	{{P��eklad|en|List of Game Shakers episodes|779235973}}
Cockatoo_Island_(New_South_Wales)	{{P��eklad|en|Cockatoo Island (New South Wales)|772762258}}
Theodor_Boveri	{{P��eklad|en|Theodor Boveri|731167145}}
Velk��_severn��_silnice	{{P��eklad|en|Great North Road (New South Wales)|779036898}}
Oscar_Hertwig	{{P��eklad|en|Oscar Hertwig|765120544}}
Mil����ek_(kniha)	{{P��eklad|en|Bel Ami|774014758}}
��esk��_Bud��jovice_ji��n��_zast��vka	{{P��eklad|pl|��esk�� Bud��jovice ji��n�� zast��vka|43790544}}
Kingston_a_historick��_oblast_Arthurova_��dol��	{{P��eklad|en|Kingston and Arthurs Vale Historic Area|778993100}}
Statek_Brickendon	{{P��eklad|en|Brickendon Estate|753926846}}
Randomizace	{{P��eklad|en|Randomization|753715368}}
Leti��t��_J��sira_Arafata	{{P��eklad|en|Yasser Arafat International Airport|756644759}}
Hodinov��_takt	{{P��eklad|en|Clock rate|758946117}}
Velk��_p��se��n��_biosf��rick��_rezervace_(Queensland)	{{P��eklad|en|Great Sandy Biosphere Reserve|777707884}}
D��rov��n��_(s����ov��n��)	{{P��eklad|en|Hole punching (networking)|773733206}}
Mezin��rodn��_leti��t��_El-Ar����	{{P��eklad|en|El Arish International Airport|774672243}}
Budj_Bim	{{P��eklad|en|Budj Bim|685859081}}
Hnut��_BDS	{{P��eklad|en|Boycott, Divestment and Sanctions|780767835}}
N��rodn��_fotbalov��_asociace_Bruneje	{{P��eklad|en|National Football Association of Brunei Darussalam|777646600}}
Swish_Swish	{{P��eklad|en|Swish Swish|781503771}}
RYB	{{P��eklad|en|RYB color model|772871895}}
Yeol_Eum_Son	{{P��eklad|en|Yeol Eum Son|782541976}}
Meizu	{{P��eklad|en|Meizu|782839570}}
��esko-n��meck��_obchodn��_a_pr��myslov��_komora	{{P��eklad|de|Deutsch-Tschechische Industrie- und Handelskammer|165961797}}
Logan_Paul	{{P��eklad|en|Logan Paul|782365881}}
Soy_Luna_en_Concierto	{{P��eklad|es|Soy Luna en Concierto|99504022}}
Marla_Maplesov��	{{P��eklad|pl|Marla Maples|47597326}}
Joseph_Ray_Watkins	{{P��eklad|en|Joseph Ray Watkins|783810519}}
Mark_Billingham	{{P��eklad|en|Mark Billingham|783571534}}
Gabriel_Harrison	{{P��eklad|en|Gabriel Harrison|765759661}}
Witness_(album,_Katy_Perry)	{{P��eklad|en|Witness (Katy Perry album)|784658913}}
Litovel_p��edm��st��	{{P��eklad|pl|Litovel p��edm��st��|43790723}}
Litovel_m��sto	{{P��eklad|pl|Litovel m��sto|43789970}}
Litovel_(��elezni��n��_stanice)	{{P��eklad|pl|Litovel (stacja kolejowa)|43790688}}
Operace_Tractable	{{P��eklad|en|Operation Tractable|776161271}}
Ku��um_Ch��n	{{P��eklad|en|Kuchum|772292543}}
Sputnik_1_testovac��_EMC/EMI_model_(Lab_model)	{{P��eklad|en|SPUTNIK-1 EMC/EMI LAB MODEL|785086415}}
American_Gods_(seri��l)	{{P��eklad|sk|American Gods|6511423}}
Programovac��_konvence	{{P��eklad|en|Coding conventions|784452346}}
Zas��l��n��_zpr��v	{{P��eklad|en|Message passing|779896367}}
Vue.js	{{P��eklad|en|Vue.js|784427134}}
Such��_utonut��	{{P��eklad|en|Dry drowning|785445566}}
Adriano_Banchieri	{{P��eklad|en|Adriano Banchieri|742695954}}