Pywikipedia-l August 2013

pywikipedia-l@lists.wikimedia.org

21 participants
36 discussions

Create additional repositories
by Dr. Trigon 28 Aug '13

28 Aug '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello all! Hello amir! I have created and maintained some other additional SVN repositiories on TS in the past [1]. [1] https://fisheye.toolserver.org/browse/drtrigon/externals Since it is not possible to include SVN repos into GIT as submodule (or else?) I fear I have to convert them (by svn2git?) to GIT repos in order to re-integrate them again into pywikibot/compat. Would it be possible to create those repos on gerrit in pywikibot too? E.g.: pywikibot/haartraining pywikibot/opencv pywikibot/pycolorname pywikibot/archive(s) (for .tar.gz files - other suggestions welcome) in order to copy them or pywikibot/other or pywikibot/thirdparty as general repos to create several such additional codes (also for future). Though later is not optimal since it can be included just as 1 submodule only. Thanks a lot for your thought and help Greetins DrTrigon -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlH4420ACgkQAXWvBxzBrDC5KwCghIA4WXfFogvMW9DZA69bfP47 Yg8An2YpPRTR+ANrqzApHw2cbPRMHtmU =BYOr -----END PGP SIGNATURE-----

3 10

Re: [Pywikipedia-l] Time lag between wikidata writes
by Chinmay Naik 24 Aug '13

24 Aug '13

Hi, I am Chinmay Naik, a Gsoc intern and operator of ProteinBoxBot(runs on pywikipedia - formerly rewrite branch). When i run the bot, there exists a sleep time of around 10 s (7s average) between two successive writes. The bot will handle around 40,000 wikidata items corresponding to gene wiki articles from http://en.wikipedia.org/wiki/Category:Human_proteins . Also , i will be uploading around 30 (claims + sources) for each wikidata item. If u take a look at the recent edits ( https://www.wikidata.org/w/index.php?title=Special:Contributions/ProteinBox…), it takes around 5 mins to completely write all claims+sources to each wikidata item. This is a very large time lag and will hugely affect deployment. Is there any way to reduce this sleep time? Any pointers on this would be helpful. Thanks, Chinmay

4 4

ascii encode error in upload.py
by AugurNZ 23 Aug '13

23 Aug '13

Hi, I was pointed here from #pywikipediabot on Freenode IRC. I am using Pywikipediabot (trunk) revision 11781 under Ubuntu 12.04. I'm having no success using the upload.py script. It keeps failing with the following error message... ---------- augur@ubuntu:/home/augur/pywikipedia# python upload.py -family:tropical -keep -filename:Pwbicon.png -noverify Pwbicon.png "The Pywikipediabot icon" No handlers could be found for logger "pywiki" Reading file Pwbicon.png The suggested description is: The Pywikipediabot icon Uploading file to tropical:en via API.... ERROR: UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in position 654: ordinal not in range(128) WARNING: Could not open 'http://www.tropicalwikis.com/w/api.php'. Maybe the server or your connection is down. Retrying in 1 minutes... ---------- I can perform other actions on www.tropicalwikis.com with my bot, such as using login.py, replace.py etc. I have also tried using a different -family: and logging in to www.thetestwiki.org (where I also have a valid bot account) and tried uploading the image file there, with the same result. I'm not a python expert, in fact I'm barely a python novice. I came across this information which may be relevant though... http://wiki.python.org/moin/UnicodeDecodeError Thanks. AugurNZ ps. sorry if this is a repeat of a previous message. I wasn't registered on the list when I first sent this, but I am now.

6 7

Re: [Pywikipedia-l] ascii encode error in upload.py
by AugurNZ 22 Aug '13

22 Aug '13

The file I'm trying to upload is the *.png thumbnail of the Pywikipediabot logo from this page... http://www.mediawiki.org/wiki/File:Pwb_icon.svg The thumbnail is called http://upload.wikimedia.org/wikipedia/commons/thumb/d/d2/Pwb_icon.svg/128px… but I have shortened the filename, as per the example below, to "Pwbicon.png" (without the quotes). So the filename should not contain any non-ASCII chars. As for the file itself, I don't know, and wouldn't even know how to find out. ;) > For me it seems that you have an HTTP error on uploading and the error > message contains a non-ASCII character. This is an old problem, nobody > could solve it by this time. > Does your uploaded file contain none-ASCII chars? > > > > 2013/8/21 AugurNZ <augurnz(a)orcon.net.nz> > >> Hi, I was pointed here from #pywikipediabot on Freenode IRC. I am using >> Pywikipediabot (trunk) revision 11781 under Ubuntu 12.04. I'm having no >> success using the upload.py script. It keeps failing with the following >> error message... >> >> ---------- >> augur@ubuntu:/home/augur/pywikipedia# python upload.py -family:tropical >> -keep -filename:Pwbicon.png -noverify Pwbicon.png "The Pywikipediabot >> icon" >> No handlers could be found for logger "pywiki" >> Reading file Pwbicon.png >> The suggested description is: >> The Pywikipediabot icon >> Uploading file to tropical:en via API.... >> ERROR: UnicodeDecodeError: 'ascii' codec can't decode byte 0x89 in >> position 654: ordinal not in range(128) >> WARNING: Could not open 'http://www.tropicalwikis.com/w/api.php'. Maybe >> the server or >> your connection is down. Retrying in 1 minutes... >> ---------- >> >> I can perform other actions on www.tropicalwikis.com with my bot, such >> as >> using login.py, replace.py etc. I have also tried using a different >> -family: and logging in to www.thetestwiki.org (where I also have a >> valid >> bot account) and tried uploading the image file there, with the same >> result. >> >> I'm not a python expert, in fact I'm barely a python novice. I came >> across >> this information which may be relevant though... >> http://wiki.python.org/moin/UnicodeDecodeError >> >> Thanks. >> AugurNZ >> >> ps. sorry if this is a repeat of a previous message. I wasn't registered >> on the list when I first sent this, but I am now. >> >> >> _______________________________________________ >> Pywikipedia-l mailing list >> Pywikipedia-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l >> > > > > -- > Bináris >

2 4

A basic pywikipedia version
by Alex Brollo 21 Aug '13

21 Aug '13

While first comparing piwikipedia and wikitools, I have been astonished to see how much light and simple are scripts to run a bot and "do basic things", ti: reading and writing pages, access to API, and so on. I'll try - consider that I'm far from skilled or "professional" - to prune off from pywikipedia any script but the basic ones, covered by wikitools, just to have some personal fun; but I encourage developers to do something similar, writing a real "pywikipedia core" containing nothing but really needed modules to run basics with personal scripts. I presume that the result will exclude 95% or more of present pwykipedia content. Alex brollo

2 2

Re: [Pywikipedia-l] About creating a bot
by Maarten Dammers 20 Aug '13

20 Aug '13

Op 20-8-2013 3:01, Chi Hong Lee schreef: > I hope that someone may help to teach me how to create a bot You could start by not sending the email to -owner address, but to the actual list and to read the manual at https://www.mediawiki.org/wiki/Pywikipediabot :-) Maarten

1 0

Re: [Pywikipedia-l] Question about externals and patches
by info＠gno.de 19 Aug '13

19 Aug '13

There is a remaining bug. Try the following e.g. from idle: >>> import Wikipedia you are asked to install patch.exe. If you decline this request you are asked a second time. If declined a second time the script raises a fatal error while importing beautifullsoup. Beautifullsoup is a central part if the screen scraping compat framework and we should not prohibit using the framework for a cosmetic patch. Please give us back that part as e.g. externals or a library script. Anyway it is not a good idea to ask for installation (twice) and fall into a fatal error if you answer is NO. There should be alternatives to patching the working copy (e.g. nightly dump etc.) Greetings xqt ----- Ursprüngliche Nachricht ----- Von: Dr. Trigon Gesendet: 03.07.2013 20:14 An: pywikipedia-l(a)lists.wikimedia.org Betreff: Re: [Pywikipedia-l] Question about externals and patches -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello to all (again)! Quit some time has elapsed since my first mail. Today I tested successfully 'patch.exe' [1] in win and in linux wine. This is another one than mentioned before, but/and as I can see works quite well even with LF (unix-style) line endings. [1] http://gnuwin32.sourceforge.net/packages/patch.htm Since I will commit these changes now the time has come to test it on other machines also. After that was successful too I will remove BeautifulSoup.py from externals because it will be downloaded and patched automatically. In order to test this on your windows machine you can e.g.: 1.) create a directory, e.g. '[...]/Desktop/patch-test' 2.) download and extract 'patch.exe' from http://downloads.sourceforge.net/project/gnuwin32/patch/2.5.9-7/patch-2.5.9… to this directory 3.) download and extract 'BeautifulSoup.py' from https://pypi.python.org/packages/source/B/BeautifulSoup/BeautifulSoup-3.2.0… to this directory 4.) download the patch 'patch-BeautifulSoup' from http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/externals/pat… to this directory 5.) open 'cmd', cd to '[...]/Desktop/patch-test' and enter "patch.exe - -p0 -d [...]/Desktop/patch-test/ -i [...]/Desktop/patch-test/patch-BeautifulSoup --binary" 6.) check the output for error messages and compare the resulting 'BeautifulSoup.py' to the one in your pywikipedia/externals directory, they should be exactly the same! IF NOT AND/OR ERRORS OCCURED PLEASE REPORT THIS TO ME! Thanks a lot to everybody helping and Greetings DrTrigon On 25.05.2013 11:40, Dr. Trigon wrote: > Hello all! > > Following issue: I am up re-organizing the whole "externals" part > in trunk as you might have recognized already. In fact this is done > now with the single exception of a generic patching system, e.g. > needed for BeautifulSoup.py. (As usual) this is no problem under > linux, but becomes a major issue under win. > > The mechanism I want to use is the well known diff-patch duo. > Therefore a "patch" executable/binary (OR python script) is needed > (for every OS). While this is kind of "built-in" in linux, win > needs extra-attention. This is what I found so far: > > * http://sourceforge.net/projects/unxutils/ The executables do only > depend on the Microsoft C-runtime (msvcrt.dll) and not on an > emulation layer like that provided by Cygwin tools - windows ONLY > not multi OS > > * http://python-patch.googlecode.com/svn/trunk/patch.py Python > script therefore multi OS - but does not support the full > diff-patch "command set" e.g. cannot create new files > > * https://code.google.com/p/google-diff-match-patch/ Not a > command-line tool like "patch" but a python library/module. Multi > OS. > > So I am stuck here and need some further knowledge, experience and > personal preferences from your side in order to make a good > decicion. > > In my opinion we should also keep further os (than just linux, > mac, win) in our mind, becuase they are very close to what we > already have too. > > Thanks for any help and Greetings DrTrigon > > _______________________________________________ Pywikipedia-l > mailing list Pywikipedia-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l > -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlHUacQACgkQAXWvBxzBrDCwOACgst3qYcS1YTqZvUvtmy8+2TOD jVkAnjBYVcMCNLSY5zBBFVG7LGg9pTDD =fDqI -----END PGP SIGNATURE----- _______________________________________________ Pywikipedia-l mailing list Pywikipedia-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

5 13

Making installation of PWB a little bit more user-freindly
by Amir Ladsgroup 19 Aug '13

19 Aug '13

As you probably know We had a workshop http://wikimania2013.wikimedia.org/wiki/DevCamp/Schedule on Wikimania's DevCamp about using PWB for people who are interested. It went very well thanks Legoktm (a BIG THANK YOU) but It was a very big problem that installation of PWB is not user-friendly (and It's worse when people try to install rewrite instead of trunk) The main cause as far as we faced during the workshop (on different computers) is dependencies. Maarten (multichill) has suggested we change setup.py and generate_user_config.py and make a auto-generated list of needed dependencies (based on OS or other things) and ask user when he/she wants to install that which one you need! and install it right away. another suggestion (my idea) is an example: When someone doesn't install i18n submodule. Codes breaks (error that says "import i18n, there is no module named "i18n"" or something like that) but we have to catch this error and ask a question and ask do you want to install i18n submodule? and maybe a user doesn't want to install it (just wants to run it in English WP) We have to let the user run the code. Best -- Amir

4 12

Pywikibot, GIT and TS
by Dr. Trigon 18 Aug '13

18 Aug '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello all! Pywikibot finished GIT migration. Now a question has anybody ever tryed to clone and use the e.g. the pywikibot/compat repo on the TS? If I do, I get and .git folder of about 200MB which nearly fills my home quota. (this is the plain clone - nothing done to/with it yet!!) Yes I could do 'git gc' I could also switch to using SVN from github (but needs to handle all externals manually)... So... what was acutally the reason for switching to GIT? Despite the fact that I have to enter about 1 million different commands now (in SVN I needed 5 ... maybe 10 - 1 command per action, in GIT I need about 5-10 command PER action) I am dealing 75% of my time with GIT issues (that SVN did not have)... In fact I am not able to to USE or further develop the GIT repo since the migration - and I am SERIOUSLY thinking about setting up an own copy on sf.net again and work there... (then we would also have the bug-tracker back in the same place) At the moment I see just 2 drawbacks: 1.) not hosted by mediawiki (really a drawback?) 2.) no review (this is the one and only advantage of git till now, and in fact it's git-review which causes issues too) The list of advantages on the other side, is that long that I will not even start making one, think of it yourself and please give me some feedback! At least I do not see if there will ever be a point in time when GIT will start to work as well as SVN did in the past. I do also not see why I have to enter 5 commands just to make a simple commit. And that will never change... Thanks for your thoughts and Greetings DrTrigon -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlIGBzQACgkQAXWvBxzBrDC40gCeNt8b7KoBLV9BDNDojjM01Upp ZsEAoIQIJ7W1FGMNysOE7LIp4qi25pyW =mvrX -----END PGP SIGNATURE-----

5 6

Re: [Pywikipedia-l] Code review in gerrit
by info＠gno.de 17 Aug '13

17 Aug '13

----- Original Nachricht ---- Von: Merlijn van Deen <valhallasw(a)arctus.nl> An: Pywikipedia discussion list <pywikipedia-l(a)lists.wikimedia.org> Datum: 11.08.2013 12:51 Betreff: Re: [Pywikipedia-l] Code review in gerrit > On 11 August 2013 12:48, Merlijn van Deen <valhallasw(a)arctus.nl> wrote: > > > I'm not really sure though how you can do this under windows easily, as > > tortoisegit doesn't seem to have any git-review capabilities... > > > > http://wiki.typo3.org/Contribution_Walkthrough_with_TortoiseGit seems to > be a guide that explains how to do gerrit with TortoiseGit. > Thanks a lot for this hint, Merlijn. I had really given up to contribute any patches to the new repository. Now can I use tortoisegit for windows and I do not need to have git-review installed which fails while missing distutils package and some other problems occured while trying to connect gerrit. Anyway I will write a small HOWTO. Best xqt

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Pywikipedia-l August 2013