Pywikipedia-l February 2012

pywikipedia-l@lists.wikimedia.org

25 participants
43 discussions

by Hannes Röst

Dear all I have posted to this mailing list in January with a library that I wanted to contribute to the codebase. This is part of an effort on my side to refactor code that accumulated over various bot-operator tasks and make it available to the community. The main part of the code deals with spellchecking using hunspell (http://hunspell.sourceforge.net/) instead of the list-based approach currently used in spellcheck.py. The second part is an interactive robot to do revision control (Sichten) in the german wikipedia. There are some api functions that use the "undo" functions of the action=edit command and an api function that uses the action=review command. So I wanted to ask whether somebody had time to have a look at the code I submitted here http://sourceforge.net/tracker/?func=detail&aid=3479070&group_id=93107&atid… (I uploaded a new file "(moved testSamples)" please us this to test, the other one seems corrupt and cannot be deleted any more as well). Thus, is there a code-review process that I can undergo or what do you suggest is the best way to get the code into trunk (if at all?). Would it be easier if I talked directly to one of you? What are the criteria to get SVN commit access -- I was just wondering what the general rules are. Greetings Hannes

11 years, 3 months

Urlencoded section titles

by Bináris

Happy Monday, There are strange people who make such links (kindof urlencoded?): [[Második világháború#Partrasz.C3.A1ll.C3.A1s Szic.C3.ADli.C3.A1ban .28Huskey hadm.C5.B1velet.29|Huskey hadműveletben]] So the section title must have been copied from the URL. Do we have a ready tool to fix these? -- Bináris

11 years, 10 months

Noisy interwiki.py

by Tuszynski, Jaroslaw W.

Running interwiki.py in the background produces infrequent but steady stream of beeps, which I can't figure out how to turn off. It seems like occasionally when program tries to display am non English character it beeps. The beep does not respond to computer's volume or mute control. I was looking for an option for less verbose output, but could not find one. Is there any other way to control it other than disabling motherboard speaker? I run interwiki.py from DOS command line in Windows XP. Jarosław Tuszyński, PhD. Science Applications International Corporation (SAIC) 703-516-7406 (Office) 571-245-5824 (BlackBerry)

12 years

Re: [Pywikipedia-l] Proposal for farewell of good old Python 2.4

by info＠gno.de

Since we have plural support for i18n py2.4 is deprecated and does not fully support it except in a default way regarding to bug #3433609. Pre 2.5 releases cannot use plural.py and IMHO we shouldn't downgrade it (look at the code for the reason why). Maybe there is only this guy who is using that old stuff. I guess we should move a copy of the current 2.4 compatible release to the archive without further updates (and maybe make it available at pywikipedia nightly for downloading). 2.6 should be the new minimum version. But we have the unicode bug with all releases from 2.5 upto 2.7.1. This means 2.7.2 is the release which is strongly recommended I think. Regards xqt

12 years, 1 month

Re: [Pywikipedia-l] Proposal for farewell of good old Python 2.4

by info＠gno.de

> > > > > Last but not least: not supported does not necessarily mean we have to > > actively remove bits that work around quirks for a certain version; > rather, > > it means we won't fix bugs due to an old python version. > > > I would be more radical. What I would like to see is a cleaner code without > unneccessary branches, and free use of some basic tools such as conditional > (x if cond. else y) that is widely used in the world of programming. But > let's see the survey first. We have fixed bugs due to old py2.4 in r9227 and r9740 but I don't like it anymore if it diverges to the rewrite branch to much (or other future branches like 3.X support). interwiki.py is a good sample for really old code stuff like nested try statements for exception handling and finally handling. And we shouldn't be forced to keep code for pre 2.4 releases like "set" and "sorted". Regards xqt

12 years, 2 months

Fwd: Pywikipediabot development and git - want to switch?

by Merlijn van Deen

Hello All, As you might know, the mediawiki repository is switching to git. Sumana contacted me to find out whether the pywikipedia community would also be interested in switching. Possible reasons for switching are mainly related to to developer workflow: it's much easier to make small (local) commits, and we can do things like review-before-merge (even though I don't think this is a large issue). In addition, it's much easier to browse the history, and something like 'svn blame' does not take two decades. The obvious disadvantage is that people will have to switch: they will have to download new software (git / tortoisegit / whatever), change their repositories, et cetera. I can remember a discussion about this, but I cannot find it in my mail archives - so I'm guessing this was on IRC. As far as I am concerned - I'd like to stick with SVN for as long as possible. Even though I *really* like git, we don't gain much by switching. We don't really have the problems with code review mediawiki has, simply because we have much less code committed. However, if wikimedia takes down svn.wikimedia.org at a certain point, we might as well consider switching to a different version control system - essentially the same as what we did when we dropped sf.net for svn.wikimedia.org. Best, Merlijn

12 years, 2 months

Proposal for farewell of good old Python 2.4

by Bináris

Every now and then we face the problems of separate needs of this old Python version. See just my previous letter about async_put or the lack of a basic conditional expression. 2.5 is a point where many things are much simpler to handle. Time is going on its way and one day we noticed the calendar showed the 2012th year. 2.4 is just as an old lumber in the middle of the room you have to go round about all the time. Let's throw it out. Last time I proposed this there were voices saying some people are forced to use this historical version. I would like to know how big this problem is and if it is still the reality. So I created a page to collect these cases here: https://www.mediawiki.org/wiki/User:Bin%C3%A1ris/Python_2.4_review I propose three steps: 1. Move the above mentiond page to public space, e.g. Manual:Pywikipediabot/Python 2.4 review 2. From now on (after a few days of discussion) wikipedia.py should check for sys.version and if it turns out to be <2.5, then display a warning and ask the user to visit this page and explain his/her reasons or just sign it. (I think we are not allowed to collect automated feedbacks so this is the only way.) 3. After 2 months we review this page and with this information we may decide what to do with 2.4, keep or deprecate. In better case we will be able to determine a deadline for upgrading, in the worse case I will sadly accept the importance of 2.4 for the future. (I think we should have been talking about Python 3000 for quite a time, not 2.4.) Btw, I see some 2.3 related code remnants in wikipedia.py lines 144 to 154 of current version, although 2.3 is no more supported, AFAIK. Are they necessary? -- Bináris

12 years, 2 months

Replace.py should save -- please comment

by Bináris

Hi! My old problem is that repalce.py can't write the pages to work on into a file on my disk. I have used a modificated version for years that does no changes but writes the title of the involved pages to a subpage on Wikipedia in automated mode, and then I can make the replacements from that page much more quickly than directly from dump or living Wikipedia. This is slow and generates a plenty of dummy edits. In other words, replace.py has a tool to get the titles from a file (-file) or from a wikipage (-links), but has no tool to generate this file. Now I am ready to rewrite it. This way we can start it and the bot will find all the possible articles to work on and save the titles without editing Wikipedia (and without artificial delay), meanwhile we can have the lunch or run a marathon or sleep. Then we make the replacements from this with -file. My idea is that replace.py should have two new parameters: -save writes the results into a new file instead of editing articles. It overwrites existing file without notice. -saveappend writes into a file or appends to the existing one. OR: -save writes and appends (primary mode) -savenew writes and overwrites The help is here: http://docs.python.org/howto/unicode.html#reading-and-writing-unicode-data So we have to import codecs. My script is: articles=codecs.open('cikkek.txt','a',encoding='utf-8') ... tutuzuzu=u'# %s\n' %page.aslink() <-- needs rewrite to the new syntax articles.write(unicode(tutuzuzu)) <-- needs further testing, if nicode() is really needed articles.flush() It works fine except '\n' is a unix-styled newline that has to be converted by lfcr.py in order to make it readable with notepad.exe. This is with constant filename, that should be developed to get from command line. Your opinions before I begin? -- Bináris

12 years, 2 months

bot page count

by Chris Watkins

Is there a way to get a bot to list pageviews for all pages in a category? I found mentions of self.counter has to do with pageviews, but I can't find a script that uses it. I was asked: "Do you have a bot that can report the page view counts for the pages in the http://www.appropedia.org/Category:JMC330_International_Mass_Communicationc…" Thanks. -- Chris Watkins Appropedia.org - Sharing knowledge to build rich, sustainable lives.

12 years, 2 months

Logout when using 'getUrl' with foreign wiki

by Dr. Trigon

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello all! I have a quite confusing situation happening to my bot when trying to access any URL that points to foreign (but mediawiki software) wiki, like this: >>> pywikibot.getSite().getUrl(foreign_wiki_url, no_hostname = >>> True) This should be more or less similar to a simple: >>> urllib.urlopen(foreign_wiki_url).read() I mean I am not trying to edit or access that other wiki in a usual way (I have not login there, no family file, ...) I just want to fetch the HTML text from that page - which actually works. But while doing this the bot seems to log-out since afterwards it is not able to edit any page anymore the "traceback" from the logs is: * Password for user DrTrigonBot on wikipedia:de: * /opt/ts/python/2.7/lib/python2.7/getpass.py:83: GetPassWarning: Can not control echo on the terminal. * passwd = fallback_getpass(prompt, stream) * Warning: Password input may be echoed. * * Unhandled exception in thread started by * * Traceback (most recent call last): * File "/home/drtrigon/pywikipedia/bot_control.py", line 317, in write * string = self._REGEX_boc.sub('', string) # make more readable * File "/home/drtrigon/pywikipedia/subster_irc.py", line 156, in main_subster * bot.run() * File "/home/drtrigon/pywikipedia/subster.py", line 220, in run * self.save(page, substed_content, head + mod % (", ".join(substed_tags)), **flags) * File "/home/drtrigon/pywikipedia/dtbext/dtbext_basic.py", line 240, in save * page.put(text, comment=comment, minorEdit=minorEdit, botflag=botflag) * File "/home/drtrigon/pywikipedia/wikipedia.py", line 1708, in put * sysop = self._getActionUser(action = 'edit', restriction = self.editRestriction, sysop = sysop) * File "/home/drtrigon/pywikipedia/wikipedia.py", line 1581, in _getActionUser * self.site().forceLogin(sysop = sysop) * File "/home/drtrigon/pywikipedia/wikipedia.py", line 5008, in forceLogin * if loginMan.login(retry = True): * File "/home/drtrigon/pywikipedia/login.py", line 307, in login * password = True) * File "/home/drtrigon/pywikipedia/wikipedia.py", line 8018, in input * data = ui.input(question, password) * File "/home/drtrigon/pywikipedia/userinterfaces/terminal_interface.py", line 238, in input * text = getpass.getpass('') * File "/opt/ts/python/2.7/lib/python2.7/getpass.py", line 83, in unix_getpass * passwd = fallback_getpass(prompt, stream) * File "/opt/ts/python/2.7/lib/python2.7/getpass.py", line 118, in fallback_getpass * return _raw_input(prompt, stream) * File "/opt/ts/python/2.7/lib/python2.7/getpass.py", line 135, in _raw_input * raise EOFError * EOFError ...any idea whats happening here?? Thanks a lot! Greetings DrTrigon -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8i0WgACgkQAXWvBxzBrDAtpwCfdge6G2fKmh+hYZIX2YQOi7lY wyIAoKN7j6QeQymclaCOqhSWdHhG1XUy =bLby -----END PGP SIGNATURE-----

12 years, 2 months

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Pywikipedia-l February 2012