[Pywikipedia-l] Rewrite status report #2

Nicolas Dumazet nicdumz at gmail.com
Fri May 9 11:52:02 UTC 2008


I would encourage any dev / bot owner fluent enough in python to give
a try to the rewrite, particularly if you use scripts fetching a lot
of data from mediawiki.

I wrote for example, a maintenance script for the French translation
project. It fetches hundreds of pages, does some mambo jumbo magic on
it, and eventually use that data to update ~200 summary pages.

I first wrote it using our trunk pywikipedia. (since editing through
the api is not yet available, I first thought that it was the only
way). It was very, very *slow*.
I wondered what improvements I would get using the rewrite, and I
rewrote my script to use the rewrite for all the page fetching part.
Well, I don't have precise figures, but I would say that the latter
version was probably 3 to 10 times faster

In addition to being faster, the more we'll use the rewrite, the more
we'll be able to detect bugs and to correct them, the easier it will
be to merge the branch when API editing will get available.

(Speaking of debugging, if you're being annoyed by the debug output
while writing your scripts,
import logging ; logging.getLogger().setLevel(logging.INFO) in your
script header will help. )

-- 
Nicolas Dumazet — NicDumZ
Deuxième année ENSIMAG.


More information about the Pywikipedia-l mailing list