Pywikipedia-l June 2008

pywikipedia-l@lists.wikimedia.org

32 participants
449 discussions

SVN: [5532] trunk/pywikipedia/commonsdelinker/delinker.py
by btongminh＠svn.wikimedia.org 08 Jun '08

08 Jun '08

Revision: 5532 Author: btongminh Date: 2008-06-08 16:51:38 +0000 (Sun, 08 Jun 2008) Log Message: ----------- Added force_complex configuration variable Modified Paths: -------------- trunk/pywikipedia/commonsdelinker/delinker.py Modified: trunk/pywikipedia/commonsdelinker/delinker.py =================================================================== --- trunk/pywikipedia/commonsdelinker/delinker.py 2008-06-08 16:47:09 UTC (rev 5531) +++ trunk/pywikipedia/commonsdelinker/delinker.py 2008-06-08 16:51:38 UTC (rev 5532) @@ -267,7 +267,7 @@ simple_replacer, match.group(2)), match.group(3)) new_text = re.sub(r_galleries, gallery_replacer, new_text) - if text == new_text: + if text == new_text or self.CommonsDelinker.config.get('force_complex', False): # All previous steps did not work, so the image is # likely embedded in a complicated template. hook = 'complex'

1 0

SVN: [5531] trunk/pywikipedia/commonsdelinker/image_replacer.py
by btongminh＠svn.wikimedia.org 08 Jun '08

08 Jun '08

Revision: 5531 Author: btongminh Date: 2008-06-08 16:47:09 +0000 (Sun, 08 Jun 2008) Log Message: ----------- Fix some None exceptions Modified Paths: -------------- trunk/pywikipedia/commonsdelinker/image_replacer.py Modified: trunk/pywikipedia/commonsdelinker/image_replacer.py =================================================================== --- trunk/pywikipedia/commonsdelinker/image_replacer.py 2008-06-08 15:35:32 UTC (rev 5530) +++ trunk/pywikipedia/commonsdelinker/image_replacer.py 2008-06-08 16:47:09 UTC (rev 5531) @@ -133,9 +133,11 @@ ('rvprop', 'timestamp|user|comment|content'), ('rvlimit', '50'), ('format', 'json'), - ('rvend', since), - ('rvexcludeuser', username.encode('utf-8')) ] + if username: + predata.append(('rvexcludeuser', username.encode('utf-8')) + if since: + predata.append(('rvend', since)) response, data = self.site.postForm(address, predata) data = simplejson.loads(data) if 'error' in data: @@ -144,7 +146,7 @@ page = data['query']['pages'].values()[0] if 'missing' in page: raise Exception('Missing page!') - return page.get('revisions', ()) + return page.get('revisions', []) def examine_revision_history(self, revisions, replacement, username): for revision in revisions:

1 0

[ pywikipediabot-Patches-1988231 ] Finnish support for noreferences.py
by SourceForge.net 08 Jun '08

08 Jun '08

Patches item #1988231, was opened at 2008-06-08 19:42 Message generated for change (Comment added) made by silvonen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1988231&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mikko Silvonen (silvonen) Assigned to: Nobody/Anonymous (nobody) Summary: Finnish support for noreferences.py Initial Comment: Here's a Finnish patch for noreferences.py. I had to change the script logic a little, because in the Finnish Wikipedia it is recommended to use the {{viitteet}} template instead of <references />. Feel free to improve the names of the variables. ---------------------------------------------------------------------- >Comment By: Mikko Silvonen (silvonen) Date: 2008-06-08 19:44 Message: Logged In: YES user_id=127947 Originator: YES File Added: noreferences_fi.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1988231&group_…

1 0

[ pywikipediabot-Patches-1988231 ] Finnish support for noreferences.py
by SourceForge.net 08 Jun '08

08 Jun '08

Patches item #1988231, was opened at 2008-06-08 19:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1988231&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mikko Silvonen (silvonen) Assigned to: Nobody/Anonymous (nobody) Summary: Finnish support for noreferences.py Initial Comment: Here's a Finnish patch for noreferences.py. I had to change the script logic a little, because in the Finnish Wikipedia it is recommended to use the {{viitteet}} template instead of <references />. Feel free to improve the names of the variables. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1988231&group_…

1 0

[ pywikipediabot-Bugs-1988198 ] replace.py crashes in szl:wiki
by SourceForge.net 08 Jun '08

08 Jun '08

Bugs item #1988198, was opened at 2008-06-08 17:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1988198&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: masti (mstmst) Assigned to: Nobody/Anonymous (nobody) Summary: replace.py crashes in szl:wiki Initial Comment: when running replace.py (r5530) with any parameters for example: python replace.py -start:! "Wp/szl/" "" -log it comes to [[Česko godka]] article and crashes with the following error: No changes were necessary in [[Česko godka]] Retrieving Allpages special page for wikipedia:szl from %E1%BA%8A%21, namespace 0 Couldn't extract allpages special page. Make sure you're using MonoBook skin. When restarting using -start:"Česko godka" it goes to the end without any problems. masti ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1988198&group_…

1 0

SVN: [5530] trunk/pywikipedia/wikipedia.py
by cosoleto＠svn.wikimedia.org 08 Jun '08

08 Jun '08

Revision: 5530 Author: cosoleto Date: 2008-06-08 15:35:32 +0000 (Sun, 08 Jun 2008) Log Message: ----------- now longpages() RE and more fixes to avoid error with numbers conversion in string Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-06-08 15:06:43 UTC (rev 5529) +++ trunk/pywikipedia/wikipedia.py 2008-06-08 15:35:32 UTC (rev 5530) @@ -4673,14 +4673,15 @@ # implemented is fairly useless # this comment applies to all the XXXXpages methods following, as well seen = set() + path = self.longpages_address(n=number) + entryR = re.compile(ur'<li>\(<a href=".+?" title=".+?">.+?</a>\) .<a href=".+?" title="(?P<title>.+?)">.+?</a> .\[(?P<length>[\d.,]+).*?\]</li>', re.UNICODE) + while True: - path = self.longpages_address(n=number) get_throttle() html = self.getUrl(path) - entryR = re.compile(ur'<li>\(<a href=".+?" title=".+?">hist</a>\) .<a href=".+?" title="(?P<title>.+?)">.+?</a> .\[(?P<length>\d+)(.+?)\]</li>', re.UNICODE) for m in entryR.finditer(html): title = m.group('title') - length = int(m.group('length')) + length = int(re.sub('[.,]', '', m.group('length'))) if title not in seen: seen.add(title) page = Page(self, title) @@ -4692,15 +4693,16 @@ """Yield Pages and lengths from Special:Shortpages.""" throttle = True seen = set() + path = self.shortpages_address(n = number) + entryR = re.compile(ur'<li>\(<a href=".+?" title=".+?">.+?</a>\) .<a href=".+?" title="(?P<title>.+?)">.+?</a> .\[(?P<length>[\d.,]+).*?\]</li>', re.UNICODE) + while True: - path = self.shortpages_address(n = number) get_throttle() html = self.getUrl(path) - entryR = re.compile(ur'<li>\(<a href=".+?" title=".+?">.+?</a>\) .<a href=".+?" title="(?P<title>.+?)">.+?</a> .\[(?P<length>[\d.,]+).*?\]</li>', re.UNICODE) for m in entryR.finditer(html): title = m.group('title') - length = int(m.group('length')) + length = int(re.sub('[., ]', '', m.group('length'))) if title not in seen: seen.add(title)

1 0

SVN: [5529] trunk/pywikipedia/wikipedia.py
by cosoleto＠svn.wikimedia.org 08 Jun '08

08 Jun '08

Revision: 5529 Author: cosoleto Date: 2008-06-08 15:06:43 +0000 (Sun, 08 Jun 2008) Log Message: ----------- RE in shortpages() was only good for English wiki sites. Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-06-08 14:13:55 UTC (rev 5528) +++ trunk/pywikipedia/wikipedia.py 2008-06-08 15:06:43 UTC (rev 5529) @@ -4696,7 +4696,8 @@ path = self.shortpages_address(n = number) get_throttle() html = self.getUrl(path) - entryR = re.compile(ur'<li>\(<a href=".+?" title=".+?">hist</a>\) .<a href=".+?" title="(?P<title>.+?)">.+?</a> .\[(?P<length>\d+)(.+?)\]</li>', re.UNICODE) + entryR = re.compile(ur'<li>\(<a href=".+?" title=".+?">.+?</a>\) .<a href=".+?" title="(?P<title>.+?)">.+?</a> .\[(?P<length>[\d.,]+).*?\]</li>', re.UNICODE) + for m in entryR.finditer(html): title = m.group('title') length = int(m.group('length'))

1 0

SVN: [5528] trunk/pywikipedia/wikipedia.py
by cosoleto＠svn.wikimedia.org 08 Jun '08

08 Jun '08

Revision: 5528 Author: cosoleto Date: 2008-06-08 14:13:55 +0000 (Sun, 08 Jun 2008) Log Message: ----------- mediawiki_message() backward compatibility fix. Special:Allmessages in XML format is available only using MediaWiki version 1.12 or higher, for previous versions now PHP format is used. I remove get_throttle() because system messages download is not frequent. Used html2unicode() to convert HTML entities in messages when BeautifulStoneSoup is used to improve uniformity between different outputs. Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-06-08 13:50:51 UTC (rev 5527) +++ trunk/pywikipedia/wikipedia.py 2008-06-08 14:13:55 UTC (rev 5528) @@ -127,7 +127,6 @@ import xmlreader from BeautifulSoup import * import simplejson -import diskcache import weakref # Set the locale to system default. This will ensure correct string @@ -4479,45 +4478,60 @@ if verbose: output( u"Retrieving mediawiki messages from Special:Allmessages") - elementtree = True - try: + # Only MediaWiki r27393/1.12 and higher support XML output for Special:Allmessages + if self.versionnumber() < 12: + usePHP = True + else: + usePHP = False + elementtree = True try: - from xml.etree.cElementTree import XML # 2.5 - except ImportError: try: - from cElementTree import XML + from xml.etree.cElementTree import XML # 2.5 except ImportError: - from elementtree.ElementTree import XML - except ImportError: - if verbose: - output(u'Elementtree was not found, using BeautifulSoup instead') - elementtree = False + try: + from cElementTree import XML + except ImportError: + from elementtree.ElementTree import XML + except ImportError: + if verbose: + output(u'Elementtree was not found, using BeautifulSoup instead') + elementtree = False if config.use_diskcache: + import diskcache _dict = lambda x : diskcache.CachedReadOnlyDictI(x, prefix = "msg-%s-%s-" % (self.family.name, self.lang)) else: _dict = dict retry_idle_time = 1 while True: - get_throttle() - xml = self.getUrl(self.get_address("Special:Allmessages") - + "&ot=xml") - # xml structure is : - # <messages lang="fr"> - # <message name="about">À propos</message> - # ... - # </messages> - if elementtree: - decode = xml.encode(self.encoding()) - tree = XML(decode) - self._mediawiki_messages = _dict([(tag.get('name').lower(), tag.text) - for tag in tree.getiterator('message')]) + if usePHP: + phppage = self.getUrl(self.get_address("Special:Allmessages") + + "&ot=php") + Rphpvals = re.compile(r"(?ms)'([^']*)' => '(.*?[^\\])',") + # Previous regexp don't match empty messages. Fast workaround... + phppage = re.sub("(?m)^('.*?' =>) '',", r"\1 ' ',", phppage) + self._mediawiki_messages = _dict([(name.strip().lower(), + html2unicode(message.replace("\\'", "'"))) + for (name, message) in Rphpvals.findall(phppage)]) else: - tree = BeautifulStoneSoup(xml) - self._mediawiki_messages = _dict([(tag.get('name').lower(), tag.string) - for tag in tree.findAll('message')]) - + xml = self.getUrl(self.get_address("Special:Allmessages") + + "&ot=xml") + # xml structure is : + # <messages lang="fr"> + # <message name="about">À propos</message> + # ... + # </messages> + if elementtree: + decode = xml.encode(self.encoding()) + tree = XML(decode) + self._mediawiki_messages = _dict([(tag.get('name').lower(), tag.text) + for tag in tree.getiterator('message')]) + else: + tree = BeautifulStoneSoup(xml) + self._mediawiki_messages = _dict([(tag.get('name').lower(), html2unicode(tag.string)) + for tag in tree.findAll('message')]) + if not self._mediawiki_messages: # No messages could be added. # We assume that the server is down.

1 0

[ pywikipediabot-Bugs-1985288 ] Substantial (language) update of featured.py
by SourceForge.net 08 Jun '08

08 Jun '08

Bugs item #1985288, was opened at 2008-06-05 12:14 Message generated for change (Settings changed) made by melancholie You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1985288&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None >Priority: 6 Private: No Submitted By: Melancholie (melancholie) Assigned to: Nobody/Anonymous (nobody) Summary: Substantial (language) update of featured.py Initial Comment: I have locally updated featured.py with the following changes: 1. Large language update (used several sources, test.wp etc.) 2. Moved cache file to folder "cache" (You can also move the file README and then delete the folder "featured") See: http://als.wikipedia.org/w/index.php?title=Benutzer:MelancholieBot/featured… Diff: http://als.wikipedia.org/w/index.php?title=Benutzer%3AMelancholieBot%2Ffeat… ---------------------------------------------------------------------- Comment By: Melancholie (melancholie) Date: 2008-06-05 12:27 Message: Logged In: YES user_id=2089773 Originator: YES A casual note: featured.py should have a "spam" filter! As some Wikipedia communities do create many thousands of articles by bots, it is possible that some communities could abuse that marking stars, declaring almost all of their articles as featured. If a small Wikipedia has even more featured articles as enwiki, this could be an indicator for that, for example. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1985288&group_…

1 0

[ pywikipediabot-Bugs-1827590 ] featured.py add empty lines
by SourceForge.net 08 Jun '08

08 Jun '08

Bugs item #1827590, was opened at 2007-11-07 16:09 Message generated for change (Comment added) made by nicdumz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1827590&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Warddr (warddr) >Assigned to: NicDumZ — Nicolas Dumazet (nicdumz) Summary: featured.py add empty lines Initial Comment: The bot always add empty lines in between two linkFA templates. example: http://nl.wikipedia.org/w/index.php?title=Schaken&diff=10073293&oldid=10070… Thanks ---------------------------------------------------------------------- >Comment By: NicDumZ — Nicolas Dumazet (nicdumz) Date: 2008-06-08 15:53 Message: Logged In: YES user_id=1963242 Originator: NO This should be fixed by r5527 ---------------------------------------------------------------------- Comment By: Melancholie (melancholie) Date: 2008-06-06 10:34 Message: Logged In: YES user_id=2089773 Originator: NO Does this still happen? ("-top") ---------------------------------------------------------------------- Comment By: Leonardo Gregianin (leogregianin) Date: 2007-11-27 00:33 Message: Logged In: YES user_id=1136737 Originator: NO This is a problem of wikipedia.replaceCategoryLinks ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1827590&group_…

1 0

← Newer
1
...
31
32
33
34
35
36
37
...
45
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

Pywikipedia-l June 2008