pywikibot May 2008

pywikibot@lists.wikimedia.org

32 participants
376 discussions

[Pywikipedia-l] SVN: [5402] trunk/pywikipedia/wikipedia.py
by nicdumz＠svn.wikimedia.org 19 May '08

19 May '08

Revision: 5402 Author: nicdumz Date: 2008-05-19 07:03:34 +0000 (Mon, 19 May 2008) Log Message: ----------- Trying to use ElementTree when cElementTree is not available. If not available, dying. Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-05-18 19:17:39 UTC (rev 5401) +++ trunk/pywikipedia/wikipedia.py 2008-05-19 07:03:34 UTC (rev 5402) @@ -4423,10 +4423,21 @@ def mediawiki_message(self, key): """Return the MediaWiki message text for key "key" """ try: - from xml.etree.cElementTree import XML + from xml.etree.cElementTree import XML # 2.5 except ImportError: - from cElementTree import XML - + try: + from cElementTree import XML + except ImportError: + output('Module cElementTree not found, using instead the slower ElementTree') + try: + from xml.etree.ElementTree import XML # 2.5 + except ImportError: + try: + from elementtree.ElementTree import XML + except ImportError: + output('ERROR: You need to install cElementTree, or ElementTree module to be able to parse mediawiki messages') + stopme() + sys.exit(1) # Allmessages is retrieved once for all in a session if not self._mediawiki_messages: if verbose:

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1965842 ] Modern skin not usable
by SourceForge.net 18 May '08

18 May '08

Bugs item #1965842, was opened at 2008-05-17 09:34 Message generated for change (Comment added) made by melancholie You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1965842&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Melancholie (melancholie) Assigned to: Nobody/Anonymous (nobody) Summary: Modern skin not usable Initial Comment: The MediaWiki skin 'modern' does not have that much HTML cruft around the actual content. Thus there would be a little bit less bot traffic with this skin. But I cannot use that skin with pyWikipedia! Error message: Traceback (most recent call last): File "interwiki.py", line 1731, in ? bot.run() File "interwiki.py", line 1482, in run self.queryStep() File "interwiki.py", line 1461, in queryStep subj.finish(self) File "interwiki.py", line 1042, in finish if self.replaceLinks(page, new, bot): File "interwiki.py", line 1200, in replaceLinks status, reason, data = page.put(newtext, comment = wikipedia.translate(page.site().lang, msg)[0] + mods) File "wikipedia.py", line 1222, in put sysop = self._getActionUser(action = 'edit', restriction = self.editRestriction, sysop = False) File "wikipedia.py", line 1114, in _getActionUser if not self.site().isAllowed(action, sysop): File "wikipedia.py", line 4013, in isAllowed return right in self._rights[index] TypeError: iterable argument required ---------------------------------------------------------------------- >Comment By: Melancholie (melancholie) Date: 2008-05-19 03:28 Message: Logged In: YES user_id=2089773 Originator: YES BTW: See also https://bugzilla.wikimedia.org/show_bug.cgi?id=3651 That skin assumedly would not work either. ---------------------------------------------------------------------- Comment By: Melancholie (melancholie) Date: 2008-05-17 09:38 Message: Logged In: YES user_id=2089773 Originator: YES After ~manipulating~ line 4013, the following message is shown: ERROR: Unexpected response from wiki server. 200 (OK) [...edit view...] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1965842&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1966693 ] Missing main page name
by SourceForge.net 18 May '08

18 May '08

Bugs item #1966693, was opened at 2008-05-19 03:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1966693&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: interwiki Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Melancholie (melancholie) Assigned to: Nobody/Anonymous (nobody) Summary: Missing main page name Initial Comment: interwiki.py prints "Missing main page name" (at least for eowiki)! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1966693&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1966691 ] ImportError: No module named cElementTree
by SourceForge.net 18 May '08

18 May '08

Bugs item #1966691, was opened at 2008-05-19 03:22 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1966691&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: interwiki Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Melancholie (melancholie) Assigned to: Nobody/Anonymous (nobody) Summary: ImportError: No module named cElementTree Initial Comment: rev5401: Changing page [[fr:Manuel d'histoire commun franco-allemand]] Dump eo (wikipedia) saved Traceback (most recent call last): File "interwiki.py", line 1733, in ? bot.run() File "interwiki.py", line 1484, in run self.queryStep() File "interwiki.py", line 1463, in queryStep subj.finish(self) File "interwiki.py", line 1044, in finish if self.replaceLinks(page, new, bot): File "interwiki.py", line 1202, in replaceLinks status, reason, data = page.put(newtext, comment = wikipedia.translate(page. site().lang, msg)[0] + mods) File "wikipedia.py", line 1263, in put newPage, self.site().getToken(sysop = sysop), sysop = sysop) File "wikipedia.py", line 1363, in _putPage if self.site().has_mediawiki_message("spamprotectiontitle")\ File "wikipedia.py", line 4472, in has_mediawiki_message v = self.mediawiki_message(key) File "wikipedia.py", line 4429, in mediawiki_message from cElementTree import XML ImportError: No module named cElementTree ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1966691&group_…

1 0

[Pywikipedia-l] SVN: [5401] trunk/pywikipedia/wikipedia.py
by nicdumz＠svn.wikimedia.org 18 May '08

18 May '08

Revision: 5401 Author: nicdumz Date: 2008-05-18 19:17:39 +0000 (Sun, 18 May 2008) Log Message: ----------- dummy #redirect[[]] plague, bis. (Undoing last commit, using a proper fix.) Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-05-18 19:09:01 UTC (rev 5400) +++ trunk/pywikipedia/wikipedia.py 2008-05-18 19:17:39 UTC (rev 5401) @@ -792,12 +792,10 @@ redirtarget = "%s#%s" % (m.group(1), self.section()) else: redirtarget = m.group(1) - #avoid dummy #REDIRECT[[]] - if redirtarget: - if get_redirect: - self._redirarg = redirtarget - elif not nofollow_redirects: - raise IsRedirectPage(redirtarget) + if get_redirect: + self._redirarg = redirtarget + elif not nofollow_redirects: + raise IsRedirectPage(redirtarget) if self.section(): # TODO: What the hell is this? Docu please. m = re.search("\.3D\_*(\.27\.27+)?(\.5B\.5B)?\_*%s\_*(\.5B\.5B)?(\.27\.27+)?\_*\.3D" % re.escape(self.section()), sectionencode(text,self.site().encoding())) @@ -5119,7 +5117,7 @@ # a label, although this is not useful. return re.compile(r'#' + redirKeywordsR + - '.*?\[\[(.*?)(?:\|.*?)?\]\]', + '.*?\[\[(.+?)(?:\|.*?)?\]\]', re.IGNORECASE | re.UNICODE | re.DOTALL) # The following methods are for convenience, so that you can access

1 0

[Pywikipedia-l] SVN: [5400] trunk/pywikipedia
by nicdumz＠svn.wikimedia.org 18 May '08

18 May '08

Revision: 5400 Author: nicdumz Date: 2008-05-18 19:09:01 +0000 (Sun, 18 May 2008) Log Message: ----------- * Correcting an es: message * Avoiding #REDIRECT[[]] redirects. They are not considered as redirects by mediawiki, and still they were raising IsRedirectPage Modified Paths: -------------- trunk/pywikipedia/reflinks.py trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/reflinks.py =================================================================== --- trunk/pywikipedia/reflinks.py 2008-05-18 15:32:13 UTC (rev 5399) +++ trunk/pywikipedia/reflinks.py 2008-05-18 19:09:01 UTC (rev 5400) @@ -52,7 +52,7 @@ 'de':u'Bot: Korrektes Referenzformat (siehe [[:en:User:DumZiBoT/refLinks]])', 'hu':u'Robot: Forráshivatkozások kibővítése a hivatkozott oldal címével', 'ko':u'봇: url만 있는 주석을 보강, (영문)[[:en:User:DumZiBoT/refLinks]] 참조', - 'es':u'Formateando las referencias que no tuvieran titulos (Pruebas por [[Wikipedia:Bot/Autorizaciones#DumZiBoT]] )', + 'es':u'Formateando las referencias que no tuvieran títulos (Pruebas por [[Wikipedia:Bot/Autorizaciones#DumZiBoT]] )', 'en':u'Bot: Converting bare references, see [[User:DumZiBoT/refLinks|FAQ]]'} deadLinkTag = {'fr':u'%s {{lien mort}}', Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-05-18 15:32:13 UTC (rev 5399) +++ trunk/pywikipedia/wikipedia.py 2008-05-18 19:09:01 UTC (rev 5400) @@ -792,10 +792,12 @@ redirtarget = "%s#%s" % (m.group(1), self.section()) else: redirtarget = m.group(1) - if get_redirect: - self._redirarg = redirtarget - elif not nofollow_redirects: - raise IsRedirectPage(redirtarget) + #avoid dummy #REDIRECT[[]] + if redirtarget: + if get_redirect: + self._redirarg = redirtarget + elif not nofollow_redirects: + raise IsRedirectPage(redirtarget) if self.section(): # TODO: What the hell is this? Docu please. m = re.search("\.3D\_*(\.27\.27+)?(\.5B\.5B)?\_*%s\_*(\.5B\.5B)?(\.27\.27+)?\_*\.3D" % re.escape(self.section()), sectionencode(text,self.site().encoding()))

1 0

[Pywikipedia-l] SVN: [5395] trunk/pywikipedia/wikipedia.py
by nicdumz＠svn.wikimedia.org 18 May '08

18 May '08

Revision: 5395 Author: nicdumz Date: 2008-05-18 10:53:07 +0000 (Sun, 18 May 2008) Log Message: ----------- making replaceCategoryInPlace and removeCategoryLinks category regexes case insensitive Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-05-18 10:42:14 UTC (rev 5394) +++ trunk/pywikipedia/wikipedia.py 2008-05-18 10:53:07 UTC (rev 5395) @@ -3388,7 +3388,7 @@ # NOTE: This assumes that language codes only consist of non-capital # ASCII letters and hyphens. catNamespace = '|'.join(site.category_namespaces()) - categoryR = re.compile(r'\[\[\s*(%s)\s*:.*?\]\][\s]*' % catNamespace) + categoryR = re.compile(r'\[\[\s*(%s)\s*:.*?\]\][\s]*' % catNamespace, re.I) text = replaceExcept(text, categoryR, '', ['nowiki', 'comment', 'math', 'pre', 'source'], marker = marker) return text.strip() @@ -3412,7 +3412,7 @@ # spaces and underscores in page titles are interchangeable, and collapsible title = title.replace(r"\ ", "[ _]+").replace(r"\_", "[ _]+") categoryR = re.compile(r'\[\[\s*(%s)\s*:\s*%s\s*((?:\|[^]]+)?\]\])' - % (catNamespace, title)) + % (catNamespace, title), re.I) if newcat is None: text = replaceExcept(oldtext, categoryR, '', ['nowiki', 'comment', 'math', 'pre', 'source'])

4 3

[Pywikipedia-l] SVN: [5399] trunk/pywikipedia/wikipedia.py
by nicdumz＠svn.wikimedia.org 18 May '08

18 May '08

Revision: 5399 Author: nicdumz Date: 2008-05-18 15:32:13 +0000 (Sun, 18 May 2008) Log Message: ----------- Less verbosity for getUrl. Anyway it is not always fetching HTML content: this warning does not make much sense when fetching XML... Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-05-18 14:51:39 UTC (rev 5398) +++ trunk/pywikipedia/wikipedia.py 2008-05-18 15:32:13 UTC (rev 5399) @@ -4287,7 +4287,8 @@ if m: charset = m.group(1) else: - output(u"WARNING: No character set found.") + if verbose: + output(u"WARNING: No character set found.") # UTF-8 as default charset = 'utf-8' # Check if this is the charset we expected

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1966402 ] wrong mediawiki_message when not using the default site lang
by SourceForge.net 18 May '08

18 May '08

Bugs item #1966402, was opened at 2008-05-18 12:18 Message generated for change (Comment added) made by nicdumz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1966402&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: General Group: None >Status: Closed >Resolution: Fixed Priority: 8 Private: No Submitted By: Nightshadow28 (nightshadow28) Assigned to: NicDumZ — Nicolas Dumazet (nicdumz) Summary: wrong mediawiki_message when not using the default site lang Initial Comment: Category.py aborts at full-protection page. [[w:ja:中田譲治]] is under full-protection, but if category.py tries to edit the article, bot will abort with following massage: Changing page [[ja:中田譲治]] ERROR: Unexpected response from wiki server. 200 (OK) (HTML) Dumping to category.dump.bz2, please wait... Traceback (most recent call last): File "category.py", line 945, in ? bot.run() File "category.py", line 432, in run inPlace=self.inPlace) File "/home/tatsujin/pywikipedia/catlib.py", line 473, in change_category article.put(text, comment) File "/home/tatsujin/pywikipedia/wikipedia.py", line 1262, in put newPage, self.site().getToken(sysop = sysop), sysop = sysop) File "/home/tatsujin/pywikipedia/wikipedia.py", line 1417, in _putPage return self._putPage(text, comment, watchArticle, minorEdit, newPage, token=self.site().getToken(sysop = sysop, getagain = True), newToken = True, sysop = sysop) File "/home/tatsujin/pywikipedia/wikipedia.py", line 1487, in _putPage raise PageNotSaved wikipedia.PageNotSaved ----- r5393 and bot account does not have adminship. ---------------------------------------------------------------------- >Comment By: NicDumZ — Nicolas Dumazet (nicdumz) Date: 2008-05-18 16:56 Message: Logged In: YES user_id=1963242 Originator: NO attempted to fix it in r5398. I tried fetching by hand several messages on several sites with different configurations : the first run is slow, since it now fetches the quite big Special:Allmessages, but at least the results match the user language... Any comments on the fix are welcome, as I'm not sure whether there's a better way to fix it or not... ---------------------------------------------------------------------- Comment By: Nightshadow28 (nightshadow28) Date: 2008-05-18 15:19 Message: Logged In: YES user_id=1971526 Originator: YES > You have set up your interface to use English messages, instead of the > default Japanese messages, have you ? Oh! Surely, I have set up English Interface for bot account because I thought that it might avoid localization problem. And I changed setting to Japanese Interface, bot succeeded skipping the page with detecting "locked" status as follows. ----- Checked for running processes. 1 processes currently running, including the current process. Reading dump from category.dump.bz2 Target page Category:東京都出身の人物 already exists! Getting [[Category:東京都出身の俳優]]... Getting 1 pages from wikipedia:ja... Sleeping for 5.2 seconds, 2008-05-18 22:07:29 Changing page [[ja:中田譲治]] Skipping 中田譲治 because page is locked Dumping to category.dump.bz2, please wait... ----- I am sorry to step the land mine... :< ---------------------------------------------------------------------- Comment By: NicDumZ — Nicolas Dumazet (nicdumz) Date: 2008-05-18 14:06 Message: Logged In: YES user_id=1963242 Originator: NO I just took a deeper look at result.txt, and now I understand things better : var wgUserLanguage = "en"; You have set up your interface to use English messages, instead of the default Japanese messages, have you ? Site.mediawiki_message('viewsource') is used to get the localized message of "view source" to match it against the page content, to know if the page is fully protected. However, it is simply checking for http://ja.wikipedia.org/wiki/MediaWiki:Viewsource , and this page is not wgUserLanguage dependant. It will always print the locale default message. In our case, it should return "view source", and it returns "wrongly" "ソースを表示" This bug is serious. It means that our messages detection is not working whenever a bot is not using the default site language. ---------------------------------------------------------------------- Comment By: NicDumZ — Nicolas Dumazet (nicdumz) Date: 2008-05-18 13:53 Message: Logged In: YES user_id=1963242 Originator: NO Thanks for your report :) I believe the fact that the page was fully protected was a coincidence here: I tried, in a python console, putting that same ja: Page, (with the intern _putPage method) and I got a LockedPage Error, instead of a PageNotSaved, which seems to indicate that the lock detection mechanism *is* properly working here. However, catlib.py::change_category was not catching all the PageNotSaved exceptions and I fixed that in r5396, meaning that such errors are not supposed to be reported to the end user anymore. If you still diagnose that some "Saving page ... failed" messages are always prompted for a particular type of full edit-protected pages, please, let us know. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1966402&group_…

1 0

[Pywikipedia-l] SVN: [5398] trunk/pywikipedia/wikipedia.py
by nicdumz＠svn.wikimedia.org 18 May '08

18 May '08

Revision: 5398 Author: nicdumz Date: 2008-05-18 14:51:39 +0000 (Sun, 18 May 2008) Log Message: ----------- Fixing #1966402 "wrong mediawiki_message when not using the default site lang" : Not fetching anymore every single message through Mediawiki:message_name, fetching instead *once* Special:Allmessages. It is slower indeed, but I think that this is the only way to get the properly localized messages. Any comments are welcome... Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2008-05-18 12:12:24 UTC (rev 5397) +++ trunk/pywikipedia/wikipedia.py 2008-05-18 14:51:39 UTC (rev 5398) @@ -146,6 +146,7 @@ except ValueError: WIDEBUILD = False + # Local exceptions class Error(Exception): @@ -4420,54 +4421,43 @@ def mediawiki_message(self, key): """Return the MediaWiki message text for key "key" """ - global mwpage, tree - if key.lower() not in self._mediawiki_messages.keys() \ - and not hasattr(self, "_phploaded"): - get_throttle() - mwpage = self.getUrl("%s?title=%s:%s&action=edit" - % (self.path(), urllib.quote( - self.namespace(8).replace(' ', '_').encode( - self.encoding())), - key)) - tree = BeautifulSoup(mwpage, - convertEntities=BeautifulSoup.HTML_ENTITIES, - parseOnlyThese=SoupStrainer("textarea")) - if tree.textarea is not None and tree.textarea.string is not None: - value = tree.textarea.string.strip() - else: - value = None - if value: - self._mediawiki_messages[key.lower()] = value - else: - self._mediawiki_messages[key.lower()] = None - # Fallback in case MediaWiki: page method doesn't work - if verbose: - output( - u"Retrieving mediawiki messages from Special:Allmessages") - retry_idle_time = 1 - while True: - get_throttle() - phppage = self.getUrl(self.get_address("Special:Allmessages") - + "&ot=php") - Rphpvals = re.compile(r"(?ms)'([^']*)' => '(.*?[^\\])',") - count = 0 - for (phpkey, phpval) in Rphpvals.findall(phppage): - count += 1 - self._mediawiki_messages[str(phpkey).lower()] = phpval - if count == 0: - # No messages could be added. - # We assume that the server is down. - # Wait some time, then try again. - output('WARNING: No messages found in Special:Allmessages. Maybe the server is down. Retrying in %i minutes...' % retry_idle_time) - time.sleep(retry_idle_time * 60) - # Next time wait longer, but not longer than half an hour - retry_idle_time *= 2 - if retry_idle_time > 30: - retry_idle_time = 30 - continue - break - self._phploaded = True + try: + from xml.etree.cElementTree import XML + except ImportError: + from cElementTree import XML + # Allmessages is retrieved once for all in a session + if not self._mediawiki_messages: + if verbose: + output( + u"Retrieving mediawiki messages from Special:Allmessages") + retry_idle_time = 1 + while True: + get_throttle() + xml = self.getUrl(self.get_address("Special:Allmessages") + + "&ot=xml") + decode = xml.encode(self.encoding()) + tree = XML(decode) + # xml structure is : + # <messages lang="fr"> + # <message name="about">À propos</message> + # ... + # </messages> + for msg in tree.getiterator('message'): + self._mediawiki_messages[msg.get('name').lower()] = msg.text + if not self._mediawiki_messages: + # No messages could be added. + # We assume that the server is down. + # Wait some time, then try again. + output('WARNING: No messages found in Special:Allmessages. Maybe the server is down. Retrying in %i minutes...' % retry_idle_time) + time.sleep(retry_idle_time * 60) + # Next time wait longer, but not longer than half an hour + retry_idle_time *= 2 + if retry_idle_time > 30: + retry_idle_time = 30 + continue + break + key = key.lower() if self._mediawiki_messages[key] is None: raise KeyError("MediaWiki key '%s' does not exist on %s"

1 0

← Newer
1
...
17
18
19
20
21
22
23
...
38
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot May 2008