pywikibot

pywikibot@lists.wikimedia.org

3 participants
6830 discussions

[Pywikipedia-l] [ pywikipediabot-Bugs-1787776 ] loading the same page twice leads to NoPage
by SourceForge.net 20 Sep '07

20 Sep '07

Bugs item #1787776, was opened at 2007-09-04 16:03 Message generated for change (Settings changed) made by wikipedian You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1787776&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Fixed Priority: 6 Private: No Submitted By: Andre Engels (a_engels) Assigned to: Nobody/Anonymous (nobody) Summary: loading the same page twice leads to NoPage Initial Comment: Note: This describes my impression of what is going on in this bug, I might be wrong as to what goes wrong. When there are two pages A and B, which both have an interwiki link to the same page C in another language, and those two pages are dealt with by interwiki.py at once, the second one will not find C and think that it is a non-existing page. I guess that points toward a mistake in the parsing of the output of [[Special:Export]]. ---------------------------------------------------------------------- Comment By: Andre Engels (a_engels) Date: 2007-09-05 09:53 Message: Logged In: YES user_id=843018 Originator: YES As a test case: interwiki.py -site:wikipedia -lang:nl -cat:Andretest -localonly This will try to work on two pages, both having an interwiki to the same (existing) page on en:. The first page will not be changed, but the second will report the English page as not existing. ---------------------------------------------------------------------- Comment By: Merlijn S. van Deen (valhallasw) Date: 2007-09-04 16:05 Message: Logged In: YES user_id=687283 Originator: NO Any test case or a 'how to reproduce'? ;) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1787776&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1798800 ] pagefromfile.py: bug with -notitle and -summary arguments
by SourceForge.net 20 Sep '07

20 Sep '07

Bugs item #1798800, was opened at 2007-09-20 16:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1798800&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roberto Zanon (qualc1) Assigned to: Nobody/Anonymous (nobody) Summary: pagefromfile.py: bug with -notitle and -summary arguments Initial Comment: The patch make working -summary (if specified -summary it uses it) and -notitle (it doesn't do anything if "-notitle" is used without this path). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1798800&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1795683 ] No error message if download is interrupted
by SourceForge.net 20 Sep '07

20 Sep '07

Bugs item #1795683, was opened at 2007-09-16 10:05 Message generated for change (Comment added) made by cosoleto You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1795683&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: General Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jani Patokallio (jpatokal) Assigned to: Nobody/Anonymous (nobody) Summary: No error message if download is interrupted Initial Comment: I'm using imagetransfer.py to download some images off a Mediawiki site, using the following very straightforward code: uo = wikipedia.MyURLopener() img = uo.open(url) file = open(targetFile, "w"); file.write(img.read()) file.close() img.close() However, I found out the hard way that there is no warning of any kind if the download is interrupted halfway through for any reason. Worse yet, there is no practical way to check if the file was downloaded successfully: the MD5 checksum function requires downloading the image and is thus subject to the same bug! The only way to determine even the file's actual size requires hacking through the file version history, and the getFileVersionHistory() command seems to break against the Mediawiki version of Wikitravel anyway. ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-20 16:14 Message: Logged In: YES user_id=181280 Originator: NO *in r4334, not r3443. ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-20 16:08 Message: Logged In: YES user_id=181280 Originator: NO getFileVersionHistory() problem fixed in revision 4305. upload_image() problems related to interrupted download improved in r3443. ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-17 18:23 Message: Logged In: YES user_id=181280 Originator: NO Try this: http://sourceforge.net/tracker/index.php?func=detail&aid=1796316&group_id=9…. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2007-09-17 04:11 Message: Logged In: NO So I managed to fix getFileVersionHistory by borrowing the code from an older version of pywikipediabot that works with Mediawiki 1.10.1, and now I can at least compare sizes. Can you make your patch available? ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-16 19:05 Message: Logged In: YES user_id=181280 Originator: NO I have tried to fix your getVersionHistory() problem. You can read file length in HTTP headers too: f.info().getheader('Content-Length') I have written a improvised and fully untested patch to wikipedia.getUrl() with check length and transfer resume feature, but I am not sure it's useful. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1795683&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1795683 ] No error message if download is interrupted
by SourceForge.net 20 Sep '07

20 Sep '07

Bugs item #1795683, was opened at 2007-09-16 10:05 Message generated for change (Comment added) made by cosoleto You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1795683&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: General Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jani Patokallio (jpatokal) Assigned to: Nobody/Anonymous (nobody) Summary: No error message if download is interrupted Initial Comment: I'm using imagetransfer.py to download some images off a Mediawiki site, using the following very straightforward code: uo = wikipedia.MyURLopener() img = uo.open(url) file = open(targetFile, "w"); file.write(img.read()) file.close() img.close() However, I found out the hard way that there is no warning of any kind if the download is interrupted halfway through for any reason. Worse yet, there is no practical way to check if the file was downloaded successfully: the MD5 checksum function requires downloading the image and is thus subject to the same bug! The only way to determine even the file's actual size requires hacking through the file version history, and the getFileVersionHistory() command seems to break against the Mediawiki version of Wikitravel anyway. ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-20 16:08 Message: Logged In: YES user_id=181280 Originator: NO getFileVersionHistory() problem fixed in revision 4305. upload_image() problems related to interrupted download improved in r3443. ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-17 18:23 Message: Logged In: YES user_id=181280 Originator: NO Try this: http://sourceforge.net/tracker/index.php?func=detail&aid=1796316&group_id=9…. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2007-09-17 04:11 Message: Logged In: NO So I managed to fix getFileVersionHistory by borrowing the code from an older version of pywikipediabot that works with Mediawiki 1.10.1, and now I can at least compare sizes. Can you make your patch available? ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-16 19:05 Message: Logged In: YES user_id=181280 Originator: NO I have tried to fix your getVersionHistory() problem. You can read file length in HTTP headers too: f.info().getheader('Content-Length') I have written a improvised and fully untested patch to wikipedia.getUrl() with check length and transfer resume feature, but I am not sure it's useful. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1795683&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Patches-1796316 ] upload.py, upload_image(), check length and resume feature
by SourceForge.net 20 Sep '07

20 Sep '07

Patches item #1796316, was opened at 2007-09-17 18:20 Message generated for change (Comment added) made by cosoleto You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1796316&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Francesco Cosoleto (cosoleto) Assigned to: Nobody/Anonymous (nobody) Summary: upload.py, upload_image(), check length and resume feature Initial Comment: Untested. ---------------------------------------------------------------------- >Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-20 15:52 Message: Logged In: YES user_id=181280 Originator: YES Fixed again, tested and applied in 4334. ---------------------------------------------------------------------- Comment By: Francesco Cosoleto (cosoleto) Date: 2007-09-18 19:06 Message: Logged In: YES user_id=181280 Originator: YES Updated. It seems work and more safe than previous code. Not fully tested with real server. File Added: upload.py.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1796316&group_…

1 0

[Pywikipedia-l] SVN: [4334] trunk/pywikipedia/upload.py
by cosoleto＠svn.wikimedia.org 20 Sep '07

20 Sep '07

Revision: 4334 Author: cosoleto Date: 2007-09-20 13:51:30 +0000 (Thu, 20 Sep 2007) Log Message: ----------- upload.py: check length and resume feature in upload_image() Modified Paths: -------------- trunk/pywikipedia/upload.py Modified: trunk/pywikipedia/upload.py =================================================================== --- trunk/pywikipedia/upload.py 2007-09-19 22:21:42 UTC (rev 4333) +++ trunk/pywikipedia/upload.py 2007-09-20 13:51:30 UTC (rev 4334) @@ -22,7 +22,7 @@ # __version__='$Id$' -import os, sys, re +import os, sys, re, time import urllib, httplib import wikipedia, config @@ -74,6 +74,7 @@ file would be overwritten or another mistake would be risked. """ + self._retrieved = False self.url = url self.urlEncoding = urlEncoding self.description = description @@ -100,22 +101,62 @@ If the upload fails, the user is asked whether to try again or not. If the user chooses not to retry, returns null. """ - if not hasattr(self, "_contents"): + + if not self._retrieved: # Get file contents + wikipedia.output(u'Reading file %s' % self.url) if '://' in self.url: - uo = wikipedia.MyURLopener() - file = uo.open(self.url) + resume = False + dt = 15 + + while not self._retrieved: + uo = wikipedia.MyURLopener() + if resume: + wikipedia.output(u"Resume download...") + uo.addheader('Range', 'bytes=%s-' % rlen) + + file = uo.open(self.url) + + if 'text/html' in file.info().getheader('Content-Type'): + print "Couldn't download the image: the requested URL was not found on this server." + return + + content_len = file.info().getheader('Content-Length') + accept_ranges = file.info().getheader('Accept-Ranges') == 'bytes' + + if resume: + self._contents += file.read() + else: + self._contents = file.read() + + file.close() + self._retrieved = True + + if content_len: + rlen = len(self._contents) + content_len = int(content_len) + if rlen < content_len: + self._retrieved = False + wikipedia.output(u"Connection closed at byte %s (%s left)" % (rlen, content_len)) + if accept_ranges and rlen > 0: + resume = True + wikipedia.output(u"Sleeping for %d seconds..." % dt) + time.sleep(dt) + if dt <= 60: + dt += 15 + elif dt < 360: + dt += 60 + else: + if wikipedia.verbose: + wikipedia.output(u"WARNING: No check length to retrieved data is possible.") else: # Opening local files with MyURLopener would be possible, but we # don't do it because it only accepts ASCII characters in the # filename. file = open(self.url,"rb") - wikipedia.output(u'Reading file %s' % self.url) - self._contents = file.read() - if self._contents.find("The requested URL was not found on this server.") != -1: - print "Couldn't download the image." - return - file.close() + self_contents = file.read() + file.close() + # Isolate the pure name filename = self.url if '/' in filename: @@ -169,7 +210,7 @@ # if user saved / didn't press Cancel if newDescription: self.description = newDescription - + formdata = {} formdata["wpUploadDescription"] = self.description # if self.targetSite.version() >= '1.5':

1 0

[Pywikipedia-l] [ pywikipediabot-Patches-1798753 ] Fixed the \n problem
by SourceForge.net 20 Sep '07

20 Sep '07

Patches item #1798753, was opened at 2007-09-20 13:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1798753&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Filnik (filnik) Assigned to: Nobody/Anonymous (nobody) Summary: Fixed the \n problem Initial Comment: Some times ago, I've asked to fix the bug in replace.py that doesn't replace \n with a newline. I've add two lines on wikipedia.py to prevent this bug (it works and shouldn't make problems also if it is done in a simple way). The patch for wikipedia.py is attached. Filnik ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1798753&group_…

1 0

[Pywikipedia-l] SVN: [4333] trunk/pywikipedia
by wikipedian＠svn.wikimedia.org 19 Sep '07

19 Sep '07

Revision: 4333 Author: wikipedian Date: 2007-09-19 22:21:42 +0000 (Wed, 19 Sep 2007) Log Message: ----------- updated scn: and nds-nl: namespaces Modified Paths: -------------- trunk/pywikipedia/families/wikipedia_family.py trunk/pywikipedia/family.py Modified: trunk/pywikipedia/families/wikipedia_family.py =================================================================== --- trunk/pywikipedia/families/wikipedia_family.py 2007-09-19 13:53:07 UTC (rev 4332) +++ trunk/pywikipedia/families/wikipedia_family.py 2007-09-19 22:21:42 UTC (rev 4333) @@ -173,6 +173,7 @@ 'nah': u'Wikipedia Discusión', 'nap': u'Discussioni Wikipedia', 'nds': u'Wikipedia Diskuschoon', + 'nds-nl': u'Overleg Wikipedie', 'new': u'विकिपिडिया खँलाबँला', 'nl': u'Overleg Wikipedia', 'nn': u'Wikipedia-diskusjon', Modified: trunk/pywikipedia/family.py =================================================================== --- trunk/pywikipedia/family.py 2007-09-19 13:53:07 UTC (rev 4332) +++ trunk/pywikipedia/family.py 2007-09-19 22:21:42 UTC (rev 4333) @@ -191,6 +191,7 @@ 'rmy':u'Uzalutno', 'ru': u'Служебная', 'sc': u'Speciale', + 'scn': u'Speciale', 'sk': u'Špeciálne', 'sl': u'Posebno', 'sq': u'Speciale', @@ -315,6 +316,7 @@ 'ru': u'Обсуждение', 'sa': u'संभाषणं', 'sc': u'Contièndha', + 'scn': u'Discussione', 'sk': u'Diskusia', 'sl': u'Pogovor', 'sq': u'Diskutim', @@ -438,6 +440,7 @@ 'ru': u'Участник', 'sa': u'योजकः', 'sc': u'Utente', + 'scn': u'Utente', 'sk': u'Redaktor', 'sl': u'Uporabnik', 'sq': u'Përdoruesi', @@ -562,6 +565,7 @@ 'ru': u'Обсуждение участника', 'sa': u'योजकसंभाषणं', 'sc': u'Utente discussioni', + 'scn': u'Discussioni utente', 'sk': u'Diskusia s redaktorom', 'sl': u'Uporabniški pogovor', 'sq': u'Përdoruesi diskutim', @@ -691,6 +695,7 @@ 'ru': u'Изображение', 'sa': u'चित्रं', 'sc': u'Immàgini', + 'scn': u'Immagine', 'sk': u'Obrázok', 'sl': u'Slika', 'sq': u'Figura', @@ -813,6 +818,7 @@ 'ru': u'Обсуждение изображения', 'sa': u'चित्रसंभाषणं', 'sc': u'Immàgini contièndha', + 'scn': u'Discussioni immagine', 'sk': u'Diskusia k obrázku', 'sl': u'Pogovor o sliki', 'sq': u'Figura diskutim', @@ -966,6 +972,7 @@ 'rmy':[u'MediyaViki vakyarimata', u'Discuţie MediaWiki'], 'ro': u'Discuţie MediaWiki', 'ru': u'Обсуждение MediaWiki', + 'scn': u'Discussioni MediaWiki', 'sk': u'Diskusia k MediaWiki', 'sl': u'Pogovor o MediaWiki', 'sq': u'MediaWiki diskutim', @@ -1201,6 +1208,7 @@ 'rmy':[u'Sikavno vakyarimata', u'Discuţie Format'], 'ro': u'Discuţie Format', 'ru': u'Обсуждение шаблона', + 'scn': u'Discussioni template', 'sk': u'Diskusia k šablóne', 'sl': u'Pogovor o predlogi', 'sq': u'Stampa diskutim', @@ -1319,6 +1327,7 @@ 'ro': u'Ajutor', 'ru': u'Справка', 'sa': u'उपकारः', + 'scn': u'Aiuto', 'sk': u'Pomoc', 'sl': u'Pomoč', 'sq': u'Ndihmë', @@ -1439,6 +1448,7 @@ 'ro': u'Discuţie Ajutor', 'ru': u'Обсуждение справки', 'sa': u'उपकारसंभाषणं', + 'scn': u'Discussioni aiuto', 'sk': u'Diskusia k pomoci', 'sl': u'Pogovor o pomoči', 'sq': u'Ndihmë diskutim', @@ -1560,6 +1570,7 @@ 'ro': u'Categorie', 'ru': u'Категория', 'sa': u'वर्गः', + 'scn': u'Categoria', 'sk': u'Kategória', 'sl': u'Kategorija', 'sr': u'Категорија', @@ -1681,6 +1692,7 @@ 'ro': u'Discuţie Categorie', 'ru': u'Обсуждение категории', 'sa': u'वर्गसंभाषणं', + 'scn': u'Discussioni categoria', 'sk': u'Diskusia ku kategórii', 'sl': u'Pogovor o kategoriji', 'sr': u'Разговор о категорији',

1 0

[Pywikipedia-l] SVN: [4332] trunk/pywikipedia/pagegenerators.py
by wikipedian＠svn.wikimedia.org 19 Sep '07

19 Sep '07

Revision: 4332 Author: wikipedian Date: 2007-09-19 13:53:07 +0000 (Wed, 19 Sep 2007) Log Message: ----------- repaired -subcat parameter Modified Paths: -------------- trunk/pywikipedia/pagegenerators.py Modified: trunk/pywikipedia/pagegenerators.py =================================================================== --- trunk/pywikipedia/pagegenerators.py 2007-09-19 12:35:52 UTC (rev 4331) +++ trunk/pywikipedia/pagegenerators.py 2007-09-19 13:53:07 UTC (rev 4332) @@ -594,7 +594,7 @@ startfrom = None cat = catlib.Category(wikipedia.getSite(), 'Category:%s' % categoryname) - return CategorizedPageGenerator(cat, start = startfrom, recurse = False) + return CategorizedPageGenerator(cat, start = startfrom, recurse = recurse) def handleArg(self, arg): gen = None

1 0

[Pywikipedia-l] SVN: [4331] trunk/pywikipedia/catlib.py
by russblau＠svn.wikimedia.org 19 Sep '07

19 Sep '07

Revision: 4331 Author: russblau Date: 2007-09-19 12:35:52 +0000 (Wed, 19 Sep 2007) Log Message: ----------- bugfix (article instead of page); break long lines Modified Paths: -------------- trunk/pywikipedia/catlib.py Modified: trunk/pywikipedia/catlib.py =================================================================== --- trunk/pywikipedia/catlib.py 2007-09-19 09:02:24 UTC (rev 4330) +++ trunk/pywikipedia/catlib.py 2007-09-19 12:35:52 UTC (rev 4331) @@ -441,10 +441,11 @@ % article.title()) except wikipedia.PageNotSaved, error: wikipedia.output(u"Saving page [[%s]] failed: %s" - % (page.title(), error.message)) + % (article.title(), error.message)) return - # This loop will replace all occurrences of the category to be changed, and remove duplicates. + # This loop will replace all occurrences of the category to be changed, + # and remove duplicates. newCatList = [] newCatSet = set() for i in range(len(cats)): @@ -455,7 +456,8 @@ sortKey = cat.sortKey if newCat: if newCat.title() not in newCatSet: - newCategory = Category(site, newCat.title(), sortKey = sortKey) + newCategory = Category(site, newCat.title(), + sortKey=sortKey) newCatSet.add(newCat.title()) newCatList.append(newCategory) elif cat.title() not in newCatSet: @@ -468,8 +470,11 @@ text = article.get(nofollow_redirects=True) try: text = wikipedia.replaceCategoryLinks(text, newCatList) - except ValueError: #Make sure that the only way replaceCategoryLinks() can return a ValueError is in the case of interwiki links to self. - wikipedia.output(u'Skipping %s because of interwiki link to self' % (article)) + except ValueError: + # Make sure that the only way replaceCategoryLinks() can return + # a ValueError is in the case of interwiki links to self. + wikipedia.output( + u'Skipping %s because of interwiki link to self' % article) try: article.put(text, comment) except wikipedia.EditConflict:

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot