pywikibot

pywikibot@lists.wikimedia.org

3 participants
6830 discussions

[Pywikipedia-l] SVN: [4623] trunk/pywikipedia/wikipedia.py
by rotem＠svn.wikimedia.org 01 Dec '07

01 Dec '07

Revision: 4623 Author: rotem Date: 2007-12-01 10:17:01 +0000 (Sat, 01 Dec 2007) Log Message: ----------- (bug 1841155) replaceImage removes templates curly brackets. Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2007-12-01 10:03:41 UTC (rev 4622) +++ trunk/pywikipedia/wikipedia.py 2007-12-01 10:17:01 UTC (rev 4623) @@ -2187,7 +2187,7 @@ fileReferenceR = re.compile(u'%s(?P<filename>(?:%s)?)' % (namespacePattern, imagePattern)) def template_replacer(match): - return fileReferenceR.sub(filename_replacer, match.group('contents')) + return fileReferenceR.sub(filename_replacer, match.group(0)) new_text = templateR.sub(template_replacer, new_text) if put:

1 0

[Pywikipedia-l] SVN: [4622] trunk/pywikipedia/wikipedia.py
by rotem＠svn.wikimedia.org 01 Dec '07

01 Dec '07

Revision: 4622 Author: rotem Date: 2007-12-01 10:03:41 +0000 (Sat, 01 Dec 2007) Log Message: ----------- Fix escaping order, so you do not escape and then put the backslashes in other brackets, in wikipedia.Page.replaceImage. This means namespaces with non-ASCII (?) chars work now. Modified Paths: -------------- trunk/pywikipedia/wikipedia.py Modified: trunk/pywikipedia/wikipedia.py =================================================================== --- trunk/pywikipedia/wikipedia.py 2007-12-01 04:44:02 UTC (rev 4621) +++ trunk/pywikipedia/wikipedia.py 2007-12-01 10:03:41 UTC (rev 4622) @@ -2134,9 +2134,8 @@ """ Creates a pattern that matches the string case-insensitively. """ - s = re.escape(s) return ur'(?:%s)' % u''.join([u'[%s%s]' - % (c.upper(), c.lower()) + % (re.escape(c.upper()), re.escape(c.lower())) for c in s]) def capitalizationPattern(s): @@ -2145,11 +2144,10 @@ the first letter case-insensitive if capitalization is switched on on the site you're working on. """ - s = re.escape(s) if self.site().nocapitalize: - return s + return re.escape(s) else: - return ur'(?:[%s%s]%s)' % (s[0].upper(), s[0].lower(), s[1:]) + return ur'(?:[%s%s]%s)' % (re.escape(s[0].upper()), re.escape(s[0].lower()), re.escape(s[1:])) namespaces = set(('Image', 'Media') + site.namespace(6, all = True) + site.namespace(-2, all = True)) # note that the colon is already included here

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1841155 ] Page's function, removeImage doesn't work.
by SourceForge.net 01 Dec '07

01 Dec '07

Bugs item #1841155, was opened at 2007-11-29 20:12 Message generated for change (Comment added) made by rotemliss You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1841155&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Filnik (filnik) Assigned to: Nobody/Anonymous (nobody) Summary: Page's function, removeImage doesn't work. Initial Comment: I've used that function, here: http://it.wikipedia.org/w/index.php?title=Massagno&diff=prev&oldid=12586322 but seems to be a really big bug... some help? :-) ---------------------------------------------------------------------- Comment By: Rotem Liss (rotemliss) Date: 2007-12-01 11:48 Message: Logged In: YES user_id=1327030 Originator: NO The function: 1. Doesn't handle some (or all?) localized namespaces properly. 2. Strips templates when it doesn't find the image. Obviously, this mustn't happen. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1841155&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Patches-1842094 ] Update namespaces for Ido
by SourceForge.net 01 Dec '07

01 Dec '07

Patches item #1842094, was opened at 2007-12-01 02:57 Message generated for change (Comment added) made by malafaya You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1842094&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Resolution: None Priority: 5 Private: No Submitted By: André Malafaya Baptista (malafaya) Assigned to: Nobody/Anonymous (nobody) Summary: Update namespaces for Ido Initial Comment: Please apply attached file containing new namespaces names for language Ido. Thanks! ---------------------------------------------------------------------- >Comment By: André Malafaya Baptista (malafaya) Date: 2007-12-01 04:45 Message: Logged In: YES user_id=1037345 Originator: YES Fixed in r4621 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1842094&group_…

1 0

[Pywikipedia-l] SVN: [4621] trunk/pywikipedia/family.py
by malafaya＠svn.wikimedia.org 01 Dec '07

01 Dec '07

Revision: 4621 Author: malafaya Date: 2007-12-01 04:44:02 +0000 (Sat, 01 Dec 2007) Log Message: ----------- (Patch 1842094) Update namespaces for Ido Modified Paths: -------------- trunk/pywikipedia/family.py Modified: trunk/pywikipedia/family.py =================================================================== --- trunk/pywikipedia/family.py 2007-11-30 22:25:25 UTC (rev 4620) +++ trunk/pywikipedia/family.py 2007-12-01 04:44:02 UTC (rev 4621) @@ -163,6 +163,7 @@ 'hu': u'Speciális', 'hy': u'Սպասարկող', 'id': u'Istimewa', + 'io': u'Specala', 'is': u'Kerfissíða', 'it': u'Speciale', 'ja': u'特別', @@ -292,6 +293,7 @@ 'hy': u'Քննարկում', 'ia': u'Discussion', 'id': [u'Pembicaraan', u'Bicara'], + 'io': u'Diskuto', 'is': u'Spjall', 'it': u'Discussione', 'ja': u'ノート', @@ -426,6 +428,7 @@ 'hy': u'Մասնակից', 'ia': u'Usator', 'id': u'Pengguna', + 'io': u'Uzanto', 'is': u'Notandi', 'it': u'Utente', 'ja': u'利用者', @@ -561,6 +564,7 @@ 'hy': u'Մասնակցի քննարկում', 'ia': u'Discussion Usator', 'id': u'Pembicaraan Pengguna', + 'io': u'Uzanto Diskuto', 'is': u'Notandaspjall', 'it': u'Discussioni utente', 'ja': u'利用者‐会話', @@ -700,6 +704,7 @@ 'hy': u'Պատկեր', 'ia': u'Imagine', 'id': [u'Berkas', u'Gambar'], + 'io': u'Imajo', 'is': u'Mynd', 'it': u'Immagine', 'ja': u'画像', @@ -832,6 +837,7 @@ 'hy': u'Պատկերի քննարկում', 'ia': u'Discussion Imagine', 'id': [u'Pembicaraan Berkas', u'Pembicaraan Gambar'], + 'io': u'Imajo Diskuto', 'is': u'Myndaspjall', 'it': u'Discussioni immagine', 'ja': u'画像‐ノート', @@ -1002,6 +1008,7 @@ 'hy': u'MediaWiki քննարկում', 'ia': u'Discussion MediaWiki', 'id': u'Pembicaraan MediaWiki', + 'io': u'MediaWiki Diskuto', 'is': u'Meldingarspjall', 'it': u'Discussioni MediaWiki', 'ja': u'MediaWiki‐ノート', @@ -1134,6 +1141,7 @@ 'hy': u'Կաղապար', 'ia': u'Patrono', 'id': u'Templat', + 'io': u'Shablono', 'is': u'Snið', 'jv': u'Cithakan', 'ka': u'თარგი', @@ -1258,6 +1266,7 @@ 'hy': u'Կաղապարի քննարկում', 'ia': u'Discussion Patrono', 'id': u'Pembicaraan Templat', + 'io': u'Shablono Diskuto', 'is': u'Sniðaspjall', 'it': u'Discussioni template', 'ja': u'Template‐ノート', @@ -1388,6 +1397,7 @@ 'hy': u'Օգնություն', 'ia': u'Adjuta', 'id': u'Bantuan', + 'io': u'Helpo', 'is': u'Hjálp', 'it': u'Aiuto', 'jv': u'Pitulung', @@ -1516,6 +1526,7 @@ 'hy': u'Օգնության քննարկում', 'ia': u'Discussion Adjuta', 'id': u'Pembicaraan Bantuan', + 'io': u'Helpo Diskuto', 'is': u'Hjálparspjall', 'it': u'Discussioni aiuto', 'ja': u'Help‐ノート', @@ -1648,6 +1659,7 @@ 'hy': u'Կատեգորիա', 'ia': u'Categoria', 'id': u'Kategori', + 'io': u'Kategorio', 'is': u'Flokkur', 'it': u'Categoria', 'jv': u'Kategori', @@ -1779,6 +1791,7 @@ 'hy': u'Կատեգորիայի քննարկում', 'ia': u'Discussion Categoria', 'id': u'Pembicaraan Kategori', + 'io': u'Kategorio Diskuto', 'is': u'Flokkaspjall', 'it': u'Discussioni categoria', 'ja': u'Category‐ノート',

1 0

[Pywikipedia-l] [ pywikipediabot-Patches-1842094 ] Update namespaces for Ido
by SourceForge.net 01 Dec '07

01 Dec '07

Patches item #1842094, was opened at 2007-12-01 02:57 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1842094&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: André Malafaya Baptista (malafaya) Assigned to: Nobody/Anonymous (nobody) Summary: Update namespaces for Ido Initial Comment: Please apply attached file containing new namespaces names for language Ido. Thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1842094&group_…

1 0

[Pywikipedia-l] SVN: [4620] trunk/pywikipedia/commonsdelinker/image_replacer.py
by btongminh＠svn.wikimedia.org 30 Nov '07

30 Nov '07

Revision: 4620 Author: btongminh Date: 2007-11-30 22:25:25 +0000 (Fri, 30 Nov 2007) Log Message: ----------- Reconnect on connection failure Modified Paths: -------------- trunk/pywikipedia/commonsdelinker/image_replacer.py Modified: trunk/pywikipedia/commonsdelinker/image_replacer.py =================================================================== --- trunk/pywikipedia/commonsdelinker/image_replacer.py 2007-11-30 05:31:33 UTC (rev 4619) +++ trunk/pywikipedia/commonsdelinker/image_replacer.py 2007-11-30 22:25:25 UTC (rev 4620) @@ -90,6 +90,8 @@ # Network error, not critical output(u'Warning! Unable to read replacement log.', False) output('%s: %s' % (e.__class__.__name__, str(e)), False) + self.site.conn.close() + self.site.conn.connect() return time.sleep(self.config['timeout']) if '{{stop}}' in text.lower():

1 0

[Pywikipedia-l] SVN: [4619] trunk/pywikipedia
by yurik＠svn.wikimedia.org 30 Nov '07

30 Nov '07

Revision: 4619 Author: yurik Date: 2007-11-30 05:31:33 +0000 (Fri, 30 Nov 2007) Log Message: ----------- Updated query library & casechacker script to use the new API Modified Paths: -------------- trunk/pywikipedia/casechecker.py trunk/pywikipedia/query.py Modified: trunk/pywikipedia/casechecker.py =================================================================== --- trunk/pywikipedia/casechecker.py 2007-11-29 16:16:45 UTC (rev 4618) +++ trunk/pywikipedia/casechecker.py 2007-11-30 05:31:33 UTC (rev 4619) @@ -74,9 +74,9 @@ langs = { 'ru': { - 'alphabet' : u'АаБбВвГгДдЕеЁёЖжЗзИиЙйКкЛлМмНнОоПпРрСсТтУуФфХхЦцЧчШшЩщЪъЫыЬьЭэЮюЯя', - 'localsuspects': u'АаВЕеКкМНОоРрСсТуХх', - 'latinsuspects': u'AaBEeKkMHOoPpCcTyXx', + 'alphabet' : u'АаБбВвГгДдЕеЁёЖжЗзИиЙйКкЛлМмНнОоПпРрСсТтУуФфХхЦцЧчШшЩщЪъЫыЬьЭэЮюЯяІі', + 'localsuspects': u'АаВЕеКкМНОоРрСсТуХхІі', + 'latinsuspects': u'AaBEeKkMHOoPpCcTyXxIi', }, 'uk': { 'alphabet' : u'АаБбВвГгҐґДдЕеЄєЖжЗзИиІіЇїЙйКкЛлМмНнОоПпРрСсТтУуФфХхЦцЧчШшЩщЮюЯяЬь', @@ -111,7 +111,6 @@ title = None replace = False stopAfter = 0 - verbose = False wikilog = None wikilogfile = 'wikilog.txt' autonomous = False @@ -136,9 +135,7 @@ self.replace = True elif arg.startswith('-limit:'): self.stopAfter = int(arg[7:]) - elif arg == '-verbose': - self.verbose = True - elif arg == '-autonomous': + elif arg == '-autonomous' or arg == '-a': self.autonomous = True elif arg.startswith('-ns:'): self.namespaces.append( int(arg[4:]) ) @@ -155,14 +152,15 @@ else: self.namespaces = [0] - self.params = {'what' : 'allpages', - 'aplimit' : self.aplimit, - 'apfilterredir' : 'nonredirects', - 'noprofile' : '' } - + self.params = { 'action' : 'query', + 'generator' : 'allpages', + 'gaplimit' : self.aplimit, + 'gapfilterredir': 'nonredirects'} + if self.links: - self.params['what'] += '|links|categories'; + self.params['prop'] = 'links|categories' + self.site = wikipedia.getSite() if self.site.lang in self.langs: @@ -200,46 +198,53 @@ count = 0 lastLetter = '' for namespace in self.namespaces: - self.params['apnamespace'] = namespace + self.params['gapnamespace'] = namespace title = None while True: # Get data - self.params['apfrom'] = self.apfrom - data = query.GetData(self.site.lang, self.params, self.verbose) + self.params['gapfrom'] = self.apfrom + data = query.GetData(self.site.lang, self.params, wikipedia.verbose, True) try: - self.apfrom = data['query']['allpages']['next'] + self.apfrom = data['query-continue']['allpages']['gapfrom'] except: self.apfrom = None # Process received data - if 'pages' in data: + if 'query' in data and 'pages' in data['query']: firstItem = True - for pageID, page in data['pages'].iteritems(): + for pageID, page in data['query']['pages'].iteritems(): printed = False title = page['title'] if firstItem: if lastLetter != title[0]: - print 'Processing ' + title + try: + print 'Processing ' + title + except: + print 'Processing unprintable title' lastLetter = title[0] firstItem = False if self.titles: err = self.ProcessTitle(title) if err: - if page['ns'] == 14: - self.WikiLog(u"* Move category content: " + err[0]) - else: - changed = False - if self.replace: - newTitle = self.PickTarget(False, title, title, err[1]) - if newTitle: - src = wikipedia.Page(self.site, title) - src.move( newTitle, wikipedia.translate(self.site, self.msgRename)) - changed = True + changed = False + if self.replace: + newTitle = self.PickTarget(False, title, title, err[1]) + if newTitle: + editSummary = wikipedia.translate(self.site, self.msgRename) + src = wikipedia.Page(self.site, title) + if page['ns'] == 14: + import category + dst = wikipedia.Page(self.site, newTitle) + bot = category.CategoryMoveRobot(src.titleWithoutNamespace(), dst.titleWithoutNamespace(), self.autonomous, editSummary, True) + bot.run() + else: + src.move(newTitle, editSummary) + changed = True - if not changed: - self.WikiLog(u"* " + err[0]) - printed = True + if not changed: + self.WikiLog(u"* " + err[0]) + printed = True if self.links: allLinks = None @@ -257,7 +262,7 @@ msg = [] for l in allLinks: - ltxt = l['*'] + ltxt = l['title'] err = self.ProcessTitle(ltxt) if err: newTitle = None Modified: trunk/pywikipedia/query.py =================================================================== --- trunk/pywikipedia/query.py 2007-11-29 16:16:45 UTC (rev 4618) +++ trunk/pywikipedia/query.py 2007-11-30 05:31:33 UTC (rev 4619) @@ -4,8 +4,9 @@ import wikipedia import simplejson import urllib +import time -def GetData( lang, params, verbose = False ): +def GetData( lang, params, verbose = False, useAPI = False, retryCount = 5 ): """Get data from the query api, and convert it into a data object """ site = wikipedia.getSite( lang ) @@ -15,7 +16,9 @@ params[k] = unicode(v) params['format'] = 'json' - params['noprofile'] = '' + + if not useAPI: + params['noprofile'] = '' for k,v in params.iteritems(): if type(v) == type(u''): @@ -30,29 +33,45 @@ data = None titlecount = 0 - path = u"/w/query.php?" + urllib.urlencode( params.items() ) + if useAPI: + path = site.api_address() + urllib.urlencode( params.items() ) + else: + path = site.query_address() + urllib.urlencode( params.items() ) if verbose: - wikipedia.output( u"Requesting %d titles from %s:%s" % (titlecount, lang, path) ) + if titlecount > 0: + wikipedia.output( u"Requesting %d titles from %s:%s" % (titlecount, lang, path) ) + else: + wikipedia.output( u"Request %s:%s" % (lang, path) ) - url = site.family.querypath(lang) - - retryCount = 1 - + lastError = None + retry_idle_time = 5 while retryCount >= 0: try: + jsontext = "Nothing received" jsontext = site.getUrl( path, retry=True, data=data ) # This will also work, but all unicode strings will need to be converted from \u notation # decodedObj = eval( jsontext ) - decodedObj = simplejson.loads( jsontext ) + return simplejson.loads( jsontext ) break except ValueError, error: retryCount -= 1 wikipedia.output( u"Error downloading data: %s" % error ) + wikipedia.output( u"Request %s:%s" % (lang, path) ) + wikipedia.debugDump('ApiGetDataParse', site, str(error) + '\n' + path, jsontext) + lastError = error + if retryCount >= 0: + wikipedia.output( u"Retrying in %i seconds..." % retry_idle_time ) + time.sleep(retry_idle_time) + # Next time wait longer, but not longer than half an hour + retry_idle_time *= 2 + if retry_idle_time > 300: + retry_idle_time = 300 + - return decodedObj + raise lastError def GetInterwikies( lang, titles, extraParams = None ): """ Usage example: data = GetInterwikies('ru','user:yurik')

1 0

[Pywikipedia-l] [ pywikipediabot-Patches-1814580 ] save page generator output
by SourceForge.net 30 Nov '07

30 Nov '07

Patches item #1814580, was opened at 2007-10-16 09:36 Message generated for change (Comment added) made by sf-robot You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1814580&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Resolution: Invalid Priority: 5 Private: No Submitted By: John Vandenberg (zeroj) Assigned to: Nobody/Anonymous (nobody) Summary: save page generator output Initial Comment: When calling pagegenerators.py directly from the command line, it is handy to save the output for post-processing or use as input into replace.py -file page generator. This patch provides a -output argument. As this patch does not handle unicode page names, it needs improvement before it should be committed. ---------------------------------------------------------------------- >Comment By: SourceForge Robot (sf-robot) Date: 2007-11-29 19:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Daniel Herding (wikipedian) Date: 2007-10-17 02:10 Message: Logged In: YES user_id=880694 Originator: NO Is this really needed? I mean, you can do this: daniel@localhost:~/projekte/pywikipedia> python pagegenerators.py -ref:Wikipedia:Pywikipediabot > output.txt Checked for running processes. 2 processes currently running, including the current process. Getting references to [[Wikipedia:Pywikipediabot]] daniel@localhost:~/projekte/pywikipedia> cat output.txt Benutzer:Head Wikipedia:Selbstlinks Benutzer:Zwobot Benutzer Diskussion:Waugsberg/Archiv/2006-8 Wikipedia:Bots [etc.] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603140&aid=1814580&group_…

1 0

[Pywikipedia-l] [ pywikipediabot-Bugs-1809991 ] welcome.py time bug
by SourceForge.net 30 Nov '07

30 Nov '07

Bugs item #1809991, was opened at 2007-10-09 00:39 Message generated for change (Comment added) made by sf-robot You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1809991&group_… Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: other Group: None >Status: Closed Resolution: Invalid Priority: 5 Private: No Submitted By: shizhao (wikishizhao) Assigned to: Leonardo Gregianin (leogregianin) Summary: welcome.py time bug Initial Comment: welcome.py use localtime() add welcome log sub page. plese use gmtime(), is UTC time. ---------------------------------------------------------------------- >Comment By: SourceForge Robot (sf-robot) Date: 2007-11-29 19:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Leonardo Gregianin (leogregianin) Date: 2007-11-15 03:05 Message: Logged In: YES user_id=1136737 Originator: NO wikipedia.py and all bots using localtime() ---------------------------------------------------------------------- Comment By: shizhao (wikishizhao) Date: 2007-10-17 17:22 Message: Logged In: YES user_id=1853316 Originator: YES localtime() => gmtime() File Added: welcome.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1809991&group_…

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

pywikibot