Pywikibot-commits June 2017

pywikibot-commits@lists.wikimedia.org

2 participants
96 discussions

[Gerrit] pywikibot/core[master]: [IMPR] Improve maintenance/wikimedia_sites.py
by jenkins-bot (Code Review) 30 Jun '17

30 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/358040 ) Change subject: [IMPR] Improve maintenance/wikimedia_sites.py ...................................................................... [IMPR] Improve maintenance/wikimedia_sites.py Change-Id: Id660b7bffd1381f4884507b9fb097b0701736257 --- M scripts/maintenance/wikimedia_sites.py 1 file changed, 2 insertions(+), 3 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/scripts/maintenance/wikimedia_sites.py b/scripts/maintenance/wikimedia_sites.py index a9df474..3a0a8e6 100755 --- a/scripts/maintenance/wikimedia_sites.py +++ b/scripts/maintenance/wikimedia_sites.py @@ -86,9 +86,8 @@ family_file_name = 'pywikibot/families/%s_family.py' % family family_file = codecs.open(family_file_name, 'r', 'utf8') family_text = family_file.read() - old = re.findall(r'(?msu)^ {8}self.languages_by_size.+?\]', - family_text)[0] - family_text = family_text.replace(old, text) + family_text = re.sub(r'(?msu)^ {8}self.languages_by_size.+?\]', + text, family_text) with codecs.open(family_file_name, 'w', 'utf8') as family_file: family_file.write(family_text) -- To view, visit https://gerrit.wikimedia.org/r/358040 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Id660b7bffd1381f4884507b9fb097b0701736257 Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Framawiki <framawiki(a)tools.wmflabs.org> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: proofreadpage.py: add support for OCR feature
by jenkins-bot (Code Review) 30 Jun '17

30 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/360575 ) Change subject: proofreadpage.py: add support for OCR feature ...................................................................... proofreadpage.py: add support for OCR feature ProofreadPage: - allow to perform OCR via https://tools.wmflabs.org/phetools tool wikisourcetext.py: - make use of the new feature to allow OCR. Bug: T159655 Change-Id: Id9fdb63da32e205bb898f184402c95fedcc176f4 --- M pywikibot/proofreadpage.py M scripts/wikisourcetext.py M tests/proofreadpage_tests.py 3 files changed, 345 insertions(+), 28 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/proofreadpage.py b/pywikibot/proofreadpage.py index 60546c4..13dbce3 100644 --- a/pywikibot/proofreadpage.py +++ b/pywikibot/proofreadpage.py @@ -1,6 +1,6 @@ # -*- coding: utf-8 -*- """ -Objects representing objects used with ProofreadPage Extension. +Objects used with ProofreadPage Extension. The extension is supported by MW 1.21+. @@ -10,9 +10,15 @@ * FullHeader * IndexPage(Page) + +OCR support of page scans via: +- https://tools.wmflabs.org/phetools/hocr_cgi.py +- https://tools.wmflabs.org/phetools/ocr.php +inspired by https://en.wikisource.org/wiki/MediaWiki:Gadget-ocr.js + """ # -# (C) Pywikibot team, 2015-2016 +# (C) Pywikibot team, 2015-2017 # # Distributed under the terms of the MIT license. # @@ -30,7 +36,10 @@ import pywikibot +from pywikibot.comms import http from pywikibot.data.api import Request + +_logger = 'proofreadpage' class FullHeader(object): @@ -97,6 +106,15 @@ p_open = re.compile(r'<noinclude>') p_close = re.compile(r'(</div>|\n\n\n)?</noinclude>') + # phe-tools ocr utility + HOCR_CMD = ('https://tools.wmflabs.org/phetools/hocr_cgi.py?' + 'cmd=hocr&book={book}&lang={lang}&user={user}') + + OCR_CMD = ('https://tools.wmflabs.org/phetools/ocr.php?' + 'cmd=ocr&url={url_image}&lang={lang}&user={user}') + + MULTI_PAGE_EXT = ['djvu', 'pdf'] + def __init__(self, source, title=''): """Instantiate a ProofreadPage object. @@ -116,12 +134,55 @@ % (self.site.proofread_levels.keys(), self.PROOFREAD_LEVELS)) + self._base, self._base_ext, self._num = self._parse_title() + self._multi_page = self._base_ext in self.MULTI_PAGE_EXT + @property def _fmt(self): if self._full_header._has_div: return self._FMT % '</div>' else: return self._FMT % '' + + def _parse_title(self): + """Get ProofreadPage base title, base extension and page number. + + Base title is the part of title before the last '/', if any, + or the whole title if no '/' is present. + + Extension is the extension of the base title. + + Page number is the part of title after the last '/', if any, + or None if no '/' is present. + + E.g. for title 'Page:Popular Science Monthly Volume 1.djvu/12': + - base = 'Popular Science Monthly Volume 1.djvu' + - extenstion = 'djvu' + - number = 12 + + E.g. for title 'Page:Original Waltzing Matilda manuscript.jpg': + - base = 'Original Waltzing Matilda manuscript.jpg' + - extenstion = 'jpg' + - number = None + + @return: (base, ext, num). + @rtype: tuple + """ + left, sep, right = self.title(withNamespace=False).rpartition('/') + if sep: + base = left + num = int(right) + else: + base = right + num = None + + left, sep, right = base.rpartition('.') + if sep: + ext = right + else: + ext = '' + + return (base, ext, num) @property def index(self): @@ -139,32 +200,33 @@ """ if not hasattr(self, '_index'): index_ns = self.site.proofread_index_ns - what_links_here = [IndexPage(page) for - page in self.getReferences(namespaces=index_ns)] + what_links_here = [IndexPage(page) for page in + set(self.getReferences(namespaces=index_ns))] if not what_links_here: self._index = (None, []) elif len(what_links_here) == 1: - self._index = (what_links_here[0], []) + self._index = (what_links_here.pop(), []) else: self._index = (None, what_links_here) - # Try to infer names form page titles. - base, sep, num = self.title(withNamespace=False).rpartition('/') - if sep == '/': + # Try to infer names from page titles. + if self._num is not None: for page in what_links_here: - if page.title(withNamespace=False) == base: + if page.title(withNamespace=False) == self._base: what_links_here.remove(page) self._index = (page, what_links_here) break page, others = self._index if others: - pywikibot.warning('Page %s is linked to several Index pages: %s.' - % (self, others)) + pywikibot.warning('%s linked to several Index pages.' % self) + pywikibot.output('{0}{1!s}'.format(' ' * 9, [page] + others)) + if page: - pywikibot.warning(' %s selected as Index.' % page) - pywikibot.warning(' %s remaining.' % others) - elif not page: + pywikibot.output('{0}Selected Index: {1}'.format(' ' * 9, page)) + pywikibot.output('{0}remaining: {1!s}'.format(' ' * 9, others)) + + if not page: pywikibot.warning('Page %s is not linked to any Index page.' % self) @@ -435,6 +497,143 @@ """ return '/* {0.status} */ '.format(self) + @property + def url_image(self): + """Get the file url of the scan of ProofreadPage. + + @return: file url of the scan ProofreadPage or None. + @rtype: str/unicode + + @raises: + - Exception in case of http errors. + """ + # wrong link fail with various possible Exceptions. + if not hasattr(self, '_url_image'): + + if self.exists(): + url = self.full_url() + else: + path = 'w/index.php?title={0}&action=edit&redlink=1' + url = self.site.base_url(path.format(self.title(asUrl=True))) + + try: + response = http.fetch(url, charset='utf-8') + except Exception: + pywikibot.error('Error fetching HTML for %s.' % self) + raise + + soup = BeautifulSoup(response.content, 'lxml') + + try: + # None if nothing is found by .find() + self._url_image = soup.find(class_='prp-page-image') + self._url_image = self._url_image.find('img') + # if None raises TypeError. + self._url_image = self._url_image['src'] + except TypeError: + raise ValueError('No prp-page-image src found for %s.' % self) + else: + self._url_image = 'https:' + self._url_image + + return self._url_image + + def _ocr_callback(self, cmd_uri, parser_func=None): + """OCR callback function. + + @return: tuple (error, text [error description in case of error]). + """ + def id(x): + return x + + if not cmd_uri: + raise ValueError('Parameter cmd_uri is mandatory.') + + if parser_func is None: + parser_func = id + + if not callable(parser_func): + raise TypeError('Keyword parser_func must be callable.') + + # wrong link fail with Exceptions + try: + response = http.fetch(cmd_uri, charset='utf-8') + except Exception as e: + pywikibot.error('Querying %s: %s' % (cmd_uri, e)) + return (True, e) + + data = json.loads(response.content) + + assert 'error' in data, 'Error from phe-tools: %s' % data + assert data['error'] in [0, 1], 'Error from phe-tools: %s' % data + + error = bool(data['error']) + if error: + pywikibot.error('Querying %s: %s' % (cmd_uri, data['text'])) + return (error, data['text']) + else: + return (error, parser_func(data['text'])) + + def _do_hocr(self): + """Do hocr using //tools.wmflabs.org/phetools/hocr_cgi.py?cmd=hocr.""" + def parse_hocr_text(txt): + """Parse hocr text.""" + soup = BeautifulSoup(txt, 'lxml') + + res = [] + for ocr_page in soup.find_all(class_='ocr_page'): + for area in soup.find_all(class_='ocr_carea'): + for par in area.find_all(class_='ocr_par'): + for line in par.find_all(class_='ocr_line'): + res.append(line.get_text()) + res.append('\n') + return ''.join(res) + + params = {'book': self.title(asUrl=True, withNamespace=False), + 'lang': self.site.lang, + 'user': self.site.user(), + } + + cmd_uri = self.HOCR_CMD.format(**params) + + return self._ocr_callback(cmd_uri, parser_func=parse_hocr_text) + + def _do_ocr(self): + """Do ocr using //tools.wmflabs.org/phetools/ocr.pmp?cmd=ocr.""" + try: + url_image = self.url_image + except ValueError: + error_text = 'No prp-page-image src found for %s.' % self + pywikibot.error(error_text) + return (True, error_text) + + params = {'url_image': url_image, + 'lang': self.site.lang, + 'user': self.site.user(), + } + + cmd_uri = self.OCR_CMD.format(**params) + + return self._ocr_callback(cmd_uri) + + def ocr(self): + """Do OCR of Proofreadpage scan. + + The text returned by this function shalle be assign to self.body, + otherwise the ProofreadPage format will not be maintained. + + It is the user's responsibility to reset quality level accordingly. + """ + if self._multi_page: + error, text = self._do_hocr() + if not error: + return text + + error, text = self._do_ocr() + if not error: + return text + else: + raise ValueError('Not possible to perform HOCR/OCR on %s.' % self) + class PurgeRequest(Request): @@ -604,11 +803,9 @@ if page in self._labels_from_page: break - # Divide page title in base title and page number. - base_title, sep, page_number = title.rpartition('/') # Sanity check if WS site use page convention name/number. - if sep == '/': - assert page_cnt == int(page_number), ( + if page._num is not None: + assert page_cnt == int(page._num), ( 'Page number %s not recognised as page %s.' % (page_cnt, title)) diff --git a/scripts/wikisourcetext.py b/scripts/wikisourcetext.py index 6ee8ae9..6ae1f92 100644 --- a/scripts/wikisourcetext.py +++ b/scripts/wikisourcetext.py @@ -4,12 +4,20 @@ This bot applies to wikisource sites to upload text. -Text is uploaded to not-(yet)-existing pages in Page ns, for a specified Index. +Text is uploaded to pages in Page ns, for a specified Index. Text to be stored, if the page is not-existing, is preloaded from the file used to create the Index page, making the upload feature independent from the format of the file, as long as it is supported by the MW ProofreadPage extension. +As alternative, if '-ocr' option is selected, https://tools.wmflabs.org/phetools +OCR tool will be used to get text. +In this case, also already existing pages with quality value 'Not Proofread' +can be treated. '-force' will override existing page in this case. + The following parameters are supported: + +# TODO: update params + handle quality level + -index:... name of the index page @@ -22,11 +30,20 @@ A -> just page A -B -> pages 1 until B + -showdiff: show difference between curent text and new text when + saving the page + + -ocr: use https://tools.wmflabs.org/phetools OCR tool to get text; + default is False, i.e. only not-(yet)-existing pages + in Page ns will be treated and text will be fetched via preload. + + -force: overwrite existing pages; + default is False; valid only if '-ocr' is selected. + -summary: custom edit summary. Use quotes if edit summary contains spaces. -always don't bother asking to confirm any of the changes. - """ # # (C) Pywikibot team, 2016-2017 @@ -44,8 +61,7 @@ from pywikibot import i18n from pywikibot.bot import SingleSiteBot -from pywikibot.proofreadpage import IndexPage -from pywikibot.tools import issue_deprecation_warning +from pywikibot.proofreadpage import IndexPage, ProofreadPage class UploadTextBot(SingleSiteBot): @@ -67,6 +83,9 @@ @type generator: generator """ self.availableOptions.update({ + 'showdiff': False, + 'force': False, + 'ocr': False, 'summary': 'Bot: uploading text' }) super(UploadTextBot, self).__init__(**kwargs) @@ -79,15 +98,31 @@ self.site, 'djvutext-creating') def treat(self, page): - """Process one page.""" - old_text = '' - new_text = page.text + """Process one ProofreadPage page. + + @param page: page to be treated. + @type page: ProofreadPage + @raises: pywikibot.Error + """ + if not isinstance(page, ProofreadPage): + raise pywikibot.Error('Page %s must be a ProofreadPage object.' + % page) summary = self.getOption('summary') + if page.exists(): + old_text = page.text + else: + old_text = '' + + if self.getOption('ocr'): + page.body = page.ocr() + + if (page.exists() and + not (self.getOption('ocr') and self.getOption('force'))): pywikibot.output('Page %s already exists, not adding!' % page) else: - self.userPut(page, old_text, new_text, summary=summary, + self.userPut(page, old_text, page.text, summary=summary, show_diff=self.getOption('showdiff')) @@ -113,11 +148,13 @@ elif arg == '-pages': pages = value elif arg == '-showdiff': - issue_deprecation_warning('The usage of -showdiff option', None, 0) + options['showdiff'] = True elif arg == '-summary': options['summary'] = value + elif arg == '-ocr': + options['ocr'] = True elif arg == '-force': - issue_deprecation_warning('The usage of -force option', None, 0) + options['force'] = True elif arg == '-always': options['always'] = True else: @@ -126,6 +163,11 @@ # index is mandatory. if not index: pywikibot.bot.suggest_help(missing_parameters=['-index']) + return False + + # '-force' can be used with '-ocr' only. + if 'force' in options and 'ocr' not in options: + pywikibot.error("'-force' can be used with '-ocr' option only.") return False site = pywikibot.Site() @@ -151,6 +193,7 @@ end = int(end) if end else index.num_pages pages[interval] = (start, end) + # gen yields ProofreadPage objects. gen_list = [] for start, end in sorted(pages): gen = index.page_gen(start=start, end=end, diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py index 49b6e75..0e17ae1 100644 --- a/tests/proofreadpage_tests.py +++ b/tests/proofreadpage_tests.py @@ -73,6 +73,62 @@ self._test_page_text() +class TestProofreadPageParseTitle(TestCase): + + """Test ProofreadPage._parse_title() function.""" + + cached = True + + # Use sites to run parametrized tests. + sites = { + '1': { + 'family': 'wikisource', 'code': 'en', + 'title': 'Page:Test.djvu/12', + 'tuple': ('Test.djvu', 'djvu', 12), + }, + '2': { + 'family': 'wikisource', 'code': 'en', + 'title': 'Page:Test djvu/12', + 'tuple': ('Test djvu', '', 12), + }, + '3': { + 'family': 'wikisource', 'code': 'en', + 'title': 'Page:Test.jpg/12', + 'tuple': ('Test.jpg', 'jpg', 12), + }, + '4': { + 'family': 'wikisource', 'code': 'en', + 'title': 'Page:Test jpg/12', + 'tuple': ('Test jpg', '', 12), + }, + '5': { + 'family': 'wikisource', 'code': 'en', + 'title': 'Page:Test.jpg', + 'tuple': ('Test.jpg', 'jpg', None), + }, + '6': { + 'family': 'wikisource', 'code': 'en', + 'title': 'Page:Test jpg', + 'tuple': ('Test jpg', '', None), + }, + } + + @classmethod + def setUpClass(cls): + """Prepare get_page dataset for tests.""" + super(TestProofreadPageParseTitle, cls).setUpClass() + + def test_parse_title(self, key): + """Test ProofreadPage_parse_title() function.""" + data = self.sites[key] + title = data['title'] + base, base_ext, num = data['tuple'] + page = ProofreadPage(self.site, title) + self.assertEqual(page._base, base) + self.assertEqual(page._base_ext, base_ext) + self.assertEqual(page._num, num) + + class TestProofreadPageValidSite(TestCase): """Test ProofreadPage class.""" @@ -89,6 +145,15 @@ 'user': 'T. Mazzei', 'header': u"{{rh|2|''THE POPULAR SCIENCE MONTHLY.''}}", 'footer': u'\n{{smallrefs}}', + 'url_image': ('https://upload.wikimedia.org/wikipedia/commons/thumb/a/ac/' + 'Popular_Science_Monthly_Volume_1.djvu/' + 'page12-1024px-Popular_Science_Monthly_Volume_1.djvu.jpg'), + } + + valid_redlink = { + 'title': 'Page:Pywikibot test page 3.jpg', + 'url_image': ('https://upload.wikimedia.org/wikisource/en/3/37/' + 'Pywikibot_test_page_3.jpg'), } existing_invalid = { @@ -218,6 +283,18 @@ page_text = page._page_to_json() self.assertEqual(json.loads(page_text), json.loads(loaded_text)) + def test_url_image(self): + """Test fetching of url image of the scan of ProofreadPage.""" + page = ProofreadPage(self.site, self.valid['title']) + self.assertEqual(page.url_image, self.valid['url_image']) + + page = ProofreadPage(self.site, self.valid_redlink['title']) + self.assertEqual(page.url_image, self.valid_redlink['url_image']) + + page = ProofreadPage(self.site, self.existing_unlinked['title']) + # test Exception in property. + self.assertRaises(ValueError, getattr, page, 'url_image') + class TestPageQuality(TestCase): -- To view, visit https://gerrit.wikimedia.org/r/360575 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Id9fdb63da32e205bb898f184402c95fedcc176f4 Gerrit-PatchSet: 6 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: [L10N] Add kbpwiki to wikipedia family file
by jenkins-bot (Code Review) 29 Jun '17

29 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/362202 ) Change subject: [L10N] Add kbpwiki to wikipedia family file ...................................................................... [L10N] Add kbpwiki to wikipedia family file - update languages_by_size - also add fallback to "fr" Bug: T169216 Change-Id: I28916b99066af93be9e49dbff0c46cdbe023a105 --- M pywikibot/families/wikipedia_family.py M pywikibot/i18n.py 2 files changed, 17 insertions(+), 17 deletions(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/families/wikipedia_family.py b/pywikibot/families/wikipedia_family.py index 0900857..6775928 100644 --- a/pywikibot/families/wikipedia_family.py +++ b/pywikibot/families/wikipedia_family.py @@ -41,30 +41,30 @@ 'hu', 'id', 'ko', 'cs', 'ro', 'sr', 'ms', 'tr', 'eu', 'eo', 'bg', 'da', 'hy', 'min', 'kk', 'sk', 'zh-min-nan', 'he', 'lt', 'hr', 'ce', 'et', 'sl', 'be', 'gl', 'nn', 'el', 'uz', 'la', 'simple', - 'ur', 'vo', 'az', 'hi', 'th', 'ka', 'ta', 'cy', 'mk', 'mg', 'oc', - 'lv', 'bs', 'new', 'tt', 'tl', 'ky', 'tg', 'te', 'sq', 'pms', 'br', - 'be-tarask', 'zh-yue', 'ht', 'ml', 'bn', 'jv', 'lb', 'ast', 'mr', + 'ur', 'az', 'vo', 'hi', 'th', 'ka', 'ta', 'cy', 'mk', 'mg', 'oc', + 'lv', 'bs', 'new', 'tl', 'tt', 'ky', 'tg', 'te', 'sq', 'pms', 'br', + 'be-tarask', 'zh-yue', 'ml', 'ht', 'bn', 'jv', 'lb', 'ast', 'mr', 'af', 'sco', 'pnb', 'is', 'ga', 'cv', 'ba', 'fy', 'su', 'sw', 'my', 'lmo', 'an', 'yo', 'ne', 'gu', 'io', 'nds', 'pa', 'scn', 'bpy', 'azb', 'als', 'ku', 'bar', 'kn', 'ia', 'qu', 'ckb', 'mn', 'arz', 'bat-smg', 'wa', 'nap', 'gd', 'bug', 'yi', 'am', 'si', 'map-bms', - 'cdo', 'or', 'fo', 'mzn', 'hsb', 'li', 'sah', 'mai', 'sa', 'vec', + 'cdo', 'or', 'fo', 'mzn', 'li', 'hsb', 'sah', 'mai', 'sa', 'vec', 'ilo', 'os', 'mrj', 'xmf', 'hif', 'mhr', 'roa-tara', 'bh', 'eml', - 'pam', 'diq', 'ps', 'sd', 'nso', 'hak', 'se', 'mi', 'bcl', 'nah', - 'nds-nl', 'gan', 'zh-classical', 'vls', 'rue', 'wuu', 'szl', 'glk', - 'bo', 'vep', 'sc', 'fiu-vro', 'co', 'lrc', 'crh', 'tk', 'kv', - 'csb', 'km', 'gv', 'frr', 'as', 'so', 'lad', 'zea', 'ace', 'ay', + 'pam', 'diq', 'ps', 'sd', 'nso', 'hak', 'se', 'bcl', 'mi', 'nah', + 'nds-nl', 'gan', 'zh-classical', 'vls', 'rue', 'wuu', 'szl', 'bo', + 'glk', 'vep', 'sc', 'fiu-vro', 'co', 'lrc', 'crh', 'tk', 'kv', + 'km', 'csb', 'frr', 'gv', 'as', 'so', 'ace', 'lad', 'zea', 'ay', 'udm', 'myv', 'lez', 'kw', 'stq', 'ie', 'nrm', 'mwl', 'pcd', 'nv', - 'koi', 'rm', 'gom', 'ug', 'lij', 'mt', 'fur', 'gn', 'dsb', - 'cbk-zam', 'dv', 'ang', 'ext', 'ln', 'kab', 'ksh', 'sn', 'gag', + 'koi', 'rm', 'gom', 'ug', 'lij', 'mt', 'gn', 'fur', 'dsb', + 'cbk-zam', 'dv', 'ang', 'ln', 'ext', 'kab', 'ksh', 'sn', 'gag', 'lo', 'frp', 'pag', 'pi', 'av', 'olo', 'xal', 'pfl', 'krc', 'haw', 'bxr', 'kaa', 'rw', 'pdc', 'pap', 'bjn', 'to', 'nov', 'kl', 'arc', - 'jam', 'kbd', 'ha', 'tyv', 'tet', 'tpi', 'ki', 'ig', 'na', 'ab', - 'lbe', 'roa-rup', 'jbo', 'ty', 'kg', 'mdf', 'za', 'wo', 'lg', 'bi', - 'dty', 'srn', 'zu', 'ltg', 'chr', 'tcy', 'sm', 'om', 'xh', 'tn', - 'chy', 'tw', 'rmy', 'cu', 'tum', 'pih', 'got', 'rn', 'pnt', 'ss', - 'bm', 'ch', 'ts', 'ady', 'iu', 'st', 'fj', 'ny', 'ee', 'ak', 'ks', - 'ik', 'sg', 've', 'dz', 'ff', 'ti', 'atj', 'cr', + 'jam', 'kbd', 'dty', 'ha', 'tyv', 'tet', 'tpi', 'ki', 'ig', 'na', + 'ab', 'lbe', 'roa-rup', 'jbo', 'ty', 'kg', 'mdf', 'za', 'wo', 'lg', + 'bi', 'srn', 'zu', 'ltg', 'chr', 'tcy', 'sm', 'om', 'xh', 'tn', + 'chy', 'rmy', 'tw', 'cu', 'tum', 'pih', 'st', 'got', 'rn', 'pnt', + 'ss', 'bm', 'ch', 'ts', 'ady', 'iu', 'fj', 'ny', 'ee', 'ks', 'ak', + 'ik', 'sg', 've', 'dz', 'ff', 'kbp', 'ti', 'cr', 'atj', ] # Sites we want to edit but not count as real languages diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py index 51432d8..5eaf069 100644 --- a/pywikibot/i18n.py +++ b/pywikibot/i18n.py @@ -175,7 +175,7 @@ if code == 'fit': return ['fi', 'sv'] # French - if code in ['atj', 'bm', 'br', 'ht', 'kg', 'ln', 'mg', 'nrm', 'pcd', + if code in ['atj', 'bm', 'br', 'ht', 'kbp', 'kg', 'ln', 'mg', 'nrm', 'pcd', 'rw', 'sg', 'ty', 'wa']: return ['fr'] if code == 'oc': -- To view, visit https://gerrit.wikimedia.org/r/362202 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I28916b99066af93be9e49dbff0c46cdbe023a105 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: API Request: set uiprop only when ensuring 'userinfo' in meta
by jenkins-bot (Code Review) 29 Jun '17

29 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/362209 ) Change subject: API Request: set uiprop only when ensuring 'userinfo' in meta ...................................................................... API Request: set uiprop only when ensuring 'userinfo' in meta uiprop does not make sense, without 'userinfo' in meta, and only generates an API warning for no benefit. Bug: T169202 Change-Id: Ic8383abf344a5e09d98eaf5ddbd8068d1b0b9b40 --- M pywikibot/data/api.py 1 file changed, 7 insertions(+), 6 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index ff4dcd9..4c28951 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -1688,12 +1688,13 @@ # Work around this by requiring userinfo only if 'tokens' and 'login' # are not both set. typep = self._params.get('type', []) - if 'userinfo' not in meta and not ('tokens' in meta and 'login' in typep): - meta = set(meta + ['userinfo']) - self._params['meta'] = sorted(meta) - uiprop = self._params.get("uiprop", []) - uiprop = set(uiprop + ["blockinfo", "hasmsg"]) - self._params['uiprop'] = sorted(uiprop) + if not ('tokens' in meta and 'login' in typep): + if 'userinfo' not in meta: + meta = set(meta + ['userinfo']) + self._params['meta'] = sorted(meta) + uiprop = self._params.get("uiprop", []) + uiprop = set(uiprop + ["blockinfo", "hasmsg"]) + self._params['uiprop'] = sorted(uiprop) if 'prop' in self._params: if self.site.has_extension('ProofreadPage'): prop = set(self._params['prop'] + ['proofread']) -- To view, visit https://gerrit.wikimedia.org/r/362209 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic8383abf344a5e09d98eaf5ddbd8068d1b0b9b40 Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Zhuyifei1999 <zhuyifei1999(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Errored: wikimedia/pywikibot-core#4153 (master - e8bdc42)
by Travis CI 28 Jun '17

28 Jun '17

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #4153 Status: Errored Duration: 2 hours, 40 minutes, and 20 seconds Commit: e8bdc42 (master) Author: Matěj Suchánek Message: [Bugfix] Use correct iterator method in interwikidata.py Bug: T168575 Change-Id: I92d13ebac9d8fcb1ff31cb925e2574a0e70e2298 View the changeset: https://github.com/wikimedia/pywikibot-core/compare/31d8cbc64e98...e8bdc421… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/247857399?utm_source=… -- You can configure recipients for build notifications in your .travis.yml file. See https://docs.travis-ci.com/user/notifications

1 0

[Gerrit] pywikibot/core[master]: Allow pywikibot to authenticate against a private wiki.
by jenkins-bot (Code Review) 28 Jun '17

28 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/361484 ) Change subject: Allow pywikibot to authenticate against a private wiki. ...................................................................... Allow pywikibot to authenticate against a private wiki. Currently, pywikibot cannot authenticate if MediaWiki is configured to require logins in order to read. This patch takes an approach suggested by user @zhuyifei1999 in ticket T153903. It tells pywikibot not to require the parameter "meta=userinfo" if 'tokens' in present in parameter 'meta' and 'login' is present in parameter 'type'. Bug: T153903 Change-Id: I55768b735dbddcf35dbb35b94e9ba493621ff51b --- M pywikibot/data/api.py 1 file changed, 7 insertions(+), 1 deletion(-) Approvals: Zhuyifei1999: Looks good to me, but someone else must approve jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py index 448394c..ff4dcd9 100644 --- a/pywikibot/data/api.py +++ b/pywikibot/data/api.py @@ -1682,7 +1682,13 @@ if self.action == 'query': meta = self._params.get("meta", []) - if "userinfo" not in meta: + # Special logic for private wikis (T153903). + # If the wiki requires login privileges to read articles, pywikibot + # will be blocked from accessing the userinfo. + # Work around this by requiring userinfo only if 'tokens' and 'login' + # are not both set. + typep = self._params.get('type', []) + if 'userinfo' not in meta and not ('tokens' in meta and 'login' in typep): meta = set(meta + ['userinfo']) self._params['meta'] = sorted(meta) uiprop = self._params.get("uiprop", []) -- To view, visit https://gerrit.wikimedia.org/r/361484 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I55768b735dbddcf35dbb35b94e9ba493621ff51b Gerrit-PatchSet: 3 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Dbarrett <mediawiki(a)blazemonger.com> Gerrit-Reviewer: Dbarrett <mediawiki(a)blazemonger.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: Zhuyifei1999 <zhuyifei1999(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: Update languages_by_size
by jenkins-bot (Code Review) 28 Jun '17

28 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/361850 ) Change subject: Update languages_by_size ...................................................................... Update languages_by_size Change-Id: I36eaea5d2e1ed1bc7a9fe5c54f4ff08a6e3333d5 --- M pywikibot/families/wikibooks_family.py M pywikibot/families/wikipedia_family.py M pywikibot/families/wikiquote_family.py M pywikibot/families/wikisource_family.py M pywikibot/families/wiktionary_family.py 5 files changed, 23 insertions(+), 23 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/families/wikibooks_family.py b/pywikibot/families/wikibooks_family.py index a2adcbc..f533eee 100644 --- a/pywikibot/families/wikibooks_family.py +++ b/pywikibot/families/wikibooks_family.py @@ -38,10 +38,10 @@ """Constructor.""" self.languages_by_size = [ 'en', 'hu', 'de', 'fr', 'ja', 'it', 'es', 'pt', 'nl', 'pl', 'he', - 'vi', 'ca', 'th', 'fi', 'id', 'sq', 'fa', 'ru', 'zh', 'cs', 'az', + 'vi', 'ca', 'th', 'fi', 'id', 'sq', 'fa', 'zh', 'ru', 'cs', 'az', 'sv', 'da', 'hr', 'sr', 'tr', 'ko', 'ar', 'no', 'gl', 'ro', 'ta', - 'tl', 'mk', 'is', 'sa', 'uk', 'ka', 'lt', 'tt', 'eo', 'sk', 'el', - 'hi', 'bg', 'bn', 'hy', 'si', 'li', 'ms', 'sl', 'ur', 'la', 'ml', + 'tl', 'mk', 'is', 'sa', 'uk', 'ka', 'lt', 'tt', 'eo', 'hi', 'sk', + 'el', 'bg', 'bn', 'hy', 'li', 'si', 'ms', 'sl', 'ur', 'la', 'ml', 'km', 'ia', 'et', 'cv', 'mr', 'eu', 'oc', 'kk', 'be', 'pa', 'ne', 'fy', 'tg', 'te', 'af', 'ku', 'ky', 'bs', 'mg', 'cy', ] diff --git a/pywikibot/families/wikipedia_family.py b/pywikibot/families/wikipedia_family.py index de3f959..0900857 100644 --- a/pywikibot/families/wikipedia_family.py +++ b/pywikibot/families/wikipedia_family.py @@ -44,7 +44,7 @@ 'ur', 'vo', 'az', 'hi', 'th', 'ka', 'ta', 'cy', 'mk', 'mg', 'oc', 'lv', 'bs', 'new', 'tt', 'tl', 'ky', 'tg', 'te', 'sq', 'pms', 'br', 'be-tarask', 'zh-yue', 'ht', 'ml', 'bn', 'jv', 'lb', 'ast', 'mr', - 'af', 'sco', 'pnb', 'is', 'ga', 'cv', 'ba', 'su', 'fy', 'sw', 'my', + 'af', 'sco', 'pnb', 'is', 'ga', 'cv', 'ba', 'fy', 'su', 'sw', 'my', 'lmo', 'an', 'yo', 'ne', 'gu', 'io', 'nds', 'pa', 'scn', 'bpy', 'azb', 'als', 'ku', 'bar', 'kn', 'ia', 'qu', 'ckb', 'mn', 'arz', 'bat-smg', 'wa', 'nap', 'gd', 'bug', 'yi', 'am', 'si', 'map-bms', @@ -52,19 +52,19 @@ 'ilo', 'os', 'mrj', 'xmf', 'hif', 'mhr', 'roa-tara', 'bh', 'eml', 'pam', 'diq', 'ps', 'sd', 'nso', 'hak', 'se', 'mi', 'bcl', 'nah', 'nds-nl', 'gan', 'zh-classical', 'vls', 'rue', 'wuu', 'szl', 'glk', - 'bo', 'vep', 'sc', 'fiu-vro', 'co', 'lrc', 'crh', 'tk', 'kv', 'km', - 'csb', 'gv', 'frr', 'as', 'so', 'lad', 'zea', 'ace', 'ay', 'udm', - 'myv', 'lez', 'kw', 'stq', 'ie', 'nrm', 'mwl', 'pcd', 'nv', 'koi', - 'rm', 'gom', 'ug', 'lij', 'mt', 'fur', 'gn', 'dsb', 'cbk-zam', - 'dv', 'ang', 'ext', 'ln', 'kab', 'ksh', 'sn', 'gag', 'lo', 'frp', - 'pag', 'pi', 'av', 'olo', 'xal', 'pfl', 'krc', 'haw', 'bxr', 'kaa', - 'rw', 'pdc', 'pap', 'bjn', 'to', 'nov', 'kl', 'arc', 'jam', 'kbd', - 'ha', 'tyv', 'tet', 'tpi', 'ki', 'ig', 'na', 'ab', 'lbe', - 'roa-rup', 'jbo', 'ty', 'kg', 'mdf', 'za', 'wo', 'lg', 'bi', 'dty', - 'srn', 'zu', 'ltg', 'chr', 'tcy', 'sm', 'om', 'xh', 'tn', 'chy', - 'tw', 'rmy', 'cu', 'tum', 'pih', 'got', 'rn', 'pnt', 'ss', 'bm', - 'ch', 'ts', 'ady', 'iu', 'fj', 'st', 'ny', 'ee', 'ak', 'ks', 'ik', - 'sg', 've', 'dz', 'ff', 'ti', 'cr', 'atj', + 'bo', 'vep', 'sc', 'fiu-vro', 'co', 'lrc', 'crh', 'tk', 'kv', + 'csb', 'km', 'gv', 'frr', 'as', 'so', 'lad', 'zea', 'ace', 'ay', + 'udm', 'myv', 'lez', 'kw', 'stq', 'ie', 'nrm', 'mwl', 'pcd', 'nv', + 'koi', 'rm', 'gom', 'ug', 'lij', 'mt', 'fur', 'gn', 'dsb', + 'cbk-zam', 'dv', 'ang', 'ext', 'ln', 'kab', 'ksh', 'sn', 'gag', + 'lo', 'frp', 'pag', 'pi', 'av', 'olo', 'xal', 'pfl', 'krc', 'haw', + 'bxr', 'kaa', 'rw', 'pdc', 'pap', 'bjn', 'to', 'nov', 'kl', 'arc', + 'jam', 'kbd', 'ha', 'tyv', 'tet', 'tpi', 'ki', 'ig', 'na', 'ab', + 'lbe', 'roa-rup', 'jbo', 'ty', 'kg', 'mdf', 'za', 'wo', 'lg', 'bi', + 'dty', 'srn', 'zu', 'ltg', 'chr', 'tcy', 'sm', 'om', 'xh', 'tn', + 'chy', 'tw', 'rmy', 'cu', 'tum', 'pih', 'got', 'rn', 'pnt', 'ss', + 'bm', 'ch', 'ts', 'ady', 'iu', 'st', 'fj', 'ny', 'ee', 'ak', 'ks', + 'ik', 'sg', 've', 'dz', 'ff', 'ti', 'atj', 'cr', ] # Sites we want to edit but not count as real languages diff --git a/pywikibot/families/wikiquote_family.py b/pywikibot/families/wikiquote_family.py index 6eb69e5..10b2d50 100644 --- a/pywikibot/families/wikiquote_family.py +++ b/pywikibot/families/wikiquote_family.py @@ -35,10 +35,10 @@ """Constructor.""" self.languages_by_size = [ 'en', 'it', 'pl', 'ru', 'cs', 'de', 'pt', 'es', 'fa', 'uk', 'sk', - 'bs', 'fr', 'tr', 'he', 'ca', 'fi', 'lt', 'bg', 'eo', 'sl', 'th', + 'fr', 'bs', 'tr', 'he', 'ca', 'fi', 'lt', 'bg', 'eo', 'sl', 'th', 'hy', 'el', 'nn', 'id', 'hr', 'zh', 'hu', 'su', 'li', 'nl', 'az', 'ko', 'ja', 'ar', 'gu', 'sv', 'gl', 'ur', 'te', 'ta', 'cy', 'la', - 'no', 'vi', 'ml', 'et', 'kn', 'ku', 'sr', 'ro', 'eu', 'be', 'hi', + 'vi', 'no', 'ml', 'et', 'kn', 'ku', 'sr', 'ro', 'eu', 'be', 'hi', 'ka', 'da', 'sa', 'is', 'sq', 'mr', 'br', 'uz', 'af', 'zh-min-nan', 'wo', 'ky', ] diff --git a/pywikibot/families/wikisource_family.py b/pywikibot/families/wikisource_family.py index 8422c2c..06fefe3 100644 --- a/pywikibot/families/wikisource_family.py +++ b/pywikibot/families/wikisource_family.py @@ -33,7 +33,7 @@ self.languages_by_size = [ 'en', 'pl', 'ru', 'de', 'fr', 'zh', 'he', 'it', 'es', 'ar', 'cs', 'pt', 'fa', 'hu', 'www', 'ml', 'ko', 'sv', 'gu', 'bn', 'sl', 'te', - 'sa', 'sr', 'ro', 'fi', 'uk', 'el', 'vi', 'ja', 'th', 'hy', 'ca', + 'sa', 'sr', 'ro', 'fi', 'el', 'uk', 'vi', 'ja', 'th', 'hy', 'ca', 'az', 'hr', 'ta', 'br', 'nl', 'is', 'la', 'no', 'vec', 'eo', 'tr', 'be', 'et', 'mk', 'yi', 'id', 'da', 'as', 'bg', 'li', 'mr', 'kn', 'or', 'bs', 'sah', 'lt', 'gl', 'cy', 'sk', 'zh-min-nan', 'fo', diff --git a/pywikibot/families/wiktionary_family.py b/pywikibot/families/wiktionary_family.py index 9bf2d3b..5594892 100644 --- a/pywikibot/families/wiktionary_family.py +++ b/pywikibot/families/wiktionary_family.py @@ -34,15 +34,15 @@ def __init__(self): """Constructor.""" self.languages_by_size = [ - 'en', 'mg', 'fr', 'sh', 'es', 'ru', 'zh', 'de', 'lt', 'nl', 'sv', + 'en', 'mg', 'fr', 'sh', 'es', 'ru', 'zh', 'de', 'nl', 'lt', 'sv', 'pl', 'ku', 'el', 'it', 'ta', 'fi', 'tr', 'hu', 'ko', 'io', 'ca', 'kn', 'pt', 'vi', 'hy', 'sr', 'chr', 'hi', 'ja', 'ro', 'no', 'th', 'ml', 'id', 'et', 'uz', 'li', 'my', 'or', 'te', 'fa', 'eo', 'cs', 'ar', 'jv', 'eu', 'az', 'gl', 'da', 'lo', 'br', 'uk', 'oc', 'hr', 'fj', 'tg', 'bg', 'ps', 'simple', 'cy', 'vo', 'is', 'wa', - 'zh-min-nan', 'sk', 'scn', 'la', 'ast', 'he', 'af', 'tl', 'sw', + 'zh-min-nan', 'sk', 'scn', 'la', 'he', 'ast', 'af', 'tl', 'sw', 'fy', 'ky', 'nn', 'lv', 'co', 'pnb', 'mn', 'ka', 'sl', 'sq', 'nds', - 'nah', 'lb', 'bs', 'sa', 'kk', 'tk', 'km', 'sm', 'mk', 'hsb', 'bn', + 'bs', 'nah', 'lb', 'sa', 'kk', 'tk', 'km', 'sm', 'mk', 'hsb', 'bn', 'be', 'pa', 'ms', 'ga', 'ur', 'an', 'wo', 'vec', 'ang', 'tt', 'sd', 'gn', 'mr', 'so', 'csb', 'ug', 'gd', 'mt', 'st', 'roa-rup', 'si', 'ia', 'ie', 'ay', 'mi', 'kl', 'fo', 'jbo', 'ln', 'zu', 'na', 'gu', -- To view, visit https://gerrit.wikimedia.org/r/361850 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I36eaea5d2e1ed1bc7a9fe5c54f4ff08a6e3333d5 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Xqt <info(a)gno.de> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: [Bugfix] Use correct iterator method in interwikidata.py
by jenkins-bot (Code Review) 28 Jun '17

28 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/361829 ) Change subject: [Bugfix] Use correct iterator method in interwikidata.py ...................................................................... [Bugfix] Use correct iterator method in interwikidata.py Bug: T168575 Change-Id: I92d13ebac9d8fcb1ff31cb925e2574a0e70e2298 --- M scripts/interwikidata.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/scripts/interwikidata.py b/scripts/interwikidata.py index 9d46edf..f4c23d5 100644 --- a/scripts/interwikidata.py +++ b/scripts/interwikidata.py @@ -111,7 +111,7 @@ 'value': self.current_page.title()} } } - for site, page in self.iwlangs.values(): + for site, page in self.iwlangs.items(): if not page.exists(): continue dbname = site.dbName() -- To view, visit https://gerrit.wikimedia.org/r/361829 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I92d13ebac9d8fcb1ff31cb925e2574a0e70e2298 Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: [IMPR] Create items with interwiki using interwikidata.py
by jenkins-bot (Code Review) 28 Jun '17

28 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/361458 ) Change subject: [IMPR] Create items with interwiki using interwikidata.py ...................................................................... [IMPR] Create items with interwiki using interwikidata.py If no foreign page had item, the script reported a conflict. Now a new item is created with the page's interwiki. I also removed InvalidTitle exception from the try statement since it's already handled in textlib.getLanguageLinks. Bug: T168575 Change-Id: Ic708890337ab8523f4fb2326771ce37b32fcc20c --- M scripts/interwikidata.py 1 file changed, 12 insertions(+), 6 deletions(-) Approvals: jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/scripts/interwikidata.py b/scripts/interwikidata.py index e2fea57..9d46edf 100644 --- a/scripts/interwikidata.py +++ b/scripts/interwikidata.py @@ -111,7 +111,14 @@ 'value': self.current_page.title()} } } - summary = (u'Bot: New item with sitelink from %s' + for site, page in self.iwlangs.values(): + if not page.exists(): + continue + dbname = site.dbName() + title = page.title() + data['sitelinks'][dbname] = {'site': dbname, 'title': title} + data['labels'][site.lang] = {'language': site.lang, 'value': title} + summary = ('Bot: New item with sitelink(s) from %s' % self.current_page.title(asLink=True, insite=self.repo)) item = pywikibot.ItemPage(self.repo) @@ -155,11 +162,10 @@ warning('Interwiki %s does not exist, skipping...' % iw_page.title(asLink=True)) continue - except pywikibot.InvalidTitle: - warning('Invalid title %s, skipping...' % - iw_page.title(asLink=True)) - continue - if len(wd_data) != 1: + if not wd_data: + # will create a new item with interwiki + return None + if len(wd_data) > 1: warning('Interwiki conflict in %s, skipping...' % self.current_page.title(asLink=True)) return False -- To view, visit https://gerrit.wikimedia.org/r/361458 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ic708890337ab8523f4fb2326771ce37b32fcc20c Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: JAn Dudík <jan.dudik(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <Ladsgroup(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] pywikibot/core[master]: [bugfix] Add colon when replacing category and file weblinks
by jenkins-bot (Code Review) 28 Jun '17

28 Jun '17

jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/361390 ) Change subject: [bugfix] Add colon when replacing category and file weblinks ...................................................................... [bugfix] Add colon when replacing category and file weblinks External links to wiki categories are replaced by internal ones without a colon before link target. This behavior is wrong as those internal links to categories without a preleading colon are used for a different approach. In addition the problem involves also links to files. Bug: T127745 Change-Id: If07182a6476498ad71ced821000bf91df2516cd8 --- M pywikibot/cosmetic_changes.py M tests/cosmetic_changes_tests.py 2 files changed, 30 insertions(+), 1 deletion(-) Approvals: Dvorapa: Looks good to me, but someone else must approve jenkins-bot: Verified Xqt: Looks good to me, approved diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py index e160467..0ae1ccf 100755 --- a/pywikibot/cosmetic_changes.py +++ b/pywikibot/cosmetic_changes.py @@ -688,7 +688,12 @@ """Convert weblinks to wikilink, fix link syntax.""" def replace_link(match): """Create a string to replace a single link.""" - replacement = '[[' + match.group('link') + replacement = '[[' + if re.match(r'(?:' + '|'.join(list(self.site.namespaces[6]) + + list(self.site.namespaces[14])) + '):', + match.group('link')): + replacement += ':' + replacement += match.group('link') if match.group('title'): replacement += '|' + match.group('title') return replacement + ']]' diff --git a/tests/cosmetic_changes_tests.py b/tests/cosmetic_changes_tests.py index c5a6397..d8cc01c 100644 --- a/tests/cosmetic_changes_tests.py +++ b/tests/cosmetic_changes_tests.py @@ -120,6 +120,10 @@ # necessary as the fixer needs the article path to fix it self.cct.site._siteinfo._cache['general'] = ( {'articlepath': '/wiki/$1'}, True) + self.cct.site._namespaces = { + 6: ['Datei', 'File'], + 14: ['Kategorie', 'Category'], + } self.assertEqual( '[[Example|Page]]\n[[Example|Page]]\n[[Example|Page]]\n' '[[Example]]\n[[Example]]\n[[Example]]\n' @@ -180,6 +184,26 @@ self.assertEqual( '[https://de.wikipedia.org/w/api.php API]', self.cct.fixSyntaxSave('[https://de.wikipedia.org/w/api.php|API]')) + self.assertEqual( + '[[:Kategorie:Example]]\n' + '[[:Category:Example|Description]]\n' + '[[:Datei:Example.svg]]\n' + '[[:File:Example.svg|Description]]\n' + '[[:Category:Example]]\n' + '[[:Kategorie:Example|Description]]\n' + '[[:File:Example.svg]]\n' + '[[:Datei:Example.svg|Description]]\n', + self.cct.fixSyntaxSave( + '[https://de.wikipedia.org/wiki/Kategorie:Example]\n' + '[https://de.wikipedia.org/wiki/Category:Example Description]\n' + '[https://de.wikipedia.org/wiki/Datei:Example.svg]\n' + '[https://de.wikipedia.org/wiki/File:Example.svg Description]\n' + '[[https://de.wikipedia.org/wiki/Category:Example]]\n' + '[[https://de.wikipedia.org/wiki/Kategorie:Example Description]]\n' + '[[https://de.wikipedia.org/wiki/File:Example.svg]]\n' + '[[https://de.wikipedia.org/wiki/Datei:Example.svg Description]]\n' + )) + del self.cct.site._namespaces def test_fixHtml(self): """Test fixHtml method.""" -- To view, visit https://gerrit.wikimedia.org/r/361390 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: If07182a6476498ad71ced821000bf91df2516cd8 Gerrit-PatchSet: 8 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Dvorapa <dvorapa(a)seznam.cz> Gerrit-Reviewer: Dvorapa <dvorapa(a)seznam.cz> Gerrit-Reviewer: Hashar <hashar(a)free.fr> Gerrit-Reviewer: JAn Dudík <jan.dudik(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com> Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com> Gerrit-Reviewer: Maven-release-user <maven-release-user(a)wikimedia.org> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: Xqt <info(a)gno.de> Gerrit-Reviewer: jenkins-bot <>

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits June 2017