Pywikibot-commits September 2015

pywikibot-commits@lists.wikimedia.org

2 participants
243 discussions

[Gerrit] [FIX] proofreadpage_tests: Use correct class - change (pywikibot/core)
by jenkins-bot (Code Review) 14 Sep '15

14 Sep '15

jenkins-bot has submitted this change and it was merged. Change subject: [FIX] proofreadpage_tests: Use correct class ...................................................................... [FIX] proofreadpage_tests: Use correct class The `TestIndexPageMappings` test added in 6db08c93 did call the wrong super class which skipped the test for bs4. Change-Id: I4c07096003f7dbf0adcbfc944702aaf5ed9e0f0a --- M tests/proofreadpage_tests.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py index bf682a2..6bf2181 100644 --- a/tests/proofreadpage_tests.py +++ b/tests/proofreadpage_tests.py @@ -323,7 +323,7 @@ @classmethod def setUpClass(cls): """Prepare get_page dataset for tests.""" - super(IndexPageTestCase, cls).setUpClass() + super(TestIndexPageMappings, cls).setUpClass() for key, site_def in cls.sites.items(): site = cls.get_site(name=key) base_title = site_def['page'] -- To view, visit https://gerrit.wikimedia.org/r/238351 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I4c07096003f7dbf0adcbfc944702aaf5ed9e0f0a Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

Broken: wikimedia/pywikibot-core#2840 (master - 3606239)
by Travis CI 14 Sep '15

14 Sep '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2840 Status: Broken Duration: 31 minutes and 17 seconds Commit: 3606239 (master) Author: jenkins-bot Message: Merge "Add whitespace in docstrings" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/5d719d4a4933...3606239b… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/80330269 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

Broken: wikimedia/pywikibot-core#2839 (master - 5d719d4)
by Travis CI 14 Sep '15

14 Sep '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2839 Status: Broken Duration: 29 minutes and 47 seconds Commit: 5d719d4 (master) Author: jenkins-bot Message: Merge "[FIX] open_compressed: Update comment" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/bb1be6dd8f73...5d719d4a… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/80328512 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

Broken: wikimedia/pywikibot-core#2838 (master - bb1be6d)
by Travis CI 14 Sep '15

14 Sep '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2838 Status: Broken Duration: 28 minutes and 34 seconds Commit: bb1be6d (master) Author: jenkins-bot Message: Merge "proofreadpage.py: Introduce IndexPage and Site.get_parsed_page()" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/229907d0e4fc...bb1be6dd… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/80326366 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

[Gerrit] Add whitespace in docstrings - change (pywikibot/core)
by jenkins-bot (Code Review) 14 Sep '15

14 Sep '15

jenkins-bot has submitted this change and it was merged. Change subject: Add whitespace in docstrings ...................................................................... Add whitespace in docstrings Fixes Sphinx errors: - exceptions module - pagegenerators module - RcListenerThread - ContextManagerWrapper - MediaWikiVersion Bug: T112491 Change-Id: I0b05d923dd5cde11bc6296ce85d3e1b832c9fe0e --- M pywikibot/comms/rcstream.py M pywikibot/exceptions.py M pywikibot/pagegenerators.py M pywikibot/tools/__init__.py 4 files changed, 28 insertions(+), 6 deletions(-) Approvals: XZise: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/comms/rcstream.py b/pywikibot/comms/rcstream.py index 438823a..68ebded 100644 --- a/pywikibot/comms/rcstream.py +++ b/pywikibot/comms/rcstream.py @@ -27,7 +27,7 @@ class RcListenerThread(threading.Thread): """ - Low-level RC Listener Thread, which reads the RC stream and pushes them to it's internal queue. + Low-level RC Listener Thread, pushing RC stream events into a queue. @param wikihost: the hostname of the wiki we want to get changes for. This is passed to rcstream using a 'subscribe' command. Pass @@ -45,6 +45,7 @@ to a single site and pushes those entries into a queue. Usage: + >>> t = RcListenerThread('en.wikipedia.org', 'stream.wikimedia.org') >>> t.start() >>> change = t.queue.get() diff --git a/pywikibot/exceptions.py b/pywikibot/exceptions.py index a8a8beb..4422660 100644 --- a/pywikibot/exceptions.py +++ b/pywikibot/exceptions.py @@ -3,6 +3,7 @@ Exception and warning classes used throughout the framework. Error: Base class, all exceptions should the subclass of this class. + - NoUsername: Username is not in user-config.py, or it is invalid. - UserBlocked: Username or IP has been blocked - AutoblockUser: requested action on a virtual autoblock user not valid @@ -16,10 +17,12 @@ - UnknownExtension: Extension is not defined for this site SiteDefinitionError: Site loading problem + - UnknownSite: Site does not exist in Family - UnknownFamily: Family is not registered PageRelatedError: any exception which is caused by an operation on a Page. + - NoPage: Page does not exist - IsRedirectPage: Page is a redirect page - IsNotRedirectPage: Page is not a redirect page @@ -31,6 +34,7 @@ PageSaveRelatedError: page exceptions within the save operation on a Page (alias: PageNotSaved). + - SpamfilterError: MediaWiki spam filter detected a blacklisted URL - OtherPageSaveError: misc. other save related exception. - LockedPage: Page is locked @@ -43,23 +47,27 @@ - NoCreateError: parameter nocreate not allow page creation ServerError: a problem with the server. + - FatalServerError: A fatal/non-recoverable server error WikiBaseError: any issue specific to Wikibase. + - CoordinateGlobeUnknownException: globe is not implemented yet. - EntityTypeUnknownException: entity type is not available on the site. DeprecationWarning: old functionality replaced by new functionality PendingDeprecationWarning: problematic code which has not yet been - fully deprecated, possibly because a replacement is not available +fully deprecated, possibly because a replacement is not available RuntimeWarning: problems developers should have fixed, and users need to - be aware of its status. +be aware of its status. + - tools._NotImplementedWarning: do not use - NotImplementedWarning: functionality not implemented UserWarning: warnings targetted at users + - config2._ConfigurationDeprecationWarning: user configuration file problems - login._PasswordFileWarning: password file problems - ArgumentDeprecationWarning: command line argument problems diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py index 8e42bf5..fa772e0 100644 --- a/pywikibot/pagegenerators.py +++ b/pywikibot/pagegenerators.py @@ -100,27 +100,36 @@ -logevents Work on articles that were on a specified Special:Log. The value may be a comma separated list of three values: + logevent,username,total + To use the default value, use an empty string. You have options for every type of logs given by the log event parameter which could be one of the following: + block, protect, rights, delete, upload, move, import, patrol, merge, suppress, review, stable, gblblock, renameuser, globalauth, gblrights, abusefilter, newusers + It uses the default number of pages 10. + Examples: + -logevents:move gives pages from move log (usually redirects) -logevents:delete,,20 gives 20 pages from deletion log -logevents:protect,Usr gives pages from protect by user Usr -logevents:patrol,Usr,20 gives 20 patroled pages by user Usr + In some cases it must be written as -logevents:"patrol,Usr,20" -namespaces Filter the page generator to only yield pages in the -namespace specified namespaces. Separate multiple namespace -ns numbers or names with commas. Examples: + -ns:0,2,4 -ns:Help,MediaWiki + If used with -newpages, -namepace/ns must be provided before -newpages. If used with -recentchanges, efficiency is improved if diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py index af11913..20779d8 100644 --- a/pywikibot/tools/__init__.py +++ b/pywikibot/tools/__init__.py @@ -386,6 +386,7 @@ Two versions are equal if their normal version and dev version are equal. A version is greater if the normal version or dev version is greater. For example: + 1.24 < 1.24.1 < 1.25wmf1 < 1.25alpha < 1.25beta1 < 1.25beta2 < 1.25-rc-1 < 1.25-rc.2 < 1.25 @@ -854,9 +855,12 @@ used as a context manager in with-statements. In such statements the value set via 'as' is directly the wrapped object. For example: - wrapped = ContextManagerWrapper(an_object) - with wrapped as another_object: - assert(another_object is an_object) + >>> class Wrapper(object): + ... def close(self): pass + >>> an_object = Wrapper() + >>> wrapped = ContextManagerWrapper(an_object) + >>> with wrapped as another_object: + ... assert another_object is an_object It does not subclass the object though, so isinstance checks will fail outside a with-statement. -- To view, visit https://gerrit.wikimedia.org/r/238086 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0b05d923dd5cde11bc6296ce85d3e1b832c9fe0e Gerrit-PatchSet: 5 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] [FIX] open_compressed: Update comment - change (pywikibot/core)
by jenkins-bot (Code Review) 14 Sep '15

14 Sep '15

jenkins-bot has submitted this change and it was merged. Change subject: [FIX] open_compressed: Update comment ...................................................................... [FIX] open_compressed: Update comment When `open_compressed` was introduced in ef438c4a it was based on the XML reader and contained a comment about uncompressed XML files. But the new function does work on any content now so reflect that in the comment. Change-Id: Iba602165d7050d9ee89064a1b992a4b93c3acabd --- M pywikibot/tools/__init__.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Mpaa: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py index af11913..5776222 100644 --- a/pywikibot/tools/__init__.py +++ b/pywikibot/tools/__init__.py @@ -955,7 +955,7 @@ else: return process.stdout else: - # assume it's an uncompressed XML file + # assume it's an uncompressed file return open(filename, 'rb') -- To view, visit https://gerrit.wikimedia.org/r/238329 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: Iba602165d7050d9ee89064a1b992a4b93c3acabd Gerrit-PatchSet: 2 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: jenkins-bot <>

1 0

[Gerrit] proofreadpage.py: Introduce IndexPage and Site.get_parsed_pa... - change (pywikibot/core)
by jenkins-bot (Code Review) 14 Sep '15

14 Sep '15

jenkins-bot has submitted this change and it was merged. Change subject: proofreadpage.py: Introduce IndexPage and Site.get_parsed_page() ...................................................................... proofreadpage.py: Introduce IndexPage and Site.get_parsed_page() Index Page page is used in Mediawiki ProofreadPage extension. Provides also helper methods to map pages (and page numbers) to page labes defined in the Index page, via <pagelist ... /> tag. A new method Site.get_parsed_page() has been introduced to fetch parsed text via action=parse, and also a corresponding private method in BasePage. Format of comments has been aligned throughout the script. Add beautifulsoup4 as extra-dependency in setup.py. Change-Id: I06b0bf2e0203957aed64fea5ea09a2e691f0536d --- M pywikibot/page.py M pywikibot/proofreadpage.py M pywikibot/site.py M setup.py M tests/proofreadpage_tests.py 5 files changed, 409 insertions(+), 13 deletions(-) Approvals: John Vandenberg: Looks good to me, approved jenkins-bot: Verified diff --git a/pywikibot/page.py b/pywikibot/page.py index 45110ef..124acbe 100644 --- a/pywikibot/page.py +++ b/pywikibot/page.py @@ -531,6 +531,13 @@ self.site.loadpageinfo(self, preload=True) return self._preloadedtext + def _get_parsed_page(self): + """Retrieve parsed text (via action=parse) and cache it.""" + # Get (cached) parsed text. + if not hasattr(self, '_parsed_text'): + self._parsed_text = self.site.get_parsed_page(self) + return self._parsed_text + def properties(self, force=False): """ Return the properties of the page. diff --git a/pywikibot/proofreadpage.py b/pywikibot/proofreadpage.py index 234dee2..0db0332 100644 --- a/pywikibot/proofreadpage.py +++ b/pywikibot/proofreadpage.py @@ -17,10 +17,14 @@ from __future__ import unicode_literals __version__ = '$Id$' -# -import re import json +import re + +try: + from bs4 import BeautifulSoup +except ImportError as e: + BeautifulSoup = e import pywikibot @@ -215,9 +219,9 @@ # Text is already cached. if hasattr(self, '_text'): return self._text - # If page does not exist, preload it + # If page does not exist, preload it. if self.exists(): - # If page exists, load it + # If page exists, load it. super(ProofreadPage, self).text else: self._text = self.preloadText() @@ -300,14 +304,14 @@ 'footer': self.footer, 'level': {'level': self.ql, 'user': self.user}, } - # ensure_ascii=False returns a unicode + # Ensure_ascii=False returns a unicode. return json.dumps(page_dict, ensure_ascii=False) - def save(self, *args, **kwargs): # see Page.save() + def save(self, *args, **kwargs): # See Page.save(). """Save page content after recomposing the page.""" summary = kwargs.pop('summary', '') summary = self.pre_summary + summary - # Save using contentformat='application/json' + # Save using contentformat='application/json'. kwargs['contentformat'] = 'application/json' kwargs['contentmodel'] = 'proofread-page' text = self._page_to_json() @@ -322,3 +326,167 @@ Status in the edit summary on wiki. """ return '/* {0.status} */ '.format(self) + + +class IndexPage(pywikibot.Page): + + """Index Page page used in Mediawiki ProofreadPage extension.""" + + # TODO: handle not existing pages when quering labels/nubers? + # Currently APIError is thrown. + def __init__(self, source, title=''): + """Instantiate a IndexPage object. + + In this class: + page number is the number in the page title in the Page namespace, if + the wikisource site adopts this convention (e.g. page_number is 12 + for Page:Popular Science Monthly Volume 1.djvu/12) or the sequential + number of the pages linked from the index section in the Index page + if the index is built via transclusion of a list of pages (e.g. like + on de wikisource). + page label is the label associated with a page in the Index page. + + Raises UnknownExtension if source Site has no ProofreadPage Extension. + """ + # Check if BeautifulSoup is imported. + if isinstance(BeautifulSoup, ImportError): + raise BeautifulSoup + + if not isinstance(source, pywikibot.site.BaseSite): + site = source.site + else: + site = source + ns = site.proofread_index_ns + super(IndexPage, self).__init__(source, title, ns=site.proofread_index_ns) + if self.namespace() != site.proofread_index_ns: + raise ValueError('Page %s must belong to %s namespace' + % (self.title(), ns)) + + self._all_page_links = set( + self.site.pagelinks(self, namespaces=self.site.proofread_page_ns)) + + # Cache results. + self._page_from_numbers = {} + self._numbers_from_page = {} + self._page_numbers_from_label = {} + self._pages_from_label = {} + self._labels_from_page_number = {} + self._labels_from_page = {} + + def _get_page_labels(self): + """Associate label and number for each page linked to the index.""" + self._parsed_text = self._get_parsed_page() + self._soup = BeautifulSoup(self._parsed_text, 'html.parser') + attrs = {'class': re.compile('prp-pagequality')} + + # Search for attribute "prp-pagequality" in tags like: + # <a class="quality1 prp-pagequality-1" + # href="/wiki/Page:xxx.djvu/n" + # title="Page:xxx.djvu/n">m + # </a> + + page_cnt = 0 + for a_tag in self._soup.find_all('a', attrs=attrs): + page_cnt += 1 + label = a_tag.text.lstrip('0') # Label is not converted to int. + title = a_tag.get('title') + + page = ProofreadPage(self.site, title) + if page not in self._all_page_links: + raise pywikibot.Error('Page %s not recognised.' % page) + + # In order to avoid to fetch other Page:title links outside + # the Pages section of the Index page; these should hopefully be + # the first ones, so if they start repeating, we are done. + if page in self._labels_from_page: + break + + # Divide page title in base title and page number. + base_title, sep, page_number = title.rpartition('/') + # Sanity check if WS site use page convention name/number. + if sep == '/': + assert page_cnt == int(page_number), ( + 'Page number %s not recognised as page %s.' + % (page_cnt, title)) + + # Mapping: numbers <-> pages. + self._page_from_numbers[page_cnt] = page + self._numbers_from_page[page] = page_cnt + # Mapping: numbers/pages as keys, labels as values. + self._labels_from_page_number[page_cnt] = label + self._labels_from_page[page] = label + # Reverse mapping: labels as keys, numbers/pages as values. + self._page_numbers_from_label.setdefault(label, set()).add(page_cnt) + self._pages_from_label.setdefault(label, set()).add(page) + + # Sanity check: all links to Page: ns must have been considered. + assert set(self._labels_from_page) == set(self._all_page_links) + + def get_label_from_page(self, page): + """Return 'page label' for page. + + There is a 1-to-1 correspondence (each page has a label). + + @param page: Page instance + @return: page label + @rtype: unicode string + """ + if not self._labels_from_page: + self._get_page_labels() + + try: + return self._labels_from_page[page] + except KeyError: + raise KeyError('Invalid Page: %s.' % page) + + def get_label_from_page_number(self, page_number): + """Return page label from page number. + + There is a 1-to-1 correspondence (each page has a label). + + @param page_number: int + @return: page label + @rtype: unicode string + """ + if not self._labels_from_page_number: + self._get_page_labels() + + try: + return self._labels_from_page_number[page_number] + except KeyError: + raise KeyError('Page number ".../%s" not range.' + % page_number) + + def _get_from_label(self, mapping_dict, label): + """Helper function to get info from label.""" + # Convert label to string if an integer is passed. + if not mapping_dict: + self._get_page_labels() + + if isinstance(label, int): + label = str(label) + + try: + return mapping_dict[label] + except KeyError: + raise KeyError('No page has label: "%s".' % label) + + def get_page_number_from_label(self, label='1'): + """Return page number from page label. + + There is a 1-to-many correspondence (a label can be the same for + several pages). + + @return: list containing page numbers corresponding to page label. + """ + return self._get_from_label(self._page_numbers_from_label, label) + + def get_page_from_label(self, label='1'): + """Return page number from page label. + + There is a 1-to-many correspondence (a label can be the same for + several pages). + + @return: list containing pages corresponding to page label. + """ + return self._get_from_label(self._pages_from_label, label) diff --git a/pywikibot/site.py b/pywikibot/site.py index e9d1100..e356fe3 100644 --- a/pywikibot/site.py +++ b/pywikibot/site.py @@ -3621,6 +3621,16 @@ raise NoPage(page) api.update_page(page, pagedata, rvgen.props) + # TODO: expand support to other parameters of action=parse? + def get_parsed_page(self, page): + """Retrieve parsed text of the page using action=parse.""" + req = self._simple_request(action='parse', page=page) + data = req.submit() + assert 'parse' in data, "API parse response lacks 'parse' key" + assert 'text' in data['parse'], "API parse response lacks 'text' key" + parsed_text = data['parse']['text']['*'] + return parsed_text + def pagelanglinks(self, page, step=None, total=None, include_obsolete=False): """Iterate all interlanguage links on page, yielding Link objects. diff --git a/setup.py b/setup.py index bbff59b..4f89d81 100644 --- a/setup.py +++ b/setup.py @@ -52,6 +52,7 @@ 'rcstream': ['socketIO-client<0.6.1'], 'security': ['requests[security]'], 'mwoauth': ['mwoauth>=0.2.4'], + 'html': ['BeautifulSoup4'], } if PY2: diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py index be73ac1..bf682a2 100644 --- a/tests/proofreadpage_tests.py +++ b/tests/proofreadpage_tests.py @@ -11,10 +11,15 @@ import json +try: + from bs4 import BeautifulSoup +except ImportError as e: + BeautifulSoup = e + import pywikibot -from pywikibot.proofreadpage import ProofreadPage from pywikibot.data import api +from pywikibot.proofreadpage import IndexPage, ProofreadPage from tests.aspects import unittest, TestCase from tests.basepage_tests import ( @@ -106,12 +111,12 @@ page = ProofreadPage(self.site, 'title') self.assertEqual(page.namespace(), self.site.proofread_page_ns) - def test_invalid_existing_page_source_in_valid_site(self): + def test_invalid_existing_page_source(self): """Test ProofreadPage from invalid existing Page as source.""" source = pywikibot.Page(self.site, self.existing_invalid['title']) self.assertRaises(ValueError, ProofreadPage, source) - def test_invalid_not_existing_page_source_in_valid_site(self): + def test_invalid_not_existing_page_source(self): """Test ProofreadPage from invalid not existing Page as source.""" # namespace is forced source = pywikibot.Page(self.site, @@ -122,19 +127,19 @@ page = ProofreadPage(fixed_source) self.assertEqual(page.title(), fixed_source.title()) - def test_invalid_not_existing_page_source_in_valid_site_wrong_ns(self): + def test_invalid_not_existing_page_source_wrong_ns(self): """Test ProofreadPage from Page not existing in non-Page ns as source.""" source = pywikibot.Page(self.site, self.not_existing_invalid['title1']) self.assertRaises(ValueError, ProofreadPage, source) - def test_invalid_link_source_in_valid_site(self): + def test_invalid_link_source(self): """Test ProofreadPage from invalid Link as source.""" source = pywikibot.Link(self.not_existing_invalid['title'], source=self.site) self.assertRaises(ValueError, ProofreadPage, source) - def test_valid_link_source_in_valid_site(self): + def test_valid_link_source(self): """Test ProofreadPage from valid Link as source.""" source = pywikibot.Link( self.valid['title'], @@ -202,6 +207,211 @@ self.assertEqual(json.loads(page_text), json.loads(loaded_text)) +class IndexPageTestCase(TestCase): + + """Run tests related to IndexPage ProofreadPage extension.""" + + @classmethod + def setUpClass(cls): + """Check if beautifulsoup4 is installed.""" + super(IndexPageTestCase, cls).setUpClass() + if isinstance(BeautifulSoup, ImportError): + raise unittest.SkipTest('beautifulsoup4 not installed') + + +class TestIndexPageInvalidSite(IndexPageTestCase): + + """Test IndexPage class.""" + + family = 'wikipedia' + code = 'en' + + cached = True + + def test_invalid_site_source(self): + """Test IndexPage from invalid Site as source.""" + self.assertRaises(pywikibot.UnknownExtension, + IndexPage, self.site, 'title') + + +class TestIndexPageValidSite(IndexPageTestCase): + + """Test IndexPage class.""" + + family = 'wikisource' + code = 'en' + + cached = True + + valid_index_title = 'Index:Popular Science Monthly Volume 1.djvu' + existing_invalid_title = 'Main Page' + not_existing_invalid_title = 'User:cannot_exists' + + def test_valid_site_as_source(self): + """Test IndexPage from valid Site as source.""" + page = IndexPage(self.site, 'title') + self.assertEqual(page.namespace(), self.site.proofread_index_ns) + + def test_invalid_existing_page_as_source(self): + """Test IndexPage from invalid existing Page as source.""" + source = pywikibot.Page(self.site, self.existing_invalid_title) + self.assertRaises(ValueError, IndexPage, source) + + def test_invalid_not_existing_page_as_source(self): + """Test IndexPage from Page not existing in non-Page ns as source.""" + source = pywikibot.Page(self.site, + self.not_existing_invalid_title) + self.assertRaises(ValueError, IndexPage, source) + + def test_invalid_link_as_source(self): + """Test IndexPage from invalid Link as source.""" + source = pywikibot.Link(self.not_existing_invalid_title, + source=self.site) + self.assertRaises(ValueError, IndexPage, source) + + def test_valid_link_as_source(self): + """Test IndexPage from valid Link as source.""" + source = pywikibot.Link(self.valid_index_title, + source=self.site, + defaultNamespace=self.site.proofread_page_ns) + page = IndexPage(source) + self.assertEqual(page.title(withNamespace=False), source.title) + self.assertEqual(page.namespace(), source.namespace) + + +class TestIndexPageMappings(IndexPageTestCase): + + """Test IndexPage class.""" + + sites = { + 'enws': { + 'family': 'wikisource', + 'code': 'en', + 'index': 'Popular Science Monthly Volume 1.djvu', + 'page': 'Popular Science Monthly Volume 1.djvu/{0}', + 'get_label': [11, 11, '1'], + 'get_number': [[1, set([11])], + ['Cvr', set([1, 9, 10, 804])], + ], + # 'get_page' is filled in setUpClass. + }, + 'dews': { # dews does not use page convention name/number. + 'family': 'wikisource', + 'code': 'de', + 'index': 'Musen-Almanach für das Jahr 1799', + 'page': 'Schiller_Musenalmanach_1799_{0:3d}.jpg', + 'get_label': [120, 120, '120'], # page no, title no, label + 'get_number': [[120, set([120])], + ], + # 'get_page' is filled in setUpClass. + }, + 'frws': { + 'family': 'wikisource', + 'code': 'fr', + 'index': 'Segard - Hymnes profanes, 1894.djvu', + 'page': 'Segard - Hymnes profanes, 1894.djvu/{0}', + 'get_label': [11, 11, '8'], + 'get_number': [[8, set([11])], + ['-', set(range(1, 4)) | set(range(101, 108))], + ], + # 'get_page' is filled in setUpClass. + }, + } + + cached = True + + @classmethod + def setUpClass(cls): + """Prepare get_page dataset for tests.""" + super(IndexPageTestCase, cls).setUpClass() + for key, site_def in cls.sites.items(): + site = cls.get_site(name=key) + base_title = site_def['page'] + + # 'get_page' has same structure as 'get_number'. + site_def['get_page'] = [] + for label, page_numbers in site_def['get_number']: + page_set = set(ProofreadPage(site, base_title.format(i)) + for i in page_numbers) + site_def['get_page'].append([label, page_set]) + + def test_get_labels(self, key): + """Test IndexPage page get_label_from_* functions.""" + data = self.sites[key] + num, title_num, label = data['get_label'] + + index_page = IndexPage(self.site, self.sites[key]['index']) + page_title = self.sites[key]['page'].format(title_num) + proofread_page = ProofreadPage(self.site, page_title) + + # Get label from number. + self.assertEqual(index_page.get_label_from_page_number(num), label) + self.assertRaises(KeyError, index_page.get_label_from_page_number, -1) + + # Get label from page. + self.assertEqual(index_page.get_label_from_page(proofread_page), label) + + # Error if page does not exists. + self.assertRaises(KeyError, index_page.get_label_from_page, None) + + def test_get_page_number(self, key): + """Test IndexPage page get_page_number functions.""" + data = self.sites[key] + index_page = IndexPage(self.site, self.sites[key]['index']) + + # Test get_page_numbers_from_label. + for label, num_set in data['get_number']: + # Get set of numbers from label with label as int or str. + self.assertEqual(index_page.get_page_number_from_label(label), + num_set) + self.assertEqual(index_page.get_page_number_from_label(str(label)), + num_set) + + # Error if label does not exists. + label, num_set = 'dummy label', [] + self.assertRaises(KeyError, index_page.get_page_number_from_label, + 'dummy label') + + # Test get_page_from_label. + for label, page_set in data['get_page']: + # Get set of pages from label with label as int or str. + self.assertEqual(index_page.get_page_from_label(label), + page_set) + self.assertEqual(index_page.get_page_from_label(str(label)), + page_set) + + # Error if label does not exists. + self.assertRaises(KeyError, index_page.get_page_from_label, 'dummy label') + + # Test consistency of page <-> numbers mapping on last page_set and + # num_set used. + for p in page_set: + n = index_page._numbers_from_page[p] + self.assertEqual(index_page._page_from_numbers[n], p) + for n in num_set: + n = index_page._page_from_numbers[p] + self.assertEqual(index_page._numbers_from_page[p], n) + + def test_page_number_mapping(self, key): + """Test consistency of page <-> mapping.""" + data = self.sites[key] + num, title_num, label = data['get_label'] + + index_page = IndexPage(self.site, self.sites[key]['index']) + page_title = self.sites[key]['page'].format(title_num) + proofread_page = ProofreadPage(self.site, page_title) + + # Get label from number. + self.assertEqual(index_page.get_label_from_page_number(num), label) + # Error if number does not exists. + self.assertRaises(KeyError, index_page.get_label_from_page_number, -1) + + # Get label from page. + self.assertEqual(index_page.get_label_from_page(proofread_page), label) + # Error if page does not exists. + self.assertRaises(KeyError, index_page.get_label_from_page, None) + + if __name__ == '__main__': try: unittest.main() -- To view, visit https://gerrit.wikimedia.org/r/233849 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I06b0bf2e0203957aed64fea5ea09a2e691f0536d Gerrit-PatchSet: 22 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com> Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com> Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com> Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de> Gerrit-Reviewer: jenkins-bot <>

1 0

Passed: wikimedia/pywikibot-core#2837 (master - 229907d)
by Travis CI 14 Sep '15

14 Sep '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2837 Status: Passed Duration: 31 minutes and 12 seconds Commit: 229907d (master) Author: jenkins-bot Message: Merge "Test paraminfo values" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/88debe6d5bbb...229907d0… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/80199461 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

Passed: wikimedia/pywikibot-core#2836 (master - 88debe6)
by Travis CI 14 Sep '15

14 Sep '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2836 Status: Passed Duration: 31 minutes and 28 seconds Commit: 88debe6 (master) Author: jenkins-bot Message: Merge "Fix whitespace around URLs in docstrings" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/c061a2aa1dca...88debe6d… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/80196445 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

Passed: wikimedia/pywikibot-core#2835 (master - c061a2a)
by Travis CI 14 Sep '15

14 Sep '15

Build Update for wikimedia/pywikibot-core ------------------------------------- Build: #2835 Status: Passed Duration: 30 minutes and 23 seconds Commit: c061a2a (master) Author: jenkins-bot Message: Merge "Fix Sphinx errors in tests" View the changeset: https://github.com/wikimedia/pywikibot-core/compare/9016caecc567...c061a2aa… View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/80195938 -- You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications

1 0

← Newer
1
...
9
10
11
12
13
14
15
...
25
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

Pywikibot-commits September 2015