jenkins-bot has submitted this change and it was merged.
Change subject: Store the 'entity' url of a globe separately
......................................................................
Store the 'entity' url of a globe separately
Currently we simply use the globe name and convert
back and forth from the entity url. This causes an issue
when we do not have the globe hardcoded. Now we store
the entity url separately and only fallback upon globes
if the entity url is not set.
Change-Id: I942d391be8298bf046cd013332ea24687339bb51
---
M pywikibot/__init__.py
1 file changed, 13 insertions(+), 14 deletions(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 73947ec..77189df 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -95,7 +95,7 @@
in the future we can use it for the GeoData extension.
"""
def __init__(self, lat, lon, alt=None, precision=None, globe='earth',
- typ="", name="", dim=None, site=None):
+ typ="", name="", dim=None, site=None, entity=''):
"""
@param lat: Latitude
@type lat: float
@@ -112,12 +112,15 @@
@type name: str
@param dim: Dimension (in meters)
@type dim: int
+ @param entity: The url entity of a Wikibase item
+ @type entity: str
"""
self.lat = lat
self.lon = lon
self.alt = alt
self._precision = precision
self.globe = globe.lower()
+ self._entity = entity
self.type = typ
self.name = name
self._dim = dim
@@ -125,15 +128,6 @@
self.site = Site().data_repository()
else:
self.site = site
- #Copied from [[mw:Extension:GeoData]]
- if not self.globe in ['earth', 'mercury', 'venus', 'moon',
- 'mars', 'phobos', 'deimos', 'ganymede',
- 'callisto', 'io', 'europa', 'mimas',
- 'enceladus', 'tethys', 'dione',
- 'rhea', 'titan', 'hyperion', 'iapetus',
- 'phoebe', 'miranda', 'ariel', 'umbriel',
- 'titania', 'oberon', 'triton', 'pluto']:
- raise ValueError(u"%s is not a supported globe." % self.globe)
def __repr__(self):
string = 'Coordinate(%s, %s' % (self.lat, self.lon)
@@ -141,6 +135,12 @@
string += ', globe="%s"' % self.globe
string += ')'
return string
+
+ @property
+ def entity(self):
+ if self._entity:
+ return self._entity
+ return self.site.globes()[self.globe]
def toWikibase(self):
"""
@@ -153,7 +153,7 @@
return {'latitude': self.lat,
'longitude': self.lon,
'altitude': self.alt,
- 'globe': self.site.globes()[self.globe],
+ 'globe': self.entity,
'precision': self.precision,
}
@@ -166,15 +166,14 @@
globekey = data['globe']
if globekey:
- # FIXME: Should probably use get() with some error handling when it's an unknown globe
- globe = globes[data['globe']]
+ globe = globes.get(data['globe'])
else:
# Default to earth or should we use None here?
globe = 'earth'
return Coordinate(data['latitude'], data['longitude'],
data['altitude'], data['precision'],
- globe, site=site)
+ globe, site=site, entity=data['globe'])
@property
def precision(self):
--
To view, visit https://gerrit.wikimedia.org/r/78623
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I942d391be8298bf046cd013332ea24687339bb51
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl>
Gerrit-Reviewer: Sk!d <swuensch(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: fix bug http://sourceforge.net/p/pywikipediabot/bugs/1671/ weblinkchecker: url incorrect parsed when using a multiline template
......................................................................
fix bug http://sourceforge.net/p/pywikipediabot/bugs/1671/
weblinkchecker: url incorrect parsed when using a multiline template
Change-Id: I33d77e9da4da609e12fe551ba6e5df18903fce17
---
M weblinkchecker.py
1 file changed, 7 insertions(+), 2 deletions(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/weblinkchecker.py b/weblinkchecker.py
index 389a967..1e61f99 100644
--- a/weblinkchecker.py
+++ b/weblinkchecker.py
@@ -143,22 +143,27 @@
while nestedTemplateR.search(text):
text = nestedTemplateR.sub(r'{{\1 \2 \3}}', text)
- # Then blow up the templates with spaces so that the | and }} will not be regarded as part of the link:.
+ # Then blow up the templates with spaces so that the | and }} will not
+ # be regarded as part of the link:.
templateWithParamsR = re.compile(r'{{([^}]*?[^ ])\|([^ ][^}]*?)}}',
re.DOTALL)
while templateWithParamsR.search(text):
text = templateWithParamsR.sub(r'{{ \1 | \2 }}', text)
- linkR = pywikibot.compileLinkR(withoutBracketed, onlyBracketed)
+ # Add <blank> at the end of a template
+ # URL as last param of multiline template would not be correct
+ text = text.replace('}}', ' }}')
# Remove HTML comments in URLs as well as URLs in HTML comments.
# Also remove text inside nowiki links etc.
text = pywikibot.removeDisabledParts(text)
+ linkR = pywikibot.compileLinkR(withoutBracketed, onlyBracketed)
for m in linkR.finditer(text):
if m.group('url'):
yield m.group('url')
else:
yield m.group('urlb')
+
class InternetArchiveConsulter:
def __init__(self, url):
self.url = url
--
To view, visit https://gerrit.wikimedia.org/r/84229
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I33d77e9da4da609e12fe551ba6e5df18903fce17
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/compat
Gerrit-Branch: master
Gerrit-Owner: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: Use the Disambiguator extension if available
......................................................................
Use the Disambiguator extension if available
Change-Id: I4842eb46bcd17334d5fb063de813558153bdd8f3
---
M pywikibot/page.py
M tests/page_tests.py
2 files changed, 18 insertions(+), 0 deletions(-)
Approvals:
Legoktm: Looks good to me, but someone else must approve
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 4b99a31..d3a393e 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -563,6 +563,11 @@
appended regardless of its existence.
"""
+
+ if self.site.hasExtension('Disambiguator', False):
+ # If the Disambiguator extension is loaded, use it
+ return 'disambiguation' in self.properties()
+
if not hasattr(self.site, "_disambigtemplates"):
try:
default = set(self.site.family.disambig('_default'))
diff --git a/tests/page_tests.py b/tests/page_tests.py
index f828c33..48f40fe 100644
--- a/tests/page_tests.py
+++ b/tests/page_tests.py
@@ -245,6 +245,19 @@
self.assertType(mainpage.previousRevision(), int)
self.assertType(mainpage.permalink(), basestring)
+ def testIsDisambig(self):
+ """
+ Test the integration with
+ Extension:Disambiguator
+ """
+ if not site.hasExtension('Disambiguator', False):
+ raise unittest.SkipTest('Disambiguator extension not loaded on test site')
+ pg = pywikibot.Page(site, 'Random')
+ pg._pageprops = {'disambiguation', ''}
+ self.assertTrue(pg.isDisambig())
+ pg._pageprops = {}
+ self.assertFalse(pg.isDisambig())
+
def testReferences(self):
count = 0
#Ignore redirects for time considerations
--
To view, visit https://gerrit.wikimedia.org/r/82792
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I4842eb46bcd17334d5fb063de813558153bdd8f3
Gerrit-PatchSet: 5
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: Catch all Exceptions in harvest_template
......................................................................
Catch all Exceptions in harvest_template
When an Exception is raised, print the exception and continue
with the next page instead of aborting.
Change-Id: I61fcf6fdd04adb92324437f2a79e9ce5ee483fc0
---
M scripts/harvest_template.py
1 file changed, 4 insertions(+), 1 deletion(-)
Approvals:
Legoktm: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/harvest_template.py b/scripts/harvest_template.py
index 40e43a3..89b034a 100755
--- a/scripts/harvest_template.py
+++ b/scripts/harvest_template.py
@@ -75,7 +75,10 @@
"""
self.templateTitles = self.getTemplateSynonyms(self.templateTitle)
for page in self.generator:
- self.procesPage(page)
+ try:
+ self.procesPage(page)
+ except Exception, e:
+ pywikibot.exception(tb=True)
def getTemplateSynonyms(self, title):
"""
--
To view, visit https://gerrit.wikimedia.org/r/84004
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I61fcf6fdd04adb92324437f2a79e9ce5ee483fc0
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: site.hasExtension: working implementation & tests
......................................................................
site.hasExtension: working implementation & tests
Changes:
1. self.siteinfo does not contain information on extensions -
it only contains the 'general' section of the siteinfo
query. Extensions are now stored as self._extensions.
2. self.hasExtension returned an Exception if the server did
not return loaded extensions. It now has a feature do define
what the return value should be instead -- this is practical
for 'must have extension' or 'must not have extension'
scenarios, where a call
if site.hasExtension('something', False):
... do something
is much more practical than
try:
if site.hasExtension('something'):
... do something
except NotImplementedError:
pass
3. Added tests to test correct behavior. This depends on the Disambiguator
extension being loaded on mysite (=enwiki).
Change-Id: Ic4f5157ae4d1c1b1f69e99a2340a95e231fb4b46
---
M pywikibot/site.py
M tests/site_tests.py
2 files changed, 42 insertions(+), 5 deletions(-)
Approvals:
Legoktm: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 43ec1df..8939b28 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -1126,12 +1126,28 @@
continue
# this is a less preferred form so it goes at the end
self._namespaces[int(item['id'])].append(item["*"])
+ if 'extensions' in sidata:
+ self._extensions = sidata['extensions']
+ else:
+ self._extensions = None
- def hasExtension(self, name):
- if not 'extensions' in self.siteinfo:
- return NotImplementedError("Feature 'hasExtension' only available in MW 1.14+")
- for ext in self.siteinfo['extensions']:
- if ext['name'] == name:
+ def hasExtension(self, name, unknown=NotImplementedError):
+ """ Determine whether extension `name` is loaded.
+
+ @param name The extension to check for
+ @param unknown The value to return if the site does not list loaded
+ extensions. Valid values are an exception to raise,
+ True or False. Default: NotImplementedError
+ """
+ if not hasattr(self, '_extensions'):
+ self._getsiteinfo()
+ if self._extensions is None:
+ if isinstance(unknown, type) and issubclass(unknown, Exception):
+ raise unknown("Feature 'hasExtension' only available in MW 1.14+")
+ else:
+ return unknown
+ for ext in self._extensions:
+ if ext['name'].lower() == name.lower():
return True
return False
diff --git a/tests/site_tests.py b/tests/site_tests.py
index 6d41c98..86f54ec 100644
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -933,6 +933,27 @@
self.assertType(rndpage, pywikibot.Page)
self.assertTrue(rndpage.namespace() in [6, 7])
+ def testExtensions(self):
+ # test automatically getting _extensions
+ del mysite._extensions
+ self.assertTrue(mysite.hasExtension('Disambiguator'))
+
+ # test case-sensitivity
+ self.assertTrue(mysite.hasExtension('disambiguator'))
+
+ self.assertFalse(mysite.hasExtension('ThisExtensionDoesNotExist'))
+
+ # test behavior for sites that do not report extensions
+ mysite._extensions = None
+ self.assertRaises(NotImplementedError, mysite.hasExtension, ('anything'))
+
+ class MyException(Exception):
+ pass
+ self.assertRaises(MyException, mysite.hasExtension, 'anything', MyException)
+
+ self.assertTrue(mysite.hasExtension('anything', True))
+ self.assertFalse(mysite.hasExtension('anything', False))
+ del mysite._extensions
if __name__ == '__main__':
try:
--
To view, visit https://gerrit.wikimedia.org/r/83838
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic4f5157ae4d1c1b1f69e99a2340a95e231fb4b46
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: pagegenerators.RedirectFilterPageGenerator
......................................................................
pagegenerators.RedirectFilterPageGenerator
Added the possibility to filter pages that are redirects or not
redirects.
Change-Id: I4e5d428062a8789485c55b07286b65ac977b9ee3
---
M pywikibot/pagegenerators.py
1 file changed, 5 insertions(+), 3 deletions(-)
Approvals:
Legoktm: Looks good to me, approved
Xqt: Looks good to me, but someone else must approve
Merlijn van Deen: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index d04b2f9..7617323 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -750,10 +750,12 @@
yield page
-def RedirectFilterPageGenerator(generator):
- """Yields pages from another generator that are not redirects."""
+def RedirectFilterPageGenerator(generator, no_redirects=True):
+ """Yields pages from another generator that are redirects or not."""
for page in generator:
- if not page.isRedirectPage():
+ if not page.isRedirectPage() and no_redirects:
+ yield page
+ elif page.isRedirectPage() and not no_redirects:
yield page
--
To view, visit https://gerrit.wikimedia.org/r/82826
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I4e5d428062a8789485c55b07286b65ac977b9ee3
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Beta16 <l.rabinelli(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot