jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/348042 )
Change subject: [Bug] Inject correct dependencies to SparqlQuery
......................................................................
[Bug] Inject correct dependencies to SparqlQuery
I057335a broke most (third-party) usages of WikidataSPARQLPageGenerator
because only providing the endpoint to the SparqlQuery interface now
throws an error.
The user can now just provide the repository if it's enough,
otherwise they need to provide the mandatory dependencies as well. The
decision about whether it's enough is now made inside SparqlQuery.
Change-Id: I2c2c383b39b3684d8aaed0375ab4253168c135e3
---
M pywikibot/pagegenerators.py
1 file changed, 9 insertions(+), 14 deletions(-)
Approvals:
Lokal Profil: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index d5a56f0..7809ee6 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -2743,12 +2743,16 @@
def WikidataSPARQLPageGenerator(query, site=None,
item_name='item', endpoint=None,
- result_type=set):
+ entity_url=None, result_type=set):
"""Generate pages that result from the given SPARQL query.
@param query: the SPARQL query string.
@param site: Site for generator results.
@type site: L{pywikibot.site.BaseSite}
+ @param endpoint: SPARQL endpoint URL
+ @type endpoint: string
+ @param entity_url: URL prefix for any entities returned in a query.
+ @type entity_url: string
@param result_type: type of the iterable in which
SPARQL results are stored (default set)
@type result_type: iterable
@@ -2759,19 +2763,10 @@
if site is None:
site = pywikibot.Site()
repo = site.data_repository()
- if endpoint is None:
- try:
- endpoint = repo.sparql_endpoint
- except NotImplementedError:
- raise NotImplementedError(
- 'Wiki version must be 1.28-wmf.23 or newer to automatically '
- 'extract the sparql endpoint. Please provide the endpoint '
- 'parameter as well.')
- if not endpoint:
- pywikibot.error('The site {0} does not provide a sparql endpoint.'
- .format(repo))
-
- query_object = sparql.SparqlQuery(endpoint=endpoint)
+ dependencies = dict(endpoint=endpoint, entity_url=entity_url)
+ if not endpoint or not entity_url:
+ dependencies['repo'] = repo
+ query_object = sparql.SparqlQuery(**dependencies)
data = query_object.get_items(query,
item_name=item_name,
result_type=result_type)
--
To view, visit https://gerrit.wikimedia.org/r/348042
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2c2c383b39b3684d8aaed0375ab4253168c135e3
Gerrit-PatchSet: 4
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: Lokal Profil <lokal.profil(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/334421 )
Change subject: site: Handle offset mismatches during chunked upload
......................................................................
site: Handle offset mismatches during chunked upload
If the server returns an error code of 'stashfailed', error JSON
contains an offset, and the provided offset is different from
the offset we have sent it, we believe the server refuse to accept
this chunk because of offset mismatch.
A condition is added to APIError handling so that if such error is
encountered, we update our own offset and continue the upload.
Bug: T156402
Change-Id: I78fff13769e887ce554e8acf8664b1ce8e8bb5f5
---
M pywikibot/site.py
1 file changed, 24 insertions(+), 0 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/site.py b/pywikibot/site.py
index f2724d9..24434a5 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -6095,6 +6095,30 @@
# TODO: catch and process foreseeable errors
if error.code == u'uploaddisabled':
self._uploaddisabled = True
+ elif error.code == u'stashfailed' and \
+ 'offset' in error.other:
+ # TODO: Ask MediaWiki to change this
+ # ambiguous error code.
+
+ new_offset = int(error.other['offset'])
+ # If the offset returned from the server
+ # (the offset it expects now) is equal to
+ # the offset we sent it, there must be
+ # something else that prevented the upload,
+ # instead of simple offset mismatch. This
+ # also prevents infinite loops when we
+ # upload the same chunk again and again,
+ # every time ApiError.
+ if offset != new_offset:
+ pywikibot.log('Old offset: {0}; Returned '
+ 'offset: {1}; Chunk size: '
+ '{2}'.format(offset, new_offset,
+ len(chunk)))
+ pywikibot.warning('Attempting to correct '
+ 'automatically from '
+ 'offset mismatch error.')
+ offset = new_offset
+ continue
raise error
if 'nochange' in data: # in simulation mode
break
--
To view, visit https://gerrit.wikimedia.org/r/334421
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I78fff13769e887ce554e8acf8664b1ce8e8bb5f5
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/281301 )
Change subject: Make WbTime from Timestamp and vice versa
......................................................................
Make WbTime from Timestamp and vice versa
To do this another parameter is introduced for WbTime.toTimestr()
Bug: T131624
Change-Id: Ia8895d29c93efd2d775144bd3baeaf347103e416
---
M pywikibot/__init__.py
M tests/wikibase_tests.py
2 files changed, 88 insertions(+), 3 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 45b08f5..60b2b9c 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -156,6 +156,7 @@
mediawikiTSFormat = "%Y%m%d%H%M%S"
ISO8601Format = "%Y-%m-%dT%H:%M:%SZ"
+ _ISO8601Format_new = '{0:+05d}-{1:02d}-{2:02d}T{3:02d}:{4:02d}:{5:02d}Z'
def clone(self):
"""Clone this instance."""
@@ -578,16 +579,68 @@
int(t[3]), int(t[4]), int(t[5]),
precision, before, after, timezone, calendarmodel, site)
- def toTimestr(self):
+ @classmethod
+ def fromTimestamp(cls, timestamp, precision=14, before=0, after=0,
+ timezone=0, calendarmodel=None, site=None):
+ """
+ Create a new WbTime object from a pywikibot.Timestamp.
+
+ @param timestamp: Timestamp
+ @type timestamp: pywikibot.Timestamp
+ @param precision: The unit of the precision of the time.
+ @type precision: int or str
+ @param before: Number of units after the given time it could be, if uncertain.
+ The unit is given by the precision.
+ @type before: int
+ @param after: Number of units before the given time it could be, if uncertain.
+ The unit is given by the precision.
+ @type after: int
+ @param timezone: Timezone information in minutes.
+ @type timezone: int
+ @param calendarmodel: URI identifying the calendar model
+ @type calendarmodel: str
+ @param site: The Wikibase site
+ @type site: pywikibot.site.DataSite
+ @rtype: pywikibot.WbTime
+ """
+ return cls.fromTimestr(timestamp.isoformat(), precision=precision,
+ before=before, after=after,
+ timezone=timezone, calendarmodel=calendarmodel,
+ site=site)
+
+ def toTimestr(self, force_iso=False):
"""
Convert the data to a UTC date/time string.
+ See fromTimestr() for differences between output with and without
+ force_iso.
+
+ @param force_iso: whether the output should be forced to ISO 8601
+ @type force_iso: bool
@return: Timestamp in a format resembling ISO 8601
@rtype: str
"""
+ if force_iso:
+ return Timestamp._ISO8601Format_new.format(
+ self.year, max(1, self.month), max(1, self.day),
+ self.hour, self.minute, self.second)
return self.FORMATSTR.format(self.year, self.month, self.day,
self.hour, self.minute, self.second)
+ def toTimestamp(self):
+ """
+ Convert the data to a pywikibot.Timestamp.
+
+ @return: Timestamp
+ @rtype: pywikibot.Timestamp
+
+ @raises ValueError: instance value can not be represented using Timestamp
+ """
+ if self.year <= 0:
+ raise ValueError('You cannot turn BC dates into a Timestamp')
+ return Timestamp.fromISOformat(
+ self.toTimestr(force_iso=True).lstrip('+'))
+
def toWikibase(self):
"""
Convert the data to a JSON object for the Wikibase API.
diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py
index fbed2b3..59e2eaf 100644
--- a/tests/wikibase_tests.py
+++ b/tests/wikibase_tests.py
@@ -249,13 +249,21 @@
dry = True
- def test_WbTime_toTimestr(self):
- """Test WbTime conversion to UTC date/time string."""
+ def test_WbTime_timestr(self):
+ """Test timestr functions of WbTime."""
repo = self.get_repo()
+ t = pywikibot.WbTime(site=repo, year=2010, month=0, day=0, hour=12,
+ minute=43)
+ self.assertEqual(t.toTimestr(), '+00000002010-00-00T12:43:00Z')
+ self.assertEqual(t.toTimestr(force_iso=True), '+2010-01-01T12:43:00Z')
+
t = pywikibot.WbTime(site=repo, year=2010, hour=12, minute=43)
self.assertEqual(t.toTimestr(), '+00000002010-01-01T12:43:00Z')
+ self.assertEqual(t.toTimestr(force_iso=True), '+2010-01-01T12:43:00Z')
+
t = pywikibot.WbTime(site=repo, year=-2010, hour=12, minute=43)
self.assertEqual(t.toTimestr(), '-00000002010-01-01T12:43:00Z')
+ self.assertEqual(t.toTimestr(force_iso=True), '-2010-01-01T12:43:00Z')
def test_WbTime_fromTimestr(self):
"""Test WbTime creation from UTC date/time string."""
@@ -275,6 +283,30 @@
day=0, hour=12, minute=43,
precision=14))
+ def test_WbTime_timestamp(self):
+ """Test timestamp functions of WbTime."""
+ repo = self.get_repo()
+ timestamp = pywikibot.Timestamp.fromISOformat('2010-01-01T12:43:00Z')
+ t = pywikibot.WbTime(site=repo, year=2010, month=0, day=0, hour=12,
+ minute=43)
+ self.assertEqual(t.toTimestamp(), timestamp)
+
+ # Roundtrip fails as Timestamp and WbTime interpret month=0 differently
+ self.assertNotEqual(
+ t, pywikibot.WbTime.fromTimestamp(timestamp, site=repo))
+
+ t = pywikibot.WbTime(site=repo, year=2010, hour=12, minute=43)
+ self.assertEqual(t.toTimestamp(), timestamp)
+
+ t = pywikibot.WbTime(site=repo, year=-2010, hour=12, minute=43)
+ self.assertRaises(ValueError, t.toTimestamp)
+
+ t = pywikibot.WbTime(site=repo, year=2010, month=1, day=1, hour=12,
+ minute=43, second=0)
+ self.assertEqual(t.toTimestamp(), timestamp)
+ self.assertEqual(
+ t, pywikibot.WbTime.fromTimestamp(timestamp, site=repo))
+
def test_WbTime_errors(self):
"""Test WbTime precision errors."""
repo = self.get_repo()
--
To view, visit https://gerrit.wikimedia.org/r/281301
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia8895d29c93efd2d775144bd3baeaf347103e416
Gerrit-PatchSet: 7
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Lokal Profil <lokal.profil(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <Ladsgroup(a)gmail.com>
Gerrit-Reviewer: Lokal Profil <lokal.profil(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>