jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/702960 )
Change subject: [doc] Update documentation
......................................................................
[doc] Update documentation
* Doc related to the move from https://travis-ci.org/
* to https://www.travis-ci.com/
Bug: T285032
Change-Id: I0606f1ec6f13f06c7332f459b6294d1dda80c46a
---
M README.rst
M tests/README.rst
2 files changed, 8 insertions(+), 8 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/README.rst b/README.rst
index d23d3eb..a00bf13 100644
--- a/README.rst
+++ b/README.rst
@@ -1,6 +1,6 @@
-.. image:: https://api.travis-ci.org/wikimedia/pywikibot.svg?branch=master
+.. image:: https://api.travis-ci.com/wikimedia/pywikibot.svg?branch=master
:alt: Travis Build Status
- :target: https://travis-ci.org/wikimedia/pywikibot
+ :target: https://travis-ci.com/github/wikimedia/pywikibot
.. image:: https://ci.appveyor.com/api/projects/status/xo2g4ctoom8k6yvw/branch/master?…
:alt: AppVeyor Build Status
:target: https://ci.appveyor.com/project/ladsgroup/pywikibot-g4xqx
diff --git a/tests/README.rst b/tests/README.rst
index 3a73ddc..4765e6d 100644
--- a/tests/README.rst
+++ b/tests/README.rst
@@ -96,22 +96,22 @@
=========
After changes are published into a GitHub repository, tests may be run on
-travis-ci.org according to the configuration in .travis.yml .
+travis-ci.com according to the configuration in .travis.yml .
When changes are merged into the main repository, they are replicated to
https://github.com/wikimedia/pywikibot , and Travis tests are run and
-published at https://travis-ci.org/wikimedia/pywikibot/builds . These tests
+published at https://travis-ci.com/github/wikimedia/pywikibot/builds . These tests
use the Wikimedia global (SUL) account 'Pywikibot-test', which has a password
securely stored in .travis.yml file. See section env:global:secure.
-Anyone can run these tests on travis-ci.org using their own GitHub account, with
+Anyone can run these tests on travis-ci.com using their own GitHub account, with
code changes that have not been merged into the main repository. To do this:
1. create a GitHub and travis-ci account
2. fork the main GitHub repository https://github.com/wikimedia/pywikibot
-3. enable builds from the Travis profile page: https://travis-ci.org/profile
+3. enable builds from the Travis profile page: https://travis-ci.com/profile
4. push changes into the forked git repository
-5. watch the build at https://travis-ci.org/<username>/pywikibot/builds
+5. watch the build at https://travis-ci.com/<username>/pywikibot/builds
Only travis-ci builds from the main repository can access the password for the
Wikimedia account 'Pywikibot-test'. All tests which require a logged in user
@@ -120,7 +120,7 @@
To enable 'user' tests on travis-ci builds for a different repository, add
a username and password to Travis:
-1. Go to https://travis-ci.org/<username>/pywikibot/settings
+1. Go to https://travis-ci.com/<username>/pywikibot/settings
2. Add a new variable named PYWIKIBOT_USERNAME and a value of a valid
Wikimedia SUL username
3. Add another variable named USER_PASSWORD, with the private password for
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/702960
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I0606f1ec6f13f06c7332f459b6294d1dda80c46a
Gerrit-Change-Number: 702960
Gerrit-PatchSet: 5
Gerrit-Owner: Meno25 <meno25mail(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-MessageType: merged
jenkins-bot has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/627243 )
Change subject: [FEAT] Minimal working example for Structured data on Commons
......................................................................
[FEAT] Minimal working example for Structured data on Commons
Supported:
- one-to-one file <-> mediainfo cooperation
- file captions
Not supported:
- statements (depicts, etc.)
- supplying structured data during upload
Bug: T213904
Bug: T223820
Change-Id: I7b019a0e311ddab12b4fd19093622f7ed56386c3
---
M pywikibot/__init__.py
M pywikibot/families/commons_family.py
M pywikibot/page/__init__.py
M pywikibot/page/_collections.py
M pywikibot/site/_apisite.py
M pywikibot/site/_datasite.py
M tests/file_tests.py
7 files changed, 144 insertions(+), 24 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index deaec89..b49938b 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -83,7 +83,7 @@
'FilePage', 'handle_args', 'html2unicode', 'input', 'input_choice',
'input_yn', 'InterwikiRedirectPage', 'InvalidTitle', 'IsNotRedirectPage',
'IsRedirectPage', 'ItemPage', 'Link', 'LockedNoPage', 'LockedPage', 'log',
- 'NoCreateError', 'NoMoveTarget', 'NoPage', 'NoUsername',
+ 'MediaInfo', 'NoCreateError', 'NoMoveTarget', 'NoPage', 'NoUsername',
'NoWikibaseEntity', 'OtherPageSaveError', 'output', 'Page',
'PageCreatedConflict', 'PageDeletedConflict', 'PageRelatedError',
'PageSaveRelatedError', 'PropertyPage', 'SectionError', 'Server414Error',
@@ -1216,6 +1216,7 @@
FilePage,
ItemPage,
Link,
+ MediaInfo,
Page,
PropertyPage,
SiteLink,
diff --git a/pywikibot/families/commons_family.py b/pywikibot/families/commons_family.py
index 67bdbb5..ee448cd 100644
--- a/pywikibot/families/commons_family.py
+++ b/pywikibot/families/commons_family.py
@@ -38,3 +38,7 @@
doc_subpages = {
'_default': (('/doc', ), ['commons']),
}
+
+ def interface(self, code):
+ """Return 'DataSite'."""
+ return 'DataSite'
diff --git a/pywikibot/page/__init__.py b/pywikibot/page/__init__.py
index 628eb62..ca64e91 100644
--- a/pywikibot/page/__init__.py
+++ b/pywikibot/page/__init__.py
@@ -15,6 +15,7 @@
#
# Distributed under the terms of the MIT license.
#
+import json as jsonlib
import logging
import os.path
import re
@@ -2509,6 +2510,24 @@
"""
return self.site.globalusage(self, total=total)
+ def data_item(self):
+ """
+ Convenience function to get the associated Wikibase item of the file.
+
+ If WikibaseMediaInfo extension is available (e.g. on Commons),
+ the method returns the associated mediainfo entity. Otherwise,
+ it falls back to behavior of BasePage.data_item.
+
+ :rtype: pywikibot.page.WikibaseEntity
+ """
+ if self.site.has_extension('WikibaseMediaInfo'):
+ if not hasattr(self, '_item'):
+ self._item = MediaInfo(self.site)
+ self._item._file = self
+ return self._item
+
+ return super().data_item()
+
class Category(Page):
@@ -3444,6 +3463,73 @@
return '{}{}'.format(self.repo.concept_base_uri, entity_id)
+class MediaInfo(WikibaseEntity):
+
+ title_pattern = r'M[1-9]\d*'
+ DATA_ATTRIBUTES = {
+ 'labels': LanguageDict,
+ # TODO: 'statements': ClaimCollection,
+ }
+
+ @property
+ def file(self) -> FilePage:
+ """Get the file associated with the mediainfo."""
+ if not hasattr(self, '_file'):
+ if self.id == '-1':
+ # if the above doesn't apply, this entity is in an invalid
+ # state which needs to be raised as an exception, but also
+ # logged in case an exception handler is catching
+ # the generic Error
+ pywikibot.error('{} is in invalid state'
+ .format(self.__class__.__name__))
+ raise Error('{} is in invalid state'
+ .format(self.__class__.__name__))
+
+ page_id = self.getID(numeric=True)
+ result = list(self.repo.load_pages_from_pageids([page_id]))
+ if not result:
+ raise Error('There is no existing page with id "{}"'
+ .format(page_id))
+
+ page = result.pop()
+ if page.namespace() != page.site.namespaces.FILE:
+ raise Error('Page with id "{}" is not a file'.format(page_id))
+
+ self._file = FilePage(page)
+
+ return self._file
+
+ def get(self, force: bool = False) -> dict:
+ if self.id == '-1':
+ if force:
+ if not self.file.exists():
+ exc = NoPageError(self.file)
+ raise NoWikibaseEntityError(self) from exc
+ # get just the id for Wikibase API call
+ self.id = 'M' + str(self.file.pageid)
+ else:
+ try:
+ data = self.file.latest_revision.slots['mediainfo']['*']
+ except NoPageError as exc:
+ raise NoWikibaseEntityError(self) from exc
+
+ self._content = jsonlib.loads(data)
+ self.id = self._content['id']
+
+ return super().get(force=force)
+
+ def getID(self, numeric=False):
+ """
+ Get the entity identifier.
+
+ :param numeric: Strip the first letter and return an int
+ :type numeric: bool
+ """
+ if self.id == '-1':
+ self.get()
+ return super().getID(numeric=numeric)
+
+
class WikibasePage(BasePage, WikibaseEntity):
"""
diff --git a/pywikibot/page/_collections.py b/pywikibot/page/_collections.py
index 93a8ab9..a8dae47 100644
--- a/pywikibot/page/_collections.py
+++ b/pywikibot/page/_collections.py
@@ -85,8 +85,10 @@
@classmethod
def fromJSON(cls, data, repo=None):
"""Construct a new LanguageDict from JSON."""
- this = cls({key: value['value'] for key, value in data.items()})
- return this
+ if data != []: # workaround for T222159
+ return cls({key: value['value'] for key, value in data.items()})
+ else:
+ return cls()
@classmethod
def normalizeData(cls, data: dict):
diff --git a/pywikibot/site/_apisite.py b/pywikibot/site/_apisite.py
index da2f734..aa9d641 100644
--- a/pywikibot/site/_apisite.py
+++ b/pywikibot/site/_apisite.py
@@ -1009,7 +1009,8 @@
def is_data_repository(self):
"""Return True if its data repository is itself."""
- return self is self.data_repository()
+ # fixme: this was an identity check
+ return self == self.data_repository()
def page_from_repository(self, item):
"""
diff --git a/pywikibot/site/_datasite.py b/pywikibot/site/_datasite.py
index 04466fb..ffe0e50 100644
--- a/pywikibot/site/_datasite.py
+++ b/pywikibot/site/_datasite.py
@@ -48,6 +48,7 @@
self._type_to_class = {
'item': pywikibot.ItemPage,
'property': pywikibot.PropertyPage,
+ 'mediainfo': pywikibot.MediaInfo,
}
def _cache_entity_namespaces(self):
@@ -164,18 +165,6 @@
return baserevid
- def data_repository(self):
- """
- Override parent method.
-
- This avoids pointless API queries since the data repository
- is this site by definition.
-
- :return: this Site object
- :rtype: pywikibot.site.DataSite
- """
- return self
-
def geo_shape_repository(self):
"""Return Site object for the geo-shapes repository e.g. commons."""
url = self.siteinfo['general'].get('wikibase-geoshapestoragebaseurl')
@@ -219,12 +208,12 @@
def preload_entities(self, pagelist, groupsize=50):
"""
- Yield subclasses of WikibasePage's with content prefilled.
+ Yield subclasses of WikibaseEntity's with content prefilled.
Note that pages will be iterated in a different order
than in the underlying pagelist.
- :param pagelist: an iterable that yields either WikibasePage objects,
+ :param pagelist: an iterable that yields either WikibaseEntity objects,
or Page objects linked to an ItemPage.
:param groupsize: how many pages to query at a time
:type groupsize: int
@@ -234,7 +223,7 @@
for sublist in itergroup(pagelist, groupsize):
req = {'ids': [], 'titles': [], 'sites': []}
for p in sublist:
- if isinstance(p, pywikibot.page.WikibasePage):
+ if isinstance(p, pywikibot.page.WikibaseEntity):
ident = p._defined_by()
for key in ident:
req[key].append(ident[key])
@@ -300,7 +289,7 @@
:param entity: Page to edit, or dict with API parameters
to use for entity identification
- :type entity: WikibasePage or dict
+ :type entity: WikibaseEntity or dict
:param data: data updates
:type data: dict
:param bot: Whether to mark the edit as a bot edit
@@ -310,7 +299,7 @@
"""
# this changes the reference to a new object
data = dict(data)
- if isinstance(entity, pywikibot.page.WikibasePage):
+ if isinstance(entity, pywikibot.page.WikibaseEntity):
params = entity._defined_by(singular=True)
if 'id' in params and params['id'] == '-1':
del params['id']
@@ -349,7 +338,7 @@
Add a claim.
:param entity: Entity to modify
- :type entity: WikibasePage
+ :type entity: WikibaseEntity
:param claim: Claim to be added
:type claim: pywikibot.Claim
:param bot: Whether to mark the edit as a bot edit
@@ -747,7 +736,7 @@
Supported actions are:
wbsetaliases, wbsetdescription, wbsetlabel and wbsetsitelink
- :param itemdef: Item to modify or create
+ :param itemdef: Entity to modify or create
:type itemdef: str, WikibaseEntity or Page connected to such item
:param action: wbset{action} to perform:
'wbsetaliases', 'wbsetdescription', 'wbsetlabel', 'wbsetsitelink'
diff --git a/tests/file_tests.py b/tests/file_tests.py
index 85b6b09..bee9347 100644
--- a/tests/file_tests.py
+++ b/tests/file_tests.py
@@ -10,7 +10,11 @@
from contextlib import suppress
import pywikibot
-from pywikibot.exceptions import NoPageError, PageRelatedError
+from pywikibot.exceptions import (
+ NoPageError,
+ NoWikibaseEntityError,
+ PageRelatedError,
+)
from tests import join_images_path
from tests.aspects import TestCase
@@ -274,6 +278,39 @@
page.download(filename)
+class TestFilePageDataItem(TestCase):
+
+ """Test structured data of FilePage."""
+
+ family = 'commons'
+ code = 'commons'
+
+ cached = True
+
+ def test_data_item(self):
+ """Test associated data item."""
+ page = pywikibot.FilePage(self.site, 'File:Albert Einstein.jpg')
+ item = page.data_item()
+ self.assertIsInstance(item, pywikibot.MediaInfo)
+ self.assertTrue(page._item is item)
+ self.assertTrue(item.file is page)
+ self.assertEqual('-1', item.id)
+ item.get()
+ self.assertEqual('M14634781', item.getID())
+ self.assertIsInstance(
+ item.labels, pywikibot.page._collections.LanguageDict)
+ del item._file
+ self.assertEqual(page, item.file)
+
+ def test_data_item_not_existing(self):
+ """Test data item associated to file that does not exist."""
+ page = pywikibot.FilePage(self.site,
+ 'File:Albert Einstein.jpg_notexisting')
+ item = page.data_item()
+ with self.assertRaises(NoWikibaseEntityError):
+ item.get()
+
+
if __name__ == '__main__': # pragma: no cover
with suppress(SystemExit):
unittest.main()
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/627243
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I7b019a0e311ddab12b4fd19093622f7ed56386c3
Gerrit-Change-Number: 627243
Gerrit-PatchSet: 9
Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Gerrit-CC: Dalba <dalba.wiki(a)gmail.com>
Gerrit-CC: Dvorapa <dvorapa(a)seznam.cz>
Gerrit-CC: Framawiki <framawiki(a)tools.wmflabs.org>
Gerrit-CC: JJMC89 <JJMC89.Wikimedia(a)gmail.com>
Gerrit-CC: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-CC: Multichill <maarten(a)mdammers.nl>
Gerrit-CC: Rubin <rubin.happy(a)gmail.com>
Gerrit-CC: Zhuyifei1999 <zhuyifei1999(a)gmail.com>
Gerrit-MessageType: merged
