jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/376058 )
Change subject: [IMPR] Cache fetched Wikibase item
......................................................................
[IMPR] Cache fetched Wikibase item
Repetitive ItemPage.fromPage calls always create
a new instance. When using -onlyif(not) filters,
the item's content can be fetched during filtering
and during treating.
Change-Id: I5b5b4d69da32c311f7d5ffe7d1767bc31e73b295
---
M pywikibot/page.py
1 file changed, 8 insertions(+), 3 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/page.py b/pywikibot/page.py
index c03c878..d54be98 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -163,7 +163,7 @@
'_contentmodel', '_langlinks', '_isredir', '_coords',
'_preloadedtext', '_timestamp', '_applicable_protections',
'_flowinfo', '_quality', '_pageprops', '_revid', '_quality_text',
- '_pageimage'
+ '_pageimage', '_item'
)
def __init__(self, source, title=u"", ns=0):
@@ -4178,6 +4178,8 @@
@raise NoPage: There is no corresponding ItemPage for the page
@raise WikiBaseError: The site of the page has no data repository.
"""
+ if hasattr(page, '_item'):
+ return page._item
if not page.site.has_data_repository:
raise pywikibot.WikiBaseError('{0} has no data repository'
''.format(page.site))
@@ -4189,7 +4191,8 @@
'_pageprops') and page.properties().get('wikibase_item'):
# If we have already fetched the pageprops for something else,
# we already have the id, so use it
- return cls(repo, page.properties().get('wikibase_item'))
+ page._item = cls(repo, page.properties().get('wikibase_item'))
+ return page._item
i = cls(repo)
# clear id, and temporarily store data needed to lazy loading the item
del i.id
@@ -4197,7 +4200,8 @@
i._title = page.title(withSection=False)
if not lazy_load and not i.exists():
raise pywikibot.NoPage(i)
- return i
+ page._item = i
+ return page._item
@classmethod
def from_entity_uri(cls, site, uri, lazy_load=False):
@@ -4309,6 +4313,7 @@
for dbname in self.sitelinks:
pg = Page(pywikibot.site.APISite.fromDBName(dbname),
self.sitelinks[dbname])
+ pg._item = self
if family is None or family == pg.site.family:
yield pg
--
To view, visit https://gerrit.wikimedia.org/r/376058
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I5b5b4d69da32c311f7d5ffe7d1767bc31e73b295
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/375833 )
Change subject: Only preload grepping generator if possible
......................................................................
Only preload grepping generator if possible
When nopreload is True, preloading should always be avoided.
It isn't problem to move this filter after preloading since
it always needs the pages to have content in order to decide.
Change-Id: I9e4c30756660542c481521f985245e5d32ad02f5
---
M pywikibot/pagegenerators.py
1 file changed, 5 insertions(+), 5 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index 1216f84..b0c6dab 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -528,20 +528,20 @@
dupfiltergen = RegexFilterPageGenerator(
dupfiltergen, self.titlenotfilter_list, 'none')
- if self.articlefilter_list:
- dupfiltergen = RegexBodyFilterPageGenerator(
- PreloadingGenerator(dupfiltergen), self.articlefilter_list)
-
if self.catfilter_list:
dupfiltergen = CategoryFilterPageGenerator(
dupfiltergen, self.catfilter_list, self.site)
- if preload and not self.nopreload:
+ if (preload or self.articlefilter_list) and not self.nopreload:
if isinstance(dupfiltergen, DequeGenerator):
dupfiltergen = DequePreloadingGenerator(dupfiltergen)
else:
dupfiltergen = PreloadingGenerator(dupfiltergen)
+ if self.articlefilter_list:
+ dupfiltergen = RegexBodyFilterPageGenerator(
+ dupfiltergen, self.articlefilter_list)
+
return dupfiltergen
@deprecated_args(arg='category')
--
To view, visit https://gerrit.wikimedia.org/r/375833
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I9e4c30756660542c481521f985245e5d32ad02f5
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Matěj Suchánek <matejsuchanek97(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Mpaa <mpaa.wiki(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/376701 )
Change subject: travis.yml: Disable OAuth jobs
......................................................................
travis.yml: Disable OAuth jobs
It's been nearly 3 weeks since these jobs started to fail.
Each of them takes nearly 50 minutes to complete and at the end there is no
useful output. They are just making the builds slow, distracting us from
things that really need attention and can be fixed.
I also sent a message to the mailing list which has had no reply so far.[1]
[1]: https://lists.wikimedia.org/pipermail/pywikibot/2017-September/009775.html
Bug: T173498
Change-Id: I45f7c4048cea4eb7a135b1c6ea1d582d43afbae7
---
M .travis.yml
1 file changed, 8 insertions(+), 6 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/.travis.yml b/.travis.yml
index c5a9928..554cdfe 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -153,18 +153,20 @@
- graphviz
- liblua5.1-0-dev
- python-ipaddr
- - python: '2.7'
- env: LANGUAGE=en FAMILY=wpbeta SITE_ONLY=1 OAUTH_DOMAIN="en.wikipedia.beta.wmflabs.org"
- - python: '3.3'
- env: LANGUAGE=zh FAMILY=wpbeta SITE_ONLY=1 OAUTH_DOMAIN="zh.wikipedia.beta.wmflabs.org"
+ # Disabled due to T173498
+ #- python: '2.7'
+ # env: LANGUAGE=en FAMILY=wpbeta SITE_ONLY=1 OAUTH_DOMAIN="en.wikipedia.beta.wmflabs.org"
+ #- python: '3.3'
+ # env: LANGUAGE=zh FAMILY=wpbeta SITE_ONLY=1 OAUTH_DOMAIN="zh.wikipedia.beta.wmflabs.org"
- python: '3.4'
env: LANGUAGE=en FAMILY=wsbeta SITE_ONLY=1
- python: '2.7'
env: LANGUAGE=wikia FAMILY=wikia PYWIKIBOT2_TEST_NO_RC=1
- python: '3.3'
env: LANGUAGE=en FAMILY=musicbrainz SITE_ONLY=1
- - python: '3.4'
- env: LANGUAGE=test FAMILY=wikipedia SITE_ONLY=1 OAUTH_DOMAIN="test.wikipedia.org"
+ # Disabled due to T173498
+ #- python: '3.4'
+ # env: LANGUAGE=test FAMILY=wikipedia SITE_ONLY=1 OAUTH_DOMAIN="test.wikipedia.org"
- python: '3.4'
env: LANGUAGE=test FAMILY=wikidata SITE_ONLY=1
- python: '3.4'
--
To view, visit https://gerrit.wikimedia.org/r/376701
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I45f7c4048cea4eb7a135b1c6ea1d582d43afbae7
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Dalba <dalba.wiki(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Magul <tomasz.magulski(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>