jenkins-bot has submitted this change and it was merged.
Change subject: Remove item count output in page generators
......................................................................
Remove item count output in page generators
Currently WikibaseSearchItemPageGenerator always
fetches all items twice in order to display an item count,
and then yield each item.
Also do not fetch all entities for test_searchitem*.
The test only needs to confirm one item is fetched,
and currently it can halt Travis builds due to fetching
~5000 items twice taking 10 mins without any output.
Also add a note to WikidataQueryPageGenerator about its item count.
Change-Id: Ia8b0e97930a94945d3ed2cdf593887ebff8a3e5c
---
M pywikibot/pagegenerators.py
M tests/pagegenerators_tests.py
2 files changed, 15 insertions(+), 5 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index 6c7a2b2..a79a6bc 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -2654,6 +2654,8 @@
wd_query = wdquery.WikidataQuery(cacheMaxAge=0)
data = wd_query.query(wd_queryset)
+ # This item count should not be copied by other generators,
+ # and should be removed when wdq becomes a real generator (T135592)
pywikibot.output(u'retrieved %d items' % data[u'status'][u'items'])
items_pages = (pywikibot.ItemPage(repo, 'Q{0}'.format(item))
for item in data[u'items'])
@@ -2710,7 +2712,6 @@
repo = site.data_repository()
data = repo.search_entities(text, language, limit=total)
- pywikibot.output(u'retrieved %d items' % len(list(data)))
for item in data:
yield pywikibot.ItemPage(repo, item['id'])
diff --git a/tests/pagegenerators_tests.py b/tests/pagegenerators_tests.py
index b1a0057..4927c65 100755
--- a/tests/pagegenerators_tests.py
+++ b/tests/pagegenerators_tests.py
@@ -970,21 +970,30 @@
gf.handleArg('-searchitem:abc')
gen = gf.getCombinedGenerator()
self.assertIsNotNone(gen)
- self.assertGreater(len(set(gen)), 0)
+ self.assertIsNotNone(next(gen))
def test_searchitem_language(self):
"""Test -searchitem with custom language specified."""
gf = pagegenerators.GeneratorFactory(site=self.site)
gf.handleArg('-searchitem:pl:abc')
+ gf.handleArg('-limit:1')
gen = gf.getCombinedGenerator()
self.assertIsNotNone(gen)
- pages = set(gen)
+ # ABC disambiguation
+ page1 = next(gen)
+ self.assertEqual(page1.title(), 'Q286874')
+
gf = pagegenerators.GeneratorFactory(site=self.site)
gf.handleArg('-searchitem:en:abc')
+ gf.handleArg('-limit:2')
gen = gf.getCombinedGenerator()
self.assertIsNotNone(gen)
- pages2 = set(gen)
- self.assertNotEqual(pages, pages2)
+ # American Broadcasting Company
+ page1 = next(gen)
+ self.assertEqual(page1.title(), 'Q169889')
+ # ABC disambiguation
+ page2 = next(gen)
+ self.assertEqual(page2.title(), 'Q286874')
class TestLogeventsFactoryGenerator(DefaultSiteTestCase,
--
To view, visit https://gerrit.wikimedia.org/r/289339
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia8b0e97930a94945d3ed2cdf593887ebff8a3e5c
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: M4tx <m4tx(a)m4tx.pl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Activate nosetrim to reduce repeated errors
......................................................................
Activate nosetrim to reduce repeated errors
The OAuth jobs for wmfbeta sites are failing due to an authentication
problem, causing mosts tests to fail.
Nose normally and helpfully emits a lot of logging information for failed jobs.
However when there is a systemic problem, as currently occurs with the
wmfbeta sites, the result is the job logs are very large, which affects
developers ability to understand the failures and it exceeds the Travis
maximum log size.
Change-Id: I048e04a9451d25bf6893755bf73007f944578191
---
M .travis.yml
M dev-requirements.txt
2 files changed, 3 insertions(+), 2 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/.travis.yml b/.travis.yml
index 90cbacb..1930065 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -91,9 +91,9 @@
nosetests --version ;
if [[ "$SITE_ONLY" == "1" ]]; then
echo "Running site tests only code ${LANGUAGE} on family ${FAMILY}" ;
- python setup.py nosetests --tests tests --verbosity=2 -a "family=$FAMILY,code=$LANGUAGE" --with-coverage --cover-package=. ;
+ python setup.py nosetests --tests tests --verbosity=2 -a "family=$FAMILY,code=$LANGUAGE" --with-trim --with-coverage --cover-package=. ;
else
- python setup.py nosetests --tests tests --verbosity=2 --with-coverage --cover-package=. ;
+ python setup.py nosetests --tests tests --verbosity=2 --with-trim --with-coverage --cover-package=. ;
fi ;
else
coverage run setup.py test ;
diff --git a/dev-requirements.txt b/dev-requirements.txt
index d990436..8243906 100644
--- a/dev-requirements.txt
+++ b/dev-requirements.txt
@@ -5,6 +5,7 @@
six
nose
+git+https://github.com/jayvdb/nosetrim@py3
pep257>=0.6
pyflakes>=0.9
--
To view, visit https://gerrit.wikimedia.org/r/289355
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I048e04a9451d25bf6893755bf73007f944578191
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>