[Pywikibot-commits] [Gerrit] Enable edit failure tests in travis builds - change (pywikibot/core)

jenkins-bot has submitted this change and it was merged. Change subject: Enable edit failure tests in travis builds ...................................................................... Enable edit failure tests in travis builds Adds new environment variable PYWIKIBOT2_TEST_WRITE_FAIL which when set to '1' will run edit failure tests, which are much safer than the other 'write' tests. Also improve documentation for travis builds. Change-Id: I85bf198c2a30b69a9e29d220d8696a92642272bc --- M .travis.yml M tests/README.rst M tests/aspects.py M tests/edit_failure_tests.py 4 files changed, 67 insertions(+), 21 deletions(-) Approvals: John Vandenberg: Looks good to me, but someone else must approve XZise: Looks good to me, approved jenkins-bot: Verified diff --git a/.travis.yml b/.travis.yml index f0c970b..1233f57 100644 --- a/.travis.yml +++ b/.travis.yml @@ -32,15 +32,20 @@ - echo "maximum_GET_length = 5000" >> ~/.pywikibot/user-config.py - echo "console_encoding = 'utf8'" >> ~/.pywikibot/user-config.py - - if [[ "$GITHUB_USER" == "wikimedia" ]]; then PYWIKIBOT2_USERNAME="Pywikibot-test"; fi + - if [[ "$GITHUB_USER" == "wikimedia" ]]; then + PYWIKIBOT2_USERNAME="Pywikibot-test" ; + else + export PYWIKIBOT2_TEST_WRITE_FAIL=1 ; + fi - - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then echo "usernames['$FAMILY']['$LANGUAGE'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py; fi - - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then echo "usernames['wikipedia']['en'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py; fi - - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then echo "usernames['wikipedia']['test'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py; fi - - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then echo "usernames['wikidata']['test'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py; fi - - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then echo "usernames['commons']['commons'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py; fi - - - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then echo "('$PYWIKIBOT2_USERNAME', '$USER_PASSWORD')" > ~/.pywikibot/passwordfile; fi + - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then + echo "usernames['$FAMILY']['$LANGUAGE'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py ; + echo "usernames['wikipedia']['en'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py ; + echo "usernames['wikipedia']['test'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py ; + echo "usernames['wikidata']['test'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py ; + echo "usernames['commons']['commons'] = '$PYWIKIBOT2_USERNAME'" >> ~/.pywikibot/user-config.py ; + echo "('$PYWIKIBOT2_USERNAME', '$USER_PASSWORD')" > ~/.pywikibot/passwordfile ; + fi - cd externals/httplib2 - python setup.py install diff --git a/tests/README.rst b/tests/README.rst index 4258016..214c110 100644 --- a/tests/README.rst +++ b/tests/README.rst @@ -83,7 +83,7 @@ use the Wikimedia global (SUL) account 'Pywikibot-test', which has a password securely stored in .travis.yml . See section env:global:secure. -Anyone can run these tests on travis-ci.org using their own account, with +Anyone can run these tests on travis-ci.org using their own github account, with code changes that have not been merged into the main repository. To do this: 1. create a github and travis-ci account @@ -103,9 +103,35 @@ 2. Add a new variable named PYWIKIBOT2_USERNAME and a value of a valid Wikimedia SUL username 3. Add another variable named USER_PASSWORD, with the private password for - the Wikimedia SUL username used in step 2 + the Wikimedia SUL username used in step 2. Check that this + environment variable has "Display value in build logs" set to OFF, so + the password does not leak into the build logs. 4. The next build should run tests that require a logged in user +While passwords in travis-ci environment variables are not leaked in normal +operations, you are responsible for your own passwords. + +It is strongly recommended that an untrusted bot account is created for +travis tests, using a password that is not shared with trusted accounts. + +There are a set of 'edit failure' tests, which attempt to write to the wikis +and **should** fail. If there is a bug in pywikibot or MediaWiki, these +tests **may** actually perform a write operation. + +These 'edit failure' tests are disabled by default for the 'wikimedia' builds, +but are enabled by default on builds by any other github account. + +To disable 'edit failure' tests in travis, add PYWIKIBOT2_TEST_WRITE_FAIL=0 + +There are also several other 'write' tests which also attempt to perform +write operations successfully. These **will** write to the wikis, and they +should always only write to 'test' wikis. + +These 'write' tests are disabled in travis builds, and currently can not be +run on travis as they require interaction using a terminal. + +To enable 'write' tests in travis, add PYWIKIBOT2_TEST_WRITE=1 + Contributing tests ================== diff --git a/tests/aspects.py b/tests/aspects.py index be21cf8..8147a70 100644 --- a/tests/aspects.py +++ b/tests/aspects.py @@ -518,24 +518,37 @@ """ Set up the test class. - Prevent test classes to write to the site and also cache results. + Reject write test classes configured with non-test wikis, or caching. - Skip the test class if environment variable PYWIKIBOT2_TEST_WRITE - does not equal 1. + Prevent test classes from writing to the site by default. + + If class attribute 'write' is -1, the test class is skipped unless + environment variable PYWIKIBOT2_TEST_WRITE_FAIL is set to 1. + + Otherwise the test class is skipped unless environment variable + PYWIKIBOT2_TEST_WRITE is set to 1. """ - if os.environ.get('PYWIKIBOT2_TEST_WRITE', '0') != '1': + super(SiteWriteMixin, cls).setUpClass() + + site = cls.get_site() + assert('test' in (site.family.name, site.code)) + + if cls.write == -1: + env_var = 'PYWIKIBOT2_TEST_WRITE_FAIL' + else: + env_var = 'PYWIKIBOT2_TEST_WRITE' + + if os.environ.get(env_var, '0') != '1': raise unittest.SkipTest( '%r write tests disabled. ' - 'Set PYWIKIBOT2_TEST_WRITE=1 to enable.' - % cls.__name__) + 'Set %s=1 to enable.' + % (cls.__name__, env_var)) if issubclass(cls, ForceCacheMixin): raise Exception( '%s can not be a subclass of both ' 'SiteEditTestCase and ForceCacheMixin' % cls.__name__) - - super(SiteWriteMixin, cls).setUpClass() class RequireUserMixin(TestCaseBase): diff --git a/tests/edit_failure_tests.py b/tests/edit_failure_tests.py index 57d3ffd..fa3c1db 100644 --- a/tests/edit_failure_tests.py +++ b/tests/edit_failure_tests.py @@ -4,6 +4,8 @@ These tests should never write to the wiki, unless something has broken badly. + +These tests use special code 'write = -1' for edit failures. """ # # (C) Pywikibot team, 2014 @@ -27,7 +29,7 @@ """Test cases for edits which should fail to save.""" - write = True + write = -1 family = 'wikipedia' code = 'test' @@ -55,7 +57,7 @@ """Test cases for actions which should fail to save.""" - write = True + write = -1 family = 'wikipedia' code = 'test' @@ -87,7 +89,7 @@ family = 'wikidata' code = 'test' - write = True + write = -1 def test_itempage_save(self): """Test ItemPage save method inherited from superclass Page.""" -- To view, visit https://gerrit.wikimedia.org/r/190635 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I85bf198c2a30b69a9e29d220d8696a92642272bc Gerrit-PatchSet: 1 Gerrit-Project: pywikibot/core Gerrit-Branch: master Gerrit-Owner: John Vandenberg &lt;jayvdb(a)gmail.com&gt; Gerrit-Reviewer: John Vandenberg &lt;jayvdb(a)gmail.com&gt; Gerrit-Reviewer: Ladsgroup &lt;ladsgroup(a)gmail.com&gt; Gerrit-Reviewer: Merlijn van Deen &lt;valhallasw(a)arctus.nl&gt; Gerrit-Reviewer: XZise &lt;CommodoreFabianus(a)gmx.de&gt; Gerrit-Reviewer: jenkins-bot <>