jenkins-bot has submitted this change and it was merged.
Change subject: Fix 'Disable pwb_tests to prevent Travis build failures'
......................................................................
Fix 'Disable pwb_tests to prevent Travis build failures'
Now with actual closing bracket.
Change-Id: Ic8522e1f08c359fff4085c172e696e62ca969a70
Fixes-Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a
---
M tests/pwb_tests.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Legoktm: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py
index af44203..5a97483 100644
--- a/tests/pwb_tests.py
+++ b/tests/pwb_tests.py
@@ -22,7 +22,7 @@
Probably the shelling out causes an issue, but we have to investigate this.
See https://gerrit.wikimedia.org/r/#/c/76486/ and
- https://gerrit.wikimedia.org/r/#/c/82370/ for details. """
+ https://gerrit.wikimedia.org/r/#/c/82370/ for details. """)
def testScriptEnvironment(self):
"""Make sure the environment is not contaminated, and is the same as
the environment we get when directly running a script."""
--
To view, visit https://gerrit.wikimedia.org/r/82422
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic8522e1f08c359fff4085c172e696e62ca969a70
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: Disable pwb_tests to prevent Travis build failures
......................................................................
Disable pwb_tests to prevent Travis build failures
I'm fairly certain this is a testing issue related to shelling
out, or possibly to environment variables. However, the test
passes for me, so I think the most reasonable option for now
is to disable the test and then to see why Travis fails.
Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a
---
M tests/pwb_tests.py
1 file changed, 5 insertions(+), 0 deletions(-)
Approvals:
Legoktm: Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py
index 4cb9759..af44203 100644
--- a/tests/pwb_tests.py
+++ b/tests/pwb_tests.py
@@ -18,6 +18,11 @@
testbasepath = os.path.join(basepath, 'tests', 'pwb')
class TestPwb(unittest.TestCase):
+ @unittest.skip("""Skipping test due to broken Travis run.
+
+Probably the shelling out causes an issue, but we have to investigate this.
+See https://gerrit.wikimedia.org/r/#/c/76486/ and
+ https://gerrit.wikimedia.org/r/#/c/82370/ for details. """
def testScriptEnvironment(self):
"""Make sure the environment is not contaminated, and is the same as
the environment we get when directly running a script."""
--
To view, visit https://gerrit.wikimedia.org/r/82370
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Iaeaaf1b34d3e2bcacf8308d2c3b74bd8f711ec6a
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: Fix and document parameters that Site.newpages was passing to Site.recentchanges
......................................................................
Fix and document parameters that Site.newpages was passing to Site.recentchanges
Also added type hinting to the various parameters.
Change-Id: Icbc9f988a6c533788f1d1d9867411eda901017c2
---
M pywikibot/site.py
1 file changed, 22 insertions(+), 1 deletion(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/site.py b/pywikibot/site.py
index 1b1f1d6..1c0b144 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -2242,29 +2242,43 @@
namespaces=None, pagelist=None, changetype=None,
showMinor=None, showBot=None, showAnon=None,
showRedirects=None, showPatrolled=None, topOnly=False,
- step=None, total=None):
+ step=None, total=None, user=None, excludeuser=None):
"""Iterate recent changes.
@param start: Timestamp to start listing from
+ @type start: pywikibot.Timestamp
@param end: Timestamp to end listing at
+ @type end: pywikibot.Timestamp
@param reverse: if True, start with oldest changes (default: newest)
+ @type reverse: bool
@param pagelist: iterate changes to pages in this list only
@param pagelist: list of Pages
@param changetype: only iterate changes of this type ("edit" for
edits to existing pages, "new" for new pages, "log" for log
entries)
+ @type changetype: basestring
@param showMinor: if True, only list minor edits; if False (and not
None), only list non-minor edits
+ @type showMinor: bool
@param showBot: if True, only list bot edits; if False (and not
None), only list non-bot edits
+ @type showBot: bool
@param showAnon: if True, only list anon edits; if False (and not
None), only list non-anon edits
+ @type showAnon: bool
@param showRedirects: if True, only list edits to redirect pages; if
False (and not None), only list edits to non-redirect pages
+ @type showRedirects: bool
@param showPatrolled: if True, only list patrolled edits; if False
(and not None), only list non-patrolled edits
+ @type showPatrolled: bool
@param topOnly: if True, only list changes that are the latest revision
(default False)
+ @type topOnly: bool
+ @param user: if not None, only list edits by this user or users
+ @type user: basestring|list
+ @param excludeuser: if not None, exclude edits by this user or users
+ @type excludeuser: basestring|list
"""
if start and end:
@@ -2312,6 +2326,13 @@
rcshow.append(filters[item] and item or ("!" + item))
if rcshow:
rcgen.request["rcshow"] = "|".join(rcshow)
+
+ if user:
+ rcgen.request['rcuser'] = user
+
+ if excludeuser:
+ rcgen.request['rcexcludeuser'] = excludeuser
+
return rcgen
@deprecate_arg("number", "limit")
--
To view, visit https://gerrit.wikimedia.org/r/82220
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Icbc9f988a6c533788f1d1d9867411eda901017c2
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
Xqt has submitted this change and it was merged.
Change subject: Updated pwb.py to better mirror direct script runs
......................................................................
Updated pwb.py to better mirror direct script runs
Before, execfile() ran the script in environment that pwb.py has, which
included os and path imports, and several other variables.
This commit uses run_python_file from Ned Batchelors' coverage.py [1],
available under the BSD license [2]. See also his blog posts on the
subject [3,4].
This commit also adds a test that shells out to compare the locals()
for runs through pwb.py as well as to a script directly.
After this change, the effects of changeset 76484 can be seen using pwb.py
[1] https://bitbucket.org/ned/coveragepy/src/b5abcee50dbe/coverage/execfile.py
[2] https://bitbucket.org/ned/coveragepy/src/2c5fb3a8b81cc56d8ad57dd1bd83ef7740…
[3] http://nedbatchelder.com/blog/200904/running_a_python_file_as_main.html
[4] http://nedbatchelder.com/blog/200905/running_a_python_file_as_main_take_2.h…
Change-Id: If9458fca50f07f08441dbb6e06f78bdbae2065de
---
M pwb.py
A tests/pwb/print_locals.py
A tests/pwb_tests.py
3 files changed, 89 insertions(+), 13 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pwb.py b/pwb.py
index ae732ed..94fef0c 100644
--- a/pwb.py
+++ b/pwb.py
@@ -12,8 +12,50 @@
# Distributed under the terms of the MIT license.
#
-import sys
+
+# The following snippet was developed by Ned Batchelder (and others)
+# for coverage.py [1], and is available under the BSD license (see [2])
+# [1] https://bitbucket.org/ned/coveragepy/src/b5abcee50dbe/coverage/execfile.py
+# [2] https://bitbucket.org/ned/coveragepy/src/2c5fb3a8b81cc56d8ad57dd1bd83ef7740…
+
+import imp
import os
+import sys
+
+
+def run_python_file(filename, args):
+ """Run a python file as if it were the main program on the command line.
+
+ `filename` is the path to the file to execute, it need not be a .py file.
+ `args` is the argument array to present as sys.argv, including the first
+ element representing the file being executed.
+
+ """
+ # Create a module to serve as __main__
+ old_main_mod = sys.modules['__main__']
+ main_mod = imp.new_module('__main__')
+ sys.modules['__main__'] = main_mod
+ main_mod.__file__ = filename
+ main_mod.__builtins__ = sys.modules['__builtin__']
+
+ # Set sys.argv and the first path element properly.
+ old_argv = sys.argv
+ old_path0 = sys.path[0]
+ sys.argv = args
+ sys.path[0] = os.path.dirname(filename)
+
+ try:
+ source = open(filename).read()
+ exec compile(source, filename, "exec") in main_mod.__dict__
+ finally:
+ # Restore the old __main__
+ sys.modules['__main__'] = old_main_mod
+
+ # Restore the old argv and path
+ sys.argv = old_argv
+ sys.path[0] = old_path0
+
+#### end of snippet
if sys.version_info[0] != 2:
raise RuntimeError("ERROR: Pywikipediabot only runs under Python 2")
@@ -33,21 +75,20 @@
os.environ["PYWIKIBOT2_DIR"] = os.path.split(__file__)[0]
if not os.path.exists(os.path.join(os.environ["PYWIKIBOT2_DIR"], "user-config.py")):
- execfile('generate_user_files.py')
+ run_python_file('generate_user_files.py', ['generate_user_files.py'])
-sys.argv.pop(0)
-if len(sys.argv) > 0:
- if not os.path.exists(sys.argv[0]):
- testpath = os.path.join(os.path.split(__file__)[0], 'scripts', sys.argv[0])
+if len(sys.argv) > 1:
+ fn = sys.argv[1]
+ args = sys.argv[1:]
+
+ if not os.path.exists(fn):
+ testpath = os.path.join(os.path.split(__file__)[0], 'scripts', fn)
if os.path.exists(testpath):
- sys.argv[0] = testpath
+ fn = testpath
else:
testpath = testpath + '.py'
if os.path.exists(testpath):
- sys.argv[0] = testpath
+ fn = testpath
else:
- raise Exception("%s not found!" % sys.argv[0])
- sys.path.append(os.path.split(sys.argv[0])[0])
- execfile(sys.argv[0])
-else:
- sys.argv.append('')
+ raise Exception("%s not found!" % fn)
+ run_python_file(fn, args)
diff --git a/tests/pwb/print_locals.py b/tests/pwb/print_locals.py
new file mode 100644
index 0000000..1b84ac0
--- /dev/null
+++ b/tests/pwb/print_locals.py
@@ -0,0 +1,4 @@
+"""docstring"""
+
+for k,v in locals().copy().iteritems():
+ print repr(k), ":", repr(v)
diff --git a/tests/pwb_tests.py b/tests/pwb_tests.py
new file mode 100644
index 0000000..4cb9759
--- /dev/null
+++ b/tests/pwb_tests.py
@@ -0,0 +1,31 @@
+# -*- coding: utf-8 -*-
+#
+# (C) Pywikipedia bot team, 2007
+#
+# Distributed under the terms of the MIT license.
+#
+__version__ = '$Id$'
+
+import os
+import sys
+import subprocess
+
+import unittest
+
+pypath = sys.executable
+basepath = os.path.split(os.path.split(__file__)[0])[0]
+pwbpath = os.path.join(basepath, 'pwb.py')
+testbasepath = os.path.join(basepath, 'tests', 'pwb')
+
+class TestPwb(unittest.TestCase):
+ def testScriptEnvironment(self):
+ """Make sure the environment is not contaminated, and is the same as
+ the environment we get when directly running a script."""
+ test = os.path.join(testbasepath, 'print_locals.py')
+
+ direct = subprocess.check_output([pypath, test])
+ vpwb = subprocess.check_output([pypath, pwbpath, test])
+ self.assertEqual(direct, vpwb)
+
+if __name__=="__main__":
+ unittest.main(verbosity=10)
--
To view, visit https://gerrit.wikimedia.org/r/76486
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: If9458fca50f07f08441dbb6e06f78bdbae2065de
Gerrit-PatchSet: 6
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: DrTrigon <dr.trigon(a)surfeu.ch>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Xqt has submitted this change and it was merged.
Change subject: Set correct language as source
......................................................................
Set correct language as source
Patch is basically the same as d90be6a8765.
Bug: https://sourceforge.net/p/pywikipediabot/feature-requests/346/
Change-Id: Ic2a6010903a3586803687f350f95ce33fa6abbb2
---
M scripts/claimit.py
1 file changed, 5 insertions(+), 5 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/claimit.py b/scripts/claimit.py
index 02bdf35..0f50db0 100755
--- a/scripts/claimit.py
+++ b/scripts/claimit.py
@@ -37,15 +37,15 @@
self.setSource(pywikibot.Site().language())
def setSource(self, lang):
- '''
+ """
Get the source
- '''
+ """
page = pywikibot.Page(self.repo, 'Wikidata:List of wikis/python')
source_values = json.loads(page.get())
source_values = source_values['wikipedia']
- for lang in source_values:
- source_values[lang] = pywikibot.ItemPage(self.repo,
- source_values[lang])
+ for source_lang in source_values:
+ source_values[source_lang] = pywikibot.ItemPage(self.repo,
+ source_values[source_lang])
if lang in source_values:
self.source = pywikibot.Claim(self.repo, 'p143')
--
To view, visit https://gerrit.wikimedia.org/r/82106
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic2a6010903a3586803687f350f95ce33fa6abbb2
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: Implementing 79569 [1] on core, making generate_user_files.py a little more user-friendly [1]: https://gerrit.wikimedia.org/r/79569
......................................................................
Implementing 79569 [1] on core, making generate_user_files.py a little more user-friendly
[1]: https://gerrit.wikimedia.org/r/79569
Change-Id: Idc4e5ae816bb10354475aa176414f866e0c8b298
---
M generate_user_files.py
1 file changed, 28 insertions(+), 6 deletions(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/generate_user_files.py b/generate_user_files.py
index ef680df..5beff0d 100644
--- a/generate_user_files.py
+++ b/generate_user_files.py
@@ -85,11 +85,20 @@
if choice == '' and default:
return default
-
+ try:
+ choice=int(choice)
+ except ValueError:
+ pass
+ if isinstance(choice, basestring):
+ if not choice in clist:
+ print("Invalid response")
+ else:
+ return choice
try:
return clist[int(choice) - 1]
except:
- print("Invalid response")
+ if not isinstance(choice, basestring):
+ print("Invalid response")
return response
@@ -127,7 +136,7 @@
set environment variables.""" % locals(), width=76)
for line in msg:
print line
- ok = raw_input("Is this OK? ([yes], [N]o) ")
+ ok = raw_input("Is this OK? ([y]es, [N]o) ")
if ok in ["Y", "y"]:
base_dir = new_base
return True
@@ -152,9 +161,22 @@
)
known_families = sorted(known_families)
fam = listchoice(known_families,
- "Select family of sites we are working on",
+ "Select family of sites we are working on, " \
+ "just enter the number not name",
default='wikipedia')
if fam not in single_wiki_families:
+ codesds=codecs.open("pywikibot/families/%s_family.py" % fam, "r","utf-8").read()
+ rre=re.compile("self\.languages\_by\_size *\= *(.+?)\]",re.DOTALL)
+ known_langs=[]
+ if not rre.findall(codesds):
+ rre=re.compile("self\.langs *\= *(.+?)\}",re.DOTALL)
+ if rre.findall(codesds):
+ import ast
+ known_langs=ast.literal_eval(rre.findall(codesds)[0]+u"}").keys()
+ else:
+ known_langs=eval(rre.findall(codesds)[0]+u"]")
+ print "This is the list of known language(s):"
+ print " ".join(sorted(known_langs))
mylang = raw_input("The language code of the site we're working on (default: 'en'): ") or 'en'
else:
mylang = fam
@@ -301,13 +323,13 @@
elif do_copy and "NO".startswith(do_copy):
break
if not os.path.isfile(os.path.join(base_dir, "user-config.py")):
- a = raw_input("Create user-config.py file? ([y]es, [N]o) ")
+ a = raw_input("Create user-config.py file? Required for running bots ([y]es, [N]o) ")
if a[:1] in ["Y", "y"]:
create_user_config()
else:
print("NOTE: user-config.py already exists in the directory")
if not os.path.isfile(os.path.join(base_dir, "user-fixes.py")):
- a = raw_input("Create user-fixes.py file? ([y]es, [N]o) ")
+ a = raw_input("Create user-fixes.py file? Optional and for advanced users ([y]es, [N]o) ")
if a[:1] in ["Y", "y"]:
create_user_fixes()
else:
--
To view, visit https://gerrit.wikimedia.org/r/81500
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Idc4e5ae816bb10354475aa176414f866e0c8b298
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot
Ladsgroup has submitted this change and it was merged.
Change subject: Overhaul README file with more information
......................................................................
Overhaul README file with more information
Now includes how to install the framework, a simple
code snippet, and a link on how to contribute patches.
Change-Id: I112b53e0a2a3767e459323a34dc012ccc05201bd
---
M README.md
1 file changed, 30 insertions(+), 6 deletions(-)
Approvals:
Ladsgroup: Verified; Looks good to me, approved
diff --git a/README.md b/README.md
index 3592013..7acbc43 100644
--- a/README.md
+++ b/README.md
@@ -1,10 +1,34 @@
-# Python Wikipedia Robot Framework
+# Pywikibot Framework
[.
+Also included are various general function scripts that can be adapted for different tasks.
-If you want to run the rewrite as a stand-alone package, please also download the
-the contents of the externals/ subdirectory. Confer externals/README for more
-information.
+## Quick start
+```
+git clone https://gerrit.wikimedia.org/r/pywikibot/core.git
+cd core
+git submodule update --init
+python pwb.py script_name
+```
+
+Our [installation guide](https://www.mediawiki.org/wiki/Manual:Pywikipediabot/Installation) has more details for advanced usage.
+
+## Usage
+
+If you wish to write your own script it's very easy to get started:
+
+```python
+import pywikibot
+site = pywikibot.Site('en', 'wikipedia') # The site we want to run our bot on
+page = pywikibot.Page(site, 'Wikipedia:Sandbox')
+text = page.get() # The current text on the page
+text = text.replace('foo', 'bar')
+page.put(text, 'Replacing "foo" with "bar"') # Saves the page
+```
+
+## Contributing
+
+Our code is maintained on Wikimedia's [Gerrit installation](https://gerrit.wikimedia.org/), [learn](https://www.mediawiki.org/wiki/Developer_access) how to get started.
+
--
To view, visit https://gerrit.wikimedia.org/r/82076
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I112b53e0a2a3767e459323a34dc012ccc05201bd
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Multichill <maarten(a)mdammers.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
](http://travis-ci.org/wikimedia/pywikibot-core)){kind=link}