Build Update for wikimedia/pywikibot-core
-------------------------------------
Build: #1737
Status: Passed
Duration: 54 minutes and 11 seconds
Commit: f4fca8f (master)
Author: Merlijn van Deen
Message: Win32 input for py3: streams are unicode not bytes
On Python 2, streams are bytes, so the unicode data returned from the
win32 api needs to be encoded (as utf-8, in this case). In Python 3,
this is all unicode, so we need to skip the encoding step.
Decoding works OK due to the 'if isinstance(x, unicode)' blocks.
Bug: T76236
Change-Id: I4121ac87816b144e334e7e47c61ce445f0b6e2c9
View the changeset: https://github.com/wikimedia/pywikibot-core/compare/b10b03cf76ca...f4fca8fd…
View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/42421861
--
You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications
jenkins-bot has submitted this change and it was merged.
Change subject: Win32 input for py3: streams are unicode not bytes
......................................................................
Win32 input for py3: streams are unicode not bytes
On Python 2, streams are bytes, so the unicode data returned from the
win32 api needs to be encoded (as utf-8, in this case). In Python 3,
this is all unicode, so we need to skip the encoding step.
Decoding works OK due to the 'if isinstance(x, unicode)' blocks.
Bug: T76236
Change-Id: I4121ac87816b144e334e7e47c61ce445f0b6e2c9
---
M pywikibot/userinterfaces/win32_unicode.py
1 file changed, 8 insertions(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/userinterfaces/win32_unicode.py b/pywikibot/userinterfaces/win32_unicode.py
index b92b273..dc85cc2 100755
--- a/pywikibot/userinterfaces/win32_unicode.py
+++ b/pywikibot/userinterfaces/win32_unicode.py
@@ -28,6 +28,9 @@
if sys.version_info[0] > 2:
unicode = str
+ PY3 = True
+else:
+ PY3 = False
if sys.platform == "win32":
import codecs
@@ -137,7 +140,11 @@
result = ReadConsoleW(self._hConsole, self.buffer, maxnum, byref(numrecv), None)
if not result:
raise Exception("stdin failure")
- return self.buffer.value[:numrecv.value].encode(self.encoding)
+ data = self.buffer.value[:numrecv.value]
+ if not PY3:
+ return data.encode(self.encoding)
+ else:
+ return data
if real_stdout or real_stderr:
# BOOL WINAPI WriteConsoleW(HANDLE hOutput, LPWSTR lpBuffer, DWORD nChars,
--
To view, visit https://gerrit.wikimedia.org/r/176371
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I4121ac87816b144e334e7e47c61ce445f0b6e2c9
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Introduce config.weblink_dead_days instead of global day
......................................................................
Introduce config.weblink_dead_days instead of global day
Using "global day" prevents from importing
linkchecker classes into another script.
Rename the variable and add it to 'config.py'
Change-Id: Ic95c31ecc7dbb557db8319b5857a4e94e4347db6
---
M config.py
M weblinkchecker.py
2 files changed, 13 insertions(+), 8 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/config.py b/config.py
index 50fafa2..19f8369 100644
--- a/config.py
+++ b/config.py
@@ -336,6 +336,9 @@
report_dead_links_on_talk = False
+# Don't alert on links days_dead old or younger
+weblink_dead_days = 7
+
############## DATABASE SETTINGS ##############
db_hostname = 'localhost'
db_username = 'wikiuser'
diff --git a/weblinkchecker.py b/weblinkchecker.py
index e7f2a90..501e01d 100644
--- a/weblinkchecker.py
+++ b/weblinkchecker.py
@@ -51,8 +51,9 @@
-notalk Overrides the report_dead_links_on_talk config variable, disabling
the feature.
--day the first time found dead link longer than x day ago, it should
- probably be fixed or removed. if no set, default is 7 day.
+
+-day Do not report broken link if the link is there only since
+ x days or less. If not set, the default is 7 days.
All other parameters will be regarded as part of the title of a single page,
and the bot will only work on that single page.
@@ -71,6 +72,9 @@
links on the article's talk page if (and ONLY if)
the linked page has been unavailable at least two
times during a timespan of at least one week.
+
+weblink_dead_days - sets the timespan (default: one week) after which
+ a dead link will be reported
Syntax examples:
python weblinkchecker.py -start:!
@@ -500,7 +504,7 @@
else:
pywikibot.output('*[[%s]] links to %s - %s.'
% (self.page.title(), self.url, message))
- self.history.setLinkDead(self.url, message, self.page, day)
+ self.history.setLinkDead(self.url, message, self.page, config.weblink_dead_days)
class History:
@@ -570,7 +574,7 @@
self.reportThread.report(url, errorReport, containingPage,
archiveURL)
- def setLinkDead(self, url, error, page, day):
+ def setLinkDead(self, url, error, page, weblink_dead_days):
"""
Adds the fact that the link was found dead to the .dat file.
"""
@@ -586,7 +590,7 @@
# if the first time we found this link longer than x day ago
# (default is a week), it should probably be fixed or removed.
# We'll list it in a file so that it can be removed manually.
- if timeSinceFirstFound > 60 * 60 * 24 * day:
+ if timeSinceFirstFound > 60 * 60 * 24 * weblink_dead_days:
# search for archived page
archiveURL = pywikibot.weblib.getInternetArchiveURL(self.site, url)
if archiveURL is None:
@@ -806,8 +810,6 @@
# that are also used by other scripts and that determine on which pages
# to work on.
genFactory = pagegenerators.GeneratorFactory()
- global day
- day = 7
for arg in pywikibot.handleArgs():
if arg == '-talk':
config.report_dead_links_on_talk = True
@@ -823,7 +825,7 @@
elif arg.startswith('-ignore:'):
HTTPignore.append(int(arg[8:]))
elif arg.startswith('-day:'):
- day = int(arg[5:])
+ config.weblink_dead_days = int(arg[5:])
elif arg.startswith('-xmlstart'):
if len(arg) == 9:
xmlStart = pywikibot.input(
--
To view, visit https://gerrit.wikimedia.org/r/175610
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic95c31ecc7dbb557db8319b5857a4e94e4347db6
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/compat
Gerrit-Branch: master
Gerrit-Owner: saper <saper(a)saper.info>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot <>
Gerrit-Reviewer: saper <saper(a)saper.info>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] HTTP: Use urlparse to get host if site not given
......................................................................
[FIX] HTTP: Use urlparse to get host if site not given
Change-Id: I18b0acbdb00ecfa474646efb5796d88664c1e173
---
M pywikibot/comms/http.py
1 file changed, 2 insertions(+), 1 deletion(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/comms/http.py b/pywikibot/comms/http.py
index f3e93c8..9e65821 100644
--- a/pywikibot/comms/http.py
+++ b/pywikibot/comms/http.py
@@ -241,6 +241,7 @@
site.ignore_certificate_error())
else:
baseuri = uri
+ host = urlparse.urlparse(uri).netloc
format_string = kwargs.setdefault("headers", {}).get("user-agent")
kwargs["headers"]["user-agent"] = user_agent(site, format_string)
@@ -260,7 +261,7 @@
raise request.data
if request.data[0].status == 504:
- raise Server504Error("Server %s timed out" % site.hostname())
+ raise Server504Error("Server %s timed out" % host)
if request.data[0].status == 414:
raise Server414Error('Too long GET request')
--
To view, visit https://gerrit.wikimedia.org/r/176331
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I18b0acbdb00ecfa474646efb5796d88664c1e173
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] Wikidataquery: raise ServerError correctly
......................................................................
[FIX] Wikidataquery: raise ServerError correctly
Change-Id: If0513951d0e31a8b1697443856556e20b402c5d5
---
M pywikibot/data/wikidataquery.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/data/wikidataquery.py b/pywikibot/data/wikidataquery.py
index c3eaf35..5ad6bed 100644
--- a/pywikibot/data/wikidataquery.py
+++ b/pywikibot/data/wikidataquery.py
@@ -567,7 +567,7 @@
data = json.loads(resp)
except ValueError:
pywikibot.warning(u"Data received from host but no JSON could be decoded")
- raise pywikibot.ServerError
+ raise pywikibot.ServerError("Data received from host but no JSON could be decoded")
return data
--
To view, visit https://gerrit.wikimedia.org/r/176330
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: If0513951d0e31a8b1697443856556e20b402c5d5
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
Build Update for wikimedia/pywikibot-core
-------------------------------------
Build: #1734
Status: Broken
Duration: 1 hour, 39 minutes, and 0 seconds
Commit: f812488 (master)
Author: Fabian Neundorf
Message: [FIX] CFD: Only allow script to be run on enwp
This script only works on the English Wikipedia, so don't even start if
not using that.
Bug: T71015
Change-Id: I3abacb1746b7f02ab654be4a76ccfb2539e10f29
View the changeset: https://github.com/wikimedia/pywikibot-core/compare/4513417a18dd...f8124888…
View the full build log and details: https://travis-ci.org/wikimedia/pywikibot-core/builds/42367712
--
You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications
jenkins-bot has submitted this change and it was merged.
Change subject: [FIX] CFD: Only allow script to be run on enwp
......................................................................
[FIX] CFD: Only allow script to be run on enwp
This script only works on the English Wikipedia, so don't even start if
not using that.
Bug: T71015
Change-Id: I3abacb1746b7f02ab654be4a76ccfb2539e10f29
---
M scripts/cfd.py
1 file changed, 6 insertions(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/cfd.py b/scripts/cfd.py
index 8687735..53a49aa 100644
--- a/scripts/cfd.py
+++ b/scripts/cfd.py
@@ -17,8 +17,9 @@
__version__ = '$Id$'
#
-import pywikibot
import re
+import pywikibot
+from pywikibot import config2 as config
import category
# The location of the CFD working page.
@@ -69,6 +70,10 @@
"""
pywikibot.handle_args(args)
+ if config.family != 'wikipedia' or config.mylang != 'en':
+ pywikibot.warning('CFD does work only on the English Wikipedia.')
+ return
+
page = pywikibot.Page(pywikibot.Site(), cfdPage)
# Variable declarations
--
To view, visit https://gerrit.wikimedia.org/r/176323
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I3abacb1746b7f02ab654be4a76ccfb2539e10f29
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot <>
Build Update for jayvdb/pywikibot-core
-------------------------------------
Build: #105
Status: Passed
Duration: 49 minutes and 17 seconds
Commit: 88ad3a4 (querygen-iter-flow-3)
Author: John Vandenberg
Message: Simplify QueryGenerator.__iter__ flow control
Rearrange the logic to use less nested branching, and
destroy the data before exiting the iteration.
Removes 'None' as a valid value for QueryGenerator.limit;
it was not able to be set using set_maximum_items().
Adds some iterator state information to help with debugging
and test validation, and dont leave data in an instance variable.
Bug: 64489
Change-Id: Ie3a9e3576a0174703bcccf0d77ed42d56da0d09f
View the changeset: https://github.com/jayvdb/pywikibot-core/compare/4513417a18dd^...88ad3a46f5…
View the full build log and details: https://travis-ci.org/jayvdb/pywikibot-core/builds/42316680
--
You can configure recipients for build notifications in your .travis.yml file. See http://docs.travis-ci.com/user/notifications