jenkins-bot has submitted this change and it was merged.
Change subject: Introduce config.weblink_dead_days instead of day
......................................................................
Introduce config.weblink_dead_days instead of day
Renamed the variable day and add it to 'config2.py'
Port of pywikibot-compat 6bf00712
Bug: T76294
Change-Id: Ic95c31ecc7dbb557db8319b5857a4e94e4347db6
---
M pywikibot/config2.py
M scripts/weblinkchecker.py
2 files changed, 15 insertions(+), 8 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index 5f82eaa..5547271 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -622,6 +622,9 @@
report_dead_links_on_talk = False
+# Don't alert on links days_dead old or younger
+weblink_dead_days = 7
+
# ############# DATABASE SETTINGS ##############
# Setting to connect the database or replica of the database of the wiki.
# db_name_format can be used to manipulate the dbName of site.
diff --git a/scripts/weblinkchecker.py b/scripts/weblinkchecker.py
index 86c0da7..5bd54da 100755
--- a/scripts/weblinkchecker.py
+++ b/scripts/weblinkchecker.py
@@ -53,8 +53,9 @@
-notalk Overrides the report_dead_links_on_talk config variable, disabling
the feature.
--day the first time found dead link longer than x day ago, it should
- probably be fixed or removed. if no set, default is 7 day.
+
+-day Do not report broken link if the link is there only since
+ x days or less. If not set, the default is 7 days.
The following config variables are supported:
@@ -70,6 +71,9 @@
links on the article's talk page if (and ONLY if)
the linked page has been unavailable at least two
times during a timespan of at least one week.
+
+weblink_dead_days - sets the timespan (default: one week) after which
+ a dead link will be reported
Syntax examples:
python pwb.py weblinkchecker -start:!
@@ -563,7 +567,8 @@
else:
pywikibot.output('*[[%s]] links to %s - %s.'
% (self.page.title(), self.url, message))
- self.history.setLinkDead(self.url, message, self.page, self.day)
+ self.history.setLinkDead(self.url, message, self.page,
+ config.weblink_dead_days)
class History(object):
@@ -637,7 +642,7 @@
self.reportThread.report(url, errorReport, containingPage,
archiveURL)
- def setLinkDead(self, url, error, page, day):
+ def setLinkDead(self, url, error, page, weblink_dead_days):
"""Add the fact that the link was found dead to the .dat file."""
self.semaphore.acquire()
now = time.time()
@@ -651,7 +656,7 @@
# if the first time we found this link longer than x day ago
# (default is a week), it should probably be fixed or removed.
# We'll list it in a file so that it can be removed manually.
- if timeSinceFirstFound > 60 * 60 * 24 * day:
+ if timeSinceFirstFound > 60 * 60 * 24 * weblink_dead_days:
# search for archived page
try:
archiveURL = get_archive_url(url)
@@ -886,7 +891,6 @@
gen = None
xmlFilename = None
HTTPignore = []
- day = 7
if isinstance(memento_client, ImportError):
warn('memento_client not imported: %s' % memento_client, ImportWarning)
@@ -905,7 +909,7 @@
elif arg.startswith('-ignore:'):
HTTPignore.append(int(arg[8:]))
elif arg.startswith('-day:'):
- day = int(arg[5:])
+ config.weblink_dead_days = int(arg[5:])
elif arg.startswith('-xmlstart'):
if len(arg) == 9:
xmlStart = pywikibot.input(
@@ -935,7 +939,7 @@
pageNumber = max(240, config.max_external_links * 2)
gen = pagegenerators.PreloadingGenerator(gen, step=pageNumber)
gen = pagegenerators.RedirectFilterPageGenerator(gen)
- bot = WeblinkCheckerRobot(gen, HTTPignore, day)
+ bot = WeblinkCheckerRobot(gen, HTTPignore, config.weblink_dead_days)
try:
bot.run()
finally:
--
To view, visit https://gerrit.wikimedia.org/r/259179
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic95c31ecc7dbb557db8319b5857a4e94e4347db6
Gerrit-PatchSet: 6
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Mhutti1 <mhutti1(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Remove dependency on pYsearch
......................................................................
Remove dependency on pYsearch
pYsearch does not work, and an alternative implementation
is not yet available.
Bug: T106062
Bug: T106085
Change-Id: I162cc2cc125dc894d8a05ca711577dc37af5dcb2
(manual cherry-pick of 2ad0dd765)
---
M pywikibot/config2.py
M pywikibot/pagegenerators.py
M requirements.txt
M setup.py
4 files changed, 6 insertions(+), 6 deletions(-)
Approvals:
John Vandenberg: Looks good to me, but someone else must approve
Yuvipanda: Looks good to me, but someone else must approve
Legoktm: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index f2d644f..16c01ba 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -612,9 +612,8 @@
# ############# SEARCH ENGINE SETTINGS ##############
-# Some scripts allow using the Yahoo! Search Web Services. To use this feature,
-# you must install the pYsearch module from http://pysearch.sourceforge.net
-# and get a Yahoo AppID from https://developer.yahoo.com/
+# Yahoo! Search Web Services are not operational.
+# See https://phabricator.wikimedia.org/T106085
yahoo_appid = ''
# To use Windows Live Search web service you must get an AppID from
diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index dd96d21..e6420a4 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -2072,7 +2072,6 @@
To use this generator, install pYsearch
"""
- # values larger than 100 fail
def __init__(self, query=None, count=100, site=None):
"""
Constructor.
@@ -2080,6 +2079,10 @@
@param site: Site for generator results.
@type site: L{pywikibot.site.BaseSite}
"""
+ raise RuntimeError(
+ 'pagegenerator YahooSearchPageGenerator is not functional.\n'
+ 'See https://phabricator.wikimedia.org/T106085')
+
self.query = query or pywikibot.input(u'Please enter the search query:')
self.count = count
if site is None:
diff --git a/requirements.txt b/requirements.txt
index f3dd315..f28f9e4 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -25,7 +25,6 @@
pydot
# core pagegenerators
-pYsearch
google >= 1.7
# scripts/script_wui.py:
diff --git a/setup.py b/setup.py
index 9a4ac85..de7bb95 100644
--- a/setup.py
+++ b/setup.py
@@ -62,7 +62,6 @@
extra_deps.update({
'csv': ['unicodecsv'],
'MySQL': ['oursql'],
- 'Yahoo': ['pYsearch'],
})
script_deps = {
--
To view, visit https://gerrit.wikimedia.org/r/258728
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I162cc2cc125dc894d8a05ca711577dc37af5dcb2
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Gergő Tisza <gtisza(a)wikimedia.org>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Legoktm <legoktm.wikipedia(a)gmail.com>
Gerrit-Reviewer: Yuvipanda <yuvipanda(a)wikimedia.org>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Change license of reflinks.py from GPL to MIT
......................................................................
Change license of reflinks.py from GPL to MIT
The Pywikibot 2.0 license was changed in the following gerrit change,
with all authors except one approving the change.
I2cee179737b45fe95c206e928ad0b8c8b560c9e2
https://gerrit.wikimedia.org/r/#/c/139294/
Change-Id: I8c4e1f9fc3ef445c8d972b84b37f0b023df90117
---
M reflinks.py
1 file changed, 1 insertion(+), 1 deletion(-)
Approvals:
John Vandenberg: Looks good to me, approved
jenkins-bot: Verified
diff --git a/reflinks.py b/reflinks.py
index 1f0519d..7cd2faa 100644
--- a/reflinks.py
+++ b/reflinks.py
@@ -34,7 +34,7 @@
# (C) 2008 - Nicolas Dumazet ( en:User:NicDumZ )
# (C) Pywikipedia bot team, 2008-2013
#
-# Distributed under the terms of the GPL
+# Distributed under the terms of the MIT license.
#
__version__ = '$Id$'
#
--
To view, visit https://gerrit.wikimedia.org/r/181357
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I8c4e1f9fc3ef445c8d972b84b37f0b023df90117
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/compat
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: DrTrigon <dr.trigon(a)surfeu.ch>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: toJSON assert correct claim type
......................................................................
toJSON assert correct claim type
continues eb9d3a80
Bug: T113212
Change-Id: I0fc771ce8509cea3439f9d029ea8b108e1deb43d
---
M pywikibot/page.py
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
Lokal Profil: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index ec7430a..22e6f33 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -4208,7 +4208,7 @@
data['qualifiers-order'] = list(self.qualifiers.keys())
for prop, qualifiers in self.qualifiers.items():
for qualifier in qualifiers:
- qualifier.isQualifier = True
+ assert qualifier.isQualifier is True
data['qualifiers'][prop] = [qualifier.toJSON() for qualifier in qualifiers]
if len(self.sources) > 0:
data['references'] = []
@@ -4217,7 +4217,7 @@
for prop, val in collection.items():
reference['snaks'][prop] = []
for source in val:
- source.isReference = True
+ assert source.isReference is True
src_data = source.toJSON()
if 'hash' in src_data:
if 'hash' not in reference:
--
To view, visit https://gerrit.wikimedia.org/r/244834
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I0fc771ce8509cea3439f9d029ea8b108e1deb43d
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Lokal Profil <lokal.profil(a)gmail.com>
Gerrit-Reviewer: Ricordisamoa <ricordisamoa(a)openmailbox.org>
Gerrit-Reviewer: Smalyshev <smalyshev(a)wikimedia.org>
Gerrit-Reviewer: Tobias47n9e <tobias47n9e(a)gmail.com>
Gerrit-Reviewer: XZise <CommodoreFabianus(a)gmx.de>
Gerrit-Reviewer: jenkins-bot <>
jenkins-bot has submitted this change and it was merged.
Change subject: Update revId upon claim change
......................................................................
Update revId upon claim change
changeTarget, removeSources: Prevents edit conflicts
addSources: No need to set every time through the loop
Conflicts:
pywikibot/page.py
Change-Id: I2d8b69a3894aaf6d58a62ecba48f5dc584617e82
(manually cherry picked from 38ae920a24be3)
---
M pywikibot/page.py
1 file changed, 4 insertions(+), 2 deletions(-)
Approvals:
John Vandenberg: Looks good to me, approved
Lokal Profil: Looks good to me, but someone else must approve
jenkins-bot: Verified
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 5be034b..313bd56 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -4115,6 +4115,7 @@
**kwargs)
# TODO: Re-create the entire item from JSON, not just id
self.snak = data['claim']['id']
+ self.on_item.lastrevid = data['pageinfo']['lastrevid']
def getTarget(self):
"""
@@ -4194,10 +4195,10 @@
@type claims: list of pywikibot.Claim
"""
data = self.repo.editSource(self, claims, new=True, **kwargs)
+ self.on_item.lastrevid = data['pageinfo']['lastrevid']
source = defaultdict(list)
for claim in claims:
claim.hash = data['reference']['hash']
- self.on_item.lastrevid = data['pageinfo']['lastrevid']
source[claim.getID()].append(claim)
self.sources.append(source)
@@ -4217,7 +4218,8 @@
@param sources: the sources to remove
@type sources: list of pywikibot.Claim
"""
- self.repo.removeSources(self, sources, **kwargs)
+ data = self.repo.removeSources(self, sources, **kwargs)
+ self.on_item.lastrevid = data['pageinfo']['lastrevid']
for source in sources:
source_dict = defaultdict(list)
source_dict[source.getID()].append(source)
--
To view, visit https://gerrit.wikimedia.org/r/243515
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I2d8b69a3894aaf6d58a62ecba48f5dc584617e82
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: John Vandenberg <jayvdb(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Lokal Profil <lokal.profil(a)gmail.com>
Gerrit-Reviewer: jenkins-bot <>