Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1306060?usp=email )
Change subject: interwiki: implement -onlylink whitelist and unify link preservation logic
......................................................................
interwiki: implement -onlylink whitelist and unify link preservation logic
Bug: T430408
Change-Id: I47baf57419859daf17f38cda7357543cc9243ce5
---
M scripts/interwiki.py
1 file changed, 37 insertions(+), 8 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/scripts/interwiki.py b/scripts/interwiki.py
index d1aeda3..a5a6955 100755
--- a/scripts/interwiki.py
+++ b/scripts/interwiki.py
@@ -239,6 +239,13 @@
treating them as correct. You can also specify a list of
site codes, separated by commas.
+-onlylink: Used as ``-onlylink:xx`` where xx is a site code, restrict
+ processing to only the specified site code(s). All other
+ interwiki links are ignored and not modified. You can specify
+ a list of site codes, separated by commas.
+
+ .. version-added:: 11.5
+
-ignore: Used as -ignore:xx:aaa where xx is a language code, and
aaa is a page title to be ignored.
@@ -466,6 +473,7 @@
minsubjects = config.interwiki_min_subjects
needlimit = 0
neverlink = []
+ onlylink = []
nobackonly = False
parenthesesonly = False
quiet = False
@@ -540,6 +548,8 @@
self.skip.update(skip_page_gen)
elif arg == 'neverlink':
self.neverlink += value.split(',')
+ elif arg == 'onlylink':
+ self.onlylink += value.split(',')
elif arg == 'ignore':
self.ignore += [pywikibot.Page(pywikibot.Site(), p)
for p in value.split(',')]
@@ -1009,10 +1019,16 @@
def isIgnored(self, page) -> bool:
"""Return True if pages is to be ignored."""
- if page.site.code in self.conf.neverlink:
+ code = page.site.code
+
+ if code in self.conf.neverlink:
pywikibot.info(f'Skipping link {page} to an ignored language')
return True
+ if self.conf.onlylink and code not in self.conf.onlylink:
+ pywikibot.info(f'Skipping link {page} to a non-selected language')
+ return True
+
if page in self.conf.ignore:
pywikibot.info(f'Skipping link {page} to an ignored page')
return True
@@ -1229,10 +1245,16 @@
for link in iw:
linkedPage = pywikibot.Page(link)
- if linkedPage.site.code in self.conf.neverlink:
- pywikibot.info(f'NOTE: {self.origin}: {page} has interwiki '
- f'link to {linkedPage} in neverlink site code,'
- ' preserving without following')
+ code = linkedPage.site.code
+ msg = (f'NOTE: {self.origin}: {page} has interwiki link to '
+ f'{linkedPage} {{}}; preserving without following')
+
+ if code in self.conf.neverlink:
+ pywikibot.info(msg.format('in neverlink site codes'))
+ continue
+
+ if self.conf.onlylink and code not in self.conf.onlylink:
+ pywikibot.info(msg.format('outside the selected site codes'))
continue
if self.conf.hintsareright and linkedPage.site in self.hintedsites:
@@ -1646,12 +1668,19 @@
# Put interwiki links into a map
old = {p.site: p for p in interwikis}
- # Preserve existing interwiki links to neverlink site codes
+ # Preserve interwiki links to neverlink or outside onlylink codes
for site, oldpage in old.items():
- if site.code in self.conf.neverlink and site not in new:
+ if site in new:
+ continue
+
+ never = site.code in self.conf.neverlink
+ only = self.conf.onlylink and site.code not in self.conf.onlylink
+
+ if never or only:
+ reason = 'in neverlink' if never else 'outside onlylink'
new[site] = oldpage
pywikibot.info(f'Preserving link to {oldpage} '
- f'(site code {site.code} is in neverlink list)')
+ f'(site code {site.code} is {reason} list)')
# Check what needs to get done
mods, mcomment, adding, removing, modifying = compareLanguages(
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1306060?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I47baf57419859daf17f38cda7357543cc9243ce5
Gerrit-Change-Number: 1306060
Gerrit-PatchSet: 3
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Ivan-r <ivanrussian15(a)gmail.com>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1307533?usp=email )
Change subject: tests: temporary skip TestProductionAndTestSite and TestMediaInfoEditing
......................................................................
tests: temporary skip TestProductionAndTestSite and TestMediaInfoEditing
Bug: T431173
Change-Id: Ib2df352a55bf1810f91007260bea295ab2b010fc
---
M tests/file_tests.py
M tests/site_tests.py
2 files changed, 2 insertions(+), 0 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
jenkins-bot: Verified
diff --git a/tests/file_tests.py b/tests/file_tests.py
index 2ae94c8..0684ee7 100755
--- a/tests/file_tests.py
+++ b/tests/file_tests.py
@@ -462,6 +462,7 @@
item.statements, pywikibot.page._collections.ClaimCollection)
+(a)unittest.skip('Skipping TimeoutError due to T431173')
class TestMediaInfoEditing(TestCase):
"""Test writing structured data of FilePage."""
diff --git a/tests/site_tests.py b/tests/site_tests.py
index 8bb3d31..473c5d4 100755
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -1162,6 +1162,7 @@
pywikibot.Site('ceb', 'wowwiki')
+(a)unittest.skip('Skipping TimeoutError due to T431173')
class TestProductionAndTestSite(AlteredDefaultSiteTestCase):
"""Test site without other production sites in its family."""
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1307533?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ib2df352a55bf1810f91007260bea295ab2b010fc
Gerrit-Change-Number: 1307533
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Xqt has submitted this change. ( https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1306106?usp=email )
Change subject: interwiki: catch OSError while writing a dump
......................................................................
interwiki: catch OSError while writing a dump
Bare except for 'Exception' is too broad and may catch unintended errors.
Use OSError instead.
Change-Id: Ibd474128cf710f0f046415fc8080d12059656b15
---
M scripts/interwiki.py
1 file changed, 2 insertions(+), 2 deletions(-)
Approvals:
Xqt: Verified; Looks good to me, approved
diff --git a/scripts/interwiki.py b/scripts/interwiki.py
index 1d0b941..d1aeda3 100755
--- a/scripts/interwiki.py
+++ b/scripts/interwiki.py
@@ -1218,8 +1218,8 @@
f.write(
f' [{config.interwiki_graph_url}{filename} graph]')
f.write('\n')
- # FIXME: What errors are we catching here?
- except Exception:
+ # Catch file operation failures while writing autonomous dump.
+ except OSError:
pywikibot.info(
'File autonomous_problems.dat open or corrupted! '
'Try again with -restore.')
--
To view, visit https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1306106?usp=email
To unsubscribe, or for help writing mail filters, visit https://gerrit.wikimedia.org/r/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: Ibd474128cf710f0f046415fc8080d12059656b15
Gerrit-Change-Number: 1306106
Gerrit-PatchSet: 2
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot