Pyfisch has submitted this change and it was merged.
Change subject: code improvements
......................................................................
code improvements
- check for a valid speedy deletion template on live wiki. The
result is hold in the class container so we need to check only
once.
- use Site instead of Site.lang as key for twn dictionaries
this should also solve that bug pointed out at
https://sourceforge.net/p/pywikipediabot/bugs/1577/
i18n library uses the right code even the key would be changed
from code to lang on twn dict in future or there might be a
fallback from code to lang on i18n library.
- test whether we are logged in as sysop because we have no
exception here like in compat
Change-Id: I39dd03bc04846f129ee20c7959a57e4685e87d94
---
M scripts/redirect.py
1 file changed, 29 insertions(+), 9 deletions(-)
Approvals:
Pyfisch: Verified; Looks good to me, approved
diff --git a/scripts/redirect.py b/scripts/redirect.py
index 0487f2a..d43d0d5 100755
--- a/scripts/redirect.py
+++ b/scripts/redirect.py
@@ -379,6 +379,7 @@
self.number = number
self.delete = delete
self.exiting = False
+ self._valid_template = None
def prompt(self, question):
if not self.always:
@@ -393,6 +394,24 @@
elif choice == 'a':
self.always = True
return True
+
+ def has_valid_template(self, twtitle):
+ """"Check whether a template from translatewiki.net does exist on real
+ wiki. We assume we are always working on self.site
+
+ @param twtitle - a sting which is the i18n key
+
+ """
+ if self._valid_template is None:
+ self._valid_template = False
+ if i18n.twhas_key(self.site, twtitle):
+ template_msg = i18n.twtranslate(self.site, twtitle)
+ template = re.findall(u'.*?{{(.*?)[|}]', template_msg)
+ if template:
+ title = template[0]
+ page = pywikibot.Page(self.site, title, ns=10)
+ self._valid_template = page.exists()
+ return self._valid_template
def delete_broken_redirects(self):
# get reason for deletion text
@@ -476,22 +495,23 @@
redir_page.title(asLink=True))):
reason = i18n.twtranslate(self.site,
'redirect-remove-broken')
- try:
+ if self.site.logged_in(sysop=True):
redir_page.delete(reason, prompt=False)
- except pywikibot.NoUsername:
- if ((i18n.twhas_key(
- targetPage.site.lang,
- 'redirect-broken-redirect-template') and
- i18n.twhas_key(targetPage.site.lang,
- 'redirect-remove-broken')) or
- targetPage.site.lang == '-'):
+ else:
+ assert targetPage.site == self.site, (
+ u'target page is on different site %s'
+ % targetPage.site)
+ if (self.has_valid_template(
+ 'redirect-broken-redirect-template') and
+ i18n.twhas_key(targetPage.site,
+ 'redirect-remove-broken')):
pywikibot.output(u"No sysop in user-config.py, "
u"put page to speedy deletion.")
content = redir_page.get(get_redirect=True)
### TODO: Add bot's signature if needed
### Not supported via TW yet
content = i18n.twtranslate(
- targetPage.site.lang,
+ targetPage.site,
'redirect-broken-redirect-template'
) + "\n" + content
redir_page.put(content, reason)
--
To view, visit https://gerrit.wikimedia.org/r/104811
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I39dd03bc04846f129ee20c7959a57e4685e87d94
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Alex S.H. Lin <alexsh(a)mail2000.com.tw>
Gerrit-Reviewer: DrTrigon <dr.trigon(a)surfeu.ch>
Gerrit-Reviewer: Huji <huji.huji(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Meno25 <meno25mail(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Pyfisch <pyfisch(a)gmail.com>
Gerrit-Reviewer: Russell Blau <russblau(a)imapmail.org>
Gerrit-Reviewer: Siebrand <siebrand(a)wikimedia.org>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
Pyfisch has submitted this change and it was merged.
Change subject: use insite parameter of Page.title() to format links on target page
......................................................................
use insite parameter of Page.title() to format links on target page
Changing the config settings is no longer needed to get sensible
backlinks. This is follow up for https://gerrit.wikimedia.org/r/#/c/105436/
Additional changes:
- rewrite the documentation tho the common format
- remove the unused -fromfamily -fromlang comments. We may just use
-lang and -family option for the source site
- use the current site paramteters as default for the target site
but raise an exception if both are equal. This means you may
ommit eigther -tolang or -tofamily option.
- DuplicateFilterGenerator is always the last generator we see
for gen. Thus we do not display the generator but the arguments.
Change-Id: I5d4cd6673140ad87f26e884007ffb6d3434ecc56
---
M scripts/transferbot.py
1 file changed, 26 insertions(+), 25 deletions(-)
Approvals:
Pyfisch: Verified; Looks good to me, approved
diff --git a/scripts/transferbot.py b/scripts/transferbot.py
index 8ab1769..1bb5ef9 100644
--- a/scripts/transferbot.py
+++ b/scripts/transferbot.py
@@ -5,15 +5,19 @@
This script transfers pages from a source wiki to a target wiki. It also
copies edit history to a subpage.
-Target site can be specified with -tofamily and -tolang
-Source site can be specified with -fromfamily and -fromlang
-Page prefix on the new site can be specified with -prefix
+-tolang: The target site code.
-Existing pages are skipped by default. Pass -overwrite to overwrite pages.
+-tosite: The target site family.
+
+-prefix: Page prefix on the new site.
+
+-overwrite: Existing pages are skipped by default. Use his option to
+ overwrite pages.
Internal links are *not* repaired!
Pages to work on can be specified using any of:
+
¶ms;
Example commands:
@@ -29,6 +33,7 @@
#
# (C) Merlijn van Deen, 2014
+# (C) pywikibot team, 2014
#
# Distributed under the terms of the MIT license.
#
@@ -46,15 +51,17 @@
fromsite = pywikibot.getSite()
tolang = fromsite.code
- tofamily = None
+ tofamily = fromsite.family.name
prefix = ''
template = None
overwrite = False
+ gen_args = []
genFactory = pagegenerators.GeneratorFactory()
for arg in tohandle:
if genFactory.handleArg(arg):
+ gen_args.append(arg)
continue
if arg.startswith('-tofamily'):
tofamily = arg[len('-tofamily:'):]
@@ -67,33 +74,22 @@
elif arg == "-overwrite":
overwrite = True
+ tosite = pywikibot.Site(tolang, tofamily)
+ if fromsite == tosite:
+ raise Exception('Target site not different from source site')
+
gen = genFactory.getCombinedGenerator()
-
- if not tofamily:
- raise Exception('Target family not specified')
-
- from pywikibot import config
-
- # we change the config family to make sure we get sensible backlinks
- # i.e. [[wikipedia:en:pagename]] instead of [[pagename]]
- # this should really be fixed in Page.title() (bug #59223)
- # we can't do this before, as the pagegenerator would work on the
- # incorrect site...
- config.mylang = tolang
- config.family = tofamily
-
- tosite = pywikibot.Site()
-
if not gen:
raise Exception('Target pages not specified')
+ gen_args = ' '.join(gen_args)
pywikibot.output(u"""
Page transfer configuration
---------------------------
Source: %(fromsite)r
Target: %(tosite)r
- Pages to transfer: %(gen)r
+ Pages to transfer: %(gen_args)s
Prefix for transferred pages: %(prefix)s
""" % locals())
@@ -101,7 +97,8 @@
for page in gen:
summary = "Moved page from %s" % page.title(asLink=True)
targetpage = pywikibot.Page(tosite, prefix + page.title())
- edithistpage = pywikibot.Page(tosite, prefix + page.title() + "/edithistory")
+ edithistpage = pywikibot.Page(tosite, prefix + page.title()
+ + "/edithistory")
if targetpage.exists() and not overwrite:
pywikibot.output(
@@ -112,12 +109,15 @@
)
continue
- pywikibot.output(u"Moving %s to %s..." % (page.title(asLink=True), targetpage.title(asLink=True)))
+ pywikibot.output(u"Moving %s to %s..."
+ % (page.title(asLink=True),
+ targetpage.title(asLink=True)))
pywikibot.log("Getting page text.")
text = page.get(get_redirect=True)
text += "<noinclude>\n\n<small>This page was moved from %s. It's edit history can be viewed at %s</small></noinclude>" % (
- page.title(asLink=True), edithistpage.title(asLink=True))
+ page.title(asLink=True, insite=targetpage.site),
+ edithistpage.title(asLink=True, insite=targetpage.site))
pywikibot.log("Getting edit history.")
historytable = page.getVersionHistoryTable()
@@ -128,6 +128,7 @@
pywikibot.log("Putting edit history.")
edithistpage.put(historytable, comment=summary)
+
if __name__ == "__main__":
try:
main()
--
To view, visit https://gerrit.wikimedia.org/r/105440
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I5d4cd6673140ad87f26e884007ffb6d3434ecc56
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Xqt <info(a)gno.de>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Pyfisch <pyfisch(a)gmail.com>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: update of MW version on Vikidia and Wikia
......................................................................
update of MW version on Vikidia and Wikia
Change-Id: Ia6d639097de42e4773731902aa0e535c54fff7ba
---
M pywikibot/families/vikidia_family.py
M pywikibot/families/wikia_family.py
2 files changed, 2 insertions(+), 2 deletions(-)
Approvals:
Xqt: Looks good to me, approved
jenkins-bot: Verified
diff --git a/pywikibot/families/vikidia_family.py b/pywikibot/families/vikidia_family.py
index 377c998..8f8a598 100644
--- a/pywikibot/families/vikidia_family.py
+++ b/pywikibot/families/vikidia_family.py
@@ -40,7 +40,7 @@
# Which version of MediaWiki is used? REQUIRED
def version(self, code):
# Replace with the actual version being run on your wiki
- return '1.16.0'
+ return '1.22.0'
def code2encoding(self, code):
"""Return the encoding for a specific language wiki"""
diff --git a/pywikibot/families/wikia_family.py b/pywikibot/families/wikia_family.py
index 4830282..0619dd2 100644
--- a/pywikibot/families/wikia_family.py
+++ b/pywikibot/families/wikia_family.py
@@ -20,7 +20,7 @@
return u'www.wikia.com'
def version(self, code):
- return "1.19.6"
+ return "1.19.9"
def scriptpath(self, code):
return ''
--
To view, visit https://gerrit.wikimedia.org/r/105667
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia6d639097de42e4773731902aa0e535c54fff7ba
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Linedwell <Linedwell(a)icloud.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: Xqt <info(a)gno.de>
Gerrit-Reviewer: jenkins-bot
jenkins-bot has submitted this change and it was merged.
Change subject: fixing_redirects.py: ported to core
......................................................................
fixing_redirects.py: ported to core
The i18n file already existed, but I don't know if the remaining messages need to be transferred to TranslateWiki as well.
Change-Id: I828fc744782c2d2ddb9afef78007eaceab0914a3
---
A scripts/fixing_redirects.py
1 file changed, 230 insertions(+), 0 deletions(-)
Approvals:
Merlijn van Deen: Looks good to me, approved
jenkins-bot: Verified
diff --git a/scripts/fixing_redirects.py b/scripts/fixing_redirects.py
new file mode 100644
index 0000000..c89d6db
--- /dev/null
+++ b/scripts/fixing_redirects.py
@@ -0,0 +1,230 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+"""
+This script has the intention to correct all redirect
+links in featured pages or only one page of each wiki.
+
+Can be using with:
+¶ms;
+
+-featured Run over featured pages
+
+Run fixing_redirects.py -help to see all the command-line
+options -file, -ref, -links, ...
+
+"""
+#
+# This script based on disambredir.py and solve_disambiguation.py
+#
+# (C) Pywikipedia team, 2004-2012
+#
+# Distributed under the terms of the MIT license.
+#
+__version__ = '$Id$'
+#
+import re
+import sys
+import pywikibot
+from pywikibot import pagegenerators
+from pywikibot import i18n
+
+# This is required for the text that is shown when you run this script
+# with the parameter -help.
+docuReplacements = {
+ '¶ms;': pagegenerators.parameterHelp,
+}
+
+featured_articles = {
+ 'ar': u'ويكيبيديا:مقالات مختارة',
+ 'cs': u'Wikipedie:Nejlepší články',
+ 'de': u'Wikipedia:Exzellente_Artikel',
+ 'en': u'Wikipedia:Featured_articles',
+ 'es': u'Wikipedia:Artículos_destacados',
+ 'fa': u'ویکیپدیا:نوشتارهای برگزیده',
+ 'fr': u'Wikipédia:Articles_de_qualité',
+ 'he': u'פורטל:ערכים_מומלצים',
+ 'is': u'Wikipedia:Úrvalsgreinar',
+ 'it': u'Wikipedia:Articoli_in_vetrina',
+ 'ja': u'Wikipedia:秀逸な記事',
+ 'nl': u'Wikipedia:Etalage',
+ 'nn': u'Wikipedia:Gode artiklar',
+ 'no': u'Wikipedia:Anbefalte artikler',
+ 'pl': u'Wikipedia:Artykuły_na_medal',
+ 'pt': u'Wikipedia:Os_melhores_artigos',
+ 'sv': u'Wikipedia:Utvalda_artiklar',
+ 'vi': u'Wikipedia:Bài_viết_chọn_lọc',
+ 'zh': u'Wikipedia:特色条目',
+}
+
+
+def firstcap(string):
+ return string[0].upper() + string[1:]
+
+
+def treat(text, linkedPage, targetPage):
+ """
+ Based on the method of the same name in solve_disambiguation.py
+ """
+ mysite = pywikibot.getSite()
+ linktrail = mysite.linktrail()
+
+ # make a backup of the original text so we can show the changes later
+ linkR = re.compile(r'\[\[(?P<title>[^\]\|#]*)(?P<section>#[^\]\|]*)?(\|(?P<label>[^\]]*))?\]\](?P<linktrail>' + linktrail + ')')
+ curpos = 0
+ # This loop will run until we have finished the current page
+ while True:
+ m = linkR.search(text, pos=curpos)
+ if not m:
+ break
+ # Make sure that next time around we will not find this same hit.
+ curpos = m.start() + 1
+ # ignore interwiki links and links to sections of the same page
+ if m.group('title').strip() == '' or \
+ mysite.isInterwikiLink(m.group('title')):
+ continue
+ else:
+ actualLinkPage = pywikibot.Page(targetPage.site, m.group('title'))
+ # Check whether the link found is to page.
+ if actualLinkPage != linkedPage:
+ continue
+
+ choice = 'y'
+
+ # The link looks like this:
+ # [[page_title|link_text]]trailing_chars
+ page_title = m.group('title')
+ link_text = m.group('label')
+
+ if not link_text:
+ # or like this: [[page_title]]trailing_chars
+ link_text = page_title
+ if m.group('section') is None:
+ section = ''
+ else:
+ section = m.group('section')
+ trailing_chars = m.group('linktrail')
+ if trailing_chars:
+ link_text += trailing_chars
+
+ if choice in "uU":
+ # unlink - we remove the section if there's any
+ text = text[:m.start()] + link_text + text[m.end():]
+ continue
+ replaceit = choice in "rR"
+
+ # remove preleading ":"
+ if link_text[0] == ':':
+ link_text = link_text[1:]
+ if link_text[0].isupper():
+ new_page_title = targetPage.title()
+ else:
+ new_page_title = targetPage.title()[0].lower() + \
+ targetPage.title()[1:]
+
+ # remove preleading ":"
+ if new_page_title[0] == ':':
+ new_page_title = new_page_title[1:]
+
+ if replaceit and trailing_chars:
+ newlink = "[[%s%s]]%s" % (new_page_title, section, trailing_chars)
+ elif replaceit or (new_page_title == link_text and not section):
+ newlink = "[[%s]]" % new_page_title
+ # check if we can create a link with trailing characters instead of a
+ # pipelink
+ elif len(new_page_title) <= len(link_text) and \
+ firstcap(link_text[:len(new_page_title)]) == \
+ firstcap(new_page_title) and \
+ re.sub(re.compile(linktrail), '', link_text[len(new_page_title):]) == '' and not section:
+ newlink = "[[%s]]%s" % (link_text[:len(new_page_title)],
+ link_text[len(new_page_title):])
+ else:
+ newlink = "[[%s%s|%s]]" % (new_page_title, section, link_text)
+ text = text[:m.start()] + newlink + text[m.end():]
+ continue
+ return text
+
+pageCache = []
+
+
+def workon(page):
+ mysite = pywikibot.getSite()
+ try:
+ text = page.get()
+ except pywikibot.IsRedirectPage:
+ pywikibot.output(u'%s is a redirect page. Skipping' % page)
+ return
+ except pywikibot.NoPage:
+ pywikibot.output(u'%s does not exist. Skipping' % page)
+ return
+ pywikibot.output(u"\n\n>>> \03{lightpurple}%s\03{default} <<<"
+ % page.title())
+ links = page.linkedPages()
+ if links is not None:
+ links = pagegenerators.PreloadingGenerator(links)
+# pywikibot.getall(mysite, links)
+ else:
+ pywikibot.output('Nothing left to do.')
+ return
+
+ for page2 in links:
+ try:
+ target = page2.getRedirectTarget()
+ except pywikibot.NoPage:
+ try:
+ target = page2.getMovedTarget()
+ except (pywikibot.NoPage, pywikibot.BadTitle):
+ continue
+ except (pywikibot.Error, pywikibot.SectionError):
+ continue
+ # no fix to user namespaces
+ if target.namespace() in [0, 1] and not page2.namespace() in [0, 1]:
+ continue
+ text = treat(text, page2, target)
+ if text != page.get():
+ comment = i18n.twtranslate(mysite, 'fixing_redirects-fixing')
+ pywikibot.showDiff(page.get(), text)
+ try:
+ page.put(text, comment)
+ except (pywikibot.Error):
+ pywikibot.error('unable to put %s' % page)
+
+
+def main():
+ featured = False
+ gen = None
+
+ # This factory is responsible for processing command line arguments
+ # that are also used by other scripts and that determine on which pages
+ # to work on.
+ genFactory = pagegenerators.GeneratorFactory()
+
+ for arg in pywikibot.handleArgs():
+ if arg == '-featured':
+ featured = True
+ else:
+ genFactory.handleArg(arg)
+
+ mysite = pywikibot.getSite()
+ if mysite.sitename() == 'wikipedia:nl':
+ pywikibot.output(
+ u'\03{lightred}There is consensus on the Dutch Wikipedia that bots should not be used to fix redirects.\03{default}')
+ sys.exit()
+
+ if featured:
+ featuredList = i18n.translate(mysite, featured_articles)
+ ref = pywikibot.Page(pywikibot.getSite(), featuredList)
+ gen = pagegenerators.ReferringPageGenerator(ref)
+ gen = pagegenerators.NamespaceFilterPageGenerator(gen, [0])
+ if not gen:
+ gen = genFactory.getCombinedGenerator()
+ if gen:
+ for page in pagegenerators.PreloadingGenerator(gen):
+ workon(page)
+ else:
+ pywikibot.showHelp('fixing_redirects')
+
+if __name__ == "__main__":
+ try:
+ main()
+ finally:
+ pywikibot.stopme()
--
To view, visit https://gerrit.wikimedia.org/r/105515
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: I828fc744782c2d2ddb9afef78007eaceab0914a3
Gerrit-PatchSet: 2
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Gerrit Patch Uploader <gerritpatchuploader(a)gmail.com>
Gerrit-Reviewer: Gerrit Patch Uploader <gerritpatchuploader(a)gmail.com>
Gerrit-Reviewer: Ladsgroup <ladsgroup(a)gmail.com>
Gerrit-Reviewer: Merlijn van Deen <valhallasw(a)arctus.nl>
Gerrit-Reviewer: jenkins-bot