Revision: 5790
Author: wikipedian
Date: 2008-08-07 10:10:46 +0000 (Thu, 07 Aug 2008)
Log Message:
-----------
Reverted r5788 by alnokta/Betacommand.
Instead, added an interwiki regex to wikipedia.replaceExcept(). You can now run replace.py with this parameter:
-exceptinsidetag:interwiki
Please try to keep things simple and don't add so many parameters to the scripts, especially when they're inconsistent with the
existing ones.
Modified Paths:
--------------
trunk/pywikipedia/replace.py
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/replace.py
===================================================================
--- trunk/pywikipedia/replace.py 2008-08-07 00:25:37 UTC (rev 5789)
+++ trunk/pywikipedia/replace.py 2008-08-07 10:10:46 UTC (rev 5790)
@@ -35,7 +35,7 @@
-requiretitle:XYZ Only do pages with titles that contain XYZ. If the -regex
argument is given, XYZ will be regarded as a regular
- expression.
+ expression.
-excepttext:XYZ Skip pages which contain the text XYZ. If the -regex
argument is given, XYZ will be regarded as a regular
@@ -212,8 +212,7 @@
and not self.isTextExcepted(entry.text):
new_text = entry.text
for old, new in self.replacements:
- new_text = wikipedia.replaceExcept(
- new_text, old, new, self.excsInside)
+ new_text = wikipedia.replaceExcept(new_text, old, new, self.excsInside, self.site)
if new_text != entry.text:
yield wikipedia.Page(self.site, entry.title)
except KeyboardInterrupt:
@@ -251,7 +250,7 @@
"""
def __init__(self, generator, replacements, exceptions={},
acceptall=False, allowoverlap=False, recursive=False,
- addedCat=None, sleep=None, exceptinterwiki=False):
+ addedCat=None, sleep=None):
"""
Arguments:
* generator - A generator that yields Page objects.
@@ -292,7 +291,6 @@
self.acceptall = acceptall
self.allowoverlap = allowoverlap
self.recursive = recursive
- self.exceptinterwiki = exceptinterwiki
if addedCat:
site = wikipedia.getSite()
cat_ns = site.category_namespaces()[0]
@@ -339,13 +337,8 @@
for old, new in self.replacements:
if self.sleep != None:
time.sleep(self.sleep)
- if self.exceptinterwiki:
- interwikis = wikipedia.getLanguageLinks(new_text)
- new_text = wikipedia.removeLanguageLinks(new_text)
new_text = wikipedia.replaceExcept(new_text, old, new, exceptions,
allowoverlap=self.allowoverlap)
- if self.exceptinterwiki:
- new_text = wikipedia.replaceLanguageLinks(new_text,interwikis)
return new_text
def run(self):
@@ -498,8 +491,6 @@
allowoverlap = False
# Do not recurse replacement
recursive = False
- #add flag to ignore interwiki links
- exceptinterwiki = False
# This factory is responsible for processing command line arguments
# that are also used by other scripts and that determine on which pages
# to work on.
@@ -551,8 +542,6 @@
sleep = float(arg[7:])
elif arg == '-always':
acceptall = True
- elif arg == '-exceptinterwiki':
- exceptinterwiki = True
elif arg == '-recursive':
recursive = True
elif arg == '-nocase':
@@ -711,7 +700,7 @@
pageNumber=20, lookahead=100)
else:
preloadingGen = pagegenerators.PreloadingGenerator(gen, pageNumber=60)
- bot = ReplaceRobot(preloadingGen, replacements, exceptions, acceptall, allowoverlap, recursive, add_cat, sleep, exceptinterwiki)
+ bot = ReplaceRobot(preloadingGen, replacements, exceptions, acceptall, allowoverlap, recursive, add_cat, sleep)
bot.run()
if __name__ == "__main__":
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2008-08-07 00:25:37 UTC (rev 5789)
+++ trunk/pywikipedia/wikipedia.py 2008-08-07 10:10:46 UTC (rev 5790)
@@ -3143,7 +3143,7 @@
# All return the modified text as a unicode object
def replaceExcept(text, old, new, exceptions, caseInsensitive=False,
- allowoverlap=False, marker = ''):
+ allowoverlap=False, marker = '', site = None):
"""
Return text with 'old' replaced by 'new', ignoring specified types of text.
@@ -3169,6 +3169,9 @@
# Hyperlink regex is defined in weblinkchecker.py
import weblinkchecker
+ if site is None:
+ site = getSite()
+
exceptionRegexes = {
'comment': re.compile(r'(?s)<!--.*?-->'),
# section headers
@@ -3203,7 +3206,10 @@
'gallery': re.compile(r'(?is)<gallery.*?>.*?</gallery>'),
# this matches internal wikilinks, but also interwiki, categories, and
# images.
- 'link': re.compile(r'\[\[(?P<title>[^\]\|]*)(\|[^\]]*)?\]\]')
+ 'link': re.compile(r'\[\[[^\]\|]*(\|[^\]]*)?\]\]'),
+ 'interwiki': re.compile(r'(?i)\[\[(%s)\s?:[^\]]*\]\][\s]*'
+ % '|'.join(site.validLanguageLinks() + site.family.obsolete.keys())),
+
}
# if we got a string, compile it as a regular expression
Feature Requests item #2028736, was opened at 2008-07-26 16:19
Message generated for change (Comment added) made by nicdumz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=2028736&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Priority: 5
Private: No
Submitted By: Mohamed M. Kassem (alnokta)
Assigned to: Nobody/Anonymous (nobody)
Summary: Option to ignore interwiki links while using replace.py
Initial Comment:
Hello,
Please add an option to replace.py (or fixes.py, you know better) so that when it is making replacements, it ignores/untouches the interwiki links and leaves them as is.
Thanks,
alnokta
----------------------------------------------------------------------
>Comment By: NicDumZ Nicolas Dumazet (nicdumz)
Date: 2008-08-07 06:28
Message:
Logged In: YES
user_id=1963242
Originator: NO
-exceptinterwiki added in r5788
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=2028736&group_…
Revision: 5787
Author: rotem
Date: 2008-08-06 09:35:24 +0000 (Wed, 06 Aug 2008)
Log Message:
-----------
versionnumber is a site method. Also removing unnecessary comments.
Modified Paths:
--------------
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2008-08-04 02:54:17 UTC (rev 5786)
+++ trunk/pywikipedia/wikipedia.py 2008-08-06 09:35:24 UTC (rev 5787)
@@ -2119,9 +2119,7 @@
predata['wpMovetalk'] = '1'
else:
predata['wpMovetalk'] = '0'
- if self.versionnumber() > 12:
- # introduced in MW 1.13 :
- # http://thread.gmane.org/gmane.science.linguistics.wikipedia.technical/39072
+ if self.site().versionnumber() >= 13:
if fixredirects:
predata['wpFixRedirects'] = '1'
else:
@@ -2153,13 +2151,13 @@
# Try to delete and move
return self.move(newtitle = newtitle, reason = reason, movetalkpage = movetalkpage, throttle = throttle, deleteAndMove = True)
except NoUsername:
- #We dont have the user rights to delete
+ # We dont have the user rights to delete
output(u'Page moved failed: Target page [[%s]] already exists.' % newtitle)
return False
elif not self.exists():
raise NoPage(u'Page move failed: Source page [[%s]] does not exist.' % newtitle)
elif Page(self.site(),newtitle).exists():
- #XXX : This might be buggy : if the move was successful, the target pase *has* been created
+ # XXX : This might be buggy : if the move was successful, the target pase *has* been created
raise PageNotSaved(u'Page move failed: Target page [[%s]] already exists.' % newtitle)
else:
output(u'Page move failed for unknown reason.')
Bugs item #2011362, was opened at 2008-07-05 20:07
Message generated for change (Comment added) made by melancholie
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=2011362&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Melancholie (melancholie)
Assigned to: Nobody/Anonymous (nobody)
Summary: featured.py: Add 'Link FA' again for hi and yi
Initial Comment:
Both, hiwiki and yiwiki do use Template:Link_FA, the pages hi:Template:Lien AdQ and yi:Template: are only redirects.
By re-adding 'Link FA' to the langs arrays, Template:Link_FA will be used by the bot (otherewise those redirects).
----------------------------------------------------------------------
>Comment By: Melancholie (melancholie)
Date: 2008-08-05 13:50
Message:
Logged In: YES
user_id=2089773
Originator: YES
added patch (working current)
File Added: featured.diff
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-07-11 07:51
Message:
Logged In: YES
user_id=2089773
Originator: YES
Maybe I was a little bit too imprecise ;-)
Here is what I mean:
templatelist = template['_default']
try:
templatelist += template[tosite.lang]
+ (u" {{%s|%s}}" % (templatelist[0],
fromsite.lang))
templatelist[0] is 'Link FA', but the localized templates used are
templatelist[1]. If there is a localized template (with Link_FA being only
a redirect) it should be used for edit. So it might be good to change the
order (make 'try:' first, add _default with +=)
For hi, yi the easiest way might be to just comment out the localized
template names, as they are only redirects to Link_FA.
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-07-10 11:59
Message:
Logged In: YES
user_id=2089773
Originator: YES
So {{Link FA}} is always the first choice?
Or does _default actually follow redirects?
----------------------------------------------------------------------
Comment By: NicDumZ Nicolas Dumazet (nicdumz)
Date: 2008-07-10 10:41
Message:
Logged In: YES
user_id=1963242
Originator: NO
I'm not sure melancholie :)
I added {{Leam VdC}} in r5707, however, since r5669 (
https://fisheye.toolserver.org/browse/pywikipedia/trunk/pywikipedia/feature…
) it uses the '_default' entry (Link FA) AND the locale entry. The list for
hi: is for example ['Link Fa', 'Lien AdQ']
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-07-10 10:28
Message:
Logged In: YES
user_id=2089773
Originator: YES
Furthermore, add http://fur.wikipedia.org/wiki/Model:Leam_VdC
[I wish I were able to do this things myself, but *still* no account yet]
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=2011362&group_…
Bugs item #2006208, was opened at 2008-06-29 23:58
Message generated for change (Comment added) made by melancholie
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=2006208&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 9
Private: No
Submitted By: Melancholie (melancholie)
Assigned to: Nobody/Anonymous (nobody)
Summary: Undo decodeEsperantoX removal (r5563)
Initial Comment:
I do not know why siebrand had problems with decodeEsperantoX(),
see http://eo.wikipedia.org/w/index.php?title=Sunfloro&diff=1775705&oldid=15592… - but now I do have that problem (not with interwiki.py, but with featured.py and replace.py)!
See http://eo.wikipedia.org/w/index.php?title=Vitamino_C&diff=prev&oldid=1819462
Characters are replaced to gxx, sxx etc., making templates unusable etc...
As decodeEsperantoX() had been used for a very long time, please undo that change:
http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/wikipedia.py?…
----------------------------------------------------------------------
>Comment By: Melancholie (melancholie)
Date: 2008-08-05 05:49
Message:
Logged In: YES
user_id=2089773
Originator: YES
Seems partially fixed:
1.
http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/families/wiki…
2.
http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/families/wiki…
But yet see:
http://sourceforge.net/tracker/index.php?func=detail&aid=2030278&group_id=9…
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-07-05 20:37
Message:
Logged In: YES
user_id=2089773
Originator: YES
Maybe this was needed before 2004 when Wikimedia wiki didn't had UTF-8 as
encoding!?
Do you know whether the example "Bordeauxx instead Bordeaux" that was
mentioned does work, currently?
If yes, I would say we could remove all the remaining
de/encodeEsperantoX() cruft, too. (description, *def* (function itself)).
----------------------------------------------------------------------
Comment By: NicDumZ Nicolas Dumazet (nicdumz)
Date: 2008-07-04 20:31
Message:
Logged In: YES
user_id=1963242
Originator: NO
encodeEsperantoX calls were removed in r5670.
I'm leaving this open, since I bet there was a reason for that X writing
convention. Does removing it complety is problematic ?
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-07-04 15:05
Message:
Logged In: YES
user_id=2089773
Originator: YES
The problem seems to be that decodeEsperantoX() has been removed, but
encodeEsperantoX() remains.
So the encodeEsperantoX() parts currently produce 'xx'! Either revert the
SVN change, or also ?remove? encodeEsperantoX().
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-06-30 00:19
Message:
Logged In: YES
user_id=2089773
Originator: YES
See also:
*
http://eo.wikipedia.org/wiki/Vikipedio:Diskutejo#.22Laboro.22_de_robotistoj
*
http://eo.wikipedia.org/w/index.php?title=Historio_de_Unui%C4%9Dinta_Re%C4%…
----------------------------------------------------------------------
Comment By: Melancholie (melancholie)
Date: 2008-06-30 00:00
Message:
Logged In: YES
user_id=2089773
Originator: YES
http://eo.wikipedia.org/wiki/Speciala:Contributions/Melancholie (r5558
works; r5639 is broken)!
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=2006208&group_…