Bugs item #1771889, was opened at 2007-08-10 19:20
Message generated for change (Comment added) made by wikipedian
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1771889&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Falk Steinhauer (falk_steinhauer)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with namespaces in wikipedia.py
Initial Comment:
I am using snapshot 2007-06-19:
In our wiki we are using title prefixes for articles that are not in german. They are Fr: (French) and En: (Englisch).
One of our French articles marks the end of a subarticle of [[Special:All Pages]] (see here: http://www.wiki-aventurica.de/index.php?title=Spezial:Alle_Seiten)
If I am using commandline option -start:! the script runs into a recursion. After Fr:xxxx is yielded the script whishes to continue with article xxxx, which is in my case alphabetically before Fr:xxxx. You can see, that this leads to a recursion. If xxxx is after Fr:xxxx, some articles might be skipped.
I detected the reponsible line of code:
wikipedia.py line 3504
# save the last hit, so that we know where to continue when we
# finished all articles on the current page. Append a '!' so that
# we don't yield a page twice.
start = Page(self,hit).titleWithoutNamespace() + '!'
Maybe this can also be fixed in titleWithoutNamespace()
Is it necessary to cut off the namespace?
----------------------------------------------------------------------
>Comment By: Daniel Herding (wikipedian)
Date: 2007-08-13 12:20
Message:
Logged In: YES
user_id=880694
Originator: NO
The timeouts are a way to reduce database server load during peak times.
See: http://www.mediawiki.org/wiki/Manual:Maxlag_parameter
Maybe your server is generally a bit slow, so try to increase the maxlag
parameter in your user-config.py, for example:
maxlag = 10
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2007-08-11 13:48
Message:
Logged In: NO
I stepped back to snapshot 2007-06-19 because of several problems with
nightly build 2007-08-10 08:39:28.
With this version my scripts were not able to change pages with
wikipedia.Page.put(). Server timeout was reported frequently, but the
server was not down.
----------------------------------------------------------------------
Comment By: Falk Steinhauer (falk_steinhauer)
Date: 2007-08-11 01:34
Message:
Logged In: YES
user_id=1810075
Originator: YES
Something is still disturbing. Our language prefixes are still cut off.
But so such pages cannot be found in namespace 0.
----------------------------------------------------------------------
Comment By: Falk Steinhauer (falk_steinhauer)
Date: 2007-08-11 01:26
Message:
Logged In: YES
user_id=1810075
Originator: YES
Thanks now it works.
Some strange thing is, that no redirects are yielded, but parameter
includeredirects of AllpagesPageGenerator() is default set to True.
----------------------------------------------------------------------
Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-10 20:55
Message:
Logged In: YES
user_id=687283
Originator: NO
Strange, as these prefixes should not be interpreted as namespaces. For
now, please update to SVN or the latest nightly (
http://tools.wikimedia.de/~valhallasw/pywiki/ ), and test if the issue
still exists.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1771889&group_…
Revision: 4033
Author: wikipedian
Date: 2007-08-13 10:19:19 +0000 (Mon, 13 Aug 2007)
Log Message:
-----------
added docu that maxlag can be set to None
Modified Paths:
--------------
trunk/pywikipedia/config.py
Modified: trunk/pywikipedia/config.py
===================================================================
--- trunk/pywikipedia/config.py 2007-08-12 22:44:33 UTC (rev 4032)
+++ trunk/pywikipedia/config.py 2007-08-13 10:19:19 UTC (rev 4033)
@@ -247,8 +247,10 @@
# than 'noisysleep' seconds, it is logged on the screen.
noisysleep = 3.0
-# Defer bot edits during periods of server lag. For details, see
+# Defer bot edits during periods of database server lag. For details, see
# http://www.mediawiki.org/wiki/Maxlag_parameter
+# You can set this variable to a number of seconds, or to None to disable
+# this behavior.
# It is recommended that you do not change this parameter unless you know
# what you are doing and have a good reason for it!
maxlag = 5
Revision: 4032
Author: wikipedian
Date: 2007-08-12 22:44:33 +0000 (Sun, 12 Aug 2007)
Log Message:
-----------
typo
Modified Paths:
--------------
trunk/pywikipedia/solve_disambiguation.py
Modified: trunk/pywikipedia/solve_disambiguation.py
===================================================================
--- trunk/pywikipedia/solve_disambiguation.py 2007-08-12 20:25:38 UTC (rev 4031)
+++ trunk/pywikipedia/solve_disambiguation.py 2007-08-12 22:44:33 UTC (rev 4032)
@@ -576,8 +576,8 @@
while True:
# Show the title of the page where the link was found.
# Highlight the title in purple.
- wikipedia.output(u"\n\n>>> \03{lightpurple}%s\03{default} \<<<" % refPage.title())
-
+ wikipedia.output(u"\n\n>>> \03{lightpurple}%s\03{default} <<<" % refPage.title())
+
# at the beginning of the link, start red color.
# at the end of the link, reset the color to default
wikipedia.output(text[m.start() - context : m.start()] + '\03{lightred}' + text[m.start() : m.end()] + '\03{default}' + text[m.end() : m.end() + context])
Revision: 4030
Author: btongminh
Date: 2007-08-11 21:11:52 +0000 (Sat, 11 Aug 2007)
Log Message:
-----------
Adding a test directory, which can be used in the future to put regression tests in.
Added Paths:
-----------
tests/
Revision: 4029
Author: btongminh
Date: 2007-08-11 20:51:10 +0000 (Sat, 11 Aug 2007)
Log Message:
-----------
Do not replace if source and destination are the same.
Modified Paths:
--------------
trunk/pywikipedia/image_replacer.py
Modified: trunk/pywikipedia/image_replacer.py
===================================================================
--- trunk/pywikipedia/image_replacer.py 2007-08-11 20:43:46 UTC (rev 4028)
+++ trunk/pywikipedia/image_replacer.py 2007-08-11 20:51:10 UTC (rev 4029)
@@ -78,7 +78,8 @@
for replacement in replacements:
res = self.examine_revision_history(
revisions, replacement, username)
- if res and self.allowed_replacement(replacement):
+ if res and self.allowed_replacement(replacement) and \
+ replacement.group(1) != replacement.group(2):
self.cursor.execute(insert, res)
text = text.replace(replacement.group(0), '')
output('Replacing %s by %s: %s' % replacement.groups())
Revision: 4028
Author: btongminh
Date: 2007-08-11 20:43:46 +0000 (Sat, 11 Aug 2007)
Log Message:
-----------
Dropping a fix that was introduced in r3837 & r3689 and caused a lot of problems lately. The problem it solved was also fixed in r3838, but afterwards forgotten.
Modified Paths:
--------------
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2007-08-11 20:20:40 UTC (rev 4027)
+++ trunk/pywikipedia/wikipedia.py 2007-08-11 20:43:46 UTC (rev 4028)
@@ -1045,10 +1045,11 @@
"""
# Fetch a page to get an edit token. If we already have
# fetched a page, this will do nothing, because get() is cached.
- try:
- self.site().sandboxpage.get(force = True, get_redirect = True)
- except NoPage:
- pass
+ # Disabled in r4027
+ #try:
+ # self.site().sandboxpage.get(force = True, get_redirect = True)
+ #except NoPage:
+ # pass
# If there is an unchecked edit restriction, we need to load the page
if self._editrestriction:
Bugs item #1771889, was opened at 2007-08-10 10:20
Message generated for change (Comment added) made by nobody
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1771889&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Falk Steinhauer (falk_steinhauer)
Assigned to: Nobody/Anonymous (nobody)
Summary: Problems with namespaces in wikipedia.py
Initial Comment:
I am using snapshot 2007-06-19:
In our wiki we are using title prefixes for articles that are not in german. They are Fr: (French) and En: (Englisch).
One of our French articles marks the end of a subarticle of [[Special:All Pages]] (see here: http://www.wiki-aventurica.de/index.php?title=Spezial:Alle_Seiten)
If I am using commandline option -start:! the script runs into a recursion. After Fr:xxxx is yielded the script whishes to continue with article xxxx, which is in my case alphabetically before Fr:xxxx. You can see, that this leads to a recursion. If xxxx is after Fr:xxxx, some articles might be skipped.
I detected the reponsible line of code:
wikipedia.py line 3504
# save the last hit, so that we know where to continue when we
# finished all articles on the current page. Append a '!' so that
# we don't yield a page twice.
start = Page(self,hit).titleWithoutNamespace() + '!'
Maybe this can also be fixed in titleWithoutNamespace()
Is it necessary to cut off the namespace?
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2007-08-11 04:48
Message:
Logged In: NO
I stepped back to snapshot 2007-06-19 because of several problems with
nightly build 2007-08-10 08:39:28.
With this version my scripts were not able to change pages with
wikipedia.Page.put(). Server timeout was reported frequently, but the
server was not down.
----------------------------------------------------------------------
Comment By: Falk Steinhauer (falk_steinhauer)
Date: 2007-08-10 16:34
Message:
Logged In: YES
user_id=1810075
Originator: YES
Something is still disturbing. Our language prefixes are still cut off.
But so such pages cannot be found in namespace 0.
----------------------------------------------------------------------
Comment By: Falk Steinhauer (falk_steinhauer)
Date: 2007-08-10 16:26
Message:
Logged In: YES
user_id=1810075
Originator: YES
Thanks now it works.
Some strange thing is, that no redirects are yielded, but parameter
includeredirects of AllpagesPageGenerator() is default set to True.
----------------------------------------------------------------------
Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-10 11:55
Message:
Logged In: YES
user_id=687283
Originator: NO
Strange, as these prefixes should not be interpreted as namespaces. For
now, please update to SVN or the latest nightly (
http://tools.wikimedia.de/~valhallasw/pywiki/ ), and test if the issue
still exists.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1771889&group_…
Feature Requests item #1771986, was opened at 2007-08-10 23:49
Message generated for change (Comment added) made by ebola_rulez
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1771986&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: Aurimas Fischer (ebola_rulez)
Assigned to: Nobody/Anonymous (nobody)
Summary: interwiky.py trusted language:page
Initial Comment:
When I try to fix interwiki conflicts, I usually check my native language wiki and remove/fix incorrect interwiki links. Then I use interwiki.py to manually choose correct interwiki links when presented with choice.
This sometimes causes to choose from as much as 14 (!) different variants and shows hundreds of rows. This slows down the process:
Try to find if the page from native wiki is in one of these groups. If it is, then choose this group. If not, then analyze different languages or view pages in a browser.
I suggest to add command line argument -trusted (only in interactive mode).
When used, this should cause interwiki.py to automatically choose correct variant number based on initial language:page combination.
For example:
interwiki.py -lang:en -trusted Cat
...
(1) Found link to [[eo:Pantero]] in:
[[da:Panter]]
[[en:Panther]]
(2) Found link to [[eo:Hejma kato]] in:
[[da:Kat]]
[[en:Cat]]
...
Should automatically choose variant 2, because all interwiki links in en:Cat are trusted.
----------------------------------------------------------------------
>Comment By: Aurimas Fischer (ebola_rulez)
Date: 2007-08-11 13:34
Message:
Logged In: YES
user_id=959303
Originator: YES
I'm not a python programmer but managed
to hack a working prototype of this functionality.
File Added: interwiki_trusted.patch
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1771986&group_…