Revision: 4203
Author: wikipedian
Date: 2007-09-06 21:35:13 +0000 (Thu, 06 Sep 2007)
Log Message:
-----------
docu
Modified Paths:
--------------
trunk/pywikipedia/interwiki.py
Modified: trunk/pywikipedia/interwiki.py
===================================================================
--- trunk/pywikipedia/interwiki.py 2007-09-06 17:15:47 UTC (rev 4202)
+++ trunk/pywikipedia/interwiki.py 2007-09-06 21:35:13 UTC (rev 4203)
@@ -173,6 +173,9 @@
Some configuration option can be used to change the working of this robot:
+interwiki_min_subjects: the minimum amount of subjects that should be processed
+ at the same time.
+
interwiki_backlink: if set to True, all problems in foreign wikis will
be reported
Revision: 4202
Author: misza13
Date: 2007-09-06 17:15:47 +0000 (Thu, 06 Sep 2007)
Log Message:
-----------
Fix ImagePage.fileUrl() - image page looks slightly different for .ogg files.
Bypassing using negative lookahead (bogus "image link" has a 'class="image"' following it).
Modified Paths:
--------------
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2007-09-05 17:18:41 UTC (rev 4201)
+++ trunk/pywikipedia/wikipedia.py 2007-09-06 17:15:47 UTC (rev 4202)
@@ -2120,7 +2120,7 @@
# The part after the | is required for copying .ogg files from en:, as they do not
# have a "full image link" div. This might change in the future; on commons, there
# is a full image link for .ogg and .mid files.
- urlR = re.compile(r'<div class="fullImageLink" id="file">.*?<a href="(?P<url>.+?)"|<span class="dangerousLink"><a href="(?P<url2>.+?)"', re.DOTALL)
+ urlR = re.compile(r'<div class="fullImageLink" id="file">.*?<a href="(?P<url>[^ ]+?)"(?! class="image")|<span class="dangerousLink"><a href="(?P<url2>.+?)"', re.DOTALL)
m = urlR.search(self.getImagePageHtml())
try:
url = m.group('url') or m.group('url2')
Support Requests item #1789036, was opened at 2007-09-06 00:10
Message generated for change (Comment added) made by malafaya
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603139&aid=1789036&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Priority: 5
Private: No
Submitted By: André Malafaya Baptista (malafaya)
Assigned to: Nobody/Anonymous (nobody)
Summary: -nobacklink option removed?
Initial Comment:
I would like to know if the option -nobacklink has been removed from interwiki.py.
I always used this option to reduce processing times and also CPU consumption (while displaying the backlink warnings, the CPU reaches 100% for some seconds, freezing other processes almost completely).
Thanks.
----------------------------------------------------------------------
>Comment By: André Malafaya Baptista (malafaya)
Date: 2007-09-06 12:21
Message:
Logged In: YES
user_id=1037345
Originator: YES
Thanks for the info, wikipedian.
I just had noticed the parameter missing from the help text but I hadn't
noticed it had been moved to user-config.py.
That surely helps. Thanks.
----------------------------------------------------------------------
Comment By: Daniel Herding (wikipedian)
Date: 2007-09-06 00:58
Message:
Logged In: YES
user_id=880694
Originator: NO
Hi André,
first of all, I have sped up the backlink processing code. It is still
slow, but not as slow as it used to be. Here is my commit message from
2007-08-13:
----
By making use of dictionaries and sets, decreased complexity from O(n^3)
to O(n^2).
For example, the backlinks report for
python interwiki.py -lang:de Indien -localonly
is now generated in 26 seconds, instead of the 190 seconds that were
needed before.
----
Also, I thought that this is something you don't want to change each time
you run the bot. Either you always want them, or you never do (IMHO). Thus,
a command-line parameter just wasn't the right thing, especially because
interwiki.py has way too many of them, which makes it hard to learn.
Instead, you can now add this in your user-config.py:
interwiki_backlink = False
Is that OK for you?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603139&aid=1789036&group_…
Bugs item #1789089, was opened at 2007-09-06 12:29
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1789089&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: John Vandenberg (zeroj)
Assigned to: Nobody/Anonymous (nobody)
Summary: UnusedFilesGenerator not working
Initial Comment:
When using unusedfiles.py on en.wikisource.org, the UnusedFilesGenerator returns nothing. I presume that this is due to MediaWiki changes in the HTML interface for "Special:Unusedimages".
My changes to make my bot functional are found here:
http://meta.wikimedia.org/wiki/User:Jayvdb
The current patch is attached, and I am happy to work on any suggestions on how this patch can be improved.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1789089&group_…
Support Requests item #1789036, was opened at 2007-09-06 01:10
Message generated for change (Comment added) made by wikipedian
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603139&aid=1789036&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: André Malafaya Baptista (malafaya)
Assigned to: Nobody/Anonymous (nobody)
Summary: -nobacklink option removed?
Initial Comment:
I would like to know if the option -nobacklink has been removed from interwiki.py.
I always used this option to reduce processing times and also CPU consumption (while displaying the backlink warnings, the CPU reaches 100% for some seconds, freezing other processes almost completely).
Thanks.
----------------------------------------------------------------------
>Comment By: Daniel Herding (wikipedian)
Date: 2007-09-06 01:58
Message:
Logged In: YES
user_id=880694
Originator: NO
Hi André,
first of all, I have sped up the backlink processing code. It is still
slow, but not as slow as it used to be. Here is my commit message from
2007-08-13:
----
By making use of dictionaries and sets, decreased complexity from O(n^3)
to O(n^2).
For example, the backlinks report for
python interwiki.py -lang:de Indien -localonly
is now generated in 26 seconds, instead of the 190 seconds that were
needed before.
----
Also, I thought that this is something you don't want to change each time
you run the bot. Either you always want them, or you never do (IMHO). Thus,
a command-line parameter just wasn't the right thing, especially because
interwiki.py has way too many of them, which makes it hard to learn.
Instead, you can now add this in your user-config.py:
interwiki_backlink = False
Is that OK for you?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603139&aid=1789036&group_…
Support Requests item #1789036, was opened at 2007-09-06 00:10
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603139&aid=1789036&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Private: No
Submitted By: André Malafaya Baptista (malafaya)
Assigned to: Nobody/Anonymous (nobody)
Summary: -nobacklink option removed?
Initial Comment:
I would like to know if the option -nobacklink has been removed from interwiki.py.
I always used this option to reduce processing times and also CPU consumption (while displaying the backlink warnings, the CPU reaches 100% for some seconds, freezing other processes almost completely).
Thanks.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603139&aid=1789036&group_…