Feature Requests item #1768728, was opened at 2007-08-06 19:47
Message generated for change (Comment added) made by valhallasw
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1768728&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 7
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: Multilingual development
Initial Comment:
English speaking at end
==Francais== Texte d'origine
Bonjour
Je voudrais vous proposer un système qui permet de rendre le robot multilingue. En effet, tous les messages envoyé à la console sont anglophone. Or le but d'un robot est de s'adapter à la multitude des languages pouvant exister de la part des utilisateurs. C'est pour cela que je vous propose le système suivant :
Création d'un nouveau répertoire 'lang'. Dans ce répertoire s'y trouverait des fichiers de type XX.py (XX étant le code ISO 639 de la langue). Donc ce répertoire contiendra 1 ficher par code de langue existant.
Lorsque les différents programmes veulent afficher un message sur la console, la commande utilisé est très souvent 'wikipedia.output' ou 'wikipedia.input'. Le travail de cette commande serait d'appeller le fichier xx.py avec le numéro du message à renvoyer en paramètre, le choix du xx serait donnée par la variable mylang de user-config.py. le fichier xx.py enverrais alors le message à afficher en tenant compte des différentes variables de type %s (ou autre) bien entendu
Exemple :
dans user-config.py, j'ai "mylang = 'fr'"
Replace.py à la ligne 375 contient la commande "wikipedia.input(u'Please enter the new text:')",
Le nouveau système coderait "wikipedia.input(u'Please enter the new text:')" par "wikipedia.input.message(284)"
appelerait donc lang/fr.py et lui demanderais de lui retourner le message n° 284 qui serait "s'il vous plais, entrez le nouveaux texte :" et le lui retourne.
Voila, en esperant avoir compris ma demande.
Je vous remercie de votre écoute
==English== Text translates since French by a machine translation system
Hello
I would like to propose you a system which allows to return the multilingual robot. Indeed, all the messages messenger in the console are English-speaking. Now the purpose of a robot is to adapt itself to the multitude of the languages which can exist on behalf of the users. It is for it that I propose you the following system:
Creation of a new directory ' lang '. In this directory would be files of type XX.py (XX there being the code ISO 639 of the language). Thus this directory will contain 1 file by existing code of language.
When the various programs want to post a message on the console, the order used is very often ' wikipedia.output ' or ' wikipedia.input '. The work of this order would be to call the xx.py file with the number of the message to be sent back in parameter, the choice of the xx would be given by the mylang variable to user-config.py
The xx.py file would send then the message to be posted(shown) by taking into account various variables of type %s (or other) naturally
Example:
In user-config.py, I have " mylang = ' fr ' "
Replace.py in the line 375 contains the command " wikipedia.input (u' Please enter the new text: ') ",
The new system would code " wikipedia.input (u' Please enter the new text: ') "by" wikipedia.input.message (284) " call thus lang/fr.py and would ask it to return it the message n° 284 which would be " s'il vous plais, entrez le nouveaux texte : ".
Here we are, by hoping to have understood my demand.
I thank you for your listening
----------------------------------------------------------------------
>Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-24 22:24
Message:
Logged In: YES
user_id=687283
Originator: NO
Update: Because of the limitations of the normal xgettext implementation,
I am writing my own version, using the python compiler package. With some
luck, it will be possible to maintain the original inputChoice format this
way (as there is no reason to use a string anymore; the string-to-translate
can be generated from a function parameter that is a list, or a dict, or
.... ).
When this implementation is done, only three functions (wikipedia.input,
wikipedia.output and wikipedia.inputChoice) need to be adapted (and
possibly we need to change wikipedia.Error to translate the error).
----------------------------------------------------------------------
Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-08 17:08
Message:
Logged In: YES
user_id=687283
Originator: NO
Check
http://svn.wikimedia.org/viewvc/pywikipedia/branches/pywikipedia/i18n/input…
and see if you like it. I wrote a new function; the system is as follows:
retval = i18nChoice("Do you want to save?", "[('yes', 'y'), ('no', 'n')]",
'yes')
instead of a key, the options *name* is given as parameter, and returned.
This returns, with a dutch translation:
Wilt u opslaan? ([j]a, [n]ee) n
then retval == 'no'
To use this, we need to use xgettext:
xgettext --keyword=i18nChoice:1,2 ic.py
and we get plural defenitions:
msgid "Do you want to save?"
msgid_plural "[('yes', 'y'), ('no', 'n')]"
msgstr[0] "Wilt u opslaan?"
msgstr[1] "[('ja', 'j'), ('nee', 'n')]"
Maybe not the nicest solution, but the best one I could find. The
alternative would be
retval = i18nChoice(_("Do you want to save?"), "[('yes', 'y'), ('no',
'n')]", 'yes')
with
xgettext --keyword=i18nChoice,2 ic.py
(and the corresponding changes in the function, of course.)
----------------------------------------------------------------------
Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-08 02:14
Message:
Logged In: YES
user_id=687283
Originator: NO
Take a look at my proposal at http://pywiki.pastey.net/71924 . Readability
still is not too good; another way would be to create a new input function
and let gettext see it as translatable strings. No idea if gettext can
handle multiple parameters, but I assume it can handle at least two ;)
----------------------------------------------------------------------
Comment By: Daniel Herding (wikipedian)
Date: 2007-08-08 00:31
Message:
Logged In: YES
user_id=880694
Originator: NO
Current status:
I have changed the text colorization system so that it is now possible to
internationalize colorized strings.
There is now a branch called i18n which has gettext support. It works
fine, and the selflink.py script can be run in German. There is a new
config variable to set the UI lang, it will default to mylang and fall back
to English if the chosen language is unsupported.
One disadvantage is that source code readability suffers. Before:
choice = wikipedia.inputChoice(u'\nWhat shall be done with this
selflink?', ['unlink', 'make bold', 'skip', 'edit', 'more context'], ['U',
'b', 's', 'e', 'm'], 'u')
After:
choice = wikipedia.inputChoice(_(u'\nWhat shall be done with this
selflink?'), [_('unlink'), _('make bold'), _('skip'), _('edit'), _('more
context')], [_('u [unlink hotkey]'), _('b [make bold hotkey]'), _('s [skip
hotkey]'), _('e [edit hotkey]'), _('m [more context hotkey]')], _('u
[unlink hotkey]'))
valhallasw and I have discussed how to modify inputChoice() to make it
less cluttered, but we didn't find a convincing solution. For example,
inputChoice could be changed so that it works like this:
choice = wikipedia.inputChoice(_(u'What shall be done with this
selflink?'), _(u'[u]nlink, make [b]old, [s]kip, [e]dit, [m]ore context'),
0)
so that it automatically parses the [brackets] to find out the hotkeys,
and returns an integer for the option that was chosen. But integers make
the code very hard to read and maintain when there are long if-elif
constructions (... elif choice == 12 ...).
So, we currently don't know how to do it better. I think having full i18n
is worth cluttering up the code a little bit.
----------------------------------------------------------------------
Comment By: Russell Blau (russblau)
Date: 2007-08-07 15:28
Message:
Logged In: YES
user_id=855050
Originator: NO
A good idea; however, it should be implemented using the standard Python
'gettext' module, rather than reinventing the wheel.
----------------------------------------------------------------------
Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-07 00:41
Message:
Logged In: YES
user_id=687283
Originator: NO
I have no idea if gettext is usable on non-unix platforms, and whether it
allows to use locally saved translations. If it does, I think we might give
it a try. We would need to change all 'text %s %s' % (self.title(),
newtitle) commands to their dictionary relatives, 'text %(title)s
%(newtitle)s' % {'title': self.title(), 'newtitle': newtitle). Not a small
job, but doable, I suppose.
If gettext is not (easily) available on win32, we might write our own
system. I do not like the idea of using integers though ;)
----------------------------------------------------------------------
Comment By: Daniel Herding (wikipedian)
Date: 2007-08-06 22:15
Message:
Logged In: YES
user_id=880694
Originator: NO
Your approach works for static strings like wikipedia.input(u'Please enter
the new text: '), but it doesn't work for dynamic ones like output(u'Page
%s moved to %s' % (self.title(), newtitle)).
Instead, we could use gettext:
http://docs.python.org/lib/module-gettext.html
What do you think?
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1768728&group_…
Feature Requests item #1780662, was opened at 2007-08-24 01:03
Message generated for change (Comment added) made by valhallasw
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1780662&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
Priority: 7
Private: No
Submitted By: Merlijn S. van Deen (valhallasw)
Assigned to: Merlijn S. van Deen (valhallasw)
Summary: interwiki.py ignoring commented iw-links
Initial Comment:
Quote:
:::::The "AI" I've been proposing for a while is simple: check whether the link is already on the page but has been commented out. That means human editors are aware the link target exists, but have made a conscious decision to not use it. Which is all the bot needs to know. Shouldn't be difficult to implement. [[User:Future Perfect at Sunrise|Fut.Perf.]] [[User talk:Future Perfect at Sunrise|☼]] 17:30, 23 August 2007 (UTC)
i.e. interwiki.py should check the page for existing interwiki links in the format
<!--[[ru-sib:...]]-->
or check for a template in the form
{{iwignore|[[ru-sib:...]]}}
which might be easier to implement.
----------------------------------------------------------------------
>Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-24 22:21
Message:
Logged In: YES
user_id=687283
Originator: YES
Added in r4106.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1780662&group_…
Revision: 4106
Author: valhallasw
Date: 2007-08-24 20:20:11 +0000 (Fri, 24 Aug 2007)
Log Message:
-----------
Ignore commented-out interwiki links per https://sourceforge.net/tracker/?func=detail&atid=603141&aid=1780662&group_…
Modified Paths:
--------------
trunk/pywikipedia/interwiki.py
Modified: trunk/pywikipedia/interwiki.py
===================================================================
--- trunk/pywikipedia/interwiki.py 2007-08-24 17:59:25 UTC (rev 4105)
+++ trunk/pywikipedia/interwiki.py 2007-08-24 20:20:11 UTC (rev 4106)
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script to check language links for general pages. This works by downloading the
@@ -954,6 +954,17 @@
# clone original newPages dictionary, so that we can modify it to the local page's needs
new = dict(newPages)
+ # remove interwiki links to ignore
+ for iw in re.finditer('<!-- *\[\[(.*?):(.*?)\]\] *-->', page.get()):
+ ignorepage = wikipedia.Page(*iw.groups()[:2])
+
+ try:
+ if (new[ignorepage.site()] == ignorepage):
+ wikipedia.output(u"Ignoring link to %(to)s for %(from)s" % {'to': ignorepage, 'from': page})
+ new.pop(ignorepage.site())
+ except KeyError:
+ pass
+
# sanity check - the page we are fixing must be the only one for that site.
pltmp = new[page.site()]
if pltmp != page:
Revision: 4105
Author: valhallasw
Date: 2007-08-24 17:59:25 +0000 (Fri, 24 Aug 2007)
Log Message:
-----------
Updated Page.get docstring to a better description of force=True
Modified Paths:
--------------
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2007-08-24 17:54:18 UTC (rev 4104)
+++ trunk/pywikipedia/wikipedia.py 2007-08-24 17:59:25 UTC (rev 4105)
@@ -485,7 +485,7 @@
SectionError: The subject does not exist on a page with a # link
Set get_redirect to True to follow redirects rather than raise an exception.
- Set force to True to ignore all exceptions (including redirects).
+ Set force to True to force a reload of all page attributes, including errors.
Set nofollow_redirects to True to not follow redirects but obey all other exceptions.
Set change_version_date to False if you have already loaded the page before and
do not check this version for changes before saving
Revision: 4103
Author: misza13
Date: 2007-08-24 16:34:02 +0000 (Fri, 24 Aug 2007)
Log Message:
-----------
Modified removeDisabledParts - the exact set of parts which are removed can now be specified and defaults to all.
Modified Paths:
--------------
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2007-08-24 15:47:51 UTC (rev 4102)
+++ trunk/pywikipedia/wikipedia.py 2007-08-24 16:34:02 UTC (rev 4103)
@@ -2618,14 +2618,24 @@
text = text[:markerpos] + marker + text[markerpos:]
return text
-def removeDisabledParts(text):
+def removeDisabledParts(text, parts=['*']):
"""
Removes those parts of a wiki text where wiki markup is disabled, i.e.
* HTML comments
* nowiki tags
* includeonly tags
+
+ The exact set of parts which are removed is passed as the 'parts' parameter
+ and defaults to all.
"""
- toRemoveR = re.compile(r'<nowiki>.*?</nowiki>|<!--.*?-->|<includeonly>.*?</includeonly>', re.IGNORECASE | re.DOTALL)
+ regexes = {
+ 'nowiki' : r'<nowiki>.*?</nowiki>',
+ 'comments' : r'<!--.*?-->',
+ 'includeonly' : r'<includeonly>.*?</includeonly>',
+ }
+ if '*' in parts:
+ parts = regexes.keys()
+ toRemoveR = re.compile('|'.join([regexes[p] for p in parts]), re.IGNORECASE | re.DOTALL)
return toRemoveR.sub('', text)
# Part of library dealing with interwiki links
Revision: 4102
Author: misza13
Date: 2007-08-24 15:47:51 +0000 (Fri, 24 Aug 2007)
Log Message:
-----------
Move ignore_bot_templates switch to wikipedia.py (to allow multiple bots under same account have different settings).
Modified Paths:
--------------
trunk/pywikipedia/config.py
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/config.py
===================================================================
--- trunk/pywikipedia/config.py 2007-08-24 15:40:08 UTC (rev 4101)
+++ trunk/pywikipedia/config.py 2007-08-24 15:47:51 UTC (rev 4102)
@@ -369,12 +369,6 @@
# foreign wiki, set cosmetic_changes_mylang_only to False, but be careful!
cosmetic_changes_mylang_only = True
-# If ignore_bot_templates is True, the bot will always ignore {{bots}}
-# and {{nobots}} templates - botMayEdit() will always return True.
-# In the default (False) state, it will honor these directives and
-# refuse to save pages that forbid it from editing.
-ignore_bot_templates = False
-
# End of configuration section
# ============================
# System-level and User-level changes.
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2007-08-24 15:40:08 UTC (rev 4101)
+++ trunk/pywikipedia/wikipedia.py 2007-08-24 15:47:51 UTC (rev 4102)
@@ -149,6 +149,15 @@
WIDEBUILD = False
+# Local settings
+
+# If ignore_bot_templates is True, the bot will always ignore {{bots}}
+# and {{nobots}} templates - botMayEdit() will always return True.
+# In the default (False) state, it will honor these directives and
+# refuse to save pages that forbid it from editing.
+ignore_bot_templates = False
+
+
# Local exceptions
class Error(Exception):
@@ -747,7 +756,8 @@
is desired to implement authorization-checking for a particular
bot, the bot must call this method before editing.
"""
- if config.ignore_bot_templates: #Check the "master ignore switch"
+ global ignore_bot_templates
+ if ignore_bot_templates: #Check the "master ignore switch"
return True
import re;
Revision: 4101
Author: misza13
Date: 2007-08-24 15:40:08 +0000 (Fri, 24 Aug 2007)
Log Message:
-----------
Added a new configuration option, "ignore_bot_templates".
Default value is False. When set to True, the bot will ignore the {{bots}} and {{nobots}} templates (botMayEdit() will always return True).
Modified Paths:
--------------
trunk/pywikipedia/config.py
trunk/pywikipedia/wikipedia.py
Modified: trunk/pywikipedia/config.py
===================================================================
--- trunk/pywikipedia/config.py 2007-08-24 04:53:17 UTC (rev 4100)
+++ trunk/pywikipedia/config.py 2007-08-24 15:40:08 UTC (rev 4101)
@@ -369,6 +369,12 @@
# foreign wiki, set cosmetic_changes_mylang_only to False, but be careful!
cosmetic_changes_mylang_only = True
+# If ignore_bot_templates is True, the bot will always ignore {{bots}}
+# and {{nobots}} templates - botMayEdit() will always return True.
+# In the default (False) state, it will honor these directives and
+# refuse to save pages that forbid it from editing.
+ignore_bot_templates = False
+
# End of configuration section
# ============================
# System-level and User-level changes.
Modified: trunk/pywikipedia/wikipedia.py
===================================================================
--- trunk/pywikipedia/wikipedia.py 2007-08-24 04:53:17 UTC (rev 4100)
+++ trunk/pywikipedia/wikipedia.py 2007-08-24 15:40:08 UTC (rev 4101)
@@ -747,6 +747,9 @@
is desired to implement authorization-checking for a particular
bot, the bot must call this method before editing.
"""
+ if config.ignore_bot_templates: #Check the "master ignore switch"
+ return True
+
import re;
p = re.compile(r"\{\{(?P<type>bots|nobots)\|?(?P<data>.*?)\}\}")
try:
Bugs item #1672346, was opened at 2007-03-02 03:33
Message generated for change (Comment added) made by nobody
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1672346&group_…
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Nobody/Anonymous (nobody)
Assigned to: Nobody/Anonymous (nobody)
Summary: uncaught socket.error exception
Initial Comment:
I got an uncaught socket error in one of my scripts today:
Traceback (most recent call last):
File "./replace-link.py", line 256, in ?
page.put(changedict[title], summ)
File "wikipedia.py", line 981, in put
return self.putPage(newtext, comment, watchArticle, minorEdit, newPage, self.site().getToken(sysop = sysop), sysop = sysop)
File "wikipedia.py", line 1049, in putPage
response, data = self.site().postForm(address, predata, sysop = sysop)
File "wikipedia.py", line 2673, in postForm
return self.postData(address, data, sysop = sysop)
File "wikipedia.py", line 2696, in postData
conn.endheaders()
File "/usr/lib/python2.4/httplib.py", line 798, in endheaders
self._send_output()
File "/usr/lib/python2.4/httplib.py", line 679, in _send_output
self.send(msg)
File "/usr/lib/python2.4/httplib.py", line 646, in send
self.connect()
File "/usr/lib/python2.4/httplib.py", line 630, in connect
raise socket.error, msg
socket.error: (110, 'Connection timed out')
Since all that socket juggling is private to wikipedia.py, I suggest that this exception be caught and re-raised as a wikipedia.Error.
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2007-08-24 01:08
Message:
Logged In: NO
Thank you very much! Sorry I was so disgruntled in my last post. I really
appreciate all the hard work you put into making this flexible framework.
----------------------------------------------------------------------
Comment By: Merlijn S. van Deen (valhallasw)
Date: 2007-08-23 09:01
Message:
Logged In: YES
user_id=687283
Originator: NO
Fixed in r4099 (socked.error is caught in postForm)
----------------------------------------------------------------------
Comment By: Nobody/Anonymous (nobody)
Date: 2007-08-23 07:44
Message:
Logged In: NO
As of SVN revision 4096, socket errors from wikipedia.py:3174-3176 are not
caught (the comment just before these lines even makes note of this
fact).
I suggest
response = conn.getresponse()
data = response.read().decode(self.encoding())
conn.close()
be changed to
try:
response = conn.getresponse()
data = response.read().decode(self.encoding())
conn.close()
except socket.error, errmsg:
raise ServerError(errmsg)
It's really annoying to have a longer bot job that is aware of
wikipedia.Error crash because of an intermittent "Connection reset by
peer".
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=603138&aid=1672346&group_…