*%(%ç%ç(%!! ... :)

sorry - was clicking too fast... here the attachement...

Greetings

Am 24.07.2011 12:19, schrieb Merlijn van Deen:

Could you post the code you are using to send emails?

On 24 July 2011 11:22, Dr. Trigon <dr.trigon@surfeu.ch mailto:dr.trigon@surfeu.ch> wrote:

Hello all!

I am searching desperately for a solution to following problem.
DrTrigonBot is written such that in case of an (not severe) error
the bot should send an email with a notification to me. BUT the
strange thing I do not understand is; userlib.py is not able to
send emails about 50% of the tries...

Some info can be found in (at the very end):
 http://toolserver.org/~drtrigon/DrTrigonBot/20110724.log

The error raised is
 'UserActionRefuse("You don't have permission to send mail",)'

Does anyone have an idea how to solve this? In my view this is
triggered by "if not self.site().isAllowed('sendemail'):" in
"userlib.sendMail" but WHY is this 'sendemail' flags sometimes
true and sometimes false...?!?

Thanks a lot for your help and time
Greetings to all!
DrTrigon

_______________________________________________
Pywikipedia-l mailing list
Pywikipedia-l@lists.wikimedia.org
<mailto:Pywikipedia-l@lists.wikimedia.org>
https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

---

Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

Since all edits are correctly assigned to the bot compare e.g. [1] and the bot log [2] (at the very end), you can see:

- edits are assigned to the bot account - last edit was at '2011/07/24 08:53:24' and bot mailing failed at '2011/07/24 08:53:38' which was 14 secs later - but the bot runned 4 hrs before without logging out...

So it may be possible but as far as I can see the bot had never problems with un-expected log-out... just with mailing...

What do you think?

[1] http://de.wikipedia.org/wiki/Spezial:Beitr%C3%A4ge/DrTrigonBot [2] http://toolserver.org/~drtrigon/DrTrigonBot/20110724.log

Am 24.07.2011 17:41, schrieb Bináris:

Try to write a line just before the e-mail sending that checks if your bot is still logged in. Is it possible that sometimes it loses the logged-in status? -- Bináris

---

Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

...

Could it be it only works from one of your accounts and not from another?

What do you exactly mean? I have 1 bot account only. This one exists for years and mailing worked fine in the past

OK. Based on the error, I suspected it was a message from the wiki. Of course, you already noted in your first mail that the message is from the framework. Silly me.

(userlib.py) 195 if not self.site().isAllowed('sendemail'): 196 raise UserActionRefuse('You don't have permission to send mail')

isAllowed is called on the site object, in your case dewikipedia:

(wikipedia.py) 5031 self._load(sysop = sysop) 5032 index = self._userIndex(sysop) (...) 5037 return right in self._rights[index]

so it might be interesting to check what yoursite._rights contains. It might be the api returns incorrect values.

so this is what I would suggest: change your code from:

def send_mailnotification(self, item): pywikibot.output(u'Sending mail "%s" to "%s" as notification!' % (error_mail[1], error_mail[0])) usr = userlib.User(pywikibot.getSite(), error_mail[0])

to something like:

def send_mailnotification(self, item): pywikibot.output(u'Sending mail "%s" to "%s" as notification!' % (error_mail[1], error_mail[0])) site = pywikibot.getSite() pywikibot.output(u'Allowed to send email: %r' % (site.isAllowed('sendemail'),) pywikibot.output(u'Permissions: %r' % (site._rights,) usr = userlib.User(pywikibot.getSite(), error_mail[0])

of course, ._rights is internal and should not generally be used like this. It's a helpful debugging tool, though.

You could also run the script through pdb to do post-mortem debugging. This would allow you to check out the problem more directly.

The recent edits to the framework that could be related are: http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/wikipedia.py?r... * (jun 5)* * http://svn.wikimedia.org/viewvc/pywikipedia/trunk/pywikipedia/wikipedia.py?r... (mar 20, does not really seem related)* * * all other recent changes are, as far as I can see, purely cosmetic.

So I would suggest: 1) see what site._rights contains, and if this does not give any useful results, 2) try to see if the problem is introduced in r9288 (see if r9287 works).

Best, Merlijn * *

...

So I would suggest: 1) see what site._rights contains, and if this does not give any useful results, 2) try to see if the problem is introduced in r9288 (see if r9287 works).

So I did the changes mentioned in 1) to get more output. Depending on those results I will either consider 2) or have to come back to you... (should NOT be a threat... ;)) Anyway thanks a lot so far for your help, hints and time!

Today was an intressting day, because of other issues, I runned the bot twice, with the same code regarding the mail stuff, here what I got the first run:

2011/08/06 04:45:45:: Sending mail "Bot ERROR" to "DrTrigon" as notification! 2011/08/06 04:45:45:: !!! Allowed to send email: True 2011/08/06 04:45:45:: !!! Permissions: [[u'move', u'autoreview', u'move-rootuserpages', u'createtalk', u'minoredit', u'movestable', u'skipcaptcha', u'nominornewtalk', u'noratelimit', u'*', u'collectionsaveasuserpage', u'abusefilter-view', u'abusefilter-log-detail', u'createpage', u'writeapi', u'createaccount', u'centralauth-merge', u'read', u'autopatrol', u'abusefilter-log', u'suppressredirect', u'user', u'collectionsaveascommunitypage', u'reupload-own', u'apihighlimits', u'autoconfirmed', u'edit', u'sendemail', u'bot', u'upload', u'purge', u'reupload'], None] 2011/08/06 04:45:45:: Email sent.

and here the second run:

2011/08/06 18:28:37:: Sending mail "Bot ERROR" to "DrTrigon" as notification! 2011/08/06 18:28:37:: !!! Allowed to send email: False 2011/08/06 18:28:37:: !!! Permissions: [['user', 'bot', 'autoconfirmed', 'read', 'createaccount', 'edit', 'upload', 'createpage', 'createtalk', 'move', 'upload', 'apihighlimits'], None] 2011/08/06 18:28:37:: !!! (<class 'userlib.UserActionRefuse'>, UserActionRefuse("You don't have permission to send mail",), <traceback object at 0x8a20cd4>) 2011/08/06 18:28:37:: !!! WARNING: mail could not be sent!

as you can see the first worked and the second not. In the first the bot got a long list of rights and in the second and short...

The point mentioned in 2) regarding revisions r9288 and r9287 does not apply here as I can see because I using the rather old r8990 anyway...

Any idea where those differences come from?

Thanks a lot (so far) and Greetings Dr.Trigon

Link to log file: http://toolserver.org/~drtrigon/DrTrigonBot/20110806.log

First of all, sorry for the slow response. I haven't had time to look into this (and I still haven't). From the top of my head, the following (hoping someone else will be able to further assist):

No problem - in fact you responded 3 days later - compared to the delays I produce regularly... ;))

I *think* the list " [['user', 'bot', 'autoconfirmed','read', 'createaccount', 'edit', 'upload', 'createpage', 'createtalk','move', 'upload', 'apihighlimits'], None] " is somewhere in the bot code (wikipedia.py:5691 and further) to take as 'default' when no response (or a bogus response) is returned from the server. If not, the server might *actually* be responding with that data, in which case the server is at fault.

This sounds reasonable. I had a look into those code parts and agree with you and am also "hoping someone else will be able to further assist"... This would be great since it looks like some core code. As far as I can see 'Site.getToken' is called in most of the page class member funcs such as get, put, ... so I am confused because they all work in my code. Do you have any idea whats the initial (first) call of 'Site.getToken' in the code - the one that seams to produce the problems we encounter here...?

If it *is* the default list, I think this should be fixed. It does not make much sense to me to apply a selective default list if no response is returned -- either retry or assume we can do anything. Then again, the latter means the bot would crash when trying to lock without lock rights.

In essence: this needs some thought.

Shall I issue a bug report in tracker on sf.net?

For your problem: try calling Site.getToken(getagain=True) when you get the exception, and retry. (this calls self._load(force=True) and throug there _getUserDataOld(force=True). It's hackish, but the best I can come up with for the moment.

Hackish? You never looked at my bot code, did you? ;))

I inserted following code lines:

if not site.isAllowed('sendemail'): pywikibot.output(u'!!! Try getting new token: %r' % \ (site.getToken(getagain=True),))

So we have to wait again... ;) (may be I have to think about how to try to force such errors...)

Thanks for all your support and time!

Greetings Dr. Trigon

Hello Dr. Trigon,

this means, the hint given:

...

...

For your problem: try calling Site.getToken(getagain=True) when you get the exception, and retry. (this calls self._load(force=True) and throug

finally works!

The last point is now to figure out why site.getToken sometimes is called properly and sometimes not...???

Intuitively, I think this is what happens: sometimes when requesting a token, the connection is reset (or something equivalent) - this is why it only happens every now and then. This is then interpreted by the framework as 'hey, we cannot get the data right now, so let's assume something more-or-less sensible'.

By calling Site.getToken(getagain=True), you force the framework to retry getting that data, which succeeds after which you can send the e-mail.

Best, Merlijn

...

Dr. Trigon dr.trigon@surfeu.ch wrote: Intuitively, I think this is what happens: sometimes when requesting a token, the connection is reset (or something equivalent) - this is why it only happens every now and then. This is then interpreted by the framework as 'hey, we cannot get the data right now, so let's assume something more-or-less sensible'.

Sound reasonable... But as we try to get a piece of productive software, honestly I do not feel good with terms like "Intuitively" and "reasonable"... ;) In my opinion the source of this (miss)behaviour should be tracked down. Thus I filed a bug report at sf.net:

https://sourceforge.net/tracker/?func=detail&aid=3397944&group_id=93...

You should never assume that the network you are using is stable and always produces desired results. It can be down to everything, like authentication problems, caching problems at wikimedia or network issues.

Sure, it might be a real problem, but it's most probably on the side of MediaWiki (or actually Wikimedia installation) rather than the framework.

//Saper

I guess this was introduced in r5037 (Reorganization of the user data retrieving: moving it to a single function.) with _getUserData() renamed to _getUserDataOld() now but I don't see the point for "Assume the user has the default rights". Maybe this is good enough for screen-scrapped pages.

Greetings xqt

----- Original Nachricht ---- Von: Merlijn van Deen valhallasw@arctus.nl An: Pywikipedia discussion list pywikipedia-l@lists.wikimedia.org Datum: 09.08.2011 17:04 Betreff: Re: [Pywikipedia-l] No permission to send mail...

Hello,

On 6 August 2011 18:55, Dr. Trigon dr.trigon@surfeu.ch wrote:

...

2011/08/06 04:45:45:: !!! Permissions: [[u'move', u'autoreview', u'move-rootuserpages', u'createtalk', u'minoredit', u'movestable', u'skipcaptcha', u'nominornewtalk', u'noratelimit', u'*', u'collectionsaveasuserpage', u'abusefilter-view', u'abusefilter-log-detail', u'createpage', u'writeapi', u'createaccount', u'centralauth-merge', u'read', u'autopatrol', u'abusefilter-log', u'suppressredirect', u'user', u'collectionsaveascommunitypage', u'reupload-own', u'apihighlimits', u'autoconfirmed', u'edit', u'sendemail', u'bot', u'upload', u'purge', u'reupload'], None] 2011/08/06 04:45:45:: Email sent.

and here the second run:

2011/08/06 18:28:37:: !!! Permissions: [['user', 'bot', 'autoconfirmed', 'read', 'createaccount', 'edit', 'upload', 'createpage', 'createtalk', 'move', 'upload', 'apihighlimits'], None]

First of all, sorry for the slow response. I haven't had time to look into this (and I still haven't). From the top of my head, the following (hoping someone else will be able to further assist): I *think* the list " [['user', 'bot', 'autoconfirmed','read', 'createaccount', 'edit', 'upload', 'createpage', 'createtalk','move', 'upload', 'apihighlimits'], None] " is somewhere in the bot code (wikipedia.py:5691 and further) to take as 'default' when no response (or a bogus response) is returned from the server. If not, the server might *actually* be responding with that data, in which case the server is at fault.

If it *is* the default list, I think this should be fixed. It does not make much sense to me to apply a selective default list if no response is returned -- either retry or assume we can do anything. Then again, the latter means the bot would crash when trying to lock without lock rights.

In essence: this needs some thought.

For your problem: try calling Site.getToken(getagain=True) when you get the exception, and retry. (this calls self._load(force=True) and throug there _getUserDataOld(force=True). It's hackish, but the best I can come up with for the moment.

Best, Merlijn

---

---

Pywikipedia-l mailing list Pywikipedia-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/pywikipedia-l

Arcor empfiehlt: Die TOP 1000 Surftipps zu allem, was im Internet interessant und spannend ist - sehen Sie hier: http://www.arcor.de/content/pc_technik/internet/surftipps/surftipps/