Hi, I believe in this case we should somehow implement 'press Save again' in the bot. I don't know if the MediaWiki API allows this, but if not, I think it should be added (to the AbuseFilter extension?). If the edit was 'blocked' by an abuse filter, things would be more complicated because we would definitely need an admin to look at the case (and thus need for notification). I think how to notify the interested parties of archiving failures is a general issue, not just for failures related to abuse filters. I created a Phabricator task for that a while ago. [2] Best, Yusuke On Sun, Jun 3, 2018 at 4:07 PM, Bináris <wikiposta(a)gmail.com> wrote:

I think that Pywikibot's save method should allow explicit overriding of a warning raised by abuse filter. It shouldn't be the default state, but it should be possible. Anyway, having a bot regulary hitting AF isn't a good idea - talk with an admin of your wiki. On Czech Wikipedia, filters that would be hit by archivebot has exception, generally !"bot" in user_groups. This allows cs.wiki to warn users that may or may not know what is happening and allow bots to do their maintenance work without being warned because their operators are expected to know what is ok and what is not. Cheers, Martin ne 3. 6. 2018 v 13:49 odesílatel billinghurst <billinghurstwiki(a)gmail.com> napsal:

2018-06-03 19:21 GMT+02:00 Dalba <dalba.wiki(a)gmail.com>om>: Generally this wouldn't be a good idea for all the bots (in framework), but actually for archivebot I don't see any risk in doing so, as it only copies existing texts from one page to another.

ne 3. 6. 2018 v 19:46 odesílatel Bináris &lt;wikiposta(a)gmail.com&gt; napsal: Well, failed sliently? Again, I run the same bot in cs.wiki and this is what I can see in logs. Looking for: {{Šablona:Archivace}} in [[cs:Diskuse:Moravská vlajka]] Processing 1 threads Only 1 (< 2) threads are old enough. Skipping Processing [[cs:Diskuse:Česko]] 13 Threads found on [[cs:Diskuse:Česko]] Looking for: {{Šablona:Archivace}} in [[cs:Diskuse:Česko]] Processing 13 threads 13 Threads found on [[cs:Diskuse:Česko/archiv-02]] Only 3 (< 10) threads are old enough. Skipping Processing [[cs:Wikipedista:ArchivačníBot/test1]] 1 Threads found on [[cs:Wikipedista:ArchivačníBot/test1]] Looking for: {{Šablona:Archivace}} in [[cs:Wikipedista:ArchivačníBot/test1]] Processing 1 threads Only 0 (< 1) threads are old enough. Skipping Processing [[cs:Wikipedista:ArchivačníBot/test3]] 1 Threads found on [[cs:Wikipedista:ArchivačníBot/test3]] Looking for: {{Šablona:Archivace}} in [[cs:Wikipedista:ArchivačníBot/test3]] Processing 1 threads Processing [[cs:Diskuse s wikipedistou:Chmee2]] 4 Threads found on [[cs:Diskuse s wikipedistou:Chmee2]] Looking for: {{Šablona:Archivace}} in [[cs:Diskuse s wikipedistou:Chmee2]] Processing 4 threads Only 0 (< 1) threads are old enough. Skipping Processing [[cs:Diskuse s wikipedistou:HypoBOT]] 1 Threads found on [[cs:Diskuse s wikipedistou:HypoBOT]] Looking for: {{Šablona:Archivace}} in [[cs:Diskuse s wikipedistou:HypoBOT]] Processing 1 threads 2 Threads found on [[cs:Diskuse s wikipedistou:HypoBOT/archiv 2017-1]] Only 1 (< 2) threads are old enough. Skipping Processing [[cs:Diskuse s wikipedistou:JAn Dudík]] 21 Threads found on [[cs:Diskuse s wikipedistou:JAn Dudík]] Looking for: {{Šablona:Archivace}} in [[cs:Diskuse s wikipedistou:JAn Dudík]] Processing 21 threads 138 Threads found on [[cs:Diskuse s wikipedistou:JAn Dudík/archiv 2017-1]] Archiving 8 thread(s). ERROR: Error occurred while processing page [[cs:Diskuse s wikipedistou:JAn Dudík]] ERROR: SpamfilterError: Edit to page [[Diskuse s wikipedistou:JAn Dudík/archiv 2017-1]] rejected by spam filter due to content: twitter.com/search ERROR: SpamfilterError: Edit to page [[Diskuse s wikipedistou:JAn Dudík/archiv 2017-1]] rejected by spam filter due to content: twitter.com/search Traceback (most recent call last): File "/shared/pywikipedia/core/scripts/archivebot.py", line 819, in main archiver.run() File "/shared/pywikipedia/core/scripts/archivebot.py", line 713, in run self.archives[a].update(comment) File "/shared/pywikipedia/core/scripts/archivebot.py", line 531, in update self.save(summary) File "/shared/pywikipedia/core/pywikibot/tools/__init__.py", line 1520, in wrapper return obj(*__args, **__kw) File "/shared/pywikipedia/core/pywikibot/tools/__init__.py", line 1520, in wrapper return obj(*__args, **__kw) File "/shared/pywikipedia/core/pywikibot/page.py", line 1264, in save cc=apply_cosmetic_changes, quiet=quiet, **kwargs) File "/shared/pywikipedia/core/pywikibot/page.py", line 136, in wrapper handle(func, self, *args, **kwargs) File "/shared/pywikipedia/core/pywikibot/page.py", line 127, in handle raise err SpamfilterError: Edit to page [[Diskuse s wikipedistou:JAn Dudík/archiv 2017-1]] rejected by spam filter due to content: twitter.com/search Framework report problems, bot operators should monitor logs. I know that this is spamfilter and not abusefilter, but Pywikibot behaves the same way even if an abusefilter has rejected given change. See the following manual example: tools.urbanecmbot@tools-bastion-02 ~ $ python Python 2.7.6 (default, Nov 23 2017, 15:49:48) [GCC 4.8.4] on linux2 Type "help", "copyright", "credits" or "license" for more information. Sleeping for 9.5 seconds, 2018-06-03 22:19:23 ERROR: editpage: abusefilter-disallowed Hit AbuseFilter: Test filter, WARNING: Page [[Wikipedista:Martin Urbanec/Pískoviště/1]] not saved Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/shared/pywikipedia/core/pywikibot/tools/__init__.py", line 1446, in wrapper return obj(*__args, **__kw) File "/shared/pywikipedia/core/pywikibot/tools/__init__.py", line 1446, in wrapper return obj(*__args, **__kw) File "/shared/pywikipedia/core/pywikibot/page.py", line 1270, in save cc=apply_cosmetic_changes, quiet=quiet, **kwargs) File "/shared/pywikipedia/core/pywikibot/page.py", line 127, in wrapper handle(func, self, *args, **kwargs) File "/shared/pywikipedia/core/pywikibot/page.py", line 118, in handle raise err pywikibot.exceptions.PageSaveRelatedError: Page [[cs:Wikipedista:Martin Urbanec/Pískoviště/1]] was not saved. Of course Pywikibot didn't report it by email, because it will be unexpectable (and, to be precise, Pywikibot do not always have a way to email you - for example, my Pywikibot bot password don't allow emailing (well, maybe it does, I'm not sure, but it isn't required to allow it).

ne 3. 6. 2018 v 22:35 odesílatel Bináris &lt;wikiposta(a)gmail.com&gt; napsal: If somebody writes "fail sliently", I expect "without any notice" automatically. There is a notice, but the notice isn't processed by the operator or their bots. But your example shows something important: Dalba's solution is not general You cannot save an edit that is being denied. This is what such filters are used to, to prevent some edits. Still, not a framework problem IMHO. >> Of course Pywikibot didn't report it by email, because it will be >> unexpectable (and, to be precise, Pywikibot do not always have a way to >> email you - for example, my Pywikibot bot password don't allow emailing >> (well, maybe it does, I'm not sure, but it isn't required to allow it). >> Pywikibot should have the option to send a mail to the owner, if he/she > wants, and setting the e-mail option for the bot account is worth for this > purpose. > Of course, it wouldn't be obligatory. > Really? Imagine we have this feature and imagine a bot operator that says "Hey, you can notify the bot operator via email, let's implement SMS as well". Then, other bot operators can require a lot of different notification types. This will make Pywikibot unsustainable. Instead, let's follow a good Unix rule. One program should focus on only one thing. If I'm writing a framework that is interacting with MW API (purpose of Pywikibot), it should interact with MW API and not monitor logs. If I want to monitor logs, I should use another program (generally called log watcher) that focuses on watching/monitoring logs and reporting some patterns to the admin. I store all logs from my Pywikibot scripts in ~/logs. If I want, I can create a maintenance job to monitor the logs. It will generally execute something like this. It would be relatively easy to modify this to monitor other bots (multiple maintenance jobs, or one running in multiple threads) and/or include the traceback directly in the mail. tail -Fn0 /data/project/urbanecmbot/logs/archivePages.log | \ while read line ; do echo "$line" | grep -q "ERROR" if [ $? = 0 ] then echo "Fatal error in ArchiveBot, see /data/project/urbanecmbot/logs/archivePages.log for details" | /usr/bin/mail -s "Error notification" martin.urbanec(a)wikimedia.cz fi done I didn't say a human should do it. I said that the bot operator is expected to monitor the logs, either themselfes or using a bot. It is the same for system administrator. The services running in the system log various events into log service, but this is the only thing it does. The log service store the events in logfiles (default setting) or to dedicated log server. If the sysadmin want to know about some events by email/SMS/whatever, they should employ a service that is supposed to monitor for given patterns in given log files. _______________________________________________

Thanks for your proposal but the easiest way for this case is to modify the edit filter to enable such edits. Check bot group, username and edit summary to allow the change. Best xqt Von meinem iPhone gesendet Am 04.06.2018 um 02:06 schrieb Yusuke Matsubara &lt;whym(a)whym.org&gt;rg>:

Is abusefilter recoverable? It can even block you without any API-side notice. Do we want to risk it or let bot operators decide on their own? Best, Martin po 4. 6. 2018 v 18:25 odesílatel Maarten Dammers &lt;maarten(a)mdammers.nl&gt; napsal:

There are no global filters that will block you, and that is purposeful decision and steward-imposed condition, though there are local filters all over the place that will do so, including meta. Now the filters have been targetted at spambots, and are well-tuned, though not perfect, so you will need to be on good terms with admins if you are going to force an edit that may be spammy. I would still recommend that you talk to admins about writing abuse filters to allow bots to be excluded fro the impact (pretty easy tweak) Note that bot edits won't get around title and spam blacklists, which I saw mentioned for twitter.com/search -- billinghurst ------ Original Message ------ From: "Martin Urbanec" &lt;martin.urbanec(a)wikimedia.cz&gt; To: "Pywikibot discussion list" &lt;pywikibot(a)lists.wikimedia.org&gt; Sent: 5/06/2018 3:08:48 PM Subject: Re: [pywikibot] Archivebot fails on abuse filter