Hi all,

It seems comments are decreasing at this point. I'd like to slowly drive this thread to a conclusion.


3. Create a plan to block clients that dont implement the (amended) User-Agent policy.
 
I think we can decide on this later. Steps 1) and 2) can be done first - they should be done anyway before 3) - and then we can see how much benefit we raise from them. If we don't get a satisfactory reaction from bot/framework maintainers, we then can go for 3). John, would you be OK with that?


If no-one else raises concerns about this, the Analytics team will:
  1. Add a mention to https://meta.wikimedia.org/wiki/User-Agent_policy, to encourage including the word "bot" (case-insensitive) in the User-Agent string, so that bots can be easily identified.

  2. Advertise the convention and reach out to bot/framework maintainers to increase the share of bots that implement the User-Agent policy.

Thanks!

On Tue, Feb 2, 2016 at 5:21 AM, Bryan Davis <bd808@wikimedia.org> wrote:
On Mon, Feb 1, 2016 at 11:42 AM, Nuria Ruiz <nuria@wikimedia.org> wrote:
>>It will take time for frameworks to implement an amended User-Agent policy.
>>For example, pywikipedia (pywikibot compat) is not actively
>>maintained.
> That doesn't imply we shouldn't have a  policy that anyone can refer to,
> these bots will not follow it until they get some maintainers.
>
>>There was a task filled against Analytics for this, but Dan Andreescu
>>removed Analytics (https://phabricator.wikimedia.org/T99373#1859170).
>
> Sorry that the tagging is confusing. I think Analytics tag was removed cause
> this is a request for data and our team doesn't do data retrieval. We
> normally tag with "analytics" phabricator items that have actionables for
> our team.
> I am cc-ing Bryan who has already done some analysis on bots requests to the
> API and can probably provide some data.

It would be possible to make some relative comparisons of pywikibot
versions using the data that is currently collected in the
wmf.webrequest data set. "Someday" I'll get T108618 [0] finished which
will make answering some of the more granular questions in T99373
easier. Kunal talked with Brad and I a few weeks ago when we were all
in SF for the DevSummit about other instrumentation that could be put
in place specifically for pywikibot so that something like
Special:ApiFeatureUsage [2] could be created for pywikibot version
tracking as well. This all seems like a fork of the topic at hand
however.

[0]: https://phabricator.wikimedia.org/T108618
[1]: https://phabricator.wikimedia.org/T99373
[2]: https://en.wikipedia.org/wiki/Special:ApiFeatureUsage

Bryan
--
Bryan Davis              Wikimedia Foundation    <bd808@wikimedia.org>
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics



--
Marcel Ruiz Forns
Analytics Developer
Wikimedia Foundation