Re: [Analytics] Meeting notes - Identifying Bots

We met briefly to weight the pros and cons of taking up a project to tag bot traffic. TL;DR While we see several projects that will benefit from a more precise bot identification we think that at this time there are workarrounds that we can do to filter bot traffic in most areas and that we should not expend the resources and computation effort that a through bot detection system will require. We think is worth spending time in quantifying our TRUE bot traffic so when management has a question like "How much of our traffic is crawling?" we can give an estimate by, say, researching bot traffic monthly, weekly and daily in one given month. At this time 15% of our pageview traffic (not requests) are detected bots, we estimate that the real bot traffic might be quite a big higher. More detailed Notes here: https://wikitech.wikimedia.org/wiki/Analytics/Bots Attendees please modify/correct notes as needed. _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics