Please cc analytics@ so the whole team sees this requests.
On Wed, Apr 22, 2015 at 3:09 PM, Dan Garry <dgarry(a)wikimedia.org> wrote:
> Hey Kevin,
>
> Task for your attention: T96926 <https://phabricator.wikimedia.org/T96926>
>
> The following patches are ready to be merged in the iOS and Android apps
> when that task is resolved:
>
> - https://gerrit.wikimedia.org/r/#/c/205980/
> - https://gerrit.wikimedia.org/r/#/c/205976/
>
> Let me know if you've got any questions.
>
> Thanks,
> Dan
>
> --
> Dan Garry
> Product Manager, Search and Discovery
> Wikimedia Foundation
>
Hey,
The Engineering Community Team is trying to define very basic Community
Metrics we want to get in place this quarter.
Taking a quick look at the listed items in the task description
https://phabricator.wikimedia.org/T94578
and sharing your expertise (pointing out bad / better ideas) in that
Phabricator task is highly welcome.
Thank you for your help!
andre
--
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/
(cc-ing analytics list)
>Heads up, that the way we test Event Logging has changed.
>Details can be found here:
https://wikitech.wikimedia.org/wiki/EventLogging/Testing/BetaLabs
Correct, analytics team will take care of maintaining these docs.
>ssh eventlogging02.eqiad.wmflabs
>cd /var/log/eventlogging
>tail -f client-side-events.log
This will give you the WHOLE stream of client side events, correct, valid
and invalid.
>For unvalidated events check out the file all-events.log instead.
This log holds VALID events both client and server side.
>2. Modify event logging code in client code
>In Android app uncomment the following line in EventLoggingEvent
>// private static final String EVENTLOG_URL = "
http://bits.beta.wmflabs.org/event.gif";
Correct
>3. Run your code, and see the output from step 1.
Correct
Thanks for testing before sending your code to prod!
Nuria
On Mon, Apr 20, 2015 at 2:03 PM, Bernd Sitzmann <bernd(a)wikimedia.org> wrote:
> Hi Nuria,
>
> I'm about to send this off to the mobile-tech list. But before I do that,
> is this info correct?
>
> Thanks,
> Bernd
>
> ---
>
> Heads up, that the way we test Event Logging has changed. Details can be
> found here:
> https://wikitech.wikimedia.org/wiki/EventLogging/Testing/BetaLabs
>
> 1.
> ssh eventlogging02.eqiad.wmflabs
> cd /var/log/eventlogging
> tail -f client-side-events.log
>
> For unvalidated events check out the file all-events.log instead.
>
> If you need access check out
> https://wikitech.wikimedia.org/wiki/EventLogging/Testing/BetaLabs#Give_peop…
> .
>
> 2. Modify event logging code in client code
> In Android app uncomment the following line in EventLoggingEvent
> // private static final String EVENTLOG_URL = "
> http://bits.beta.wmflabs.org/event.gif";
>
> Please don't commit that change to keep the production value as is.
>
> 3. Run your code, and see the output from step 1.
>
>
> As an aside:
>
> Confused about beta and labs, like I was? Check out
> https://wikitech.wikimedia.org/wiki/Labs_labs_labs. For some reason when
> I joined it was called beta labs and that stuck with me, and for a lot of
> others, like the creator of the page
> https://wikitech.wikimedia.org/wiki/EventLogging/Testing/BetaLabs.
>
> Cheers,
> Bernd
>
Hi Analytics,
We have found a problem that has been affecting the EventLogging data for
one month. Since March 22, 2015 there have been several gaps (of around 1-2
hours length each) without data in all schema tables.
You can see the details in the following links:
Phabricator task:
https://phabricator.wikimedia.org/T96082
Incident documentation:
https://wikitech.wikimedia.org/wiki/Incident_documentation/20150409-EventLo…
Technical discussion:
https://lists.wikimedia.org/pipermail/analytics/2015-April/003775.html
The problem still persists, although with less frequency due to sampling
added to the Edit schema events, that have reduced EL throughput.
Next steps:
* The backfilling of the data gaps will be carried out this week.
* Implement a patch to avoid the problem ASAP.
* Implement a consistent solution to EventLogging scaling problems.
* Backfill any gaps that occur during implementation of the solutions.
Marcel
Hi all,
The media file request count files for upload.wikimedia.org has got a
missing file for April 14, 2015. [1] There should be a file called
"mediacounts.top1000.2015-04-14.v00.csv.zip", but it was apparently not
generated and skipped.
Can someone look into this? Thank you.
[1]: https://dumps.wikimedia.org/other/mediacounts/daily/2015/
--
Best regards,
Hydriz Scholz
Oliver: Two months is fine. Thank you so much for your help!
> On Apr 13, 2015, at 4:40 PM, analytics-request(a)lists.wikimedia.org wrote:
>
> Send Analytics mailing list submissions to
> analytics(a)lists.wikimedia.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.wikimedia.org/mailman/listinfo/analytics
> or, via email, send a message with subject or body 'help' to
> analytics-request(a)lists.wikimedia.org
>
> You can reach the person managing the list at
> analytics-owner(a)lists.wikimedia.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Analytics digest..."
>
>
> Today's Topics:
>
> 1. Re: Page views on a more frequent than hourly basis (Pine W)
> 2. Re: Page views on a more frequent than hourly basis (Hirav Gandhi)
> 3. Re: Page views on a more frequent than hourly basis (Oliver Keyes)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 13 Apr 2015 13:34:23 -0700
> From: Pine W <wiki.pine(a)gmail.com>
> To: "A mailing list for the Analytics Team at WMF and everybody who
> has an interest in Wikipedia and analytics."
> <analytics(a)lists.wikimedia.org>
> Subject: Re: [Analytics] Page views on a more frequent than hourly
> basis
> Message-ID:
> <CAF=dyJjZMdfTHZ+0+LwnHb9m8xUOd4WetGCFUXYB9Qyf7CyC5Q(a)mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Oliver, re ccing people who are on list, this is the protocol we
> followed in IEGCom to ping people who are subscribed and mentioned in
> certain emails but, like many of us, may automatically move emails from
> lists directly to folders where they may be unread for days. So there is a
> reason to do this.
>
> Thanks,
>
> Pine
>
Hi Sean,
Here's Marcel from Analytics.
I'd like to comment with you some strange behaviors that we've observed on
EventLogging database (m4-master.equiad.wmnet).
1) There are some time spans where there is no data in any table. Examples
follow:
- 2015-04-09 17:20 -> 18:35
- 2015-04-11 03:35 -> 05:20
- 2015-04-11 14:00 -> 15:20
- 2015-04-11 19:00 -> 20:00
- 2015-04-12 14:30 -> 15:40
- 2015-04-13 11:35 -> 12:30
- 2015-04-13 16:30 -> 17:35
This is happening a lot, as you can see, so we are really concerned about
it.
These outages are not explained by any of the EventLogging logs (processor
logs, consumer logs) which confirm that the events were actually sent
normally to the database (and written) for those time spans.
2) Maybe it has something to do with it, or not. But we observed that
sometimes the last events inserted in the tables date from 30mins -
1h30mins in the past. At other moments, the last inserts date from seconds
ago. This is maybe expected due to slave replication but we're not sure.
Do you have some opinion on what can be happening? Any thought would help a
lot.
Thanks!
Marcel
(cc-ing analytics list)
Jon:
To get a prompt response please cc analytics or analytics-internal on
e-mails that way -given that we operate in three timezones- someone is
bound to see your e-mail and respond in a timely fashion.
>Right now the Gather feature is only available/discoverable on mobile web
beta for english wikipedia.
I assume "mobile web beta" is deployed to enwiki in production just like
an special "skin" or 'extension.
>Mobile web beta does not have any special url. It is triggered by a
cookie.
If the that identifies 'mobile-web-beta' is stripped off in varnish
(something you can ask your devs about) then there is no way for you to see
that traffic. (this is the most-likely case I was looking around and
couldn't find any code that would be persisting this cookies further than
varnish).
However, if the cookie value is persisted to x-analytics field then records
are "findable".
Now, if there are any javascript/css files ONLY used by the "mobile web
beta feature" you could do a (very, very) rough estimation of requests by
looking at those. Probably you want to team up with a developer to find out
whether this is possible.
On Thu, Apr 2, 2015 at 4:33 PM, Jon Katz <jkatz(a)wikimedia.org> wrote:
> Hi Nuria,
> Right now the Gather feature is only available/discoverable on mobile web
> beta for english wikipedia. As such, it is really important that I know
> what the overall traffic is on mobile web beta. How can I discover this
> using web request logs? Mobile web beta does not have any special url.
>
> It is triggered by a cookie. Here is how a user opts in. Any help would
> be greatly appreciated!!!
>
> [image: Inline image 1]
>
>
>
> Best,
>
> Jon
>
Hi Analytics Team,
First of, thank you for the pagecount-all-sites data that you've been
publishing. I've been using it for doing some visualization experiments,
and have found it quite helpful.
Prior to yesterday, I'd observed that the pagecount data gets generated in
about 1.5 to 2 hours (eg: the timestamp on pagecounts-20150413-000000.gz
shows 13-Apr-2015 01:38).
However, this lag appears to have been rising since yesterday, and
currently it stands at around 7.5 hours.
http://i.imgur.com/B9C1UgM.png
Would you know what might be causing this? Thanks in advance for your help.
Regards
Antony
