The mobile stuff comes only from varnish though, right?

On Aug 26, 2013, at 12:49 PM, Diederik van Liere <dvanliere@wikimedia.org> wrote:

Yes, we get log lines from Varnish, Squid and Nginx servers.

D

On Mon, Aug 26, 2013 at 3:32 PM, Toby Negrin <tnegrin@wikimedia.org> wrote:

Do we process squid logs?

On Aug 26, 2013, at 12:31 PM, Diederik van Liere <dvanliere@wikimedia.org> wrote:

On Mon, Aug 26, 2013 at 3:23 PM, Matthew Walker <mwalker@wikimedia.org> wrote:

Whist investigating an orthogonal logging issue I encountered a couple of differences between squid/varnish which I didn't know about:

* Varnish does not give subsecond request time information

Yes that's correct

@ottomata: maybe we can add subsecond request time information to varnish requests?

* Varnish does give subsecond request processing time info

Yes that's correct. 

* Varnish calls it 'hit/200' or 'miss/302' instead of 'TCP_MEM_HIT/200' or 'TCP_MISS/302'

Yes that's correct. 

* Varnish does not URL encode the user agent field

We never URL encoded the user agent field as far as I know, we did replace space with %20 but have stopped doing that as we are using the tab as separator now. Maybe that patch has not been removed from the squids?

Example log lines:

amssq41.esams.wikimedia.org

1013692039

2013-07-31T23:00:02.331

0

XXX

TCP_MEM_HIT/200

614

GET

http://meta.wikimedia.org/XXX

NONE/-

image/png

http://en.wikipedia.org/XXX

-

Mozilla/5.0%20(Wind...

en-US

en;q=0.8

-

cp1006.eqiad.wmnet

1442176851

2013-07-31T23:00:02.452

0

XXX

TCP_MISS/302

406

GET

http://meta.wikimedia.org/XXX

NONE/-

-

http://en.m.wikipedia.org/XXX

-

Mozilla/5.0%20(i....

en-us

-

cp3012.esams.wikimedia.org

823553992

2013-07-31T23:00:02

0.000119448

XXX

hit/200

20

GET

http://meta.m.wikimedia.org/XXX

-

image/png

http://de.m.wikipedia.org/XXX

XXX

Mozilla/5.0 (iPho...

de-de

-

~Matt Walker

Wikimedia Foundation

Fundraising Technology Team

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________
Analytics mailing list
Analytics@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics