On Tue, Sep 01, 2015 at 12:42:35PM -0400, John wrote:
Could they be sending a non-standard header of "-"
They could.
But if a request comes in without a User-Agent header, the logging pipeline silently translates it into "-".
Have fun, Christian
P.S.: The relevant configuration (for webrequests) is at
https://github.com/wikimedia/operations-puppet/blob/production/modules/role/...
That long line contains '%{User-Agent@user_agent}i', which means
log the request's User-Agent header
but no default value is provided. As no default value is provided, varnishkafka uses the pre-set default value, which is "-":
https://github.com/wikimedia/varnishkafka/blob/master/varnishkafka.c#L246
This conversion from the empty string to "-" does not kill relevant information and is useful for some researchers when manually inspecting TSVs, or manually browsing Hive output.