Remind me again why a production setup is logging
every header of every
request?
That's ludicrous. Please reread our udplog format documentation and this
entire thread carefully, especially the first message before commenting any
further.
Also, if you are logging every header, then the
amount of data
added by a single extra header would be insignificant compared to the rest
of the request.
*--*
*Tyler Romeo*
Stevens Institute of Technology, Class of 2015
Major in Computer Science
www.whizkidztech.com | tylerromeo(a)gmail.com <javascript:;>
On Sun, Feb 3, 2013 at 5:12 AM, Asher Feldman
<afeldman@wikimedia.org<javascript:;>
wrote:
That's not at all true in the real world.
Look at the actual requests for
google analytics on a high percentage of sites, etc.
Setting new request headers for mobile that map to new inflexible fields
in
> the log stream that must be set on all non mobile requests ("\t-\t-")
> equals gigabytes of unnecessarily log data every day (that we want
> to save 100% of) for no good reason. Wanting to keep query params "pure"
> isn't a good reason.
>
> On Sunday, February 3, 2013, Tyler Romeo wrote:
>
> > Considering that the query component of a URI is meant to identify the
> > resource whereas HTTP headers are meant to tell the server additional
> > information about the request, I think a header approach is much more
> > appropriate than a no-op query parameter.
> >
> > If the X- is removed, I'd have no problem with the addition of these
> > headers, but what is the advantage of having two over one. Wouldn't a
> > header like:
> > MobileFrontend: 1/2 a/b/s
> > work just as fine?
> >
> > *--*
> > *Tyler Romeo*
> > Stevens Institute of Technology, Class of 2015
> > Major in Computer Science
> >
www.whizkidztech.com | tylerromeo(a)gmail.com
<javascript:;><javascript:;>
> >
> >
> > On Sun, Feb 3, 2013 at 4:35 AM, Asher Feldman
<afeldman@wikimedia.org<javascript:;>
> <javascript:;>
> >
wrote:
> >
> > > Regarding varnish cacheability of mobile API requests with a logging
> > query
> > > param - it would probably be worth making frontend varnishes strip
out
all
> occurrences of that query param and its value from their backend
requests
> > so they're all the same to the caching instances. A generic param
name
> that
> > can take any value would allow for adding as many extra log values as
> > needed, limited only by the uri log field length.
> >
> > &l=mft2&l=mfstable etc.
> >
> > So still an edge cache change but the result is more flexible
> > while avoiding changing the fixed field length log format across
> unrelated
> > systems like text squids or image caches.
> >
> > On Sunday, February 3, 2013, Asher Feldman wrote:
> >
> > > If you want to differentiate categories of API requests in logs,
add
> > > > descriptive noop query params to the requests. I.e &mfmode=2.
Doing
> > this
> > > in
> > > > request headers and altering edge config is unnecessary and a bad
> > design
> > > > pattern. On the analytics side, if parsing query params seems
> > challenging
> > > > vs. having a fixed field to parse, deal.
> > > >
> > > > On Sunday, February 3, 2013, David Schoonover wrote:
> > > >
> > > >> Huh! News to me as well. I definitely agree with that decision.
> > Thanks,
> > > >> Ori!
> > > >>
> > > >> I've already written the Varnish code for setting X-MF-Mode
so it
> can
> > be
> > > >> captured by varnishncsa. Is there agreement to switch to
> Mobile-Mode,
> > or
> > > >> at
> > > >> least, MF-Mode?
> > > >>
> > > >> Looking especially to hear from Arthur and Matt.
> > > >>
> > > >> --
> > > >> David Schoonover
> > > >> dsc(a)wikimedia.org
> > > >>
> > > >>
> > > >> On Sat, Feb 2, 2013 at 2:16 PM, Diederik van Liere
> > > >> <dvanliere(a)wikimedia.org>wrote;wrote:
> > > >>
> > > >> > Thanks Ori, I was not aware of this
> > > >> > D
> > > >> >
> > > >> > Sent from my iPhone
> > > >> >
> > > >> > On 2013-02-02, at 16:55, Ori Livneh
<ori(a)wikimedia.org> wrote:
> > > >> >
> > > >> > >
> > > >> > >
> > > >> > > On Saturday, February 2, 2013 at 1:36 PM, Platonides
wrote:
> > > >> > >
> > > >> > >> I don't like it's cryptic nature.
> > > >> > >>
> > > >> > >> Someone looking at the headers sent to his browser
would be
> very
> > > >> > >> confused about what's the point of «X-MF-Mode:
b».
> > > >> > >>
> > > >> > >> Instead something like this would be much more
descriptive:
> > > >> > >> X-Mobile-Mode: stable
> > > >> > >> X-Mobile-Request: secondary
> > > >> > >>
> > > >> > >> But that also means sending more bytes through the
wire :S
> > > >> > > Well, you can (and should) drop the 'X-' :-)
> > > >> > >
> > > >> > > See
http://tools.ietf.org/html/rfc6648: Deprecating the
"X-"
> > Prefix
> > > >> and
> > > >> > Similar Constructs in Application Protocols
> > > >> > >
> > > >> > >
> > > >> > > --
> > > >> > > Ori Livneh
> > > >> > >
> > > >> > >
> > >