Awesome Oliver, thanks! Having a look.
On Sat, Mar 21, 2015 at 4:32 PM, Oliver Keyes okeyes@wikimedia.org wrote:
Alrighty; existing implementation updated with https://gerrit.wikimedia.org/r/#/c/198489/ which also exposes the isAppPageview method and associates a UDF with it (Marcel, this means you'll be able to do your stuff. Rockin'!)
Let me know when y'all have the necessary varnish and app changes proposed/committed/merged/etc so I can work on the updates to this method and the documentation in parallel.
On 20 March 2015 at 21:11, Oliver Keyes okeyes@wikimedia.org wrote:
Okay. So, distinct header, registering refresh=1 or [nothing] in the x-analytics field when it hits the varnish layer? Rocking! Let me know what happens/where it goes/when it goes/etc.
The RESTbase services, yeah; they're going to make an impact. Not just in format either (I have literally no idea how the caching setup there works, if there is a caching setup, or which varnish cluster any such caching setup would go through and so if we're even picking up those requests at all).
On 20 March 2015 at 19:11, Bernd Sitzmann bernd@wikimedia.org wrote:
Adam: Thanks for including us Android devs. I think a distinct header is probably preferable.
Oliver: I think you already got this, so just to confirm: for Android you can
use
sections=0 for counting pageviews, and sections=all for counting saved
page
refreshes.
As a heads-up we're experimenting with Node.js/RESTBase services. Not
sure
when and if those will be used in production. We're at an early stage.
Just
wanted to mention that this since it will change the way we request
pages
from the server significantly (actually it would be from different
servers,
too). That's probably another pro for using HTTP request headers.
The downside is that we're not using this special request header yet.
Bernd
On Fri, Mar 20, 2015 at 3:17 PM, Oliver Keyes okeyes@wikimedia.org
wrote:
Sounds good for me, although I'd just go for the first option. There's nothing contained in the second option not found in the first that we need (that is: yes, it has device version and OS and all of that, which we need for other things, but then so does the user agent, so it's probably extraneous work to include all of that a /second/ time)
On 20 March 2015 at 15:42, Adam Baso abaso@wikimedia.org wrote:
Either use of a distinct header (e.g., X-WMF-Refresh: 1) or use of a distinct parameter (e.g., wprov=rfsi1 for "refresh from saved pages
on
iOS, version 1") from the client would work. The use of a distinct
parameter
is much easier.
Then the VCL code could be updated to look for the field and enrich
the
X-Analytics header accordingly.
Okay?
On Fri, Mar 20, 2015 at 11:37 AM, Oliver Keyes <okeyes@wikimedia.org
wrote:
Just a note that I'm started on a related patch now, using the (we
now
know, not-futureproof) logic of:
- if it's got sections=0 it's a pageview;
- if it's got sections=all and is from iOS it's a pageview.
Would appreciate some feedback on the idea of sending refresh=1 in
the
x_analytics header, so we know what to expect there.
On 20 March 2015 at 00:44, Oliver Keyes okeyes@wikimedia.org
wrote:
> In the long-term we'll just be relying on x_analytics, yes,
because
> the app will be sending through the pageID and namespace in the
same
> way as desktop and the mobile web do; so it'll be > pageid=50;ns=0;refresh-1 or pageid=50;ns=0, respectively. That's a > different patch. In the short-term I'm not seeing how building
out a
> complex ecosystem for this is a valuable use of time given that we > know what we'll be switching to (and that it's standardised not
just
> across apps, but also for desktop and the mobile web, to boot).
All
> we > care about distinguishing that we won't get for free as part of > already-scheduled work is refreshes from pageviews. > > On 19 March 2015 at 23:59, Gergo Tisza gtisza@wikimedia.org
wrote:
>> On Thu, Mar 19, 2015 at 8:06 PM, Oliver Keyes <
okeyes@wikimedia.org>
>> wrote: >>> >>> As I see it, we basically have two possibilities here: >>> >>> 1. Make the URLs distinguishable; >>> 2. Add additional metadata in a non-URL place >>> >>> 1 is undesirable because it ruins caching, and we like caching.
So
>>> we >>> look at 2, which realistically means the x_analytics field. Why >>> don't >>> we add a parameter there? refresh=1. And then, our app check
boils
>>> down to (in pseudocode): >>> >>> if(other_checks & urlContains("sections=(0|all)" & >>> !xAnalyticsContains("refresh")){ >>> return true; >>> } >>> return false; >>> >>> Nice and simple and easy. It'll require some coordination with >>> Ottomata because it means modifying the UDF parameters, and
we're
>>> using said UDF in production so it'll have to be synced to a
change
>>> in >>> the relevant Oozie job, but it should be totally doable, and I >>> can't >>> see an easier way of doing it. >>> >>> Thoughts, people? >> >> >> Why use anything other than X-Analytics at all? Source of the >> pageview >> is >> exactly the kind of information it is meant for. Just set >> source=AndroidAppPageView / source=IosAppSectionView / >> source=AndroidAppSavedPageRefresh etc. (Or set up a mapping to >> three-letter >> acronyms if you want to be nice on the servers.) That puts
logging
>> completely in the hands of the app developers, so there are no >> information >> flow problems and less organizational overhead; it also makes
rules
>> more >> explicit (and thus harder to mess up / easier to spot errors). >> >> _______________________________________________ >> Analytics mailing list >> Analytics@lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/analytics >> > > > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
-- Oliver Keyes Research Analyst Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
-- Oliver Keyes Research Analyst Wikimedia Foundation
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics