I got the impression from this discussion that the mobile apps aren't currently in use so the CUs have had no experience working with them. It sounds like I was mistaken.
Toby, what UA data do CUs currently see from edits made through the mobile apps?
CUs, is the information that you're currently getting from edits from the new mobile apps a good balance the concerns raised previously in this discussion?
Pine
Date: Thu, 27 Mar 2014 15:36:16 -0700 From: Toby Negrin tnegrin@wikimedia.org To: "A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics." analytics@lists.wikimedia.org Cc: mobile-l mobile-l@lists.wikimedia.org, Oliver Keyes ironholds@gmail.com, Yuvi Panda yuvipanda@gmail.com Subject: Re: [Analytics] [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) Message-ID: CAAjh0EyqpFDke3P6Q5FxecOnE5yc7uSs_VS9HcB5khoxZ6-Yng@mail.gmail.com Content-Type: text/plain; charset="iso-8859-1"
Hey folks -- we aren't considering changing any of the data that goes into checkuser. That tool will be unchanged.
This discussion only concerns backend logging EventLogging and page view analytics.
thanks,
-Toby
The mobile apps are definitely in the wild, my understanding (from some checks in the past and from the discussion in the other thread) was that it was just the 'normal' useragent for the phone with the addition of a WikipediaMobile identifier at the front (so that you could tell it was from the app). They aren't very common but I have seen some which I knew were app driven.
The reason this came up was that one of the discussions that came up in that other thread was to rewrite the user agent to be less unique which would make it harder for CUs to tell users apart (especially once they found out, which they often do and these discussions are on public lists). The compromise that Yuvi proposed was to send a 'special' user agent for checkuser so that it didn't lose the fingerprinting ability as much.
James
James Alexander Legal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:27 PM, ENWP Pine deyntestiss@hotmail.com wrote:
I got the impression from this discussion that the mobile apps aren't currently in use so the CUs have had no experience working with them. It sounds like I was mistaken.
Toby, what UA data do CUs currently see from edits made through the mobile apps?
CUs, is the information that you're currently getting from edits from the new mobile apps a good balance the concerns raised previously in this discussion?
Pine
Date: Thu, 27 Mar 2014 15:36:16 -0700 From: Toby Negrin tnegrin@wikimedia.org To: "A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics." analytics@lists.wikimedia.org Cc: mobile-l mobile-l@lists.wikimedia.org, Oliver Keyes ironholds@gmail.com, Yuvi Panda yuvipanda@gmail.com Subject: Re: [Analytics] [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) Message-ID: CAAjh0EyqpFDke3P6Q5FxecOnE5yc7uSs_VS9HcB5khoxZ6-Yng@mail.gmail.com Content-Type: text/plain; charset="iso-8859-1"
Hey folks -- we aren't considering changing any of the data that goes
into
checkuser. That tool will be unchanged.
This discussion only concerns backend logging EventLogging and page view analytics.
thanks,
-Toby
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Thanks James.
No UA data is recorded from any platform for non-edit actions like pageviews and watchlisting, even if an editor is logged in, right?
Pine
From: jalexander@wikimedia.org Date: Thu, 27 Mar 2014 23:36:01 -0700 Subject: Re: [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) To: deyntestiss@hotmail.com CC: analytics@lists.wikimedia.org; mobile-l@lists.wikimedia.org
The mobile apps are definitely in the wild, my understanding (from some checks in the past and from the discussion in the other thread) was that it was just the 'normal' useragent for the phone with the addition of a WikipediaMobile identifier at the front (so that you could tell it was from the app). They aren't very common but I have seen some which I knew were app driven.
The reason this came up was that one of the discussions that came up in that other thread was to rewrite the user agent to be less unique which would make it harder for CUs to tell users apart (especially once they found out, which they often do and these discussions are on public lists). The compromise that Yuvi proposed was to send a 'special' user agent for checkuser so that it didn't lose the fingerprinting ability as much.
JamesJames AlexanderLegal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:27 PM, ENWP Pine deyntestiss@hotmail.com wrote:
I got the impression from this discussion that the mobile apps aren't currently in use so the CUs have had no experience working with them. It sounds like I was mistaken.
Toby, what UA data do CUs currently see from edits made through the mobile apps?
CUs, is the information that you're currently getting from edits from the new mobile apps a good balance the concerns raised previously in this discussion?
Pine
Date: Thu, 27 Mar 2014 15:36:16 -0700 From: Toby Negrin tnegrin@wikimedia.org To: "A mailing list for the Analytics Team at WMF and everybody who
has an interest in Wikipedia and analytics." analytics@lists.wikimedia.org Cc: mobile-l mobile-l@lists.wikimedia.org, Oliver Keyes
ironholds@gmail.com, Yuvi Panda yuvipanda@gmail.com Subject: Re: [Analytics] [WikimediaMobile] Mobile and CheckUser (Was
[mobile-l] Wikipedia App User Agents) Message-ID: CAAjh0EyqpFDke3P6Q5FxecOnE5yc7uSs_VS9HcB5khoxZ6-Yng@mail.gmail.com
Content-Type: text/plain; charset="iso-8859-1"
Hey folks -- we aren't considering changing any of the data that goes into checkuser. That tool will be unchanged.
This discussion only concerns backend logging EventLogging and page view analytics.
thanks,
-Toby
_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
I'll let someone more knowledgeable about current practices then me answer that question to a better extent so that I don't butcher it but I know that some of that info gets in with differing levels of sampling and filtering and retention. The UA data, for example, is how we know information like what browsers visit the site ( http://stats.wikimedia.org/wikimedia/squids/SquidReportClients.htm ).
James
James Alexander Legal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:45 PM, ENWP Pine deyntestiss@hotmail.com wrote:
Thanks James.
No UA data is recorded from any platform for non-edit actions like pageviews and watchlisting, even if an editor is logged in, right?
Pine
From: jalexander@wikimedia.org Date: Thu, 27 Mar 2014 23:36:01 -0700 Subject: Re: [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) To: deyntestiss@hotmail.com CC: analytics@lists.wikimedia.org; mobile-l@lists.wikimedia.org
The mobile apps are definitely in the wild, my understanding (from some checks in the past and from the discussion in the other thread) was that it was just the 'normal' useragent for the phone with the addition of a WikipediaMobile identifier at the front (so that you could tell it was from the app). They aren't very common but I have seen some which I knew were app driven.
The reason this came up was that one of the discussions that came up in that other thread was to rewrite the user agent to be less unique which would make it harder for CUs to tell users apart (especially once they found out, which they often do and these discussions are on public lists). The compromise that Yuvi proposed was to send a 'special' user agent for checkuser so that it didn't lose the fingerprinting ability as much.
James
James Alexander Legal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:27 PM, ENWP Pine deyntestiss@hotmail.comwrote:
I got the impression from this discussion that the mobile apps aren't currently in use so the CUs have had no experience working with them. It sounds like I was mistaken.
Toby, what UA data do CUs currently see from edits made through the mobile apps?
CUs, is the information that you're currently getting from edits from the new mobile apps a good balance the concerns raised previously in this discussion?
Pine
Date: Thu, 27 Mar 2014 15:36:16 -0700 From: Toby Negrin tnegrin@wikimedia.org To: "A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics." analytics@lists.wikimedia.org Cc: mobile-l mobile-l@lists.wikimedia.org, Oliver Keyes ironholds@gmail.com, Yuvi Panda yuvipanda@gmail.com Subject: Re: [Analytics] [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) Message-ID: CAAjh0EyqpFDke3P6Q5FxecOnE5yc7uSs_VS9HcB5khoxZ6-Yng@mail.gmail.com Content-Type: text/plain; charset="iso-8859-1"
Hey folks -- we aren't considering changing any of the data that goes
into
checkuser. That tool will be unchanged.
This discussion only concerns backend logging EventLogging and page view analytics.
thanks,
-Toby
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
OK. I'm less worried about aggregate data than retaining information about an identifiable device or user. For example the information that the English Wikipedia article "Crimea" has 251 watchers is ok to record and publish, but we probably shouldn't log that user:BarackObama logged in at 10:04:34 EST with UA xyz after linking to the Wikipedia page from the the Huffington Post, watchlisted the article at 10:05:40, and typed "Angela Merkel" into the Wikipedia search box, and ended his session at 10:06:12. This concern may get into subjects that are covered by the new Wikimedia Privacy Policy and I would be interested in hearing from an expert about what non-edit data is logged and tied to a specific user including non-logged-in users who are browsing Wikimedia sites.
If no changes are being made to mobile that would affect checkusers then should we move this discussion to Analytics-l only?
Thanks,
Pine
From: jalexander@wikimedia.org Date: Thu, 27 Mar 2014 23:58:03 -0700 Subject: Re: [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) To: deyntestiss@hotmail.com CC: analytics@lists.wikimedia.org; mobile-l@lists.wikimedia.org
I'll let someone more knowledgeable about current practices then me answer that question to a better extent so that I don't butcher it but I know that some of that info gets in with differing levels of sampling and filtering and retention. The UA data, for example, is how we know information like what browsers visit the site ( http://stats.wikimedia.org/wikimedia/squids/SquidReportClients.htm ).
JamesJames AlexanderLegal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:45 PM, ENWP Pine deyntestiss@hotmail.com wrote:
Thanks James.
No UA data is recorded from any platform for non-edit actions like pageviews and watchlisting, even if an editor is logged in, right?
Pine
From: jalexander@wikimedia.org Date: Thu, 27 Mar 2014 23:36:01 -0700
Subject: Re: [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) To: deyntestiss@hotmail.com CC: analytics@lists.wikimedia.org; mobile-l@lists.wikimedia.org
The mobile apps are definitely in the wild, my understanding (from some checks in the past and from the discussion in the other thread) was that it was just the 'normal' useragent for the phone with the addition of a WikipediaMobile identifier at the front (so that you could tell it was from the app). They aren't very common but I have seen some which I knew were app driven.
The reason this came up was that one of the discussions that came up in that other thread was to rewrite the user agent to be less unique which would make it harder for CUs to tell users apart (especially once they found out, which they often do and these discussions are on public lists). The compromise that Yuvi proposed was to send a 'special' user agent for checkuser so that it didn't lose the fingerprinting ability as much.
JamesJames AlexanderLegal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:27 PM, ENWP Pine deyntestiss@hotmail.com wrote:
I got the impression from this discussion that the mobile apps aren't currently in use so the CUs have had no experience working with them. It sounds like I was mistaken.
Toby, what UA data do CUs currently see from edits made through the mobile apps?
CUs, is the information that you're currently getting from edits from the new mobile apps a good balance the concerns raised previously in this discussion?
Pine
Date: Thu, 27 Mar 2014 15:36:16 -0700 From: Toby Negrin tnegrin@wikimedia.org To: "A mailing list for the Analytics Team at WMF and everybody who
has an interest in Wikipedia and analytics." analytics@lists.wikimedia.org Cc: mobile-l mobile-l@lists.wikimedia.org, Oliver Keyes
ironholds@gmail.com, Yuvi Panda yuvipanda@gmail.com Subject: Re: [Analytics] [WikimediaMobile] Mobile and CheckUser (Was
[mobile-l] Wikipedia App User Agents) Message-ID: CAAjh0EyqpFDke3P6Q5FxecOnE5yc7uSs_VS9HcB5khoxZ6-Yng@mail.gmail.com
Content-Type: text/plain; charset="iso-8859-1"
Hey folks -- we aren't considering changing any of the data that goes into checkuser. That tool will be unchanged.
This discussion only concerns backend logging EventLogging and page view analytics.
thanks,
-Toby
_______________________________________________
Mobile-l mailing list
Mobile-l@lists.wikimedia.org
The old apps have been in the wild for ages but they had no editing ability. The new apps have editing but aren't in the wild yet - they are only being alpha tested by about 100 users at best at the moment. Wider availability this coming month. Since this is the first time apps will have editing I wanted to get both CU and UA for analytics right. On Mar 28, 2014 12:16 PM, "ENWP Pine" deyntestiss@hotmail.com wrote:
Thanks James.
No UA data is recorded from any platform for non-edit actions like pageviews and watchlisting, even if an editor is logged in, right?
Pine
From: jalexander@wikimedia.org Date: Thu, 27 Mar 2014 23:36:01 -0700 Subject: Re: [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) To: deyntestiss@hotmail.com CC: analytics@lists.wikimedia.org; mobile-l@lists.wikimedia.org
The mobile apps are definitely in the wild, my understanding (from some checks in the past and from the discussion in the other thread) was that it was just the 'normal' useragent for the phone with the addition of a WikipediaMobile identifier at the front (so that you could tell it was from the app). They aren't very common but I have seen some which I knew were app driven.
The reason this came up was that one of the discussions that came up in that other thread was to rewrite the user agent to be less unique which would make it harder for CUs to tell users apart (especially once they found out, which they often do and these discussions are on public lists). The compromise that Yuvi proposed was to send a 'special' user agent for checkuser so that it didn't lose the fingerprinting ability as much.
James
James Alexander Legal and Community Advocacy Wikimedia Foundation (415) 839-6885 x6716 @jamesofur
On Thu, Mar 27, 2014 at 11:27 PM, ENWP Pine deyntestiss@hotmail.comwrote:
I got the impression from this discussion that the mobile apps aren't currently in use so the CUs have had no experience working with them. It sounds like I was mistaken.
Toby, what UA data do CUs currently see from edits made through the mobile apps?
CUs, is the information that you're currently getting from edits from the new mobile apps a good balance the concerns raised previously in this discussion?
Pine
Date: Thu, 27 Mar 2014 15:36:16 -0700 From: Toby Negrin tnegrin@wikimedia.org To: "A mailing list for the Analytics Team at WMF and everybody who has an interest in Wikipedia and analytics." analytics@lists.wikimedia.org Cc: mobile-l mobile-l@lists.wikimedia.org, Oliver Keyes ironholds@gmail.com, Yuvi Panda yuvipanda@gmail.com Subject: Re: [Analytics] [WikimediaMobile] Mobile and CheckUser (Was [mobile-l] Wikipedia App User Agents) Message-ID: CAAjh0EyqpFDke3P6Q5FxecOnE5yc7uSs_VS9HcB5khoxZ6-Yng@mail.gmail.com Content-Type: text/plain; charset="iso-8859-1"
Hey folks -- we aren't considering changing any of the data that goes
into
checkuser. That tool will be unchanged.
This discussion only concerns backend logging EventLogging and page view analytics.
thanks,
-Toby
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Mobile-l mailing list Mobile-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/mobile-l
Hi Pine,
On Thu, Mar 27, 2014 at 11:45:59PM -0700, ENWP Pine wrote:
No UA data is recorded from any platform for non-edit actions like pageviews and watchlisting, even if an editor is logged in, right?
no, that is not correct.
Currently, for each request [1] to the text and mobile caches udp2log holds [2] the User-Agent (column 14), and URL (column 9). This data gets stored away into files (some parts sampled, some parts unsampled). I and some others have access to this data.
Same for kafka and mobile caches. There it is always unsampled.
You could do all kinds of bad™ things with those data sets, if you wanted.
I do see that Ops might need that data. They should have it.
But I hope that my access to this /raw/ data, and access of other fellow Analytics team members to this raw data get's killed in the foreseeable future, and that we effectively make it impossible to fingerprint/track people around.
Best regards, Christian
[1] Regardless of whether it is edit or non-edit. Regardless of the action. Regardless of the platform.
[2] https://wikitech.wikimedia.org/wiki/Cache_log_format