stats.grok.se used in study about Snowden and internet traffic

List overview All Threads
Download

newer

older

EventLogging RL modules can take a...

Share your experience and feedback...

Andrew Otto

18 Jan 2017 18 Jan '17

6 p.m.

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass-surveillance-breed…

...

From the paper

<https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao

Attachments:

attachment.htm (text/html — 1.5 KB)

Show replies by thread

Leila Zia

18 Jan 18 Jan

6:56 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

+ Juliet, as this is something Communications may want to follow up given that stats.groke.se is not maintained by a Wikimedia Foundation member. Thanks for sharing this. Leila Leila Zia Senior Research Scientist Wikimedia Foundation On Wed, Jan 18, 2017 at 9:00 AM, Andrew Otto <acotto(a)gmail.com> wrote:

...

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass- surveillance-breeds-meekness-fear-and-self-censorship/ From the paper <https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Lodewijk

6:59 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

For the record/archive: The link is http://stats.grok.se/ but the tool is no longer maintained, it seems (internal server error when entering an article name). Interesting nonetheless. Lodewijk 2017-01-18 18:56 GMT+01:00 Leila Zia <leila(a)wikimedia.org>rg>:

...

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass-sur veillance-breeds-meekness-fear-and-self-censorship/ From the paper <https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Dan Andreescu

19 Jan 19 Jan

8:09 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

Very interesting, I hope legal reads this. And I hope comms figures out a way to counter-act the public opinion that Wikipedia traffic is monitored by the government. Ops goes to such lengths to make sure that's not possible, they should get credit for that (and apparently the blog posts about the https switch are not enough). On a technical note: we should revisit the idea of backfilling old data into AQS so people can do this type of research project on top of the pageview API, now that stats.grok is completely down. On Wed, Jan 18, 2017 at 12:59 PM, Lodewijk <lodewijk(a)effeietsanders.org> wrote:

...

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass-sur veillance-breeds-meekness-fear-and-self-censorship/ From the paper <https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Jonathan Morgan

8:12 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

On Thu, Jan 19, 2017 at 11:09 AM, Dan Andreescu <dandreescu(a)wikimedia.org> wrote:

...

On a technical note: we should revisit the idea of backfilling old data into AQS so people can do this type of research project on top of the pageview API, now that stats.grok is completely down.

I would love to have historical pageview data through the pageview API, and I know many other researchers and educators who would appreciate this addition immensely. I'd be happy to help with publicizing/championing this change; let me know if I can help in any way. - J

...

On Wed, Jan 18, 2017 at 12:59 PM, Lodewijk <lodewijk(a)effeietsanders.org> wrote:

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass-sur veillance-breeds-meekness-fear-and-self-censorship/ From the paper <https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

-- Jonathan T. Morgan Senior Design Researcher Wikimedia Foundation User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)>

Federico Leva (Nemo)

9:47 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

Dan Andreescu, 19/01/2017 20:09:

...

now that stats.grok is completely down.

Lodewijk

11:39 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

Thanks for clarification. I created a ticket T155785 <https://phabricator.wikimedia.org/T155785> to disable 'impossible' queries and explain this on the tool page. Not sure who could do that, so if someone could add that person as subscriber and add the right tag, that'd be great. Thanks, Lodewijk 2017-01-19 21:47 GMT+01:00 Federico Leva (Nemo) <nemowiki(a)gmail.com>om>:

...

Dan Andreescu, 19/01/2017 20:09:

now that stats.grok is completely down.

It's not, AFAICT: http://stats.grok.se/en/200712/Britney_Spears Only the new data is missing (since January 2016), as stated on the FAQ < https://en.wikipedia.org/wiki/User:Killiondude/stats#Are_ there_known_dates_for_which_complete_sets_have_not_been_ compiled_although_the_data_seems_to_be_available> Nemo _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Dan Andreescu

11:42 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

Federico Leva (Nemo)

20 Jan 20 Jan

12:01 a.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

Dan Andreescu, 19/01/2017 23:42:

...

there are no ways to know if data is missing or there are actual gaps.

These were documented on the FAQ though, also based on Erik Zachte's analysis. Nemo

Dan Andreescu

1:49 a.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

Dan Andreescu, 19/01/2017 23:42:

...

there are no ways to know if data is missing or there are actual gaps.

These were documented on the FAQ though, also based on Erik Zachte's analysis. Erik's analysis I trust, but the FAQ wasn't maintained either so I wouldn't go by it.

Jeremy Baron

7:22 a.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

On Jan 19, 2017 14:09, "Dan Andreescu" <dandreescu(a)wikimedia.org> wrote:

...

figures out a way to counter-act the public opinion that Wikipedia traffic is monitored by the government. Ops goes to such lengths to make sure that's not possible, they should get credit for that (and apparently the blog posts about the https switch are not enough).

I hope they don't go too far with that, AFAIK we're still quite vulnerable to traffic analysis by a passive listener. (no MITM, and notwithstanding any vulnerability mentioned elsewhere on this thread (because we're not using quic?)) and we leak SNI too. see threads: Zack started a few different threads on ~ 2013-08-16: https://lists.wikimedia.org/pipermail/wikitech-l/2013-August/071262.html "take two" (2014-06-05): https://lists.wikimedia.org/pipermail/wikitech-l/2014-June/076876.html

...

On a technical note: we should revisit the idea of backfilling old data into AQS so people can do this type of research project on top of the pageview API,

I would use it. :-) -Jeremy

Alex Druk

9:27 a.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

It seems to me that decline in terrorism related pageviews started long before Snowden revelations (2011-2012?). See enclosed graph from wikipediatrends.com. [image: Inline image 1] On Wed, Jan 18, 2017 at 6:00 PM, Andrew Otto <acotto(a)gmail.com> wrote:

...

-- Thank you. Alex Druk alex.druk(a)gmail.com (775) 237-8550 Google voice

Leon Ziemba

11:17 a.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

stats.grok.se is advertised as being maintained by User:Henrik <https://en.wikipedia.org/wiki/User:Henrik>, who has been inactive since 2014. I have my doubts any requests to update the tool will get any where. Anyway, I am very pleased to see there more interest in backfilling historical data into AQS. Access to this data is by far the most common request I've gotten in working on the Pageviews Analysis tools. I, too, am happy to help with this effort in any way! :) ~Leon On Fri, Jan 20, 2017 at 3:27 AM, Alex Druk <alex.druk(a)gmail.com> wrote:

...

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass-sur veillance-breeds-meekness-fear-and-self-censorship/ From the paper <https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

-- Thank you. Alex Druk alex.druk(a)gmail.com (775) 237-8550 Google voice _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

Avner Kantor

22 Jan 22 Jan

6:52 p.m.

New subject: stats.grok.se used in study about Snowden and internet traffic

I wrote a small R application that helps working with stats.grok.se api . It works with every language (unlike wikipediatrends.com). [1] https://avnerkantor.shinyapps.io/WikipediaTrends/ [2] https://github.com/avnerkantor/WikipediaTrendsR/ Avner On Fri, Jan 20, 2017 at 12:17 PM, Leon Ziemba <musikanimal(a)wikimedia.org> wrote:

...

Saw this on reddit: https://theintercept.com/2016/04/28/new-study-shows-mass-sur veillance-breeds-meekness-fear-and-self-censorship/ From the paper <https://poseidon01.ssrn.com/delivery.php?ID=753074071124064067015124108126089006010045004048003005075122091087089064095002083029126119017036023013055086074065017070067091045045047076049103090096065089019029088069014094126005065070066069027097007119094008090029020087004112003090106003067003084&EXT=pdf> : "This case study uses data on English language Wikipedia article view counts from the online service stats.grok.se, a portal maintained by a Wikimedia Foundation member. This portal provides access to a range of Wikipedia analytics, stats, and data.86 In particular, the portal aggregates Wikipedia article view data on a daily and monthly basis.87 This data at stats.grok.se has been used in a range of research, including studies involving market trends, health information access, and social-political change.88” Just thought it might be of interest, especially considering WMF’s NSA lawsuit <https://blog.wikimedia.org/2015/03/10/wikimedia-v-nsa/>. -Ao _______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

_______________________________________________ Analytics mailing list Analytics(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics

2644

days inactive

2648

days old

analytics@lists.wikimedia.org

Manage subscription

13 comments

10 participants

tags (0)

participants (10)

Alex Druk
Andrew Otto
Avner Kantor
Dan Andreescu
Federico Leva (Nemo)
Jeremy Baron
Jonathan Morgan
Leila Zia
Leon Ziemba
Lodewijk