Wikitech-l September 2016

wikitech-l@lists.wikimedia.org

105 participants
116 discussions

Fwd: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning
by Pine W 03 Apr '17

03 Apr '17

Forwarding. Pine ---------- Forwarded message ---------- From: "Jordan Adler" <jmadler(a)google.com> Date: Aug 11, 2016 13:06 Subject: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning To: "commons-l(a)wikimedia.org" <commons-l(a)lists.wikimedia.org> Cc: "Ray Sakai" <rsakai(a)reactive.co.jp>, "Ram Ramanathan" < ramramanathan(a)google.com>, "Kazunori Sato" <kazsato(a)google.com> Hey folks! A few months back a colleague of mine was looking for some unstructured images to analyze as part of a demo for the Google Cloud Vision API <https://cloud.google.com/blog/big-data/2016/05/explore-the-galaxy-of-images…>. Luckily, I knew just the place <https://commons.wikimedia.org/wiki/Category:Media_needing_categories>, and the resulting demo <http://vision-explorer.reactive.ai/>, built by Reactive Inc., is pretty awesome. It was shared on-stage by Jeff Dean during the keynote <https://www.youtube.com/watch?v=HgWHeT_OwHc&feature=youtu.be&t=2h1m19s> at GCP NEXT 2016. I wanted to quickly share the data from the programmatically identified images so it could be used to help categorize the media in the Commons. There's about 80,000 images worth of data: - map.txt <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/map.txt> (5.9MB): A single text file mapping id to filename in a "id : filename" format, one per line - results.tar.gz <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/results.tar.…> (29.6MB): a tgz'd directory of json files representing the output of the API <https://cloud.google.com/vision/reference/rest/v1/images/annotate#response-…>, in the format "${id}.jpg.json" We're making this data available under the CC0 license, and these links will likely be live for at least a few weeks. If you're interested in working with the Cloud Vision API to tag other images in the Commons, talk to the WMF Community Tech team. Thanks for your help! _______________________________________________ Commons-l mailing list Commons-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l

2 1

Introduction for Gsoc 2017
by MAYANK JINDAL 10 Dec '16

10 Dec '16

Hi, My name is Mayank Jindal. I am third year undergraduate student currently studying at Indian Institute of Technology, Kharagpur. I want to take part in Gsoc-2017 from Wikimedia. I have knowledge of C, C++, JAVA, Python, Android app development and Web development and beginner in *Machine Learning, Artificial Intelligence.* I am very enthusiastic to learn new skills which would be required. Kindly guide me to proceed further. -- Kind Regards, Mayank Jindal, Third year undergraduate student, Indian Institute of Technology Kharagpur Mobile : +91- 7076670299 || 8875432718

4 5

Deploying the UrlShortener extension to Wikimedia wikis
by Legoktm 12 Nov '16

12 Nov '16

Hi, YuviPanda, prtksxna, and myself (with help from Tim and Aaron) have been working the UrlShortener extension, which is designed to implement the URL shortener RfC[1] (specifically Tim's implementation suggestion). I've filed T108557[2] to deploy the extension to Wikimedia wikis. We'd like to use the "w.wiki" short domain, which the WMF is already in control of. A test wiki has been set up mimicking what Wikimedia's configuration would be like: http://urlshortener.wmflabs.org/, and has an accompanying "short" domain at us.wmflabs.org (e.g. http://us.wmflabs.org/3). Please play with it and report any bugs you might find :) [1] https://www.mediawiki.org/wiki/Requests_for_comment/URL_shortener [2] https://phabricator.wikimedia.org/T108557 Thanks, -- Legoktm

3 5

Looking for WikiDev Summit main topics and volunteers
by Quim Gil 20 Oct '16

20 Oct '16

Hi, this is a request from the organizers of the Wikimedia Developer Summit 2016 (San Francisco, January 9-11). We are looking for candidates to become main topics of the next Summit. Ideally complex topics with a high user impact (direct or indirect) and ramifications in multiple technical areas. Deciding a few main topics beforehand will help us inviting the people needing to be involved, especially non-WMF contributors requiring travel sponsorship. We are also looking for volunteers who want to get involved in the organization of the Summit, or want to provide feedback to improve our plans. If you have proposals for main topics and/or want to volunteer, please reply. Context: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit_2017 & https://lists.wikimedia.org/pipermail/wikitech-l/2016-September/086476.html -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

5 7

Public Event Streams (AKA RCStream replacement) question
by Andrew Otto 20 Oct '16

20 Oct '16

Hi all, We’ve been busy working on building a replacement for RCStream. This new service would expose recentchanges as a stream as usual, but also other types of event streams that we can make public. But we’re having a bit of an existential crisis! We had originally chosen to implement this using an up to date socket.io server, as RCStream also uses socket.io. We’re mostly finished with this, but now we are taking a step back and wondering if socket.io/websockets are the best technology to use to expose stream data these days. The alternative is to just use ‘streaming’ HTTP chunked transfer encoding. That is, the client makes a HTTP request for a stream, and the server declares that it will be sending back data indefinitely in the response body. Clients just read (and parse) events out of the HTTP response body. There is some event tooling built on top of this (namely SSE / EventSource), but the basic idea is a never ending streamed HTTP response body. So, I’m reaching out to to gather some input to help inform a decision. What will be easier for you users of RCStream in the future? Would you prefer to keep using socket.io (newer version), or would you prefer to work directly with HTTP? There seem to be good clients for socket.io and for SSE/EventSource in many languages. https://phabricator.wikimedia.org/T130651 has more context, but don’t worry about reading it; it is getting a little long. Feel free to chime in there or on this thread. Thanks! -Andrew Otto

8 15

Need review on ContentHandler implementation, Newsletter extension
by Tony Thomas 18 Oct '16

18 Oct '16

Hello all, We have a patch https://gerrit.wikimedia.org/r/#/c/295670/, which had its last review almost 28 days back, and is a major blocker for the deployment of Newsletter extension in production. The shift is tracked at T138462[1]. The patch is bit lengthy, and enable the extension to use ContentHandler, which help us use a lot of in-wiki features. I had pulled the change to the labs wiki at http://newsletter-test.wmflabs.org/, which is broken as of now, tracked in T138686[2]. It would be great if you devs can take a look at both the labs instance and the contenthandler change. [1] https://phabricator.wikimedia.org/T138462 [1] https://phabricator.wikimedia.org/T138686 Thanks, Tony Thomas <https://www.mediawiki.org/wiki/User:01tonythomas> Home <http://www.thomastony.me> | Blog <http://blog.thomastony.me> | ThinkFOSS <http://www.thinkfoss.com>

2 3

[Node.JS] Deprecation of support for Node 0.1x
by Marko Obrovac 04 Oct '16

04 Oct '16

Hello, The Wikimedia Services team~[1] is strongly considering dropping the support for Node 0.1x~[2], which would soon impact all MediaWiki installations that use RESTBase~[3]. Eventually, the support for 0.1x would also be dropped for Parsoid~[4]. We want to know if making such a move would not be favourable to administrators running such installations. If this impacts you, please let us know on the task~[2] so we can discuss in detail the time-lines and possible solutions. Note that there are already services that require Node 4 or higher, such as Change Propagation~[5], Mathoid~[6] and the Mobile Content Service~[7]. If you are running any of these in your production environment, you are not affected by the outcome of this discussion in any way. Cheers, Marko [1] https://www.mediawiki.org/wiki/Wikimedia_Services [2] https://phabricator.wikimedia.org/T134200 [3] https://www.mediawiki.org/wiki/RESTBase [4] https://www.mediawiki.org/wiki/Parsoid [5] https://www.mediawiki.org/wiki/Change_propagation [6] https://www.mediawiki.org/wiki/Mathoid [7] https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/RESTBase_services_for_ap… -- Marko Obrovac, PhD Senior Services Engineer Wikimedia Foundation

1 1

Recently proposed patchsets by new contributors awaiting code review
by Andre Klapper 04 Oct '16

04 Oct '16

Your help is welcome to provide feedback and guidance: == in "mediawiki/tools/mwdumper": == since 2016-09-08: Major refactoring https://gerrit.wikimedia.org/r/#/c/309314/ Thanks in advance for your reviews. Of last weeks' 5 listed patches, 4 got merged & 1 got reviewed. Thanks to Amire80, Daniel, FlorianSW, Jdlrobson, MatmaRex, Smalyshev, Tjones! andre -- Andre Klapper | Wikimedia Bugwrangler http://blogs.gnome.org/aklapper/

3 4

[RFC] Giving actual CSRF tokens to not logged in users (T40417)
by Brian Wolff 30 Sep '16

30 Sep '16

Hi everyone: Currently, the CSRF token for anonymous users are very predictable. This potentially allows someone to make CSRF attacks against non-logged in users. I would like to propose we change that. Since this is a sort of major change, I'd appreciate everyone's feedback. There are multiple proposals on the bug - https://phabricator.wikimedia.org/T40417#2612673 . Its unclear which solution we should chose. Personally, my preferred solution [0] [I might be biased in evaluating them] would be to base the CSRF token on a session cookie if one exists. If one does not exist, use a HMAC of the users IP addressed, keyed using a server side secret (The only state an Anon has is what IP address it is, so this should be safe). This way it will work for users without cookies (Maybe none exist, but I like the idea you can edit wikipedia without cookies) and for users who have rapidly changing IPs. It will also have minimal breakage, as you won't have to adjust any existing usages of tokens (For example, on special pages). It also ensures that users are not forced to skip varnish cache (from session cookie) unless they really need to. Anyways, I'd appreciate everyone's comments. I would really like to settle on a specific implementation, and then go do it. [0] https://phabricator.wikimedia.org/T40417#2034118 -- Bawolff

5 5

Call for participation: Wikimedia Developer Summit 2017
by Quim Gil 30 Sep '16

30 Sep '16

Hi, the call for participation for the Wikimedia Developer Summit 2017 is now open: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/Call_for_particip… We welcome especially proposals related to these main topics: * A plan for the Community Wishlist 2016 top results * Handling wiki content beyond plaintext * A unified vision for editorial collaboration * Building a sustainable user experience together * Useful, consistent, and well documented APIs * How to manage our technical debt * Artificial Intelligence to build and navigate content * How to grow our technical community If you want to propose an activity pre-scheduled in the Summit program, you have time until Monday, October 31. There is no deadline to propose Unconference sessions. ABOUT The Wikimedia Developer Summit is the annual meeting to push the evolution of MediaWiki and other technologies supporting the Wikimedia movement. We welcome all Wikimedia technical contributors and third party developers using the Wikimedia APIs or MediaWiki. https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit (This information wants to be forwarded!) -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

1 0

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l September 2016