Wikitech-l April 2017

wikitech-l@lists.wikimedia.org

81 participants
87 discussions

Fwd: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning
by Pine W 03 Apr '17

03 Apr '17

Forwarding. Pine ---------- Forwarded message ---------- From: "Jordan Adler" <jmadler(a)google.com> Date: Aug 11, 2016 13:06 Subject: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning To: "commons-l(a)wikimedia.org" <commons-l(a)lists.wikimedia.org> Cc: "Ray Sakai" <rsakai(a)reactive.co.jp>, "Ram Ramanathan" < ramramanathan(a)google.com>, "Kazunori Sato" <kazsato(a)google.com> Hey folks! A few months back a colleague of mine was looking for some unstructured images to analyze as part of a demo for the Google Cloud Vision API <https://cloud.google.com/blog/big-data/2016/05/explore-the-galaxy-of-images…>. Luckily, I knew just the place <https://commons.wikimedia.org/wiki/Category:Media_needing_categories>, and the resulting demo <http://vision-explorer.reactive.ai/>, built by Reactive Inc., is pretty awesome. It was shared on-stage by Jeff Dean during the keynote <https://www.youtube.com/watch?v=HgWHeT_OwHc&feature=youtu.be&t=2h1m19s> at GCP NEXT 2016. I wanted to quickly share the data from the programmatically identified images so it could be used to help categorize the media in the Commons. There's about 80,000 images worth of data: - map.txt <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/map.txt> (5.9MB): A single text file mapping id to filename in a "id : filename" format, one per line - results.tar.gz <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/results.tar.…> (29.6MB): a tgz'd directory of json files representing the output of the API <https://cloud.google.com/vision/reference/rest/v1/images/annotate#response-…>, in the format "${id}.jpg.json" We're making this data available under the CC0 license, and these links will likely be live for at least a few weeks. If you're interested in working with the Cloud Vision API to tag other images in the Commons, talk to the WMF Community Tech team. Thanks for your help! _______________________________________________ Commons-l mailing list Commons-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l

2 1

CirrusSearch forceSearchIndex.php issue
by Aran 03 Apr '17

03 Apr '17

Hello, I've found that running forceSearchIndex.php with --skipLinks --indexOnSkip options is only processing about 1000-1500 pages and then stopping. It's not due to error or anything because I can set --fromId and run it again and it does another batch. I've tried setting --limit and --batchSize and all sorts but nothing allows it to do more than this amount at a time.... anyone have any idea what might be going on here? (It's happening on both MW 1.27 with Elastic Search 1.75 and MW 1.28 with elastic 2.4.4) Thanks, Aran

2 1

Phabricator monthly statistics - 2017-03
by communitymetrics＠wikimedia.org 03 Apr '17

03 Apr '17

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2017-03): 332 Active users (any activity) in (2017-03): 891 Task authors in (2017-03): 503 Users who have closed tasks in (2017-03): 274 Projects which had at least one task moved from one column to another on their workboard in (2017-03): 294 Tasks created in (2017-03): 2645 Tasks closed in (2017-03): 2417 Open and stalled tasks in total: 34232 Median age in days of open tasks by priority: Unbreak now: 0 Needs Triage: 262 High: 443 Normal: 638 Low: 909 Lowest: 844 (How long tasks have been open, not how long they have had that priority) TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on iridium at Sat Apr 1 00:00:14 UTC 2017)

3 3

Selenium tests in Node.js
by Željko Filipin 03 Apr '17

03 Apr '17

# TL;DR You can now write Selenium tests in Node.js! Learn more about it at https://www.mediawiki.org/wiki/Selenium/Node.js # Introduction Five years ago we introduced browser tests using Selenium and a Ruby based stack. It has worked great for some teams, and not so great for others. Last year we talked to people from several teams and ran a survey[0]. The outcome is a preference toward using a language developers are familiar with: JavaScript/Node.Js. After several months of research and development, we are proud to announce support for writing tests in Node.js. We have decided to use WebdriverIO[1]. It is already available in MediaWiki core and supports running tests for extensions. You can give it a try in MediaWiki-Vagrant[2]: vagrant up vagrant ssh sudo apt-get install chromedriver export PATH=$PATH:/usr/lib/chromium cd /vagrant/mediawiki xvfb-run npm run selenium # Documentation Extensive details are available on the landing page: https://www.mediawiki.org/wiki/Selenium/Node.js # Future We plan to replace the majority of Selenium tests written in Ruby with tests in Node.js in the next 6 months. We can not force anybody to rewrite existing tests, but we will offer documentation and pairing sessions for teams that need help. After 6 months, teams that want to continue using Ruby framework will be able to do so, but without support from Release Engineering team. I have submitted a skill share session for Wikimedia Hackathon 2017 in Vienna[3]. If you would like to pair on Selenium tests in person, that would be a great time. The list of short term actions is in task T139740[4]. # Thanks I would like to thank several people for reviews, advice and code: Jean-Rene Branaa, Dan Duvall, Antoine Musso, Jon Robson, Timo Tijhof. (Names are sorted alphabetically by last name. Apologies to people I have forgot.) # References [0] https://www.mediawiki.org/wiki/Browser_testing_user_satisfaction_survey [1] http://webdriver.io/ [2] https://www.mediawiki.org/wiki/Selenium/Node.js/Inside_MediaWiki-Vagrant [3] https://phabricator.wikimedia.org/T159945 [4] https://phabricator.wikimedia.org/T139740 Željko Filipin

1 0

GSOC project for multiple watch lists - looking for mentors
by Techman224 01 Apr '17

01 Apr '17

Hello, Sorry if this is cutting it close to the deadline, but I’m interested in doing a GSOC project over the summer to implement multiple watchlists for users. What I have in mind is first, deliver a minimum viable product where users can add/remove/change custom watchlists. If time permits, it can be integrated into other components, such as notifications, a way to add pages into custom watch lists from the page itself, the API etc. I’m looking for potential mentors. In particular, someone who knows how to get around Mediawiki’s codebase and can help me guide coding and development. If there is any interest let me know! Thanks, Techman224 Task: https://phabricator.wikimedia.org/T3492

1 0

ArchCom Minutes 2017-03-29
by Daniel Kinzler 01 Apr '17

01 Apr '17

Hi all! Here are the minutes from this week's ArchCom meeting. You can also find the minutes at <https://www.mediawiki.org/wiki/Architecture_committee/2017-03-29>. See also the ArchCom status page at <https://www.mediawiki.org/wiki/Architecture_committee/Status> and the RFC board <https://phabricator.wikimedia.org/tag/mediawiki-rfcs/>. Here are the minutes, for your convenience: On the ArchCom’s Mind: * REST API 1.0 to be announced as stable! <https://www.mediawiki.org/wiki/REST_API> * The services team updated their requirements document for the container pipeline: <https://docs.google.com/document/d/1UNl3P4Ky_K_qu3Md700xeDMEyThF07jXqr1eIH1…> * The proposal to get rid of LocalizationUpdate seems to have been replace by a drive to rewrite it to integrate with git more closely, see the discussion on <https://phabricator.wikimedia.org/T158360> * Ladsgroup is working on using Redis as a distributed lock managing mechanism for Wikidata change dispatching, see https://phabricator.wikimedia.org/T151993 and <https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/docs/change-…> * On a related note, Services team is discussing replacing Redis with a Kafka based solution for some use cases. Ask GWicke for details. There is a related discussion about replacing the JobQueue with changeprop and EventBus. (note: Wikidata change dispatching is not (yet) “changeprop”). See <https://phabricator.wikimedia.org/T149408> and <https://phabricator.wikimedia.org/T157088> * As the new MediaWiki Platform team is taking shape, it has taken ownership of some previously orphaned parts of the MediaWiki code base, see <https://www.mediawiki.org/w/index.php?diff=2428275&oldid=2382496&title=Deve…> and <https://www.mediawiki.org/wiki/MediaWiki_Platform_team> Input wanted: * Input wanted on “Provide standard-compatible way to load multi-file packages” <https://phabricator.wikimedia.org/T133462> * The idea of building a standalone password verification service is being revived, input wanted: <https://phabricator.wikimedia.org/T120484> RFC discussions: * We discussed “Canonical data URIs and URLs for machine readable page content” <https://phabricator.wikimedia.org/T161527> on Wednesday. There was rough consensus for URLs of the form https://commons.wikimedia.org/data/Data:Avignon_City_Wall.map redirecting to action=raw URLs. Full log: <https://tools.wmflabs.org/meetbot/wikimedia-office/2017/wikimedia-office.20…>. Daniel will refine the writeup and then announce the RFC for a Last Call. If no new pertinent concerns are raised during the last call period, the RFC will be approved an April 12. * There is no public RFC discussion scheduled for next Wednesday, April 5. -- Daniel Kinzler Principal Platform Engineer Wikimedia Deutschland Gesellschaft zur Förderung Freien Wissens e.V.

1 0

Draft Proposal : Adding Data storage feature and upgrading Quiz extension - Gsoc17
by Harjot Bhatia 01 Apr '17

01 Apr '17

Hi, I'd like to share my Project proposal for Gsoc17 Draft Proposal Link: https://docs.google.com/document/d/1P-R68dlBrKaWrYZWts8twIztOj5mHjKJ8BsDFkQ… Phabricator Link : https://phabricator.wikimedia.org/T160257 I'd appreciate comments, feedback and suggestions. Thanks, Harjot Singh

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l April 2017