Wikitech-l April 2017

wikitech-l@lists.wikimedia.org

81 participants
82 discussions

2017-04-05 Scrum of Scrums meeting notes
by Grace Gellerman 05 Apr '17

05 Apr '17

https://www.mediawiki.org/wiki/Scrum_of_scrums/2017-04-05 = 2017-04-05 = contact: https://www.mediawiki.org/wiki/Wikimedia_Engineering == Call outs == * Analytics: Planning wikistats 2.0 prototype consultation - Please share your thoughts on the UI prototype https://analytics-prototype.wmflabs.org (emails with where to share soon) * Parsoid drops node 0.10 support -- announcement and details coming this week. * ParserMigration extension now live on all wikis -- announcement and details coming next week. * Services: New version of the PDF render service still awaits checking from WMDE *** https://en.wikipedia.beta.wmflabs.org/api/rest_v1/#!/Page_content/get_page_… *** https://phabricator.wikimedia.org/T160764 * RelEng: May need Ops help deploying 3d2png via scap3 (worked on Beta Cluster), need some +2 in puppet * FR-tech's new job runner / cron complement script to replace our Jenkins instance is working like a charm. Anyone else want to use it for scheduled jobs? ** https://github.com/wikimedia/wikimedia-fundraising-process-control == Product == === Reading === ==== Web ==== * Last week: ** Reading Off-site * Next week: ** Deploy PagePreviews to Hungarian and Hebrew wikipedias. ** PagePreviews instrumentation related bugfixes. ** Improved print styles for MobileFrontend ==== Android ==== * Last week: ** Reading Off-site. ** A nasty little crash in production for logged-in users caused by a small, unexpected change to the userjs response format. () * This week / early next week: a new release to beta and production thereafter, including the following: ** A substantially better offline experience! (our Q3 goal: https://phabricator.wikimedia.org/tag/android-app-goals-2017-jan-mar-q3/) ** Wikidata description editing expanded from three languages to all except the top 10 by page views as measured in the app. * Kanban: https://phabricator.wikimedia.org/project/view/2352/ ====Reading Infrastructure==== * TemplateStyles: ** CSS sanitizer library https://gerrit.wikimedia.org/r/#/q/project:css-sanitizer ** will be taken over by MediaWiki Platform * Mobile Content Service: https://phabricator.wikimedia.org/project/view/2445/ ** Deployed: Added 13 more languages to the featured article. ==== iOS ==== * Last Week ** Reading Offsite * This week ** 5.4.1 - https://phabricator.wikimedia.org/project/view/2600/ *** Monitor crash reports, fix major issues ** 5.5 - https://phabricator.wikimedia.org/project/view/2602/ *** Places *** JavaScript consolidation with Android *** Image caching fixes ==== Community Tech ==== * Cookie blocks deployed to English Wikipedia, more wikis to follow * LoginNotify in testing on beta cluster * CodeMirror bug fixes, colour tweaks * Various improvements to the new XTools, nothing to show * No blockers === Editing === ==== Parsing ==== * ParserMigration extension has now been deployed on all wikis. You can enable it on the Editing tab of your Special:Preferences page. Clicking it will add a sidebar link called "Edit with migration tool". This opens a page for editing that shows you a side-by-side preview of the page with the default (production) parser and an experimental / proposed-migration of the parser. For example, currently, this lets you compare output with Tidy and output with RemexHTML (a PHP-only replacement for Tidy that is based on HTML5 semantics). You should also be able to directly exercise it via https://en.wikipedia.org/w/index.php?title=PAGE-TITLE&action=parsermigratio…. For example, https://en.wikipedia.org/w/index.php?title=Typhoon_Parma&action=parsermigra… shows that the page renders differently with RemexHTML ( because of https://phabricator.wikimedia.org/T161341#3129283 ). We will send out a note on mailing lists (and possibly other forums like Tech News) next week about this and about how to leverage the Linter extension and Visual Diffing results for fixing wiki pages * Filed https://phabricator.wikimedia.org/T162275 to make visual diffing output more useful for editors for fixing up pages. This can help us track progress of wikitext and template fixup and tell us when it is safe to flip the switch on Tidy. * We are continuing to work on https://www.mediawiki.org/wiki/Parsoid/Language_conversion/Preprocessor_fix… to document impact of cleanup to the preprocessor to remove some edge cases ( https://gerrit.wikimedia.org/r/#/c/333997/ ). We'll use the results of this investigation to figure out followup work. * We released Parsoid debian 0.7.0 package y'day. This release drops support for node 0.10x and requires node v4x. However, we forgot to update the dependencies in the control file which still states a >= v0.8.0 requirement. Will release a 0.7.1 deb that fixes this. * In November, we announced a deprecation of node 0.10 for Parsoid ( https://lists.wikimedia.org/pipermail/wikitech-l/2016-November/087005.html ), Going forward, node 0.10 is no longer supported in Parsoid. * Bumping service-runner in Parsoid as requested by Services team for the upcoming DC switchover this month. ==== Collaboration ==== * RCFilters beta feature went live on plwiki, ptwiki last week; next release to trwiki, fawiki, ruwiki and frwiki next week (Apr 11); then enwiki + most non-ORES wikis on Apr 24; and all remaining wikis May 9 * Working with Aaron H on figuring out ORES behavior for RCFilters for the next releases ==== Language ==== * Work on OOjs UI migration continue. == Technology == === Analytics === * Reportcard migrated to new infra - Limn is dead * new AQS metrics (historical projectcounts) have a bug - still vetting before announcing * user agent map in EventLogging replaces user agent string (fully deployed) * All hadoop workers have been reimaged to jessie - masters left (next week) * Planning wikistats 2.0 prototype consultation ( https://analytics-prototype.wmflabs.org") * Eventlogging MariaDB custom replication procedure has been updated (should avoid long delays between master and slave) * db1047 is to be decommissioned - https://phabricator.wikimedia.org/T156844#3107357 === Security === * Reviews: ** iOS 2-factor wrapping up ** Ex:WikibaseMediaInfo ** Ex:CodeMirror === Services === * Blockers: none * Updates: ** New version of the PDF render service still awaits checking from WMDE *** https://en.wikipedia.beta.wmflabs.org/api/rest_v1/#!/Page_content/get_page_… *** https://phabricator.wikimedia.org/T160764 ** All SCB services have been redeployed in preparation to DC switchover to pick up short-term DNS caching === RelEng === *Blockers: ** May need Ops help deploying 3d2png via scap3 (worked on Beta Cluster), need some +2 in puppet === Discovery === * No blockers * Discussing strategy for enabling live updates for completion suggester - probably starting with new pages * New Polish analyzers deployed, Ukrainian is next, working on Chinese * Working on Wikidata search improvement * Working on adding non-exact search capabilities to Special:Undelete * Working on improving noop detection plugins for Elasticsearch * Preparing for active-active switch === Fundraising Tech === * Moved all our scheduled jobs off Jenkins with surprisingly little complication: https://phabricator.wikimedia.org/T161569 ** Anybody need a job scheduler / cron wrapper script? ** yaml job description files with command, timeout, environment vars ** Ensures jobs run as a specified user ** non-ops can be given the ability to run defined jobs on demand ** can generate cron.d entries automatically ** See more here: https://github.com/wikimedia/wikimedia-fundraising-process-control * More work on Central Notice banner sequence, and general admin UI refactor: https://phabricator.wikimedia.org/T144453

1 0

[Reminder] CREDIT showcase, Wednesday, 5-April-2017
by Lani Goto 05 Apr '17

05 Apr '17

Just a reminder, CREDIT is starting in 30 minutes! ---------- Forwarded message ---------- From: Lani Goto <lgoto(a)wikimedia.org> Date: Thu, Mar 30, 2017 at 5:24 PM Subject: CREDIT showcase, Wednesday, 5-April-2017 To: wikitech-l(a)lists.wikimedia.org Hi! I'd like to welcome you to join us at the CREDIT showcase next week, Wednesday, 5-April-2017 at 1800 UTC / 1100 Pacific Time. We'd like to see your demos, whether they're rough works in progress or polished production material, or even just a telling of something you've been studying recently. For more information on the upcoming event, as well as recordings of previous events, please visit the following page: https://www.mediawiki.org/wiki/CREDIT_showcase And if you'd like to share the news about the upcoming CREDIT showcase, here's some suggested verbiage. Thanks! *Hi <FNAME>* *I hope all is well with you! I wanted to let you know about CREDIT, a monthly demo series that we’re running to showcase open source tech projects from Wikimedia’s Community, Reading, Editing, Discovery, Infrastructure and Technology teams. * *CREDIT is open to the public, and we welcome questions and discussion. The next CREDIT will be held on April 5th at 11am PT / 2pm ET / 18:00 UTC. * *There’s more info on MediaWiki, and on Etherpad, which is where we take notes and ask questions. You can also ask questions on IRC in the Freenode chatroom #wikimedia-office (web-based access here). Links to video will become available at these locations shortly before the event.* *Please feel free to pass this information along to any interested folks. Our projects tend to focus on areas that might be of interest to folks working across the open source tech community: language detection, numerical sort, large data visualizations, maps, and all sorts of other things.* *If you have any questions, please let me know! Thanks, and I hope to see you at CREDIT.* *YOURNAME* -- Lani Goto Project Assistant, Engineering Admin -- Lani Goto Project Assistant, Engineering Admin

1 0

Re: [Wikitech-l] Fantastic MediaWikis - Track at Wikimedia Hackathon - CfP
by Markus Glaser 05 Apr '17

05 Apr '17

Hello all, just a reminder: the call for participation for "Fantastic MediaWikis and How to Maintain Them", a one-day track at the Wikimedia Hackathon in Vienna, ends today. So if you intend to submit a presentation, please add yourself to this list: https://www.mediawiki.org/wiki/MediaWiki_Stakeholders%27_Group/Fantastic_Me… All the best, Markus (User:Mglaser) MediaWiki Stakeholders' Group -----Ursprüngliche Nachricht----- Von: Wikitech-l [mailto:wikitech-l-bounces@lists.wikimedia.org] Im Auftrag von Markus Glaser Gesendet: Montag, 20. März 2017 15:30 An: Wikimedia developers <wikitech-l(a)lists.wikimedia.org> Betreff: [Wikitech-l] Fantastic MediaWikis - Track at Wikimedia Hackathon - CfP [This sender failed our fraud detection checks and may not be who they appear to be. Learn about spoofing at http://aka.ms/LearnAboutSpoofing] Hello everyone, "Fantastic MediaWikis and How to Maintain Them" is a one-day conference track designed for people who work with the open source software MediaWiki in their organisation, company or business [1]. The conference track is curated by the MediaWiki Stakeholders Group and hosted by Wikimedia Austria. The track is part of Wikimedia's biggest annual tech-event, the Wikimedia Hackathon [2]: For three days each year, about 200 developers come together to improve MediaWiki. Among the participants are coders from the Wikimedia Foundation as well as volunteers from all over the world. We're using this opportunity to connect the people who work with MediaWiki in their enterprise, company, or organisation to the people who actually develop the code. We invite all MediaWiki stakeholders - power-users, maintainers, and service providers, who have a professional or semi-professional interest in the software - to join us for a day full of MediaWiki knowledge and networking! This is a call for participation. We are specifically looking for contributions of your experience in running MediaWikis. These include but are not confined to: * Use Cases. Tell us how and why you use MediaWiki in a specific area or to solve a specific problem. * Best Practices. Talk about the most effective way you have found to use particular MediaWiki features. * Enhancements. Have you improved MediaWiki to fit your needs? Show us your solution. * Challenges. How can we all add to MediaWiki functionality? Do you have specific ideas you want to collaborate on? Share your experience and submit a presentation here: https://www.mediawiki.org/wiki/MediaWiki_Stakeholders%27_Group/Fantastic_Me… Important dates: * 20th March 2017: Call for Participation opens * 5th April 2017: Submission deadline * 10th April 2017: Notification of acceptance * 19th - 21st May 2017: Vienna Hackathon We are looking forward to your submissions! Best, Markus (User:Mglaser) MediaWiki Stakeholders' Group [1] https://www.wikimedia.at/hackathon/fantastic-mediawikis/ [2] https://www.mediawiki.org/wiki/Wikimedia_Hackathon_2017 _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

1 0

Introducing the MediaWiki Platform Team!
by Victoria Coleman 05 Apr '17

05 Apr '17

Hello everyone, I am pleased to share with you the news about the formation of the latest team in the Technology department, the MediaWiki Platform team![1] The MediaWiki Platform team will be tasked with leading maintenance and improvements related to the core MediaWiki platform codebase. That includes encouraging future development of the MediaWiki platform and addressing the technical debt that has accumulated during the 15-year history of MediaWiki. MediaWiki is an amazing, powerful, and complex open-source software platform. The number and variety of extensions, and the wide variety of communities who have adopted MediaWiki as their method for knowledge collection and dissemination, are a testament to its strength as a software platform. Like any significant codebase with a long development history, there are remnants of design choices and experiments that are no longer in use, and some areas of code are in need of modernization. However, at its core is a large amount of highly functional, secure, performant code, capable of supporting a robust platform through the use of extensions and hooks. There is also a great amount of flexibility to adapt to new requirements. This team will have a more focused purpose than the previous MediaWiki Core team.[2]. While the previous team was at times spread too thin, many areas are now covered by dedicated teams like Security and Performance. The new MediaWiki Platform team will center their efforts on the core codebase. The team will also have a dedicated Product Manager who will be creating the platform roadmap in collaboration with the team, the Architecture Committee and the MediaWiki user community. Specific goals for this team are to: * Assist and encourage development of features for MediaWiki by providing developers with a strong core. * Undertake feature development work which is primarily architectural in nature. * Facilitate the development and publication of MediaWiki's roadmap to assist coordination between internal and external users. * Maintain and promote guidelines and standards for the MediaWiki core. I am thrilled that Tim Starling has agreed to lead the team, reporting directly to me. He will be joined by Brion Vibber, Kunal Mehta and Brad Jorsch. The team officially launches on Monday April 3, and will complete the hiring and onboarding of additional team members in the coming months. Their initial workplan will include core support for multi content revisions for the Structured Data on Commons project and will be discussed in more detail during the upcoming consultation for the Wikimedia Foundation 2017-2018 annual plan. I am excited by this latest evolution in the structure of the Foundation's Engineering group. We will continue to learn from our collective knowledge and expertise, and make adjustments to our composition and plans. I appreciate the input provided by many in the community that helped inform this decision. I also want to thank the members of the Wikimedia Foundation's Product, Technology, and Community Engagement departments who were involved in this process. In particular, I would like to thank Toby Negrin, Adam Baso, and Trevor Parscal - whose support was critical in bringing this plan together. Join me in welcoming and celebrating our new team! Victoria [1] https://www.mediawiki.org/wiki/MediaWiki_Platform_team <https://www.mediawiki.org/wiki/MediaWiki_Platform_team> [2] https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team <https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team>

7 6

Questions over usage of ObjectCache
by Nischay Nahata 04 Apr '17

04 Apr '17

Hi all, I have been trying to add certain rate limiting actions for my extension's API and noticed that it doesn't work with $wgMainCacheType set to CACHE_NONE This is because User::pingLimiter() uses ObjectCache::getLocalClusterInstance() to store the current rate limit counts. Is this the expected behavior? Looks pretty deceitful to me as $wgMainCacheType is assumed to define the cache type but is actually disabling a core functionality like rate limiting and maybe few others? Regards, Nischay Nahata

2 3

Fwd: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning
by Pine W 03 Apr '17

03 Apr '17

Forwarding. Pine ---------- Forwarded message ---------- From: "Jordan Adler" <jmadler(a)google.com> Date: Aug 11, 2016 13:06 Subject: [Commons-l] Programmatically categorizing media in the Commons with Machine Learning To: "commons-l(a)wikimedia.org" <commons-l(a)lists.wikimedia.org> Cc: "Ray Sakai" <rsakai(a)reactive.co.jp>, "Ram Ramanathan" < ramramanathan(a)google.com>, "Kazunori Sato" <kazsato(a)google.com> Hey folks! A few months back a colleague of mine was looking for some unstructured images to analyze as part of a demo for the Google Cloud Vision API <https://cloud.google.com/blog/big-data/2016/05/explore-the-galaxy-of-images…>. Luckily, I knew just the place <https://commons.wikimedia.org/wiki/Category:Media_needing_categories>, and the resulting demo <http://vision-explorer.reactive.ai/>, built by Reactive Inc., is pretty awesome. It was shared on-stage by Jeff Dean during the keynote <https://www.youtube.com/watch?v=HgWHeT_OwHc&feature=youtu.be&t=2h1m19s> at GCP NEXT 2016. I wanted to quickly share the data from the programmatically identified images so it could be used to help categorize the media in the Commons. There's about 80,000 images worth of data: - map.txt <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/map.txt> (5.9MB): A single text file mapping id to filename in a "id : filename" format, one per line - results.tar.gz <https://storage.googleapis.com/gcs-samples2-explorer/reprocess/results.tar.…> (29.6MB): a tgz'd directory of json files representing the output of the API <https://cloud.google.com/vision/reference/rest/v1/images/annotate#response-…>, in the format "${id}.jpg.json" We're making this data available under the CC0 license, and these links will likely be live for at least a few weeks. If you're interested in working with the Cloud Vision API to tag other images in the Commons, talk to the WMF Community Tech team. Thanks for your help! _______________________________________________ Commons-l mailing list Commons-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l

2 1

CirrusSearch forceSearchIndex.php issue
by Aran 03 Apr '17

03 Apr '17

Hello, I've found that running forceSearchIndex.php with --skipLinks --indexOnSkip options is only processing about 1000-1500 pages and then stopping. It's not due to error or anything because I can set --fromId and run it again and it does another batch. I've tried setting --limit and --batchSize and all sorts but nothing allows it to do more than this amount at a time.... anyone have any idea what might be going on here? (It's happening on both MW 1.27 with Elastic Search 1.75 and MW 1.28 with elastic 2.4.4) Thanks, Aran

2 1

Phabricator monthly statistics - 2017-03
by communitymetrics＠wikimedia.org 03 Apr '17

03 Apr '17

Hi Community Metrics team, This is your automatic monthly Phabricator statistics mail. Accounts created in (2017-03): 332 Active users (any activity) in (2017-03): 891 Task authors in (2017-03): 503 Users who have closed tasks in (2017-03): 274 Projects which had at least one task moved from one column to another on their workboard in (2017-03): 294 Tasks created in (2017-03): 2645 Tasks closed in (2017-03): 2417 Open and stalled tasks in total: 34232 Median age in days of open tasks by priority: Unbreak now: 0 Needs Triage: 262 High: 443 Normal: 638 Low: 909 Lowest: 844 (How long tasks have been open, not how long they have had that priority) TODO: Numbers which refer to closed tasks might not be correct, as described in https://phabricator.wikimedia.org/T1003 . Yours sincerely, Fab Rick Aytor (via community_metrics.sh on iridium at Sat Apr 1 00:00:14 UTC 2017)

3 3

Selenium tests in Node.js
by Željko Filipin 03 Apr '17

03 Apr '17

# TL;DR You can now write Selenium tests in Node.js! Learn more about it at https://www.mediawiki.org/wiki/Selenium/Node.js # Introduction Five years ago we introduced browser tests using Selenium and a Ruby based stack. It has worked great for some teams, and not so great for others. Last year we talked to people from several teams and ran a survey[0]. The outcome is a preference toward using a language developers are familiar with: JavaScript/Node.Js. After several months of research and development, we are proud to announce support for writing tests in Node.js. We have decided to use WebdriverIO[1]. It is already available in MediaWiki core and supports running tests for extensions. You can give it a try in MediaWiki-Vagrant[2]: vagrant up vagrant ssh sudo apt-get install chromedriver export PATH=$PATH:/usr/lib/chromium cd /vagrant/mediawiki xvfb-run npm run selenium # Documentation Extensive details are available on the landing page: https://www.mediawiki.org/wiki/Selenium/Node.js # Future We plan to replace the majority of Selenium tests written in Ruby with tests in Node.js in the next 6 months. We can not force anybody to rewrite existing tests, but we will offer documentation and pairing sessions for teams that need help. After 6 months, teams that want to continue using Ruby framework will be able to do so, but without support from Release Engineering team. I have submitted a skill share session for Wikimedia Hackathon 2017 in Vienna[3]. If you would like to pair on Selenium tests in person, that would be a great time. The list of short term actions is in task T139740[4]. # Thanks I would like to thank several people for reviews, advice and code: Jean-Rene Branaa, Dan Duvall, Antoine Musso, Jon Robson, Timo Tijhof. (Names are sorted alphabetically by last name. Apologies to people I have forgot.) # References [0] https://www.mediawiki.org/wiki/Browser_testing_user_satisfaction_survey [1] http://webdriver.io/ [2] https://www.mediawiki.org/wiki/Selenium/Node.js/Inside_MediaWiki-Vagrant [3] https://phabricator.wikimedia.org/T159945 [4] https://phabricator.wikimedia.org/T139740 Željko Filipin

1 0

GSOC project for multiple watch lists - looking for mentors
by Techman224 01 Apr '17

01 Apr '17

Hello, Sorry if this is cutting it close to the deadline, but I’m interested in doing a GSOC project over the summer to implement multiple watchlists for users. What I have in mind is first, deliver a minimum viable product where users can add/remove/change custom watchlists. If time permits, it can be integrated into other components, such as notifications, a way to add pages into custom watch lists from the page itself, the API etc. I’m looking for potential mentors. In particular, someone who knows how to get around Mediawiki’s codebase and can help me guide coding and development. If there is any interest let me know! Thanks, Techman224 Task: https://phabricator.wikimedia.org/T3492

1 0

← Newer
1
2
3
4
5
6
7
8
9
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l April 2017