Wikitech-l August 2014

wikitech-l@lists.wikimedia.org

133 participants
134 discussions

MediaWiki Language Extension Bundle 2014.08 release
by Kartik Mistry 28 Aug '14

28 Aug '14

Hello all, I would like to announce the release of MediaWiki Language Extension Bundle 2014.08. This bundle is compatible with MediaWiki 1.23.x and MediaWiki 1.22.x releases. * Download: https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2014.08.tar… * sha256sum: f4663910facd2b34d4afb1af930f66adad22b2746f9395f7706f07751e633857 Quick links: * Installation instructions are at: https://www.mediawiki.org/wiki/MLEB * Announcements of new releases will be posted to a mailing list: https://lists.wikimedia.org/mailman/listinfo/mediawiki-i18n * Report bugs to: https://bugzilla.wikimedia.org * Talk with us at: #mediawiki-i18n @ Freenode Release notes for each extension are below. -- Kartik Mistry == Babel, CLDR, CleanChanges and LocalisationUpdate == * Only localisation updates. == Translate == === Noteworthy changes === * Breaking change: Dropped support for MediaWiki 1.21 (2014.05 is the latest version of MLEB as a whole which supports 1.21) * Avoid losing translators' edits: improved handling of expired sessions ({{bug|69314}}). ** Status has yellow background if the translation has not been saved. ** Show an alert to the user if session has expired. ** Prevent saving translations as an anonymous user accidentally. * Restore progress icon for the current translation page (regression from MLEB 2014.07). * Update associated translation pages when moving translation units. * Show definition diff for all messages if available, not just for outdated messages. * Special:AggregateGroups: Group selector is now input field with autocompletion instead of combobox. * Special:PageMigration: Don't add the created pages to watchlist. * Special:PagePreparation: This is a new special page to assist in preparing pages for translation == UniversalLanguageSelector == * Change the autonym for Algerian Arabic. * Localisation updates. ==== Input Methods ==== * Tibetian-ewts: Use of unicode NFC instead of NFD. * Fixed description in Bengali Probhat layout. ==== Fonts ==== * Added Comic Neue font. It is available for English and other languages using the latin alphabet. * Updated UnifrakturMaguntia font to the latest version. -- Kartik Mistry/કાર્તિક મિસ્ત્રી | IRC: kart_ {kartikm, 0x1f1f}.wordpress.com

1 0

dev.wikimedia.org (was Re: build.wikimedia.org)
by Quim Gil 28 Aug '14

28 Aug '14

On Tuesday, August 26, 2014, MZMcBride <z(a)mzmcbride.com> wrote: > Legoktm wrote: > >This is now at [[dev.wikimedia.org]]? That sounds like a much better > >name than "build", which I thought was going to be some CI automated > >builds server from your email title :) > dev.wikimedia.org it is, yes. Moving on, please. We > already have a publishing platform that we can and should leverage and it > lives at www.mediawiki.org > This is basically decided as well. However, we still need the best of our technical knowledge to solve this problem: how to extract API documentation from source code repositories and import it to wiki pages. Your ideas and help are welcome -- see the ongoing discussion at http://fab.wmflabs.org/T491 -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

1 0

Moving Vector and MonoBook to separate repositories and what it means for you
by Bartosz Dziewoński 28 Aug '14

28 Aug '14

tl;dr: I am going to break your workflow and your wiki. Skip to the last section and read on. == Background == As you know, I've been working on a GSoC project to better separate skins and core MediaWiki [1]. Moving Vector and MonoBook to separate repositories is the final step of it, and it's exactly as scary as it sounds. Until recently MediaWiki was heavily interconnected with the core skins, particularly Vector – right now it is only slightly interconnected and I have some patches pending to make it not so at all [2]. [1] https://www.mediawiki.org/wiki/Separating_skins_from_core_MediaWiki [2] https://gerrit.wikimedia.org/r/#/q/topic:skinning,n,z == The plan == * Fix up some remaining issues with Vector being required (I have already fixed most) * Stop always loading MonoBook and Vector [patch pending: https://gerrit.wikimedia.org/r/148509 + dependencies] * Use a special fallback when no skins are installed [patch pending: https://gerrit.wikimedia.org/r/148508] * Move MonoBook and Vector to separate repositories * Ship MonoBook, Vector and some more skins with the installer tarball * Enable them during the installation [patch merged: https://gerrit.wikimedia.org/r/138652] == What it means for you == If you're upgrading a wiki or you're a developer working on master, THIS IS A BACKWARDS INCOMPATIBLE CHANGE. Your wiki will continue to work, it will just look ugly (no styles). When we do this and you upgrade, you're going to get a big helpful message asking you to install and enable some skins and explaining how to do this (see https://gerrit.wikimedia.org/r/148508 for implementation; basically, put the files/repository under skins/ and require_once in LocalSettings). Try it out with this patch: https://gerrit.wikimedia.org/r/148509 After you do this, you'll be able to switch to slightly older MediaWiki versions without things breaking, as the new skins will work with old MediaWiki (the new directory names are different… unless you're using a case-insensitive OS). I hope this sounds reasonable to everyone, and I hope to have this done around the time of Wikimania (possibly during it; I'll be there). I don't think there is a less disruptive way to do this, other than not doing it at all; if you come up with one, please share it. -- Matma Rex

14 35

OAuth and callbacks
by Chris Steipp 28 Aug '14

28 Aug '14

For those who run one of our 76(!) approved OAuth apps, or are using OAuth extension on their own wiki.. We have a patch [1] from Mitar to allow OAuth apps to pass a configurable callback during the OAuth handshake. This will probably make a lot of app author's lives easier, but can also open up a couple avenues of abuse-- specifically, it's needed for covert redirect attacks [2]. If OAuth app authors chose loose callback requirements, which we can assume will happen if we make approvals automatic (bug 65750), and we ever allow public consumers (huggle was asking for that for a long time), then it would be possible for attackers to abuse our OAuth setup. So far, I've been really conservative about how we use OAuth (there are two other features we would have to enable to make this attack likely). I'd like to hear other's thoughts about: * Assuming we implement one or two of: dynamic callbacks, automatic approval of apps, or public consumers, but not all three, which are most desired? * If we do implement all three, we can limit how the callback can differ from what is registered. I put some suggestions on the gerrit patch, but would that cause more confusion than help? [1] - https://gerrit.wikimedia.org/r/153983 [2] - http://tetraph.com/covert_redirect/oauth2_openid_covert_redirect.html

2 1

Pre-release announcement for MediaWiki releases 1.22.10 and 1.23.3
by Markus Glaser 27 Aug '14

27 Aug '14

Hello everyone, this is a notice that on 27th August between 20:00-22:00 UTC we will release maintenance updates for current and legacy branches of the MediaWiki software. Downloads and patches will be available at that time. Best, Markus Wiki Release Team

3 2

Call for reviewers: structured logging
by Bryan Davis 27 Aug '14

27 Aug '14

Thanks to a lot of hard work by Antoine [0], Zuul and Jenkins are now testing things using the mediawiki/vendor repository. This unblocks my patches that implement PSR-3 logging for the structured logging RFC [1]: * Add a PSR-3 based logging interface - https://gerrit.wikimedia.org/r/#/c/119940/ * Use MWLogger logging for legacy logging methods - https://gerrit.wikimedia.org/r/#/c/119941/ * Use MWLogger logging for wfLogProfilingData - https://gerrit.wikimedia.org/r/#/c/119942/ * Add logging context to database logs - https://gerrit.wikimedia.org/r/#/c/141599/ I'd love to see these reviewed and merged. :) [0]: http://www.gossamer-threads.com/lists/wiki/wikitech/498938 [1]: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

2 1

Wikimedia engineering report, July 2014
by Guillaume Paumier 27 Aug '14

27 Aug '14

Hi, The report covering Wikimedia engineering activities in July 2014 is now available. Wiki version: https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July Blog version: https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/ We're also proposing a shorter version of this report focusing on priority goals for this quarter: https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July/summa… Below is the HTML text of the report. As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them. ------------------------------------------------------------------ Major news in July include: - a recap of how the Operations team collaborated with the RIPE NCC <https://blog.wikimedia.org/2014/07/09/how-ripe-atlas-helped-wikipedia-users/> to measure the delivery of Wikimedia sites to users in Asia and elsewhere; - an analysis of the impact of the San Francisco data center <https://blog.wikimedia.org/2014/07/11/making-wikimedia-sites-faster/> on the speed of Wikimedia sites; - the launch of the new native Wikipedia app for iOS <https://blog.wikimedia.org/2014/07/31/official-wikipedia-app-available-on-i…> ; - a first look at the content translation tool <https://blog.wikimedia.org/2014/07/16/first-look-at-the-content-translation…> . *Note: We’re also providing a shorter and translatable version of this report <https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July/summa…>.* Engineering metrics in July: - 164 unique committers contributed patchsets of code to MediaWiki. - The total number ofunresolved commits <https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z> went from around 1575 to about 1642. - About 31 shell requests <https://www.mediawiki.org/wiki/Shell_requests>were processed. Contents - Personnel <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Personn…> - Work with us <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Work_wi…> - Announcements <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Announc…> - Technical Operations <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Technic…> - Features Engineering <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Feature…> - Editor retention: Editing tools <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/_Editing…> - Services <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Services> - Core Features <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Core_Fe…> - Growth <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Growth> - Mobile <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Mobile> - Language Engineering <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Languag…> - Platform Engineering <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Platfor…> - MediaWiki Core <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#MediaWi…> - Release Engineering <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Release…> - Multimedia <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Multime…> - Engineering Community Team <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Enginee…> - Analytics <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Analyti…> - Kiwix <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Kiwix> - 10 Wikidata <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Wikidata> - 11 Future <https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Future> PersonnelWork with us <https://wikimediafoundation.org/wiki/Work_with_us> Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. - VP of Engineering <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Front-end (VisualEditor) <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Services <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Front-end <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Maps & Geo <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Mobile – iOS <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - QA Tester <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Software Engineer – Full Stack <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Lean/Agile Coach <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Product Manager <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Product Manager – Language Engineering <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Operations Security Engineer <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - UX Senior Designer <http://grnh.se/veo5n8> - UX Senior Design Researcher <http://grnh.se/38vlip> - UX User Research Recruiter <http://grnh.se/kg42d7> - Project Coordinator – Engineering <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Mobile Partnerships Regional Manager <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> - Program Evaluation Internship <http://grnh.se/2ekx4h> Announcements - Arthur Richards is now Team Practices Manager (announcement <http://lists.wikimedia.org/pipermail/wikimediaannounce-l/2014-July/000956.h…> ). - Kristen Lans joined the Team Practices Group as Scrum Master ( announcement <http://lists.wikimedia.org/pipermail/teampractices/2014-July/000437.html> ). - Joel Sahleen joined the Language Engineering team as Software Engineer (announcement <http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077829.html>). Technical Operations *Dallas data center* Throughout July, the cabling work of all racked servers and other equipment was nearly completed. We’re still awaiting the installation of the first connectivity to the rest of our US network in early August before we can begin installation of servers and services. *San Francisco data center* Due to a necessary upgrade to power & cooling infrastructure in our San Francisco data center (which we call *ulsfo*), our racks have been migrated to a new floor within the same building on July 9. The move completed in a very smooth fashion without user impact, and the site was brought back online serving all user traffic again in less than 24 hours. *PFS enabled* Through the help of volunteer work and research, our staff enabled Perfect Forward Secrecy <https://en.wikipedia.org/wiki/en:Perfect_Forward_Secrecy> on our SSL infrastructure, significantly increasing the security of encrypted user traffic. Labs metrics in July: - Number of projects: 173 - Number of instances: 464 - Amount of RAM in use (in MBs): 1,933,824 - Amount of allocated storage (in GBs): 20,925 - Number of virtual CPUs in use: 949 - Number of users: 3,500 *Wikimedia Labs* We’ve made several minor updates to Wikitech: we added OAuth support, fixed a few user interface issues, and purged the obsolete ‘local-*’ terminology for service groups. OPW Intern Dinu Sandaru has set forms for structured project documentation. This should will help match new volunteers with existing projects, and will make communication with project administrators more straightforward. Sean Pringle is in the process of updating the Tool Labs replica databases to MariaDB version 10.0. This may reduce replag, and should improve performance and reliability. We’re setting up new storage hardware for the project dumps. This will resolve our ongoing problems with full drives and out-of-date dumps. Features Engineering <https://www.mediawiki.org/wiki/Wikimedia_Features_engineering> Editor retention: Editing tools *VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>* In July, the team working on VisualEditor converged the design for mobile and desktop, made it possible to see and edit HTML comments, improved access to re-using citations, and fixed over 120 bugs and tickets <https://bugzilla.wikimedia.org/buglist.cgi?list_id=333021&order=priority%2C…> . The new design, with controls focussed at the top of each window in consistent positions, was made possible due to the significant progress made in cross-platform support in the UI library, which now provides responsively-sized windows that can work on desktop, tablet and phone with the same code. HTML comments are occasionally used on a few articles to alert editors to contentious or problematic issues without disrupting articles as they are read, so making them prominently visible avoids editors accidentally stepping over expected limits. Re-using citations is now provided with its simple dialog available in the toolbar so that it is easier for users to find. Other improvements include an array of performance fixes targeted at helping mobile users especially, fixing a number of minor instances where VisualEditor would corrupt the page, and installing better monitoring of corruptions if they occur, and better support for right-to-left languages, displaying icons with the right orientation based on context. The mobile version of VisualEditor, currently available for beta testers, moved towards stable release, fixing a number of bugs and editing issues and improving loading performance. Our work to support languages made some significant gains, nearing the completion of a major task to support IME users, and the work to support Internet Explorer uncovered some more issues as well as fixes. The deployed version of the code was updated five times in the regular release cycle (1.24-wmf12 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf8#VisualEditor>, 1.24-wmf13 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf9#VisualEditor> , 1.24-wmf14 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf10#VisualEditor>, 1.24-wmf15 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf15#VisualEditor> and 1.24-wmf16 <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf16#VisualEditor>). In wider news, the team expanded its scope to cover all MediaWiki editing tools as well, as the new Editing Team (covered below). *Editing <https://www.mediawiki.org/wiki/Editing>* In July, the newly re-named and re-scoped Editing Team <https://www.mediawiki.org/wiki/Editing> was formed from the VisualEditor Team. We are responsible for extending and improving the editing tools used at Wikimedia – primarily VisualEditor and maintenance for WikiEditor. We exist to support new and existing editors alike; our current work is mostly on desktop, and we are working with Mobile to take responsibility for all editing across desktop, tablet and phone platforms, spanning approximately 50 different areas of MediaWiki and extensions related to editing. We will continue to report progress on VisualEditor separately. The biggest Editing change this month was in the Cite extension (for footnotes) – this now automatically shows a references list at the end of the page if you forget to put in a <references /> tag, instead of displaying an ugly error message. The Math extension (for formulæ) was improved with more rigorous error handling and LaTeX formula checking, as part of the long-term volunteer-led work to introduce MathML-based display and editing. The TemplateData GUI editor was deployed to a further six wikis – the English, French, Italian, Russian, Finnish and Dutch Wikipedias. A lot of work was done on libraries and infrastructure for the Editing Team and others. The OOjs UI <https://www.mediawiki.org/wiki/OOjs_UI> library was extensively modified to bring in a new window management system for comprehensive combined desktop, tablet and phone support, as well as other updates to improve Internet Explorer compatibility and accessibility of controls. In the next few months the team will continue working on OOUI to support other teams’ needs and implement a consistent look-and-feel in collaboration with the Design team. The OOjs <https://www.mediawiki.org/wiki/OOjs> library was updated to fix a minor bug, with a new version (v1.0.11) released and pushed downstream into MediaWiki, VisualEditor and OOjs UI. The ResourceLoader framework was extended to allow skins to set the “skinStyles” property themselves, rather than rely on faux dependencies, as part of wider efforts led jointly by a volunteer and a team member to improve MediaWiki’s skin support. *Parsoid <https://www.mediawiki.org/wiki/Parsoid>* In July, the Parsoid team continued with ongoing bug fixes and bi-weekly deployments. With an eye towards supporting Parsoid-driven page views, the Parsoid team strategized on addressing Cite extension rendering differences that arise from site-messages based customizations and is considering a pure CSS-based solution for addressing the common use cases. We also finished work developing the test setup for doing mass visual diff tests between PHP parser rendering and Parsoid rendering. It was tested locally and we started preparations for deploying that on our test servers. This will go live end-July or early-August. The GSoC 2014 LintTrap project continued to make good progress. We had productive conversations with Project WikiCheck about integrating LintTrap with WikiCheck in a couple different ways. We hope to develop this further over the coming months. Overall, this was also a month of reduced activity with Gabriel now officially full time in the Services team and Scott focused on the PDF service deployment that went live a couple days ago. The full team is also spending a week at a off-site meeting working and spending time together in person prior to Wikimania in London. Services *Services and REST API <https://www.mediawiki.org/wiki/Services>* The brand new Services <https://www.mediawiki.org/wiki/Services> group (currently Matt Walker and Gabriel Wicke) started July with two main projects: 1. PDF render <https://www.mediawiki.org/wiki/PDF_rendering> service deployment 2. Design and prototyping work on the storage service <https://www.mediawiki.org/wiki/Requests_for_comment/Storage_service> and REST API <https://www.mediawiki.org/wiki/Requests_for_comment/Content_API> The PDF render service is now deployed in production, and can be selected as a render backend in Special:Book. The renderer does not work perfectly on all pages yet, but the hope is that this will soon be fixed in collaboration with the other primary author of this service, C. Scott Ananian. Prototyping work on the storage service <https://github.com/gwicke/rashomon> and REST API <https://github.com/gwicke/restface> is progressing well. The storage service now has early support for bucket creation and multiple bucket types. We decided to configure the storage service as a backend for the REST API server. This means that all requests will be sent to the REST API, which will then route them to the appropriate storage service without network overhead. This design lets us keep the storage service buckets very general by adding entry point specific logic in front-end handlers. The interface is still well-defined in terms of HTTP requests, so it remains straightforward to run the storage service as a separate process. We refined the bucket design to allow us to add features very similar to Amazon DynamoDB <https://en.wikipedia.org/wiki/Amazon_DynamoDB> in a future iteration. There is also an early design for light-weight HTTP transaction support. Matt Walker is sadly leaving the Foundation by the end of this month to follow his passion of building flying cars. This means that we currently have three positions open <http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&pag…> in the service group, which we hope to start filling soon. Core Features *Flow <https://www.mediawiki.org/wiki/Flow/Project_information>* In July, the Flow team built the ability for users to subscribe to individual Flow discussions, instead of following an entire page of conversations. Subscribing to an individual thread is automatic for users who create or reply to the thread, and users can choose to subscribe (or unsubscribe) by clicking a star icon in the conversation’s header box. Users who are subscribed to a thread receive notifications about any replies or activity in that thread. To support the new subscription/notification system, the team created a new namespace, Topic, which is the new “permalink” URL for discussion threads; when a user clicks on a notification, the target link will be the Topic page, with the new messages highlighted with a color. The team is currently building a new read/unread state for Flow notifications, to help users keep track of the active discussion topics that they’re subscribed to. Growth *Growth <https://www.mediawiki.org/wiki/Growth>* In July, the Growth team completed its second round of A/B testing ofsignup invitations for anonymous editors <https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites> on English Wikipedia, includingdata analysis <https://meta.wikimedia.org/wiki/Research:Asking_anonymous_editors_to_regist…>. The team also built the first API and interface prototypes for task recommendations <https://www.mediawiki.org/wiki/Task_recommendations>. This new system, first aimed at brand new editors, makes suggestions based on a user’s previous edits. Mobile <https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering> *Wikimedia Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>* Following on from the successful launch to Android, the Mobile Apps team released the new native Wikipedia app to iOS on July 31. The app is the iOS counterpart to the Android app, with many of the same features such as editing, saving pages for offline reading, and browsing history. The iOS app also contains an onboarding screen that is shown the first time the app is launched, asking users to sign up, a feature which was also launched on Android this month (see below). On Android this month we released to production accessibility and styling features which were requested by our users, such as a night mode for reading in the dark and a font size selector. We also released an onboarding screen that asks users to sign up. Our plan for next month is to get user feedback from Wikimania, wrap up our styling fixes, and begin work on an onboarding screen the first time that someone taps edit. *Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>* This month, the team continued to focus on wrapping up the collaboration with the Editing team to bring VisualEditor to tablet users on the mobile site. We also began working to design and prototype our first new Wikidata contribution stream, which we will build and test with users on the beta site in the coming month. *Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>* During the last month, the team worked on software architecture features that allow for expansion of the Wikipedia Zero footprint on partner networks and that get users to content faster with support for lowered cache fragmentation on Varnish caches. Whereas the previous system supported one-size-fits-all configuration for heterogeneous partner networks, inhibiting some zero-rated access, the new system supports multiple configurations for disparate IP addresses and connection profiles per operator. Additionally, lightweight script and GIF-ified Wikipedia Zero banner support has been added and is being tested; in time this should drastically reduce Varnish cache fragmentation, making pages be served faster and reducing Varnish server load. A faster landing page was introduced for “zerodot” (zero.wikipedia.org, legacy text-only experience) landing pages when operators have multiple popular languages in their geography. Work on compression proxy traffic analysis for header enrichment conformance with the official Wikipedia Zero configurations was also performed after more diagnostic logging code was added to the system. Finally, watchlist thumbnails, although low bandwidth, were removed from the zerodot user experience, as was the higher bandwidth MediaViewer feature for zerodot; mdot will have these features, though. In side project work, the team spent time on API continuation queries, Android IP editing notices, Amazon Kindle and other non-Google Play distribution, and Google Play reviews (now that the Android launch dust has settled, mobile apps product management will be triaging the reviews). In partnerships work, the team met with Mozilla to talk about future plans for the Firefox OS HTML5 app (e.g., repurposing the existing mobile website, but without any feature reduction) and how Wikimedia search might be further integrated into Firefox OS, and also spoke with Canonical about how Wikipedia might be better integrated into the forthcoming Ubuntu Phone OS. Routine pre- and post-launch configuration changes were made to support operator zero-rating, with routine technical assistance provided to operators and the partner management team to help add zero-rating and address anomalies. The team also continued its search for a third Partners engineering teammate. *Wikipedia Zero (partnerships)* We served an estimated 68 million free page views in July through Wikipedia Zero. We continue to bring new partners into the program, though none launched in July. Adele Vrana met with prospective partners and local Wikimedians in Brazil. We published our operating principles <https://wikimediafoundation.org/wiki/Wikipedia_Zero_Operating_Principles> to increase transparency. Language Engineering <https://www.mediawiki.org/wiki/Wikimedia_Language_engineering> *Language tools <https://www.mediawiki.org/wiki/Language_tools>* CLDR extension was updated to use CLDR 25 <http://cldr.unicode.org/index/downloads/cldr-25>; this work was mostly done by Ryan Kaldari. The team made various internationalization fixes in core, MobileFrontend, Wikipedia Android app, Flow, VisualEditor and other features. In the Translate extension, Niklas Laxström fixed ElasticSearchTTMServer <https://gerrit.wikimedia.org/r/145769> to provide translation memory suggestions longer than one word; and improved translation memory suggestions for translation units containing variables (bug 67921 <https://bugzilla.wikimedia.org/show_bug.cgi?id=67921>). *Language Engineering Communications and Outreach <https://www.mediawiki.org/wiki/Language_engineering_communications_and_outr…>* We announced the initial availability of the Content translation tool <https://blog.wikimedia.org/2014/07/16/first-look-at-the-content-translation…> with limited feature support. We are focusing on supporting Spanish to Catalan translations for this initial release. You can read a report on thefeedback received since deployment <https://www.mediawiki.org/wiki/Content_translation/Updates/31July2014>. *Content translation <https://www.mediawiki.org/wiki/Content_translation>* An initial version <http://es.wikipedia.beta.wmflabs.org/wiki/Especial:ContentTranslation> was released on Beta Labs; it supports machine translation between Spanish and Catalan. The machine translation API leverages open source machine translation with Apertium. The tool supports experimental template adaptation between languages. Numerous bug fixes were made based on testing and user feedback. We worked on matching the Apertium version to the cluster, and planning for the next round of development has started. Platform Engineering <https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering> MediaWiki Core *HHVM <https://www.mediawiki.org/wiki/HHVM>* The Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster> is running HHVM. The latest MediaWiki-Vagrant <https://www.mediawiki.org/wiki/MediaWiki-Vagrant> andLabs-vagrant <https://wikitech.wikimedia.org/wiki/Labs-vagrant> use HHVM by default. *Admin tools development <https://www.mediawiki.org/wiki/Admin_tools_development>* Most admin tools resources are currently diverted towards SUL finalisation <https://www.mediawiki.org/wiki/SUL_finalisation>, which will greatly help in reducing the admin tools backlog. July saw the deployment of the global rename tool (bug 14862 <https://bugzilla.wikimedia.org/show_bug.cgi?id=14862>), and core fixes including the creation of the “viewsuppressed” userright (bug 20476 <https://bugzilla.wikimedia.org/show_bug.cgi?id=20476>). *Search <https://www.mediawiki.org/wiki/Search>* Our deployment of CirrusSearch to larger wikis as the primary search back-end turned out to be too ambitious. After encountering performance issues, we rolled back this change. We are now addressing the root of the problem, by getting more servers (nearly doubling the cluster size) and putting together more optimizations to the portion of Cirrus that fell over (working set). If everything goes as planned, it’ll be reduced by about 80%, by reducing indexing performance in return of search performance. These optimizations will slightly change result relevance; please let us know if you notice any issues. *Auth systems <https://www.mediawiki.org/wiki/Auth_systems>* Most work was spent on SUL Finalization tasks. Phpunit and browser tests were added for CentralAuth, global rename was deployed, and lots of small fixes were made to CentralAuth to clean up user accounts in preparation for finalization. *SUL finalisation <https://www.mediawiki.org/wiki/SUL_finalisation>* In July, the SUL finalisation team began work on completing the necessary feature work to support the SUL finalisation. To help users with local-only accounts that are going to be forcibly renamed due to the SUL finalisation, the team is working on a form that lets those users request a rename. These requests will be forwarded onto the stewards <https://meta.wikimedia.org/wiki/Stewards> to handle. The SUL team is currently in consultation with the stewards about how they would like this tool to work. When this consultation is wrapped up, the team will begin design and implementation. To help users get globally renamed without having to request renames on potentially hundreds of wikis, the team implemented and deployed GlobalRenameUser, a tool which renames users globally. As the tool is designed to work post-finalisation, it only performs renames where the current name is global, and the requested name is totally untaken (no global account and no local accounts exist with that name). To help users who get renamed by the finalisation and, despite our best efforts to reach out to them, did not get the chance to request a rename before the finalisation, the team is working on a feature to let users log in with their old credentials. The feature will display an interstitial when they log in, informing them that they logged in with old credentials and that they need to use new ones. We are also considering a persistent banner for those users, so that they definitely know they need to use their new credentials. An early beta version of this feature is complete, and now needs design and product refinements to be completed. To help users who get renamed by the finalisation and, as a result, have several accounts that were previously local-only turned into separate global accounts, the team is working on a tool to merge global accounts. We chose to merge accounts as it was the easiest way to satisfy the use case without causing further local-global account clashes that would cause us to have to perform a second finalisation. The tool is in its preliminary stages. The team also globalised some accounts that were not globalised but had no clashes. These accounts were either created in this local-only form due to bugs, or are accounts from before CentralAuth <https://www.mediawiki.org/wiki/Extension:CentralAuth> was deployed where the user never globalised. As these accounts had no clashes, there were no repercussions to globalising these accounts, so we did this immediately. At present, no date has been chosen for the finalisation. The team plans to have the necessary engineering work done by the end of the quarter (end of September 2014), and have a date chosen by then. Next month the team plans to continue work on these features. *Security auditing and response <https://www.mediawiki.org/wiki/Security_auditing_and_response>* MediaWiki 1.23.2 was released, fixing 3 security bugs. Security reviews were made for BounceHandler and Petition extensions, and the password API was merged. Release Engineering <https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team> *Release Engineering <https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team>* This month, the Release and QA Team became the Release Engineering Team, mostly reflecting the transition of this team from being made up of members of other distinct teams to that of a coherent self-contained (mostly) team. This will, hopefully, allow better coordination of “Release” and “QA” things (broadly spreaking). A lot of progress was made on making Phabricator suitable as a task/bug tracking system for Wikimedia projects. You can see the work to be sorted and completed at this workboard <http://fab.wmflabs.org/project/board/31/>. The Beta Cluster <https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep> now runs with HHVM, bringing us much closer to full HHVM deployment. In addition, the Language Team deployed the newContent translation <https://www.mediawiki.org/wiki/Content_translation> system on the Beta Cluster with the help of the Release Engineering team. The second round of public RFP for third-party MediaWiki release management was conducted and concluded. We now no longer use the third-party Cloudbees service for any of our Jenkins jobs and run all jobs locally. This will enable us to better diagnose issues with our build process, especially as it pertains to our browser tests (which still mostly run on SauceLabs). *Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>* This month, the QA team finished two significant achievements: after porting all the remaining browser tests from the browsertestsrepository to the repositories of the extensions being tested in June, as well as porting a significant set of tests to MediaWiki core itself, we completely retired the Jenkins instance running on a third-party host in favor of running test builds from the Wikimedia Jenkins instance, and we deleted the /qa/browsertests code repository. These moves are the result of more than two years of work. In addition, we have added more functions to the API wrapper used by browser tests, improved support for testing in Vagrant virtual machines, added new Jenkins builds for extensions, and improved the function of the beta labs test environments by preventing database locks and stopping users from being logged out by accident. *Browser testing <https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>* The browser tests are now all integrated with builds on the Wikimedia Jenkins host. We added browser tests for MediaWiki core that will validate the correctness of a MediaWiki installation regardless of language, or of what extensions may or may not exist on the wiki, so that the tests may be packaged with the distribution of MediaWiki itself and used on arbitrary wikis. We saw a lot of browser test activity for Flow development, and we are preparing to support even more extensions and features in the very near future. Multimedia *Multimedia <https://www.mediawiki.org/wiki/Multimedia>* <https://commons.wikimedia.org/wiki/File:Media_Viewer_-_New_Design_Proposal_…> Media Viewer’s new ‘minimal design’. In July, the multimedia team reviewed more feedback aboutMedia Viewer <https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer>, from three separate Requests for Comments on the English and German Wikipedias, as well as on Wikimedia Commons. Based on this community feedback, the team worked to make the tool more useful for readers, while addressing editor concerns. We are now considering a new ‘minimal design’, which would include: a much more visible link to the File: page; an even easier way to disable the tool; a caption or description right below the image; removing additional metadata below the image, directing users to the File: page instead. As described in our improvements plan <https://docs.google.com/a/wikimedia.org/document/d/1iCDNOUK14D7xb47o33k1p0D…>, these new features are being prototyped and will be carefully tested with target users in August, so we can validate their effectiveness before developing and deploying them in September. You can see some of our thinking in this presentation <https://docs.google.com/a/wikimedia.org/presentation/d/1FNGLEzVsoELZqxiso_1…> . This month, we continued to work on the Structured Data <https://commons.wikimedia.org/wiki/Commons:Structured_data> project with theWikidata team <https://www.wikidata.org/wiki/> and many community members, to implement machine-readable data on Wikimedia Commons. We prepared to host a range on online and in-person discussions to plan this project with our communities, and aim to develop our first experiments in October, based on their recommendations. We also continued a major code refactoring for the UploadWizard <https://www.mediawiki.org/wiki/UploadWizard>, as well as fixed a number of bugs for some of our other multimedia tools. Last but not least, we prepared seven different multimedia roundtables and presentations <https://wikimania2014.wikimedia.org/wiki/Multimedia_Events> for Wikimania 2014, which we will report on in more depth in August. For now, you can keep up with our work by joining themultimedia mailing list <https://lists.wikimedia.org/mailman/listinfo/multimedia>. Engineering Community Team <https://www.mediawiki.org/wiki/Engineering_Community_Team> *Bug management <https://www.mediawiki.org/wiki/Bug_management>* At the Pywikibot bugdays, 189 reports received updates <http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077828.html>. Technically, Jan enabled invalidating the CSS cache <https://bugzilla.wikimedia.org/show_bug.cgi?id=49720> and strict transport security <https://gerrit.wikimedia.org/r/#/c/127256/>, Matanya updated Bugzilla’s cipher_suite <https://gerrit.wikimedia.org/r/#/c/144427/> and cleaned up a template <https://gerrit.wikimedia.org/r/#/c/144442/>, and Danieldeleted an unused config file <https://gerrit.wikimedia.org/r/145496>. Tyler and Andre added requested components to Bugzilla <https://bugzilla.wikimedia.org/buglist.cgi?bug_id=67661,67721,68549,68144>. Planning of an exposed “easy bug of the week” continued, summarized on a wikipage <https://www.mediawiki.org/wiki/Bug_management/Bug_of_the_week>. *Phabricator migration <https://www.mediawiki.org/wiki/Phabricator/Migration>* Phabricator’s “Legalpad” application (a tool to manage trusted users <http://fab.wmflabs.org/T364>) was set up on a separate server <https://legalpad.wikimedia.org/>. This instance provides WMF Single-User Login authentication <http://fab.wmflabs.org/T314>. Mukunda implemented restricting access to tasks in a certain project <http://fab.wmflabs.org/T95>which can be tested on fab.wmflabs.org. As a followup, he investigatedenforcing security policy also on files and attachments <http://fab.wmflabs.org/T477> and replacing the IRC bots by Phab’s chatbot <http://fab.wmflabs.org/T221>. Chase worked on initial migration code to import data from Bugzilla reports into Phabricator tasks (and ran into missing API code in Phabricator <http://fab.wmflabs.org/T497>), investigated configuring Exim for mail <http://fab.wmflabs.org/T407>, set up a data backup system for Phabricator <http://fab.wmflabs.org/T411>, and upgraded the dedicated Phabricator server to Ubuntu Trusty <http://fab.wmflabs.org/T405>. Quim starteddocumenting Phabricator <https://www.mediawiki.org/wiki/Phabricator/Help>. Andre helped making decisions on defining field values and how to handle certain Bugzilla fields in the import script <http://fab.wmflabs.org/T423> and sent a summary email to wikitech-l <http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077684.html> about the Phabricator migration status. *Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>* All Google Summer of Code <https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014> and FOSS Outreach Program for Women <https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8>projects continued their development toward a successful end. For details, check the reports: - Tools for mass migration of legacy translated wiki content <https://www.mediawiki.org/wiki/Extension:Translate/Mass_migration_tools/Pro…> - Wikidata annotation tool <https://www.mediawiki.org/wiki/Wikidata_annotation_tool/updates> - Email bounce handling to MediaWiki with VERP <https://www.mediawiki.org/wiki/VERP/GSOC_Progress_Rerport> - Google Books, Internet Archive, Commons upload cycle <https://www.mediawiki.org/wiki/Google_Books,_Internet_Archive,_Commons_uplo…> - UniversalLanguageSelector fonts for Chinese wikis <https://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector/Fonts_fo…> - MassMessage page input list improvements <https://www.mediawiki.org/wiki/Extension:MassMessage/Page_input_list_improv…> - Book management in Wikibooks/Wikisource <https://meta.wikimedia.org/wiki/Book_management_2014/Progress> - Parsoid-based online-detection of broken wikitext <https://www.mediawiki.org/wiki/User:Hardik95/GSoC_2014_Progress_Report> - Usability improvements for the Translate extension <https://www.mediawiki.org/wiki/User:Kunalgrover05/Progress_Report> - A modern, scalable and attractive skin for MediaWiki <https://www.mediawiki.org/wiki/User:Jack_Phoenix/GSoC_2014> - Automatic cross-language screenshots for user documentation <https://www.mediawiki.org/wiki/Automatic_cross-language_screenshots/progress> - Separating skins from core MediaWiki <https://www.mediawiki.org/wiki/Separating_skins_from_core_MediaWiki/Progress> - Chemical Markup support for Wikimedia Commons <https://www.mediawiki.org/wiki/Chemical_Markup_support_for_Wikimedia_Common…> - Improving URL citations on Wikimedia <https://www.mediawiki.org/wiki/User:Mvolz/Weekly_Reports> - Historical OpenStreetMap <https://www.mediawiki.org/wiki/User:JaimeLyn/Weekly_Reports> - Welcoming new contributors to Wikimedia Labs and Tool Labs <https://www.mediawiki.org/wiki/Welcome_to_labs/Progress_Reports> - Evaluating, documenting, and improving MediaWiki web API client libraries <https://www.mediawiki.org/wiki/Evaluating_and_Improving_MediaWiki_web_API_c…> - Feed the Gnomes – Wikidata Outreach <https://www.mediawiki.org/wiki/User:Thepwnco/OPW_Reporting> - Template Matching for RDFIO <https://www.mediawiki.org/wiki/Extension:RDFIO/Template_matching_for_RDFIO/…> - Switching Semantic Forms Autocompletion to Select2 <https://www.mediawiki.org/wiki/Extension:Semantic_Forms/Select2_for_autocom…> - Catalogue for Mediawiki Extensions <https://www.mediawiki.org/wiki/User:Adi.iiita/Gsoc2014/Report#Weekly_Report> - Generic, efficient localisation update service <https://www.mediawiki.org/wiki/Extension:LocalisationUpdate/LUv2/Updates> . *Technical communications <https://www.mediawiki.org/wiki/Technical_communications>* <https://commons.wikimedia.org/wiki/File:Flesch_reading_ease_chart_for_Tech_…> Chart showing historical Flesch reading ease <https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests> data for Tech News, a measure of the newsletter’s readability <https://meta.wikimedia.org/wiki/Tech/News/Readability>. Higher scores indicate material that is easier to read. A score of 60–70 corresponds to content easily understood by 13- to 15-year-old students. Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom> collaborated with authors of the Education newsletter <https://outreach.wikimedia.org/wiki/Education/News>to set it up <https://outreach.wikimedia.org/wiki/Education/Newsletter/Newsroom/Multiling…> for multilingual delivery, using a script <https://meta.wikimedia.org/wiki/Module:Assemble_multilingual_message> similar to the one used for Tech News <https://meta.wikimedia.org/wiki/Tech/News>. He also wrote adetailed how-to <https://meta.wikimedia.org/wiki/Newsletters/Translation> to accompany the script for people who want to send a multilingual message across wikis. In preparation for the Wikimania session <https://wikimania2014.wikimedia.org/wiki/Submissions/Tech_news> about Tech News, he updated the readability <https://meta.wikimedia.org/wiki/Tech/News/Readability> andsubscribers <https://meta.wikimedia.org/wiki/Tech/News/Subscribers> metrics. He also continued to provide ongoing communications support <https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity> for the engineering staff, and to prepare and distribute Tech News every week. *Volunteer coordination and outreach <https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>* We focused on the preparation of the Wikimania Hackathon <https://wikimania2014.wikimedia.org/wiki/Hackathon>, encouraging all registered participants to propose topics and sign up to interesting sessions. We also organized a Q&A session <https://plus.google.com/events/c0fgci542f8cn58o606gng6avio> with potential organizers of the Wikimedia Hackathon 2015. We organized two Tech Talks: *Hadoop and Beyond. An overview of Analytics infrastructure* <https://plus.google.com/u/0/b/103470172168784626509/events/c53ho5esd0luccd0…> and*HHVM in production: what that means for Wikimedia developers* <https://plus.google.com/events/cp5mjf6jrihevtdje8lmu5hvm1k>. More activities hosted in July can be found at Project:Calendar/2014/07 <https://www.mediawiki.org/wiki/Project:Calendar/2014/07>. *Architecture and Requests for comment process <https://www.mediawiki.org/wiki/Architecture_process>* Developers finished the security architecture guidelines <https://www.mediawiki.org/wiki/Security_for_developers/Architecture>, and discussed several requests for comment <https://www.mediawiki.org/wiki/Requests_for_comment> in online architecture meetings <https://www.mediawiki.org/wiki/Architecture_meetings>: - 2014-07-10 — Frontend standardization discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-10> focusing onRequests for comment/Redo skin framework <https://www.mediawiki.org/wiki/Requests_for_comment/Redo_skin_framework> ; - 2014-07-16 — RfC discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-16> focusing on Requests for comment/Vertical writing support <https://www.mediawiki.org/wiki/Requests_for_comment/Vertical_writing_support> ; - 2014-07-23 — RfC discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-23> focusing on Requests for comment/Composer managed libraries for use on WMF cluster <https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_librar…>, in which the architecture committee approved the RfC; - 2014-07-30 — RfC discussion <https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-30> focusing on Requests for comment/CentralNotice Caching Overhaul – Frontend Proxy <https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_O…> . *dev.wikimedia.org <https://www.mediawiki.org/wiki/Dev.wikimedia.org>* In July, Quim Gil sorted the tasks necessary for the first hub prototype <http://fab.wmflabs.org/project/board/60/> into a Phabricator board, and Sumana Harihareswara determined which three APIs <http://fab.wmflabs.org/T479> she would document first. Analytics <https://www.mediawiki.org/wiki/Analytics> *Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>* Wikimetrics can now generate vital sign metrics for every project daily. Rolling Monthly Active Editor metric has been implemented; the reports are in JSON format, in a logical path hosted on a file server and downloadable. The team also worked on backfilling data for the daily reports on Newly Registered and Rolling Active Editor, and numerous optimizations to backfill the data quickly. *Data Processing <https://www.mediawiki.org/wiki/Analytics/Data_Processing>* New nodes were added to the cluster this month and all machines were upgraded to run CDH5. The team decided not to preserve any data on the cluster during the upgrade and started fresh. The team hosted a Tech Talk on our Hadoop installation (see video <https://plus.google.com/u/0/events/c53ho5esd0luccd09a1c30rlrmg> and slides <https://docs.google.com/a/wikimedia.org/presentation/d/1ZPmfN-kmfqWEJUMIRg2…>). Duplicate monitoring has also been implemented in Hadoop to monitor the incoming Varnish logs. *Editor Engagement Vital Signs <https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs>* The culmination of our efforts this month can be visualized in a prototype built for Wikimania <https://metrics-staging.wmflabs.org/static/public/dash/>. This was made possible thanks to many back-end enhancements (optimizations) to Wikimetrics, along with research and selection of the optimal technologies to implement the stack to display a dashboard <https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs/Dash…> . *EventLogging <https://www.mediawiki.org/wiki/Analytics/EventLogging>* EventLogging monitoring is now in graphite, and we can see which schemas cause spikes in traffic (example <https://graphite.wikimedia.org/render/?width=588&height=311&_salt=140491272…> ). *Research and Data <https://www.mediawiki.org/wiki/Analytics/Research_and_Data>* This month, we completed the documentation for the Active Editor Model <https://meta.wikimedia.org/wiki/Research:Modeling_monthly_active_editors>, a set of metrics for observing sub-population trends and setting product team goals. We also engaged in further work on the new pageviews definition. An interim solution for Limited-duration Unique Client Identifiers (LUCIDs) was also developed and passed to the Analytics Engineering team for review. We analyzed trends in mobile readership and contributions, with a particular focus on the tablet switchover and the release of the native Android app. We found <https://meta.wikimedia.org/wiki/Research:Mobile_editor_engagement/Editor_ac…> that in the first half of 2014, mobile surpassed desktop in the rate at which new registered users become first-time editors and first-time active editors in many major projects, including the English Wikipedia. An update on mobile trends <https://commons.wikimedia.org/wiki/File:Wikimedia_Mobile_Trends.pdf> will be presented at the upcoming Monthly Metrics meeting on July 31. Development of a standardised toolkit <https://github.com/Ironholds/WMUtils> for geolocation, user agent parsing and accessing pageviews data was completed. We supported the multimedia team in developing a research study <https://meta.wikimedia.org/wiki/Research:Media_Viewer_preference_elicitation> to objectively measure the preference of Wikipedia editor and readers. We hosted the July research showcase <https://www.mediawiki.org/wiki/Analytics/Research_and_Data/Showcase#July_20…> with a presentation <https://commons.wikimedia.org/wiki/File:Halfak%27s_wiki_research_libraries_…> by Aaron Halfaker of 4 Python libraries for data analysis, and a guest talk by Center for Civic Media’s Nathan Matias <http://natematias.com/> on the use of open data <https://commons.wikimedia.org/wiki/File:Using_Open_Data_and_Stories_to_Broa…> to increase the diversity of collaboratively created content. We prepared 8 presentations <https://www.mediawiki.org/wiki/Analytics/Wikimania_2014> that we will be giving or co-presenting next week at Wikimania in London. We also organized the next WikiResearch hackathon <https://meta.wikimedia.org/wiki/Research:Labs2/Hackathons/August_6-7th,_2014> that will be jointly hosted in London (UK) <https://meta.wikimedia.org/wiki/Research:Labs2/Hackathons/August_6-7th,_201…> (during the pre-conference Wikimania Hackathon) and in Philadelphia (USA) <https://meta.wikimedia.org/wiki/Research:Labs2/Hackathons/August_6-7th,_201…> on August 6-7, 2014. We filled the fundraising research analyst position: the new member of the Research & Data team will join us in September and we’ll post an announcement on the lists shortly before his start date. Lastly, we gave presentations on current research at the Wikimedia Foundation at the Institute for Scientific Interchange <https://en.wikipedia.org/wiki/Institute_for_Scientific_Exchange> (Turin) and at theDesignDensity <http://www.densitydesign.org/> lab (Milan). Kiwix <http://www.kiwix.org/> <https://commons.wikimedia.org/wiki/File:Project_Gutenberg_screenshot.png> Screenshot of the first Project Gutenberg <http://www.gutenberg.org/> ZIM file *The Kiwix project is funded and executed by Wikimedia CH <https://meta.wikimedia.org/wiki/Wikimedia_CH>.* We have pre-release binaries <http://download.kiwix.org/bin/0.9/> of the next 0.9 (final) release. Except for OSX everything seems to work file as far. The support of RaspberryPi <http://www.raspberrypi.org/> was finally merged to the kiwix-plug master branch <https://sourceforge.net/p/kiwix/other/ci/master/tree/plug/>; this offers new perspectives because the price to create a Kiwix-Plug <http://www.kiwix.org/wiki/Kiwix-plug> has dropped to around USD 100. We also started an engineering collaboration with ebook reader manufacturer Bookeen <http://www.bookeen.com/en/> (in the scope of the Malebooks <http://malebooks.ml/> project) to be able offer an offline version of Wikipedia on e-ink devices. We participated in the Google Serve Day at Google Zurich. The goal was to meet Google engineers during one day and have them work on open source projects. The result was a dozen of fixed bugs and implemented features, mostly on Kiwix for Android <https://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile>, but also in Kiwix for desktop and MediaWiki. Four developers had a one-week hackathon in Lyon, France to develop an offline version of the Gutenberg library <http://www.gutenberg.org/>. We’re currently polishing the code and plan a release soon; our partners and sponsors plan the first deployments in Africa in Autumn. Last but not least, a proof-of-concept of a Kiwix iOS app was made, so we might release a first app before the end of the year. Wikidata <https://meta.wikimedia.org/wiki/Wikidata> *The Wikidata project is funded and executed by Wikimedia Deutschland <https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.* The biggest improvement around Wikidata in July is the release of the entity suggester <https://lists.wikimedia.org/pipermail/wikidata-l/2014-July/004148.html>. It makes it a lot easier to see what kind of information is missing on an item. Helen and Anjali, Wikidata’s Outreach Program for Women interns, continued improving user documentation and outreach around Wikidata as well as worked on a new design for the main page <https://www.wikidata.org/wiki/Wikidata:Portal_Redesign>. Guided Tours <https://www.wikidata.org/wiki/Wikidata:Tours>were published, helping newcomers find their way around the site. The developers further worked on supporting badges (like “featured article”), redirects between items, the monolingual text datatype (to be able to express things like the motto of a country) as well as the first implementation steps for the new user interface design. Additionally the first JSON dumps <https://lists.wikimedia.org/pipermail/wikidata-l/2014-July/004216.html> were published. FutureThe engineering management team continues to update the *Deployments <https://wikitech.wikimedia.org/wiki/Deployments>*page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals>*, listing ongoing and future Wikimedia engineering efforts. ------------------------------ *This article was written collaboratively by Wikimedia engineers and managers. See revision history <https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/20…> and associated status pages. A wiki version <https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/July> is also available.* -- Guillaume Paumier Technical Communications Manager — Wikimedia Foundation

6 7

mediawiki now being tested with vendor repo
by Antoine Musso 27 Aug '14

27 Aug '14

Hello, We have a mediawiki/vendor repository that holds third party libraries. Since MediaWiki core is going to eventually rely on them I have crafted a new Jenkins job (mediawiki-vendor-integration) which clones both repositories, checkout the appropriate patch / branch and run the whole MediaWiki PHPUnit test suite. The job is triggered when a patch is proposed on either mediawiki/core or mediawiki/vendor but only for the master and REL* branches for now. I have made it non voting. The job takes roughly 6 minutes that will slightly delay the report to Gerrit. If the job looks fine. The next steps are: - make it pass on wmf branches - trigger it run on CR+2 Then, I will be able to phase out the grouped PHPUnit groups and the sqlite installer tester. Long term: have the tests to run using HHVM as well. -- Antoine "hashar" Musso

3 4

build.wikimedia.org
by Quim Gil 27 Aug '14

27 Aug '14

(This is the project formerly known as Wikimedia Data & Developer Hub) build.wikimedia.org aims to become a hub for developers and researchers interested in Wikimedia data and the APIs to interact with it. Wikimedia has developed a number of repositories of free and open datasets, and has made its APIs freely available. However, these APIs and their documentation are currently scattered across several projects, with no central location for people to learn about and build upon them. Let's fix this problem! Wikimedia has so much to offer. https://www.mediawiki.org/wiki/build.wikimedia.org http://fab.wmflabs.org/project/board/60/ Currently we are developing a prototype.[1] We are still discussing the technologies to be used in the final implementation,[2] including the specific use of MediaWiki. Your ideas and help are welcome. 1 - http://fab.wmflabs.org/T478 2 - http://fab.wmflabs.org/T491 -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

5 4

Make port 22 on gerrit.wikimedia.org default for git (same as what 29418 is now)
by Petr Bena 26 Aug '14

26 Aug '14

gerrit.wikimedia.org is a default git gateway for all of wikimedia projects, but it still has a number of issues compared to other (IMHO better) providers, like GitHub. One of major issues I am now having is, that git needs to be accessed using non-standard port because regular ssh access is still being served on port 22. This is a problem on restricted networks where random ports are prohibited and blocked for various reasons, while standard ports like 21, 22, 80 etc are open. I propose to move the current ssh port (22) to some non-standard port, as staff which needs to use it is far smaller than developer community that needs to access git (the server itself is for git only) and open port 22 as a gerrit (git) port. The current port 29418 can be used as well, so that nobody needs to update their repo's which were already cloned. This is technically possible and I think it would make usage of gerrit much less of a pain in the ****. Thanks for your considerations :P

2 1

← Newer
1
2
3
4
5
6
7
...
14
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l August 2014