Wikimedia engineering report, July 2014 - Wikitech-l

26 Aug 2014


      Hi,
The report covering Wikimedia engineering activities in July 2014 is now
available.
Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July
Blog version:
https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/
We're also proposing a shorter version of this report focusing on priority
goals for this quarter:
https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July/summar...
Below is the HTML text of the report.
As always, feedback is appreciated on the usefulness of the report and its
summary, and on how to improve them.
------------------------------------------------------------------
Major news in July include:
- a recap of how the Operations team collaborated with the RIPE NCC
   https://blog.wikimedia.org/2014/07/09/how-ripe-atlas-helped-wikipedia-users/
to
   measure the delivery of Wikimedia sites to users in Asia and elsewhere;
   - an analysis of the impact of the San Francisco data center
   https://blog.wikimedia.org/2014/07/11/making-wikimedia-sites-faster/ on
   the speed of Wikimedia sites;
   - the launch of the new native Wikipedia app for iOS
   https://blog.wikimedia.org/2014/07/31/official-wikipedia-app-available-on-ios-and-android/
   ;
   - a first look at the content translation tool
   https://blog.wikimedia.org/2014/07/16/first-look-at-the-content-translation-tool/
   .
*Note: We’re also providing a shorter and translatable version of this
report
https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/July/summary.*
Engineering metrics in July:
- 164 unique committers contributed patchsets of code to MediaWiki.
   - The total number ofunresolved commits
   https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z
went
   from around 1575 to about 1642.
   - About 31 shell requests
https://www.mediawiki.org/wiki/Shell_requestswere
   processed.
Contents
- Personnel
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Personnel
      - Work with us
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Work_with_us
      - Announcements
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Announcements
   - Technical Operations
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Technical_Operations
   - Features Engineering
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Features_Engineering
      - Editor retention: Editing tools
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/_Editing_tools
      - Services
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Services
      - Core Features
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Core_Features
      - Growth
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Growth
   - Mobile
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Mobile
   - Language Engineering
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Language_Engineering
   - Platform Engineering
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Platform_Engineering
      - MediaWiki Core
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#MediaWiki_Core
      - Release Engineering
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Release_Engineering
      - Multimedia
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Multimedia
      - Engineering Community Team
      https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Engineering_Community_Team
   - Analytics
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Analytics
   - Kiwix
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Kiwix
   - 10 Wikidata
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Wikidata
   - 11 Future
   https://blog.wikimedia.org/2014/08/26/engineering-report-july-2014/#Future
PersonnelWork with us https://wikimediafoundation.org/wiki/Work_with_us
Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.
- VP of Engineering
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ods8Xfwu
   - Software Engineer – Front-end (VisualEditor)
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=o8jyYfwH
   - Software Engineer – Services
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oAhYYfwx
   - Software Engineer – Front-end
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oxgWYfwr
   - Software Engineer – Maps & Geo
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ojIlZfw5
   - Software Engineer – Mobile – iOS
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=ovD8YfwY
   - QA Tester
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oKIUYfw4
   - Software Engineer – Full Stack
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=orQ2Yfw1
   - Lean/Agile Coach
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oVKlZfwJ
   - Product Manager
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oQf8YfwV
   - Product Manager – Language Engineering
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=osiMYfwe
   - Operations Security Engineer
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oT6cYfwT
   - UX Senior Designer http://grnh.se/veo5n8
   - UX Senior Design Researcher http://grnh.se/38vlip
   - UX User Research Recruiter http://grnh.se/kg42d7
   - Project Coordinator – Engineering
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oUleZfwc&s
   - Mobile Partnerships Regional Manager
   http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oTviZfwp&s
   - Program Evaluation Internship http://grnh.se/2ekx4h
Announcements
- Arthur Richards is now Team Practices Manager (announcement
   http://lists.wikimedia.org/pipermail/wikimediaannounce-l/2014-July/000956.html
   ).
   - Kristen Lans joined the Team Practices Group as Scrum Master (
   announcement
   http://lists.wikimedia.org/pipermail/teampractices/2014-July/000437.html
   ).
   - Joel Sahleen joined the Language Engineering team as Software Engineer
   (announcement
   http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077829.html).
Technical Operations
*Dallas data center*
Throughout July, the cabling work of all racked servers and other equipment
was nearly completed. We’re still awaiting the installation of the first
connectivity to the rest of our US network in early August before we can
begin installation of servers and services.
*San Francisco data center*
Due to a necessary upgrade to power & cooling infrastructure in our San
Francisco data center (which we call *ulsfo*), our racks have been migrated
to a new floor within the same building on July 9. The move completed in a
very smooth fashion without user impact, and the site was brought back
online serving all user traffic again in less than 24 hours.
*PFS enabled*
Through the help of volunteer work and research, our staff enabled Perfect
Forward Secrecy https://en.wikipedia.org/wiki/en:Perfect_Forward_Secrecy on
our SSL infrastructure, significantly increasing the security of encrypted
user traffic.
Labs metrics in July:
- Number of projects: 173
   - Number of instances: 464
   - Amount of RAM in use (in MBs): 1,933,824
   - Amount of allocated storage (in GBs): 20,925
   - Number of virtual CPUs in use: 949
   - Number of users: 3,500
*Wikimedia Labs*
We’ve made several minor updates to Wikitech: we added OAuth support, fixed
a few user interface issues, and purged the obsolete ‘local-*’ terminology
for service groups. OPW Intern Dinu Sandaru has set forms for structured
project documentation. This should will help match new volunteers with
existing projects, and will make communication with project administrators
more straightforward. Sean Pringle is in the process of updating the Tool
Labs replica databases to MariaDB version 10.0. This may reduce replag, and
should improve performance and reliability. We’re setting up new storage
hardware for the project dumps. This will resolve our ongoing problems with
full drives and out-of-date dumps.
 Features Engineering
https://www.mediawiki.org/wiki/Wikimedia_Features_engineering Editor
retention: Editing tools
*VisualEditor https://www.mediawiki.org/wiki/VisualEditor*
In July, the team working on VisualEditor converged the design for mobile
and desktop, made it possible to see and edit HTML comments, improved
access to re-using citations, and fixed over 120 bugs and tickets
https://bugzilla.wikimedia.org/buglist.cgi?list_id=333021&order=priority%2Cbug_severity&product=VisualEditor&query_format=advanced&resolution=FIXED&target_milestone=VE-deploy-2014-07-03&target_milestone=VE-deploy-2014-07-10&target_milestone=VE-deploy-2014-07-17&target_milestone=VE-deploy-2014-07-24&target_milestone=VE-deploy-2014-07-31
.
The new design, with controls focussed at the top of each window in
consistent positions, was made possible due to the significant progress
made in cross-platform support in the UI library, which now provides
responsively-sized windows that can work on desktop, tablet and phone with
the same code. HTML comments are occasionally used on a few articles to
alert editors to contentious or problematic issues without disrupting
articles as they are read, so making them prominently visible avoids
editors accidentally stepping over expected limits. Re-using citations is
now provided with its simple dialog available in the toolbar so that it is
easier for users to find.
Other improvements include an array of performance fixes targeted at
helping mobile users especially, fixing a number of minor instances where
VisualEditor would corrupt the page, and installing better monitoring of
corruptions if they occur, and better support for right-to-left languages,
displaying icons with the right orientation based on context.
The mobile version of VisualEditor, currently available for beta testers,
moved towards stable release, fixing a number of bugs and editing issues
and improving loading performance. Our work to support languages made some
significant gains, nearing the completion of a major task to support IME
users, and the work to support Internet Explorer uncovered some more issues
as well as fixes. The deployed version of the code was updated five times
in the regular release cycle (1.24-wmf12
https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf8#VisualEditor,
1.24-wmf13 https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf9#VisualEditor
, 1.24-wmf14
https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf10#VisualEditor,
1.24-wmf15
https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf15#VisualEditor and
1.24-wmf16
https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf16#VisualEditor).
In wider news, the team expanded its scope to cover all MediaWiki editing
tools as well, as the new Editing Team (covered below).
*Editing https://www.mediawiki.org/wiki/Editing*
In July, the newly re-named and re-scoped Editing Team
https://www.mediawiki.org/wiki/Editing was formed from the VisualEditor
Team. We are responsible for extending and improving the editing tools used
at Wikimedia – primarily VisualEditor and maintenance for WikiEditor. We
exist to support new and existing editors alike; our current work is mostly
on desktop, and we are working with Mobile to take responsibility for all
editing across desktop, tablet and phone platforms, spanning approximately
50 different areas of MediaWiki and extensions related to editing. We will
continue to report progress on VisualEditor separately.
The biggest Editing change this month was in the Cite extension (for
footnotes) – this now automatically shows a references list at the end of
the page if you forget to put in a <references /> tag, instead of
displaying an ugly error message. The Math extension (for formulæ) was
improved with more rigorous error handling and LaTeX formula checking, as
part of the long-term volunteer-led work to introduce MathML-based display
and editing. The TemplateData GUI editor was deployed to a further six
wikis – the English, French, Italian, Russian, Finnish and Dutch Wikipedias.
A lot of work was done on libraries and infrastructure for the Editing Team
and others. The OOjs UI https://www.mediawiki.org/wiki/OOjs_UI library
was extensively modified to bring in a new window management system for
comprehensive combined desktop, tablet and phone support, as well as other
updates to improve Internet Explorer compatibility and accessibility of
controls. In the next few months the team will continue working on OOUI to
support other teams’ needs and implement a consistent look-and-feel in
collaboration with the Design team. The OOjs
https://www.mediawiki.org/wiki/OOjs library was updated to fix a minor
bug, with a new version (v1.0.11) released and pushed downstream into
MediaWiki, VisualEditor and OOjs UI. The ResourceLoader framework was
extended to allow skins to set the “skinStyles” property themselves, rather
than rely on faux dependencies, as part of wider efforts led jointly by a
volunteer and a team member to improve MediaWiki’s skin support.
*Parsoid https://www.mediawiki.org/wiki/Parsoid*
In July, the Parsoid team continued with ongoing bug fixes and bi-weekly
deployments.
With an eye towards supporting Parsoid-driven page views, the Parsoid team
strategized on addressing Cite extension rendering differences that arise
from site-messages based customizations and is considering a pure CSS-based
solution for addressing the common use cases. We also finished work
developing the test setup for doing mass visual diff tests between PHP
parser rendering and Parsoid rendering. It was tested locally and we
started preparations for deploying that on our test servers. This will go
live end-July or early-August.
The GSoC 2014 LintTrap project continued to make good progress. We had
productive conversations with Project WikiCheck about integrating LintTrap
with WikiCheck in a couple different ways. We hope to develop this further
over the coming months.
Overall, this was also a month of reduced activity with Gabriel now
officially full time in the Services team and Scott focused on the PDF
service deployment that went live a couple days ago. The full team is also
spending a week at a off-site meeting working and spending time together in
person prior to Wikimania in London.
Services
*Services and REST API https://www.mediawiki.org/wiki/Services*
The brand new Services https://www.mediawiki.org/wiki/Services group
(currently Matt Walker and Gabriel Wicke) started July with two main
projects:
1. PDF render https://www.mediawiki.org/wiki/PDF_rendering service
   deployment
   2. Design and prototyping work on the storage service
   https://www.mediawiki.org/wiki/Requests_for_comment/Storage_service
    and REST API
   https://www.mediawiki.org/wiki/Requests_for_comment/Content_API
The PDF render service is now deployed in production, and can be selected
as a render backend in Special:Book. The renderer does not work perfectly
on all pages yet, but the hope is that this will soon be fixed in
collaboration with the other primary author of this service, C. Scott
Ananian.
Prototyping work on the storage service https://github.com/gwicke/rashomon
 and REST API https://github.com/gwicke/restface is progressing well. The
storage service now has early support for bucket creation and multiple
bucket types. We decided to configure the storage service as a backend for
the REST API server. This means that all requests will be sent to the REST
API, which will then route them to the appropriate storage service without
network overhead. This design lets us keep the storage service buckets very
general by adding entry point specific logic in front-end handlers. The
interface is still well-defined in terms of HTTP requests, so it remains
straightforward to run the storage service as a separate process. We
refined the bucket design to allow us to add features very similar to Amazon
DynamoDB https://en.wikipedia.org/wiki/Amazon_DynamoDB in a future
iteration. There is also an early design for light-weight HTTP transaction
support.
Matt Walker is sadly leaving the Foundation by the end of this month to
follow his passion of building flying cars. This means that we currently
have three positions open
http://hire.jobvite.com/CompanyJobs/Careers.aspx?c=qSa9VfwQ&cs=9UL9Vfwt&page=Job%20Description&j=oAhYYfwx
in
the service group, which we hope to start filling soon.
Core Features
*Flow https://www.mediawiki.org/wiki/Flow/Project_information*
In July, the Flow team built the ability for users to subscribe to
individual Flow discussions, instead of following an entire page of
conversations. Subscribing to an individual thread is automatic for users
who create or reply to the thread, and users can choose to subscribe (or
unsubscribe) by clicking a star icon in the conversation’s header box.
Users who are subscribed to a thread receive notifications about any
replies or activity in that thread. To support the new
subscription/notification system, the team created a new namespace, Topic,
which is the new “permalink” URL for discussion threads; when a user clicks
on a notification, the target link will be the Topic page, with the new
messages highlighted with a color. The team is currently building a new
read/unread state for Flow notifications, to help users keep track of the
active discussion topics that they’re subscribed to.
Growth
*Growth https://www.mediawiki.org/wiki/Growth*
In July, the Growth team completed its second round of A/B testing ofsignup
invitations for anonymous editors
https://www.mediawiki.org/wiki/Anonymous_editor_acquisition/Signup_invites on
English Wikipedia, includingdata analysis
https://meta.wikimedia.org/wiki/Research:Asking_anonymous_editors_to_register.
The team also built the first API and interface prototypes for task
recommendations https://www.mediawiki.org/wiki/Task_recommendations. This
new system, first aimed at brand new editors, makes suggestions based on a
user’s previous edits.
 Mobile https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering
*Wikimedia Apps https://www.mediawiki.org/wiki/Wikimedia_Apps*
Following on from the successful launch to Android, the Mobile Apps team
released the new native Wikipedia app to iOS on July 31. The app is the iOS
counterpart to the Android app, with many of the same features such as
editing, saving pages for offline reading, and browsing history. The iOS
app also contains an onboarding screen that is shown the first time the app
is launched, asking users to sign up, a feature which was also launched on
Android this month (see below).
On Android this month we released to production accessibility and styling
features which were requested by our users, such as a night mode for
reading in the dark and a font size selector. We also released an
onboarding screen that asks users to sign up.
Our plan for next month is to get user feedback from Wikimania, wrap up our
styling fixes, and begin work on an onboarding screen the first time that
someone taps edit.
*Mobile web projects https://www.mediawiki.org/wiki/Mobile_web_projects*
This month, the team continued to focus on wrapping up the collaboration
with the Editing team to bring VisualEditor to tablet users on the mobile
site. We also began working to design and prototype our first new Wikidata
contribution stream, which we will build and test with users on the beta
site in the coming month.
*Wikipedia Zero https://www.mediawiki.org/wiki/Wikipedia_Zero*
During the last month, the team worked on software architecture features
that allow for expansion of the Wikipedia Zero footprint on partner
networks and that get users to content faster with support for lowered
cache fragmentation on Varnish caches. Whereas the previous system
supported one-size-fits-all configuration for heterogeneous partner
networks, inhibiting some zero-rated access, the new system supports
multiple configurations for disparate IP addresses and connection profiles
per operator. Additionally, lightweight script and GIF-ified Wikipedia Zero
banner support has been added and is being tested; in time this should
drastically reduce Varnish cache fragmentation, making pages be served
faster and reducing Varnish server load. A faster landing page was
introduced for “zerodot” (zero.wikipedia.org, legacy text-only experience)
landing pages when operators have multiple popular languages in their
geography. Work on compression proxy traffic analysis for header enrichment
conformance with the official Wikipedia Zero configurations was also
performed after more diagnostic logging code was added to the system.
Finally, watchlist thumbnails, although low bandwidth, were removed from
the zerodot user experience, as was the higher bandwidth MediaViewer
feature for zerodot; mdot will have these features, though.
In side project work, the team spent time on API continuation queries,
Android IP editing notices, Amazon Kindle and other non-Google Play
distribution, and Google Play reviews (now that the Android launch dust has
settled, mobile apps product management will be triaging the reviews). In
partnerships work, the team met with Mozilla to talk about future plans for
the Firefox OS HTML5 app (e.g., repurposing the existing mobile website,
but without any feature reduction) and how Wikimedia search might be
further integrated into Firefox OS, and also spoke with Canonical about how
Wikipedia might be better integrated into the forthcoming Ubuntu Phone OS.
Routine pre- and post-launch configuration changes were made to support
operator zero-rating, with routine technical assistance provided to
operators and the partner management team to help add zero-rating and
address anomalies. The team also continued its search for a third Partners
engineering teammate.
*Wikipedia Zero (partnerships)*
We served an estimated 68 million free page views in July through Wikipedia
Zero. We continue to bring new partners into the program, though none
launched in July. Adele Vrana met with prospective partners and local
Wikimedians in Brazil. We published our operating principles
https://wikimediafoundation.org/wiki/Wikipedia_Zero_Operating_Principles to
increase transparency.
 Language Engineering
https://www.mediawiki.org/wiki/Wikimedia_Language_engineering
*Language tools https://www.mediawiki.org/wiki/Language_tools*
CLDR extension was updated to use CLDR 25
http://cldr.unicode.org/index/downloads/cldr-25; this work was mostly
done by Ryan Kaldari. The team made various internationalization fixes in
core, MobileFrontend, Wikipedia Android app, Flow, VisualEditor and other
features. In the Translate extension, Niklas Laxström fixed
ElasticSearchTTMServer https://gerrit.wikimedia.org/r/145769 to provide
translation memory suggestions longer than one word; and improved
translation memory suggestions for translation units containing variables (bug
67921 https://bugzilla.wikimedia.org/show_bug.cgi?id=67921).
*Language Engineering Communications and Outreach
https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach*
We announced the initial availability of the Content translation tool
https://blog.wikimedia.org/2014/07/16/first-look-at-the-content-translation-tool/
with
limited feature support. We are focusing on supporting Spanish to Catalan
translations for this initial release. You can read a report on thefeedback
received since deployment
https://www.mediawiki.org/wiki/Content_translation/Updates/31July2014.
*Content translation https://www.mediawiki.org/wiki/Content_translation*
An initial version
http://es.wikipedia.beta.wmflabs.org/wiki/Especial:ContentTranslation was
released on Beta Labs; it supports machine translation between Spanish and
Catalan. The machine translation API leverages open source machine
translation with Apertium. The tool supports experimental template
adaptation between languages. Numerous bug fixes were made based on testing
and user feedback. We worked on matching the Apertium version to the
cluster, and planning for the next round of development has started.
 Platform Engineering
https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering MediaWiki
Core
*HHVM https://www.mediawiki.org/wiki/HHVM*
The Beta cluster https://www.mediawiki.org/wiki/Beta_cluster is running
HHVM. The latest MediaWiki-Vagrant
https://www.mediawiki.org/wiki/MediaWiki-Vagrant andLabs-vagrant
https://wikitech.wikimedia.org/wiki/Labs-vagrant use HHVM by default.
*Admin tools development
https://www.mediawiki.org/wiki/Admin_tools_development*
Most admin tools resources are currently diverted towards SUL finalisation
https://www.mediawiki.org/wiki/SUL_finalisation, which will greatly help
in reducing the admin tools backlog. July saw the deployment of the global
rename tool (bug 14862
https://bugzilla.wikimedia.org/show_bug.cgi?id=14862), and core fixes
including the creation of the “viewsuppressed” userright (bug 20476
https://bugzilla.wikimedia.org/show_bug.cgi?id=20476).
*Search https://www.mediawiki.org/wiki/Search*
Our deployment of CirrusSearch to larger wikis as the primary search
back-end turned out to be too ambitious. After encountering performance
issues, we rolled back this change. We are now addressing the root of the
problem, by getting more servers (nearly doubling the cluster size) and
putting together more optimizations to the portion of Cirrus that fell over
(working set). If everything goes as planned, it’ll be reduced by about
80%, by reducing indexing performance in return of search performance.
These optimizations will slightly change result relevance; please let us
know if you notice any issues.
*Auth systems https://www.mediawiki.org/wiki/Auth_systems*
Most work was spent on SUL Finalization tasks. Phpunit and browser tests
were added for CentralAuth, global rename was deployed, and lots of small
fixes were made to CentralAuth to clean up user accounts in preparation for
finalization.
*SUL finalisation https://www.mediawiki.org/wiki/SUL_finalisation*
In July, the SUL finalisation team began work on completing the necessary
feature work to support the SUL finalisation.
To help users with local-only accounts that are going to be forcibly
renamed due to the SUL finalisation, the team is working on a form that
lets those users request a rename. These requests will be forwarded onto
the stewards https://meta.wikimedia.org/wiki/Stewards to handle. The SUL
team is currently in consultation with the stewards about how they would
like this tool to work. When this consultation is wrapped up, the team will
begin design and implementation.
To help users get globally renamed without having to request renames on
potentially hundreds of wikis, the team implemented and deployed
GlobalRenameUser, a tool which renames users globally. As the tool is
designed to work post-finalisation, it only performs renames where the
current name is global, and the requested name is totally untaken (no
global account and no local accounts exist with that name).
To help users who get renamed by the finalisation and, despite our best
efforts to reach out to them, did not get the chance to request a rename
before the finalisation, the team is working on a feature to let users log
in with their old credentials. The feature will display an interstitial
when they log in, informing them that they logged in with old credentials
and that they need to use new ones. We are also considering a persistent
banner for those users, so that they definitely know they need to use their
new credentials. An early beta version of this feature is complete, and now
needs design and product refinements to be completed.
To help users who get renamed by the finalisation and, as a result, have
several accounts that were previously local-only turned into separate
global accounts, the team is working on a tool to merge global accounts. We
chose to merge accounts as it was the easiest way to satisfy the use case
without causing further local-global account clashes that would cause us to
have to perform a second finalisation. The tool is in its preliminary
stages.
The team also globalised some accounts that were not globalised but had no
clashes. These accounts were either created in this local-only form due to
bugs, or are accounts from before CentralAuth
https://www.mediawiki.org/wiki/Extension:CentralAuth was deployed where
the user never globalised. As these accounts had no clashes, there were no
repercussions to globalising these accounts, so we did this immediately.
At present, no date has been chosen for the finalisation. The team plans to
have the necessary engineering work done by the end of the quarter (end of
September 2014), and have a date chosen by then.
Next month the team plans to continue work on these features.
*Security auditing and response
https://www.mediawiki.org/wiki/Security_auditing_and_response*
MediaWiki 1.23.2 was released, fixing 3 security bugs. Security reviews
were made for BounceHandler and Petition extensions, and the password API
was merged.
Release Engineering
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team
*Release Engineering
https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team*
This month, the Release and QA Team became the Release Engineering Team,
mostly reflecting the transition of this team from being made up of members
of other distinct teams to that of a coherent self-contained (mostly) team.
This will, hopefully, allow better coordination of “Release” and “QA”
things (broadly spreaking).
A lot of progress was made on making Phabricator suitable as a task/bug
tracking system for Wikimedia projects. You can see the work to be sorted
and completed at this workboard http://fab.wmflabs.org/project/board/31/.
The Beta Cluster
https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-prep now
runs with HHVM, bringing us much closer to full HHVM deployment. In
addition, the Language Team deployed the newContent translation
https://www.mediawiki.org/wiki/Content_translation system on the Beta
Cluster with the help of the Release Engineering team.
The second round of public RFP for third-party MediaWiki release management
was conducted and concluded.
We now no longer use the third-party Cloudbees service for any of our
Jenkins jobs and run all jobs locally. This will enable us to better
diagnose issues with our build process, especially as it pertains to our
browser tests (which still mostly run on SauceLabs).
*Quality Assurance https://www.mediawiki.org/wiki/Quality_Assurance*
This month, the QA team finished two significant achievements: after
porting all the remaining browser tests from the browsertestsrepository to
the repositories of the extensions being tested in June, as well as porting
a significant set of tests to MediaWiki core itself, we completely retired
the Jenkins instance running on a third-party host in favor of running test
builds from the Wikimedia Jenkins instance, and we deleted the
/qa/browsertests code repository. These moves are the result of more than
two years of work. In addition, we have added more functions to the API
wrapper used by browser tests, improved support for testing in Vagrant
virtual machines, added new Jenkins builds for extensions, and improved the
function of the beta labs test environments by preventing database locks
and stopping users from being logged out by accident.
*Browser testing
https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing*
The browser tests are now all integrated with builds on the Wikimedia
Jenkins host. We added browser tests for MediaWiki core that will validate
the correctness of a MediaWiki installation regardless of language, or of
what extensions may or may not exist on the wiki, so that the tests may be
packaged with the distribution of MediaWiki itself and used on arbitrary
wikis. We saw a lot of browser test activity for Flow development, and we
are preparing to support even more extensions and features in the very near
future.
Multimedia
*Multimedia https://www.mediawiki.org/wiki/Multimedia*
https://commons.wikimedia.org/wiki/File:Media_Viewer_-_New_Design_Proposal_-_Rapa_Nui.png
Media Viewer’s new ‘minimal design’.
In July, the multimedia team reviewed more feedback aboutMedia Viewer
https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer, from three
separate Requests for Comments on the English and German Wikipedias, as
well as on Wikimedia Commons. Based on this community feedback, the team
worked to make the tool more useful for readers, while addressing editor
concerns. We are now considering a new ‘minimal design’, which would
include: a much more visible link to the File: page; an even easier way to
disable the tool; a caption or description right below the image; removing
additional metadata below the image, directing users to the File: page
instead.
As described in our improvements plan
https://docs.google.com/a/wikimedia.org/document/d/1iCDNOUK14D7xb47o33k1p0D48688EIGjERZZomwpYEA/edit,
these new features are being prototyped and will be carefully tested with
target users in August, so we can validate their effectiveness before
developing and deploying them in September. You can see some of our
thinking in this presentation
https://docs.google.com/a/wikimedia.org/presentation/d/1FNGLEzVsoELZqxiso_1wE2sYR2JO6PzQd_aeYdTyXx0/edit#slide=id.g3711baa7d_3_346
.
This month, we continued to work on the Structured Data
https://commons.wikimedia.org/wiki/Commons:Structured_data project with
theWikidata team https://www.wikidata.org/wiki/ and many community
members, to implement machine-readable data on Wikimedia Commons. We
prepared to host a range on online and in-person discussions to plan this
project with our communities, and aim to develop our first experiments in
October, based on their recommendations. We also continued a major code
refactoring for the UploadWizard
https://www.mediawiki.org/wiki/UploadWizard, as well as fixed a number of
bugs for some of our other multimedia tools.
Last but not least, we prepared seven different multimedia roundtables and
presentations https://wikimania2014.wikimedia.org/wiki/Multimedia_Events for
Wikimania 2014, which we will report on in more depth in August. For now,
you can keep up with our work by joining themultimedia mailing list
https://lists.wikimedia.org/mailman/listinfo/multimedia.
Engineering Community Team
https://www.mediawiki.org/wiki/Engineering_Community_Team
*Bug management https://www.mediawiki.org/wiki/Bug_management*
At the Pywikibot bugdays, 189 reports received updates
http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077828.html.
Technically, Jan enabled invalidating the CSS cache
https://bugzilla.wikimedia.org/show_bug.cgi?id=49720 and strict transport
security https://gerrit.wikimedia.org/r/#/c/127256/, Matanya updated
Bugzilla’s
cipher_suite https://gerrit.wikimedia.org/r/#/c/144427/ and cleaned up a
template https://gerrit.wikimedia.org/r/#/c/144442/, and Danieldeleted an
unused config file https://gerrit.wikimedia.org/r/145496. Tyler and Andre
added requested components to Bugzilla
https://bugzilla.wikimedia.org/buglist.cgi?bug_id=67661,67721,68549,68144.
Planning of an exposed “easy bug of the week” continued, summarized on a
wikipage https://www.mediawiki.org/wiki/Bug_management/Bug_of_the_week.
*Phabricator migration
https://www.mediawiki.org/wiki/Phabricator/Migration*
Phabricator’s “Legalpad” application (a tool to manage trusted users
http://fab.wmflabs.org/T364) was set up on a separate server
https://legalpad.wikimedia.org/. This instance provides WMF Single-User
Login authentication http://fab.wmflabs.org/T314.
Mukunda implemented restricting access to tasks in a certain project
http://fab.wmflabs.org/T95which can be tested on fab.wmflabs.org. As a
followup, he investigatedenforcing security policy also on files and
attachments http://fab.wmflabs.org/T477 and replacing the IRC bots by
Phab’s chatbot http://fab.wmflabs.org/T221. Chase worked on initial
migration code to import data from Bugzilla reports into Phabricator tasks
(and ran into missing API code in Phabricator http://fab.wmflabs.org/T497),
investigated configuring Exim for mail http://fab.wmflabs.org/T407, set
up a data backup system for Phabricator http://fab.wmflabs.org/T411,
and upgraded
the dedicated Phabricator server to Ubuntu Trusty
http://fab.wmflabs.org/T405. Quim starteddocumenting Phabricator
https://www.mediawiki.org/wiki/Phabricator/Help.
Andre helped making decisions on defining field values and how to handle
certain Bugzilla fields in the import script http://fab.wmflabs.org/T423 and
sent a summary email to wikitech-l
http://lists.wikimedia.org/pipermail/wikitech-l/2014-July/077684.html about
the Phabricator migration status.
*Mentorship programs https://www.mediawiki.org/wiki/Mentorship_programs*
All Google Summer of Code
https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014 and FOSS
Outreach Program for Women
https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8projects
continued their development toward a successful end. For details, check the
reports:
- Tools for mass migration of legacy translated wiki content
   https://www.mediawiki.org/wiki/Extension:Translate/Mass_migration_tools/Project_updates
   - Wikidata annotation tool
   https://www.mediawiki.org/wiki/Wikidata_annotation_tool/updates
   - Email bounce handling to MediaWiki with VERP
   https://www.mediawiki.org/wiki/VERP/GSOC_Progress_Rerport
   - Google Books, Internet Archive, Commons upload cycle
   https://www.mediawiki.org/wiki/Google_Books,_Internet_Archive,_Commons_upload_cycle/Progress
   - UniversalLanguageSelector fonts for Chinese wikis
   https://www.mediawiki.org/wiki/Extension:UniversalLanguageSelector/Fonts_for_Chinese_wikis#Weekly_Report
   - MassMessage page input list improvements
   https://www.mediawiki.org/wiki/Extension:MassMessage/Page_input_list_improvements/Progress_reports
   - Book management in Wikibooks/Wikisource
   https://meta.wikimedia.org/wiki/Book_management_2014/Progress
   - Parsoid-based online-detection of broken wikitext
   https://www.mediawiki.org/wiki/User:Hardik95/GSoC_2014_Progress_Report
   - Usability improvements for the Translate extension
   https://www.mediawiki.org/wiki/User:Kunalgrover05/Progress_Report
   - A modern, scalable and attractive skin for MediaWiki
   https://www.mediawiki.org/wiki/User:Jack_Phoenix/GSoC_2014
   - Automatic cross-language screenshots for user documentation
   https://www.mediawiki.org/wiki/Automatic_cross-language_screenshots/progress
   - Separating skins from core MediaWiki
   https://www.mediawiki.org/wiki/Separating_skins_from_core_MediaWiki/Progress
   - Chemical Markup support for Wikimedia Commons
   https://www.mediawiki.org/wiki/Chemical_Markup_support_for_Wikimedia_Commons/Internship_Report
   - Improving URL citations on Wikimedia
   https://www.mediawiki.org/wiki/User:Mvolz/Weekly_Reports
   - Historical OpenStreetMap
   https://www.mediawiki.org/wiki/User:JaimeLyn/Weekly_Reports
   - Welcoming new contributors to Wikimedia Labs and Tool Labs
   https://www.mediawiki.org/wiki/Welcome_to_labs/Progress_Reports
   - Evaluating, documenting, and improving MediaWiki web API client
   libraries
   https://www.mediawiki.org/wiki/Evaluating_and_Improving_MediaWiki_web_API_client_libraries/Progress_Reports
   - Feed the Gnomes – Wikidata Outreach
   https://www.mediawiki.org/wiki/User:Thepwnco/OPW_Reporting
   - Template Matching for RDFIO
   https://www.mediawiki.org/wiki/Extension:RDFIO/Template_matching_for_RDFIO/Reports
   - Switching Semantic Forms Autocompletion to Select2
   https://www.mediawiki.org/wiki/Extension:Semantic_Forms/Select2_for_autocompletion/Progress_Report
   - Catalogue for Mediawiki Extensions
   https://www.mediawiki.org/wiki/User:Adi.iiita/Gsoc2014/Report#Weekly_Report
   - Generic, efficient localisation update service
   https://www.mediawiki.org/wiki/Extension:LocalisationUpdate/LUv2/Updates
   .
*Technical communications
https://www.mediawiki.org/wiki/Technical_communications*
https://commons.wikimedia.org/wiki/File:Flesch_reading_ease_chart_for_Tech_News.svg
Chart showing historical Flesch reading ease
https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests data
for Tech News, a measure of the newsletter’s readability
https://meta.wikimedia.org/wiki/Tech/News/Readability. Higher scores
indicate material that is easier to read. A score of 60–70 corresponds to
content easily understood by 13- to 15-year-old students.
Guillaume Paumier https://www.mediawiki.org/wiki/User:Guillom collaborated
with authors of the Education newsletter
https://outreach.wikimedia.org/wiki/Education/Newsto set it up
https://outreach.wikimedia.org/wiki/Education/Newsletter/Newsroom/Multilingual_message
for
multilingual delivery, using a script
https://meta.wikimedia.org/wiki/Module:Assemble_multilingual_message similar
to the one used for Tech News https://meta.wikimedia.org/wiki/Tech/News.
He also wrote adetailed how-to
https://meta.wikimedia.org/wiki/Newsletters/Translation to accompany the
script for people who want to send a multilingual message across wikis. In
preparation for the Wikimania session
https://wikimania2014.wikimedia.org/wiki/Submissions/Tech_news about Tech
News, he updated the readability
https://meta.wikimedia.org/wiki/Tech/News/Readability andsubscribers
https://meta.wikimedia.org/wiki/Tech/News/Subscribers metrics. He also
continued to provide ongoing communications support
https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity
for
the engineering staff, and to prepare and distribute Tech News every week.
*Volunteer coordination and outreach
https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach*
We focused on the preparation of the Wikimania Hackathon
https://wikimania2014.wikimedia.org/wiki/Hackathon, encouraging all
registered participants to propose topics and sign up to interesting
sessions. We also organized a Q&A session
https://plus.google.com/events/c0fgci542f8cn58o606gng6avio with potential
organizers of the Wikimedia Hackathon 2015. We organized two Tech
Talks: *Hadoop
and Beyond. An overview of Analytics infrastructure*
https://plus.google.com/u/0/b/103470172168784626509/events/c53ho5esd0luccd09a1c30rlrmg
 and*HHVM in production: what that means for Wikimedia developers*
https://plus.google.com/events/cp5mjf6jrihevtdje8lmu5hvm1k. More
activities hosted in July can be found at Project:Calendar/2014/07
https://www.mediawiki.org/wiki/Project:Calendar/2014/07.
*Architecture and Requests for comment process
https://www.mediawiki.org/wiki/Architecture_process*
Developers finished the security architecture guidelines
https://www.mediawiki.org/wiki/Security_for_developers/Architecture, and
discussed several requests for comment
https://www.mediawiki.org/wiki/Requests_for_comment in online architecture
meetings https://www.mediawiki.org/wiki/Architecture_meetings:
- 2014-07-10 — Frontend standardization discussion
   https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-10
focusing
   onRequests for comment/Redo skin framework
   https://www.mediawiki.org/wiki/Requests_for_comment/Redo_skin_framework
   ;
   - 2014-07-16 — RfC discussion
   https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-16
focusing
   on Requests for comment/Vertical writing support
   https://www.mediawiki.org/wiki/Requests_for_comment/Vertical_writing_support
   ;
   - 2014-07-23 — RfC discussion
   https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-23
focusing
   on Requests for comment/Composer managed libraries for use on WMF cluster
   https://www.mediawiki.org/wiki/Requests_for_comment/Composer_managed_libraries_for_use_on_WMF_cluster,
   in which the architecture committee approved the RfC;
   - 2014-07-30 — RfC discussion
   https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-07-30
focusing
   on Requests for comment/CentralNotice Caching Overhaul – Frontend Proxy
   https://www.mediawiki.org/wiki/Requests_for_comment/CentralNotice_Caching_Overhaul_-_Frontend_Proxy
   .
*dev.wikimedia.org https://www.mediawiki.org/wiki/Dev.wikimedia.org*
In July, Quim Gil sorted the tasks necessary for the first hub prototype
http://fab.wmflabs.org/project/board/60/ into a Phabricator board, and
Sumana Harihareswara determined which three APIs
http://fab.wmflabs.org/T479 she would document first.
 Analytics https://www.mediawiki.org/wiki/Analytics
*Wikimetrics https://www.mediawiki.org/wiki/Analytics/Wikimetrics*
Wikimetrics can now generate vital sign metrics for every project daily.
Rolling Monthly Active Editor metric has been implemented; the reports are
in JSON format, in a logical path hosted on a file server and downloadable.
The team also worked on backfilling data for the daily reports on Newly
Registered and Rolling Active Editor, and numerous optimizations to
backfill the data quickly.
*Data Processing https://www.mediawiki.org/wiki/Analytics/Data_Processing*
New nodes were added to the cluster this month and all machines were
upgraded to run CDH5. The team decided not to preserve any data on the
cluster during the upgrade and started fresh. The team hosted a Tech Talk
on our Hadoop installation (see video
https://plus.google.com/u/0/events/c53ho5esd0luccd09a1c30rlrmg and slides
https://docs.google.com/a/wikimedia.org/presentation/d/1ZPmfN-kmfqWEJUMIRg2feSstFPY45js4AnYaf3NbLNE/edit#slide=id.p).
Duplicate monitoring has also been implemented in Hadoop to monitor the
incoming Varnish logs.
*Editor Engagement Vital Signs
https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs*
The culmination of our efforts this month can be visualized in a prototype
built for Wikimania
https://metrics-staging.wmflabs.org/static/public/dash/. This was made
possible thanks to many back-end enhancements (optimizations) to
Wikimetrics, along with research and selection of the optimal technologies
to implement the stack to display a dashboard
https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs/Dashboard
.
*EventLogging https://www.mediawiki.org/wiki/Analytics/EventLogging*
EventLogging monitoring is now in graphite, and we can see which schemas
cause spikes in traffic (example
https://graphite.wikimedia.org/render/?width=588&height=311&_salt=1404912729.182&target=eventlogging.overall.valid.rate&target=eventlogging.schema.MobileWebClickTracking.rate&target=eventlogging.schema.MediaViewer.rate&from=00%3A00_20140630&until=23%3A59_20140709
).
*Research and Data
https://www.mediawiki.org/wiki/Analytics/Research_and_Data*
This month, we completed the documentation for the Active Editor Model
https://meta.wikimedia.org/wiki/Research:Modeling_monthly_active_editors,
a set of metrics for observing sub-population trends and setting product
team goals. We also engaged in further work on the new pageviews
definition. An interim solution for Limited-duration Unique Client
Identifiers (LUCIDs) was also developed and passed to the Analytics
Engineering team for review.
We analyzed trends in mobile readership and contributions, with a
particular focus on the tablet switchover and the release of the native
Android app. We found
https://meta.wikimedia.org/wiki/Research:Mobile_editor_engagement/Editor_activation
that
in the first half of 2014, mobile surpassed desktop in the rate at which
new registered users become first-time editors and first-time active
editors in many major projects, including the English Wikipedia. An update
on mobile trends
https://commons.wikimedia.org/wiki/File:Wikimedia_Mobile_Trends.pdf will
be presented at the upcoming Monthly Metrics meeting on July 31.
Development of a standardised toolkit
https://github.com/Ironholds/WMUtils for
geolocation, user agent parsing and accessing pageviews data was completed.
We supported the multimedia team in developing a research study
https://meta.wikimedia.org/wiki/Research:Media_Viewer_preference_elicitation
to
objectively measure the preference of Wikipedia editor and readers.
We hosted the July research showcase
https://www.mediawiki.org/wiki/Analytics/Research_and_Data/Showcase#July_2014
with
a presentation
https://commons.wikimedia.org/wiki/File:Halfak%27s_wiki_research_libraries_-_WMF_R%26D_showcase_(Jul._2014).pdf
by
Aaron Halfaker of 4 Python libraries for data analysis, and a guest talk by
Center for Civic Media’s Nathan Matias http://natematias.com/ on the use
of open data
https://commons.wikimedia.org/wiki/File:Using_Open_Data_and_Stories_to_Broaden_Crowd_Content.pdf
to
increase the diversity of collaboratively created content.
We prepared 8 presentations
https://www.mediawiki.org/wiki/Analytics/Wikimania_2014 that we will be
giving or co-presenting next week at Wikimania in London. We also organized
the next WikiResearch hackathon
https://meta.wikimedia.org/wiki/Research:Labs2/Hackathons/August_6-7th,_2014
that
will be jointly hosted in London (UK)
https://meta.wikimedia.org/wiki/Research:Labs2/Hackathons/August_6-7th,_2014/Meetups/London,_UK
(during
the pre-conference Wikimania Hackathon) and in Philadelphia (USA)
https://meta.wikimedia.org/wiki/Research:Labs2/Hackathons/August_6-7th,_2014/Meetups/Philadelphia,_PA,_US
on
August 6-7, 2014.
We filled the fundraising research analyst position: the new member of the
Research & Data team will join us in September and we’ll post an
announcement on the lists shortly before his start date.
Lastly, we gave presentations on current research at the Wikimedia
Foundation at the Institute for Scientific Interchange
https://en.wikipedia.org/wiki/Institute_for_Scientific_Exchange (Turin)
and at theDesignDensity http://www.densitydesign.org/ lab (Milan).
 Kiwix http://www.kiwix.org/
https://commons.wikimedia.org/wiki/File:Project_Gutenberg_screenshot.png
Screenshot of the first Project Gutenberg http://www.gutenberg.org/ ZIM
file
*The Kiwix project is funded and executed by Wikimedia CH
https://meta.wikimedia.org/wiki/Wikimedia_CH.*
We have pre-release binaries http://download.kiwix.org/bin/0.9/ of the
next 0.9 (final) release. Except for OSX everything seems to work file as
far. The support of RaspberryPi http://www.raspberrypi.org/ was finally
merged to the kiwix-plug master branch
https://sourceforge.net/p/kiwix/other/ci/master/tree/plug/; this offers
new perspectives because the price to create a Kiwix-Plug
http://www.kiwix.org/wiki/Kiwix-plug has dropped to around USD 100. We
also started an engineering collaboration with ebook reader manufacturer
Bookeen http://www.bookeen.com/en/ (in the scope of the Malebooks
http://malebooks.ml/ project) to be able offer an offline version of
Wikipedia on e-ink devices. We participated in the Google Serve Day at
Google Zurich. The goal was to meet Google engineers during one day and
have them work on open source projects. The result was a dozen of fixed
bugs and implemented features, mostly on Kiwix for Android
https://play.google.com/store/apps/details?id=org.kiwix.kiwixmobile, but
also in Kiwix for desktop and MediaWiki. Four developers had a one-week
hackathon in Lyon, France to develop an offline version of the Gutenberg
library http://www.gutenberg.org/. We’re currently polishing the code and
plan a release soon; our partners and sponsors plan the first deployments
in Africa in Autumn. Last but not least, a proof-of-concept of a Kiwix iOS
app was made, so we might release a first app before the end of the year.
 Wikidata https://meta.wikimedia.org/wiki/Wikidata
*The Wikidata project is funded and executed by Wikimedia Deutschland
https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en.*
The biggest improvement around Wikidata in July is the release of the
entity suggester
https://lists.wikimedia.org/pipermail/wikidata-l/2014-July/004148.html.
It makes it a lot easier to see what kind of information is missing on an
item. Helen and Anjali, Wikidata’s Outreach Program for Women interns,
continued improving user documentation and outreach around Wikidata as well
as worked on a new design for the main page
https://www.wikidata.org/wiki/Wikidata:Portal_Redesign. Guided Tours
https://www.wikidata.org/wiki/Wikidata:Tourswere published, helping
newcomers find their way around the site. The developers further worked on
supporting badges (like “featured article”), redirects between items, the
monolingual text datatype (to be able to express things like the motto of a
country) as well as the first implementation steps for the new user
interface design. Additionally the first JSON dumps
https://lists.wikimedia.org/pipermail/wikidata-l/2014-July/004216.html were
published.
 FutureThe engineering management team continues to update the *Deployments
https://wikitech.wikimedia.org/wiki/Deployments*page weekly, providing
up-to-date information on the upcoming deployments to Wikimedia sites, as
well as the *annual goals
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals*,
listing ongoing and future Wikimedia engineering efforts.
------------------------------
*This article was written collaboratively by Wikimedia engineers and
managers. See revision history
https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/2014/July&action=history
and
associated status pages. A wiki version
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/July is
also available.*
-- 
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation