Hi,
The report covering Wikimedia engineering activities in September 2014 is
now available:
https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September
Below is the HTML text of the report.
------------------------------------------------------------------
Major news in September include:
- a call for candidates
<https://blog.wikimedia.org/2014/09/18/new-foss-outreach-internships-female-technical-contributors/>
for the Free and Open Source Software Outreach Program for Women;
- a roundtable discussion
<https://blog.wikimedia.org/2014/09/29/round-table-with-editors-from-the-catalan-wikipedia/>
between the Language engineering team and editors from the Catalan
language Wikipedia, focusing on the Content Translation tool.
Engineering metrics in September:
- 151 unique committers contributed patchsets of code to MediaWiki.
- About 27 shell requests <https://www.mediawiki.org/wiki/Shell_requests>
were processed.
Contents [hide
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#>
]
- 1 Upcoming events
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Upcoming_events>
- 2 Personnel
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Personnel>
- 2.1 Work with us
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Work_with_us>
- 2.2 Announcements
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Announcements>
- 3 Technical Operations
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Technical_Operations>
- 4 Features Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Features_Engineering>
- 4.1 Editor retention: Editing tools
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Editor_retention:_Editing_tools>
- 4.2 Services
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Services>
- 4.3 Core Features
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Core_Features>
- 5 Mobile
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Mobile>
- 6 Language Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Language_Engineering>
- 7 Platform Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Platform_Engineering>
- 7.1 MediaWiki Core
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#MediaWiki_Core>
- 7.2 Multimedia
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Multimedia>
- 7.3 Engineering Community Team
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Engineering_Community_Team>
- 8 Analytics
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Analytics>
- 9 Kiwix
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Kiwix>
- 10 Wikidata
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Wikidata>
- 11 Future
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/Report/2014/September#Future>
Upcoming events[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=1>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=1>
]
There are many opportunities for you to *get involved and contribute
<https://www.mediawiki.org/wiki/How_to_contribute>* to MediaWiki and
technical activities to improve Wikimedia sites, both for coders and
contributors with other talents.
For a more complete and up-to-date list, check out the Project:Calendar
<https://www.mediawiki.org/wiki/Project:Calendar>.
[edit or add events
<https://www.mediawiki.org/wiki/Special:MyLanguage/Project:Calendar>] [refresh
table
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=purge#>]
[view credits
<https://www.mediawiki.org/wiki/Special:MyLanguage/Project:Visual_identity>]
DateTypeEventContactOct 1–Oct 3[image: IRL (physical) meet-ups and
conferences] <https://www.mediawiki.org/wiki/Events>SMWCon Fall 2014
<https://www.semantic-mediawiki.org/wiki/SMWCon_Fall_2014> (Vienna,
Austria)Conference
talk <https://www.semantic-mediawiki.org/wiki/Talk:SMWCon_Fall_2014>Oct
6[image:
Online meetings] <https://www.mediawiki.org/wiki/Meetings>Tech Talk: The
Dashboarding Problem
<https://plus.google.com/u/0/b/103470172168784626509/events/ch8uuivq05nqejqlivrqni6v1n0>
1900-2000 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Tech+Talk%3A+The+Dashboarding+Problem&iso=20141006T19&p1=1440&ah=1>
in #wikimedia-office <irc://irc.freenode.net/wikimedia-office>connect
<http://webchat.freenode.net/?channels=#wikimedia-office>.Nuria Ruiz
<https://www.mediawiki.org/wiki/User_talk:NRuiz_(WMF)>Oct 8–Oct 8[image:
Old bugs] <https://www.mediawiki.org/wiki/Bug_management>Collection/Book/PDF
Bug Day <https://www.mediawiki.org/wiki/Bug_management/Triage/201410> (exact
date pending)Talk
<https://www.mediawiki.org/wiki/Thread:Talk:PDF_rendering/Test_instance_and_bug_triage>Oct
22[image: Online meetings] <https://www.mediawiki.org/wiki/Meetings>Tech
Talk: Design Research in Product Development
<https://plus.google.com/u/0/b/103470172168784626509/events/caiiagf75bvddr09nf4jbgccn30>
1900-2000 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Tech+Talk%3A+Design+Research+in+Product+Development&iso=20141022T19&p1=1440&ah=1>
in #wikimedia-office <irc://irc.freenode.net/wikimedia-office>connect
<http://webchat.freenode.net/?channels=#wikimedia-office>.Abbey Ripstra
<https://www.mediawiki.org/wiki/User_talk:ARipstra_(WMF)>Oct 22[image:
Online meetings] <https://www.mediawiki.org/wiki/Meetings>Meetup: Exploratory
Testing for Complex Software; Lessons from Cloud Foundry
<https://plus.google.com/u/0/b/103470172168784626509/events/c9r9lqtbpn52vhivkc12usps55c>
0130-0300 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Cloudfoundry+Meetup&iso=20141022T0130&ah=1&am=30>
in #wikimedia-office <irc://irc.freenode.net/wikimedia-office>connect
<http://webchat.freenode.net/?channels=#wikimedia-office>.
Personnel[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=2>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=2>
]Work with us <https://wikimediafoundation.org/wiki/Work_with_us>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=3>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=3>
]
Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.
- Senior Software Engineer - Services <http://grnh.se/748mr2>
- Software Engineer - Maps & Geo - Mobile <http://grnh.se/4w5iyb>
- Software Engineer - Mobile - iOS <http://grnh.se/j4gmk5>
- Release Engineer <http://grnh.se/5fw24x>
- Technical Writer <http://grnh.se/d86hti>
- Full Stack Developer - Analytics <http://grnh.se/vfgr0t>
- Research Analyst <http://grnh.se/r8ukg1>
- Agile Coach/ScrumMaster - Team Practices Group <http://grnh.se/5h4jdv>
- Senior Technical Product Manager <http://grnh.se/bm5neu>
- Community Liaison <http://grnh.se/i2jvpo>
- Community Liaison (PT Contract) <http://grnh.se/kyra75>
- Operations Security Engineer <http://grnh.se/m65tn8>
- UX Senior Designer <http://grnh.se/47xrkn>
- UX Senior Design Researcher <http://grnh.se/x2nsqv>
- UX Visual Design Fellowship <http://grnh.se/783knf>
- Mobile Partnerships Regional Manager <http://grnh.se/v75upy>
Announcements[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=4>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=4>
]
- Damon Sicore joined the Wikimedia Foundation as Vice President of
Engineering (announcement
<https://lists.wikimedia.org/pipermail/wikimediaannounce-l/2014-September/000987.html>
).
- Rachel Farrand joined the Engineering Community Team as Events
Coordinator (announcement
<https://lists.wikimedia.org/pipermail/wikitech-l/2014-September/078488.html>
).
- Jeff Hobson joined the Wikipedia Zero engineering team (announcement
<https://lists.wikimedia.org/pipermail/wikitech-l/2014-September/078638.html>
).
- Daisy Chen joined the UX Research team as Associate Design Researcher (
announcement
<https://lists.wikimedia.org/pipermail/design/2014-September/002095.html>
).
Technical Operations[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=5>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=5>
]
*Dallas data center*
In September we have setup (backup) replication of most project data,
including core databases and external storage. Work on Swift images and
system backups was still ongoing into October. Essential system
infrastructure such as an installation server, DNS, LVS, NTP etc. has been
deployed as well.
*Tampa data center*
We started the last push to get the remaining services & systems out of our
Tampa data center, with a deadline for shutdown of all systems on October
1st. The remaining services included PDF generation, mail servers,
noc.wikimedia.org and LDAP.
Labs metrics in September:
- Number of projects: 146
- Number of instances: 415
- Amount of RAM in use (in MBs): 1,996,288
- Amount of allocated storage (in GBs): 20,435
- Number of virtual CPUs in use: 977
- Number of users: 4,083
*Wikimedia Labs*
Wikitech (the Labs web interface) is now managed
<https://lists.wikimedia.org/pipermail/labs-l/2014-September/002870.html> via
the standard WMF deployment system. This should allow for more frequent
MediaWiki updates and overall greater stability.The last historic remaining
dependencies on our old Tampa datacenter (e.g. LDAP and Labs DNS backup
servers) were finally stamped out and replaced with dependencies on Dallas
hardware.One of the labs virtualization hosts (virt1006) was suffering
intermittent problems, so all affected instances were migrated to other
hosts in order to stave off possible future disaster. Consequently, Labs is
a bit short on virtualization space, but new hardware procurement is under
way.Several long-unused instances and projects were cleaned up in order to
free up more space.The last of the ToolLabs replica DB servers was upgraded
to MariaDB 10.Features Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=6>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=6>
]Editor retention: Editing tools[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=7>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=7>
]
*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>* [edit
<https://www.mediawiki.org/w/index.php?title=VisualEditor/status&veaction=edit>
]
In September, the team working on VisualEditor expanded browser support,
improved some features, and fixed nearly 60 bugs and tickets
<https://bugzilla.wikimedia.org/buglist.cgi?list_id=348697&order=priority%2Cbug_severity&product=VisualEditor&query_format=advanced&resolution=FIXED&target_milestone=VE-deploy-2014-09-04&target_milestone=VE-deploy-2014-09-11&target_milestone=VE-deploy-2014-09-181&target_milestone=VE-deploy-2014-09-25&target_milestone=VE-deploy-2014-10-02>
.
Users of Internet Explorer 10, who we were previously preventing from using
VisualEditor due to some major bugs, will now be able to use VisualEditor;
this follows on from Internet Explorer 11 support last month. When editing
a template with a required field, VisualEditor now warns you to avoid
leaving it blank, and you can now create auto-numbered links using
VisualEditor.
Improvements and updates were made to a number of interface messages as
part of our work with translators to improve the software for all users,
based on feedback from users and user testing
<https://www.mediawiki.org/wiki/VisualEditor/Design/User_testing>. We made
progress on table structure editing and auto-filled citations, both of
which will be coming soon.
The deployed version of the code was updated five times in the regular
release cycle (1.24-wmf20
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf20#VisualEditor>,
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf17#VisualEditor>
1.24-wmf21
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf20#VisualEditor>,
1.24-wmf22
<https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf22#VisualEditor>,
1.25-wmf1 <https://www.mediawiki.org/wiki/MediaWiki_1.25/wmf1#VisualEditor>
and 1.25-wmf2
<https://www.mediawiki.org/wiki/MediaWiki_1.25/wmf2#VisualEditor>).
*Editing <https://www.mediawiki.org/wiki/Editing>* [edit
<https://www.mediawiki.org/w/index.php?title=Editing/status&veaction=edit>]
In September, the Editing Team made substantial progress on front-end
standardisation, as well as the work on VisualEditor which is reported
separately. The team welcomed Bartosz "MatmaRex" Dziewoński
<https://meta.wikimedia.org/wiki/User:Bartosz_Dziewo%C5%84ski_(WMF)> as a
new team member, and existing student member Moriel Schottlender
<https://meta.wikimedia.org/wiki/User:MSchottlender-WMF> converted to
full-time status.
The team's work on front-end standardisation is focussed on improving
libraries and infrastructure, and in particular, the OOjs UI
<https://www.mediawiki.org/wiki/OOjs_UI> library. This included the
creation of a MediaWiki theme in collaboration with the Design team, which
can be explored in the online demo
<https://tools.wmflabs.org/oojs-ui/oojs-ui/demos/index.html#widgets-mediawiki-vector-ltr>;
this will be deployed into MediaWiki's use of OOUI in the next few weeks. A
number of bugs were fixed, including working around window and popup
sizing, over-flow item placement, and working around some browser bugs in
Firefox and Safari. The code documentation has a number of minor issues
corrected, and the build process was extended to create a minified
distribution. The OOjs <https://www.mediawiki.org/wiki/OOjs> library was
updated to fix a minor bug in oo.Compare, with a new version (v1.1.1)
released and pushed downstream into MediaWiki, VisualEditor and OOjs UI.
The TemplateData extension now supports the "autovalue" parameter property,
a wikitext value that a parameter can be set to have inserted by default if
desired. Also, the specification for TemplateData
<http://git.wikimedia.org/blob/mediawiki%2Fextensions%2FTemplateData.git/HEAD/Specification.md>
was re-written to be clearer and more consistent. Next month the
TemplateData GUI editor will be made available on all Wikimedia wikis.
*Parsoid <https://www.mediawiki.org/wiki/Parsoid>* [edit
<https://www.mediawiki.org/w/index.php?title=Parsoid/status&veaction=edit>]
In September, we continued to fix bugs, upgraded libraries, and made
additional progress towards improving compatibility with PHP parser + Tidy
rendering. Specifically, Parsoid's paragraph wrapping now targets the PHP
parser + Tidy output rather than PHP parser output. We also continued to
update Parsoid's CSS / rendering to more closely match the current
rendering. We also improved Parsoid's robustness handling edge case
scenarios (pathological backtracking, parsing of very large wikitext
tables). Part of the Parsoid team was also busy with launching the PDF
rendering service which was successfully launched end of September.
Services[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=8>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=8>
]
*Services and REST API <https://www.mediawiki.org/wiki/Services>* [edit
<https://www.mediawiki.org/w/index.php?title=Services/status&veaction=edit>]
September saw a lot of activity on the RESTBase
<https://github.com/gwicke/restbase> storage and API service. A new
'pagecontent' composite bucket type using revisioned blob buckets was
introduced. This uses the by-now fairly rich table storage backend to
provide functionality similar to MediaWiki's revision table, and supports
any number of revisioned types of content (like HTML, wikitext, JSON
metadata) associated with each revision.
Work on secondary index updates
<https://github.com/gwicke/restbase-cassandra/blob/master/doc/SecondaryIndexes.md>
continued at full steam, and is now close to being merged.
Core Features[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=9>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=9>
]
*Flow <https://www.mediawiki.org/wiki/Flow/Project_information>* [edit
<https://www.mediawiki.org/w/index.php?title=Flow/Project_information/status&veaction=edit>
]
In September, the Flow team enabled new test pages on French WP and Hebrew
WP. The French test is for the Forum des Nouveaux, a Help space for new
contributors (similar to the Teahouse on English WP). The Forum des
Nouveaux hosts reached out to the Flow team after Wikimania, excited to try
out the new discussions system. The Hebrew WP test is helping the team
diagnose problems for Right-to-Left languages, and general i18n issues. The
team also refined the new Echo notifications functionality, with lots of
feedback from contributors on
Mediawiki.org and En.wp. New topic
notifications are now bundled in Echo, and we fixed several bugs related to
the behavior of the Alerts and Messages tabs, and getting excess mention
notifications.
*Growth <https://www.mediawiki.org/wiki/Growth>* [edit
<https://www.mediawiki.org/w/index.php?title=Growth/status&veaction=edit>]
In September, the Growth team shut down, with workflows shifting into the
mainstream of other teams.
Mobile <https://www.mediawiki.org/wiki/Wikimedia_mobile_engineering>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=10>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=10>
]
*Wikimedia Apps <https://www.mediawiki.org/wiki/Wikimedia_Apps>* [edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Apps/status&veaction=edit>
]
In September, the Mobile Apps Team released a new version of the iOS app
containing the Nearby feature which shows you articles about things that
are near your location, and a references panel that pops up whenever you
tap a reference. The team also released an iOS 8 compatibility build to
market. The team also spent time performing code quality improvements and
refactoring on both the iOS and Android apps.
*Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>* [
edit
<https://www.mediawiki.org/w/index.php?title=Mobile_web_projects/status&veaction=edit>
]
This month the mobile web team focused on the first prototype of WikiGrok,
a new contribution feature that asks users who are reading Wikipedia
articles to help add Wikidata that is missing about the article subject.
Over the course of the month, we built and user-tested the first
experimental interface for allowing users to input Wikidata: a simple
binary question mode that provides the user with a suggested occupation on
biographies that are missing this information in Wikidata but contain a
possible occupation in the Wikipedia article. In this early test phase we
are storing the replies in a separate database, not pushing to Wikidata. We
plan to add suggestions for more Wikidata fields and test this version
against a slightly more complex tagging interface in beta in October.
*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>* [edit
<https://www.mediawiki.org/w/index.php?title=Wikipedia_Zero/status&veaction=edit>
]
During September 2014, the team built more Partners Portal architecture,
including Graph extension integration components for eventual display of
aggregate statistics to zero-rating partners (it already works and is being
reviewed in house). The team also grew support for dynamic zero-rating
banners while enhancing JSON configuration extension code and issuing
bugfixes. Additionally, the team shrunk the size of the Wikipedia favicon
to reduce bandwidth usage by users across the web. And on the partner side,
we launched Telenor Myanmar in September. Finally, the team welcomed its
newest software engineer, Jeff Hobson, to the Wikimedia Foundation!
Language Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=11>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=11>
]
*Language tools <https://www.mediawiki.org/wiki/Language_tools>* [edit
<https://www.mediawiki.org/w/index.php?title=Language_tools/status&veaction=edit>
]
The CLDR extension was updated to version 26 and entries identical to CLDR
were removed from LocalNamesEn.php. The team made RTL fixes in core, Echo
and Wikibase, and tested Flow for RTL support. Maintenance of the Translate
extension continued, and the performance of translation memory was improved
on ElasticSearch with the help of Nik Everett
*Language Engineering Communications and Outreach
<https://www.mediawiki.org/wiki/Language_engineering_communications_and_outreach>*
[edit
<https://www.mediawiki.org/w/index.php?title=Language_engineering_communications_and_outreach/status&veaction=edit>
]
The Language Engineering team hosted a round-table with Catalan Wikipedia
editors
<http://blog.wikimedia.org/2014/09/29/round-table-with-editors-from-the-catalan-wikipedia/>
who use ContentTranslation to gather feedback about their experience with
the tool. OPW mentors started coordinating with candidates interested
in internationalization
projects
<https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_9#Internationalization_and_localization>
*Content translation <https://www.mediawiki.org/wiki/Content_translation>* [
edit
<https://www.mediawiki.org/w/index.php?title=Content_translation/status&veaction=edit>
]
The second version of the tool was released
<https://www.mediawiki.org/wiki/Content_translation/Announcement-October2014>.
This version has not yet been deployed due to technical issues in the Labs
setup. This is currently being resolved with the Ops team.
Notable improvements include:
- a basic formatting toolbar (for Chrome);
- more accurate warnings for unchanged machine translated content;
- design improvements for the top bar and progress bar;
- bi-directional support for Spanish-Portuguese machine translation;
- link adaptation improvements.
The team is performing ongoing tests with users for Spanish-Portuguese,
Portuguese-Spanish translations, and we started planning for the third
release
<https://www.mediawiki.org/wiki/Content_translation/Roadmap/CX02Release>.
Platform Engineering
<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=12>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=12>
]MediaWiki Core[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=13>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=13>
]
*Admin tools development
<https://www.mediawiki.org/wiki/Admin_tools_development>* [edit
<https://www.mediawiki.org/w/index.php?title=Admin_tools_development/status&veaction=edit>
]
The majority of admin tools resources are currently diverted towards the SUL
finalisation <https://www.mediawiki.org/wiki/SUL_finalisation> project. In
September, minor UX enhancements were applied to the Special:CentralAuth
<https://www.mediawiki.org/wiki/Special:CentralAuth> page.
*Search <https://www.mediawiki.org/wiki/Search>* [edit
<https://www.mediawiki.org/w/index.php?title=Search/status&veaction=edit>]
In September we worked to mitigate the performance bottleneck that we found
in August. We found there to be no silver bullet but used the information
we learned to pick and order appropriate hardware to handle the remaining
wikis. We also implemented out significantly improved wikitext Regular
Expression search. In October we've begun rolling out the wikitext Regular
Expression search and received some of the hardware we need to finish
cutting over the remaining wikis. We believe we'll get it all installed in
October and cut the remaining wikis over in November.
*SUL finalisation <https://www.mediawiki.org/wiki/SUL_finalisation>* [edit
<https://www.mediawiki.org/w/index.php?title=SUL_finalisation/status&veaction=edit>
]
In September, the team wrapped up the feature development for SUL
finalisation. One part of the work (the steward end of the rename request
form) is outstanding and will be finished in October. In October, the team
is planning to proceed into deployment and testing of the features.
*Security auditing and response
<https://www.mediawiki.org/wiki/Security_auditing_and_response>* [edit
<https://www.mediawiki.org/w/index.php?title=Security_auditing_and_response/status&veaction=edit>
]
We published the 1.23.4 security release, and completed review for the
Graph and Imagemetrics extensions.
Multimedia[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=14>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=14>
]
*Multimedia <https://www.mediawiki.org/wiki/Multimedia>* [edit
<https://www.mediawiki.org/w/index.php?title=Multimedia/status&veaction=edit>
]
<https://www.mediawiki.org/wiki/File:Media_Viewer_-_New_Design_-_Default_Lightbox.png>
Media Viewer's new user interface, with a 'More details' button.
In September, the multimedia team developed and released a first round of new
improvements
<https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Improvements> to
Media Viewer, based on feedback from our recent community consultation
<https://meta.wikimedia.org/wiki/Community_Engagement_(Product)/Media_Viewer_consultation>
and ongoing user research
<https://www.mediawiki.org/wiki/Media_Viewer_Research_Round_2_(August_2014)>
.
These improvements aim to make Media Viewer easier to use by readers and
casual editors, with these features: a more prominent "More Details" button
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/830>,
linking to the File: page; separate icons for "Download"
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/841>
and "Share or Embed"
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/834>
features;
and an easier way to enlarge images by clicking on them
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/831>.
Next, we plan to work on an easier way to disable Media Viewer for personal
use
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/836>
and a caption or description right below the image
<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/589>.
We would like to thank all the community members who suggested these
improvements. Our research suggests that they offer a better user
experience, that is both clearer and simpler.
This month, we also ramped up the Structured Data
<https://commons.wikimedia.org/wiki/Commons:Structured_data> project, in
collaboration with community members and the Wikidata team
<https://www.wikidata.org/wiki/>: in October, we will start developing a
first prototype for a high-end API that can read and write machine-readable
data on Wikimedia Commons, to be followed by a wider deployment in coming
months. In parallel, the foundation is also launching a file metadata
cleanup drive <https://meta.wikimedia.org/wiki/File_metadata_cleanup_drive> to
add machine-readable attributions and licenses on files that lack them,
spearheaded by Guillaume Paumier
<https://www.mediawiki.org/wiki/User:Guillaume_(WMF)>. To learn more, join
our Structured Data Q&A
<https://commons.wikimedia.org/wiki/Commons:Structured_data#Discussions>
on Thursday,
October 16 at 18:00 UTC
<http://www.timeanddate.com/worldclock/fixedtime.html?msg=Structured+Data+IRC+discussion&iso=20141016T18&ah=1>,
for an office hours
<https://meta.wikimedia.org/wiki/IRC_office_hours#Upcoming_office_hours> chat
on #wikimedia-office <irc://irc.freenode.net/wikimedia-office>connect
<http://webchat.freenode.net/?channels=#wikimedia-office>(Freenode IRC).
We also continued our code refactoring for the UploadWizard
<https://www.mediawiki.org/wiki/UploadWizard>, and started to collect
metrics for an upload funnel analysis
<https://www.mediawiki.org/wiki/UploadWizard/Funnel_analysis>, to find out
how many users drop out at each step of the upload and where failure is
occurring, so we can prioritize bug fixes. For more information about our
work, join the multimedia mailing list
<https://lists.wikimedia.org/mailman/listinfo/multimedia>.
Engineering Community Team
<https://www.mediawiki.org/wiki/Engineering_Community_Team>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=15>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=15>
]
*Bug management <https://www.mediawiki.org/wiki/Bug_management>* [edit
<https://www.mediawiki.org/w/index.php?title=Bug_management/status&veaction=edit>
]
The "Commons App" product was closed as no further development is planned
currently. A new product was created for the new PDF renderer infrastructure
<https://bugzilla.wikimedia.org/show_bug.cgi?id=69603> and numerous
components were created
<https://bugzilla.wikimedia.org/buglist.cgi?bug_id=70535,70500,70432,70622,71103,71049,70951,70846>
.
*Phabricator migration
<https://www.mediawiki.org/wiki/Phabricator/Migration>* [edit
<https://www.mediawiki.org/w/index.php?title=Phabricator/Migration/status&veaction=edit>
]
phabricator.wikimedia.org got set up with tickets imported from the
previous Labs instance (public registration will be enabled once all
remaining tasks <https://phabricator.wikimedia.org/T463> have been sorted
out). Restricting access to Phabricator tasks based on project membership
<https://phabricator.wikimedia.org/T50> was implemented. Inbound email was
<https://phabricator.wikimedia.org/T244> configured so Phabricator
lets you interact
with external (non-Phabricator) users via email
<https://phabricator.wikimedia.org/T52>. A certificate for
phab.wmfusercontent.org <https://phabricator.wikimedia.org/T373> for file
uploads was set up and Operations set up SNI on misc-web-lb
<https://rt.wikimedia.org/Ticket/Display.html?id=8345>, made it work with
nginx, added IPv6 access
<https://bugzilla.wikimedia.org/show_bug.cgi?id=71270>, and fixed an error
when trying to log in via HTTP <https://phabricator.wikimedia.org/T457>.
The legal footer required was set up with its license
<https://phabricator.wikimedia.org/T171>, terms of use
<https://phabricator.wikimedia.org/T198> and correct links
<https://phabricator.wikimedia.org/T505>. Reedy added phabricator to the
interwiki map <https://phabricator.wikimedia.org/T454>, and Helder started
to the 'tracked' gadget on MediaWiki to query the Phabricator API
<https://phabricator.wikimedia.org/T177>. Many further smaller fixes took
place. Furthermore, Quim improved the Phabricator documentation and help
<https://www.mediawiki.org/wiki/Phabricator/Help>. Andre showed the very
basics of Phabricator <http://www.youtube.com/watch?v=-fpkHyCGX1Y> in a
video. Daniel and Yuvi set up a new Phabricator test instance on
https://phab-01.wmflabs.org/ that anybody is welcome to test.
Analytics <https://www.mediawiki.org/wiki/Analytics>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=16>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=16>
]
*Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>* [edit
<https://www.mediawiki.org/w/index.php?title=Analytics/Wikimetrics/status&veaction=edit>
]
Work was done on the following metrics:
- Rolling New Active Editor - Implemented
- Rolling Surviving New Active Editor - Implemented
- Pages Created and Edits - Updated to include reporting configuration
to include changes to deleted pages (this is a default).
- Metrics with ‘Namespaces’ as a parameter let you specify “All
Namespaces.” Leave the input field blank to do so.
- Rolling recurring old active editor is implemented, but does not
perform sufficiently rapidly for us to enable it on the production servers.
- The status of the implementation of Standardized Metrics defined by
the Research Team is here:
https://meta.wikimedia.org/wiki/Research:Metrics_standardization/Implementa…
*Data Processing <https://www.mediawiki.org/wiki/Analytics/Data_Processing>*
[edit
<https://www.mediawiki.org/w/index.php?title=Analytics/Data_Processing/status&veaction=edit>
]
A terrific weekly summary is posted to the Analytics mailing list with a
summary at the top of each email. Here are the links to related posts in
the archives.
- 2014-09-01--2014-09-07
<https://lists.wikimedia.org/pipermail/analytics/2014-September/002493.html>
- 2014-09-08--2014-09-14
<https://lists.wikimedia.org/pipermail/analytics/2014-September/002496.html>
- 2014-09-15--2014-09-21
<https://lists.wikimedia.org/pipermail/analytics/2014-September/002522.html>
- 2014-09-22--2014-09-28
<https://lists.wikimedia.org/pipermail/analytics/2014-October/002539.html>
*Editor Engagement Vital Signs
<https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs>* [
edit
<https://www.mediawiki.org/w/index.php?title=Analytics/Editor_Engagement_Vital_Signs/status&veaction=edit>
]
The Vital Signs <https://metrics.wmflabs.org/static/public/dash/> dashboard
is now live. We are calling it “Vital Signs” because it will eventually
display content and readership metrics, not just Editor Engagement metrics.
Vital Signs was presented at the Analytics Quarterly Review as well as the
October WMF Metrics meeting
<https://meta.wikimedia.org/wiki/WMF_Metrics_and_activities_meetings/2014-10>
.
*EventLogging <https://www.mediawiki.org/wiki/Analytics/EventLogging>* [edit
<https://www.mediawiki.org/w/index.php?title=Analytics/EventLogging/status&veaction=edit>
]
Work <https://bugzilla.wikimedia.org/show_bug.cgi?id=68931> was performed
to clean up some EventLogging tables per the privacy policy.
*Research and Data
<https://www.mediawiki.org/wiki/Analytics/Research_and_Data>* [edit
<https://www.mediawiki.org/w/index.php?title=Analytics/Research_and_Data/status&veaction=edit>
]
This month we onboarded Ellery Wulczyn
<https://meta.wikimedia.org/wiki/User:Ewulczyn_(WMF)> as the newest
addition to the Research & Data team. Ellery recently finished a Computer
Science Masters program at Stanford and joins us as a full-time research
analyst after completing a summer fellowship with University of Chicago's Data
Science for Social Good <http://dssg.io/> program. His focus at WMF is
going to be fundraising research and analytics. Welcome, Ellery!
We completed the definitions, documentation and requirements for a new set
of metrics
<https://meta.wikimedia.org/wiki/Research:Metrics_standardization#q1-2015> to
be implemented in Vital Signs
<https://www.mediawiki.org/wiki/Analytics/Editor_Engagement_Vital_Signs>.
We completed a first draft of a page view definition
<https://meta.wikimedia.org/wiki/Research:Page_view>, which is currently
being discussed. We supported the mobile team with baseline traffic reports
<https://trello.com/c/8OyGYWEM/379-mobile-apps-traffic-analytics> for Apps
and Mobile Web.
We participated in the preparatory sessions for the design of an open
consultation led by the Community Liaison team as well as in regular
meetings to support the strategy consultation process.
We held our Q1-2015 quarterly review
<https://www.mediawiki.org/wiki/File:Analytics_Quarterly_Review_Q1_2014-15_(Research_and_Data).pdf>,
reviewed the team's progress against Q1 goals and posted our proposed Q2
goals
<https://www.mediawiki.org/wiki/Analytics/Research_and_Data/2014-15_Goals#Q2:_Oct_-_Dec_2014>
.
Kiwix <http://www.kiwix.org/>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=17>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=17>
]
*The Kiwix project is funded and executed by Wikimedia CH
<https://meta.wikimedia.org/wiki/Wikimedia_CH>.*
<https://www.mediawiki.org/wiki/File:Kiwix_bookeen_1.png>
Kiwix and Wikipedia offline will soon be available on an e-ink device.
We made progress in our work with our partner Bookeen
<http://www.bookeen.com/> to get an e-ink reader able to read Wikipedia
offline. We managed to get a first version of the device firmware working,
and it will be tested in the field as part of the Malebooks
<http://malebooks.ml/> pilot deployment.As a consequence of a bug fixing
sprint with Parsoid and Wikisource developers at Wikimania, we were finally
able to generate *usable* ZIM files from Wikisource (dxample with
fr.wikisource.org <http://library.kiwix.org/wikisource_fr_all_09_2014/>).Work
on the offline <https://github.com/kiwix/gutenberg> project Gutenberg
<http://gutenberg.org/> continued and we are now almost ready to release. A
few ZIM files are in testing, for example in German
<http://library.kiwix.org/gutenberg_de_all_10_2014/> and in Spanish
<http://library.kiwix.org/gutenberg_es_all_10_2014/>.Kiwix was represented
at the Selenium conference <http://seleniumconf.org/> where we held a 2-day
bug hunting session: 120 bugs were reported, of which 50% were fixed
<https://github.com/Bug-Bash/selenium-kiwix/issues>.mwoffliner
<https://sourceforge.net/p/kiwix/other/ci/master/tree/mwoffliner/> was
improved to make it easier to use for everyone, in particular to make ZIM
files for only a selection of articles. As a demonstration, we prepared a
ZIM files containing all Wikipedia articles about medicine
<http://download.kiwix.org/zim/wikipedia/wikipedia_en_medicine_09_2014.zim.torrent>
.After many years, a new version of a tool to generate a live CD
<http://kiwix.org/wiki/Live_CD> including Kiwix and Wikipedia content was
released.Wikidata <https://meta.wikimedia.org/wiki/Wikidata>[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=18>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=18>
]
*The Wikidata project is funded and executed by Wikimedia Deutschland
<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.*
In September, the Wikidata team focused on improving performance, doing
groundwork for the new user interface design, and making it possible to
track where data from Wikidata is used. Next to that, they worked on tests
and prepared for a week-long meeting with the WMF multimedia team and
volunteers to discuss and plan structured data support for Wikimedia
Commons.Future[edit
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&veaction=edit&vesection=19>
| edit source
<https://www.mediawiki.org/w/index.php?title=Wikimedia_Engineering/Report/2014/September&action=edit§ion=19>
]The engineering management team continues to update the *Deployments
<https://wikitech.wikimedia.org/wiki/Deployments>* page weekly, providing
up-to-date information on the upcoming deployments to Wikimedia sites, as
well as the *annual goals
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/2014-15_Goals>*,
listing ongoing and future Wikimedia engineering efforts.
--
Guillaume Paumier