[Wikimedia-l] [Wikimedia Announcements] Wikimedia engineering January 2013 report

Guillaume Paumier gpaumier at wikimedia.org
Thu Feb 7 19:57:53 UTC 2013 

Hi,

The report covering Wikimedia engineering activities in January 2013 is now
available.

Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/January
Blog version:
https://blog.wikimedia.org/2013/02/07/engineering-january-2013-report/

We're also proposing a shorter, simpler and translatable version of this
report that does not assume specialized technical knowledge:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/January/summary

Below is the full HTML text of the report, as previously requested.

As always, feedback is appreciated about the usefulness of the report and
its summary, and on how to improve them.

------------------------------------------------------------------

Major news in January include:

   - the successful migration of our main
services<https://blog.wikimedia.org/2013/01/19/wikimedia-sites-move-to-primary-data-center-in-ashburn-virginia/>to
our data center in Ashburn, Virginia;
   - new features<https://blog.wikimedia.org/2013/01/11/mobile-beta-a-sandbox-for-new-experimental-features/>available
in our mobile beta;
   - progress on input
methods<https://blog.wikimedia.org/2013/01/25/language-engineering-progress-with-input-methods-and-translation-editor/>and
our upcoming translation
   interface<https://blog.wikimedia.org/2013/01/11/a-more-efficient-translation-interface/>
   ;
   - the announcement of
GeoData<https://blog.wikimedia.org/2013/01/31/geodata-a-new-age-of-geotagging-on-wikipedia/>,
   a feature to attach geo-coordinates to Wikipedia and Wikivoyage articles;
   - a testing event to assess how VisualEditor handles non-Latin
characters<https://blog.wikimedia.org/2013/01/28/help-us-test-and-investigate-visualeditor/>
   .

*Note: We're also proposing a shorter, simpler and translatable version of
this report<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/January/summary>that
does not assume specialized technical knowledge.
*
Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us>

Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.

   - Software Engineer - Editor
Engagement<http://hire.jobvite.com/Jobvite/Job.aspx?j=ovvXWfwD>
   - Technical Writer -
(Contract)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGH5Wfw8>
   - Software Developer -
Fundraising<http://hire.jobvite.com/Jobvite/Job.aspx?j=oF2EWfw1>
   - Software Engineer
(Partners)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oX2hWfwW>
   - Software Engineer
(Apps)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oqU0Wfw0>
   - Software Developer General
(Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4cKWfwG>
   - Software Engineer -
Multimedia<http://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3>
   - Software Engineer
(Search)<http://hire.jobvite.com/Jobvite/Job.aspx?j=ogk1Wfwh>
   - Product Manager
(Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGWJWfw1>
   - Director of User
Experience<http://hire.jobvite.com/Jobvite/Job.aspx?j=otv0WfwE>
   - Visual Designer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oomJWfw9>
   - Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf>
   - Operations Engineer/Database
Administrator<http://hire.jobvite.com/Jobvite/Job.aspx?j=obMOWfwr>
   - Site Reliability
Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=o7k2Wfw9>
   - Tools Lab Operations Engineer
(Contractor)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o7y3Wfwo>

 Announcements

   - Yuvaraj Pandian re-joined the Mobile engineering
team<https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering>as
Software developer (
   announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065636.html>).
   He joined the newly created Mobile App team with Brion Vibber and Shankar
   Narayan.
   - Munagala Ramanath (Ram) joined the MediaWiki core team of the Platform
   engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>group
as Senior Software Engineer (
   announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065698.html>
   ).
   - Runa Bhattacharjee joined the Language
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering>team
as Outreach and QA coordinator (
   announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/066030.html>
   ).

 Technical Operations

*Production Site
Switchover<http://wikitech.wikimedia.org/view/Eqiad_Migration_Planning>
*
The Wikimedia Foundation switched over its primary data
center<http://blog.wikimedia.org/2013/01/19/wikimedia-sites-move-to-primary-data-center-in-ashburn-virginia/>from
Tampa, Florida to Ashburn, Virginia on January 22. Given the scale
and complexity<http://wikitech.wikimedia.org/view/Eqiad_Migration_Planning/Steps>of
the migration, we scheduled three 8-hour windows to perform the
migration, but we were able to complete
it<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065892.html>on
the first attempt. Because the switchover involved, among other
things,
moving over the master databases from Tampa to Ashburn, the site was set to
'read-only' mode for about 32 minutes. During that period, the site was
available but no new contents were created, edited or uploaded. As
expected, there was some minor fallout of the migration, mostly due to
configuration changes, but they were quickly contained by the Engineering
and Operation teams.With this migration, Tampa data center will now be our
fail-over site and we plan to perform site fail-over tests every few
months. There are remaining small non-core applications still using Tampa
as the primary site, such as RT, etherpad and Bugzilla. They too will be
migrated in the coming months.

*Site infrastructure*
One of the main concerns of the migration was serving traffic from the new
data center using empty memcached servers: the spike in load on the Apache
and database servers could have been disastrous to the site. To address it,
Tim Starling improved on the single instance implementation of 'Parser
Cache' persistent store in Tampa (to 3 sharded instances), and Asher
Feldman built and replicated the databases across the 2 data centers.Another
improvement, done by Asher and Peter Youngmeister, was the implementation
of MHA <http://code.google.com/p/mysql-master-ha/> (Master High
Availability) on our MySQL clusters. Its primary objective is to automate
the promotion of a slave database in a master database fail-over scenario
and to to reduce downtime, without suffering from replication integrity
problems, without prolong database latency, and without changing existing
deployments.Faidon Liambotis and Mark Bergsma continued to work on the Ceph
file object store. With Domas Mituzas' help, they identified a performance
issue with the RAID card which caused severe read/write latency on the Ceph
cluster. Faidon has confirmed with the vendor that it is a known problem
and no fix is available yet. We have ordered and substituted those RAID
cards, and test results seem to indicate that the performance issue is
solved.

*Fundraising*
Fundraising bastion hosts were deployed in the Ashburn and Tampa data
centers. We also tweaked and tuned central logging and monitoring, and
converted the remaining fundraising MuISAM tables to InnoDB, which should
fix dump-induced replication lag.

*Data Dumps <https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps>*
This month, we had a look at the process of using the XML dumps to create a
local copy of a Wikimedia site: it turned out to be painful and cumbersome
at best, and unfathomable for the end-user in the worst case. As part of an
attempt to improve this situation, there is now a new experimental
tool<https://gerrit.wikimedia.org/r/gitweb?p=operations/dumps.git;a=tree;f=xmlfileutils;h=47995d9e8aea8a1d0dc7fb12c4152f2010daeb6a;hb=ariel>available
for *nix platforms, for generating MySQL tables from the XML stub
and page content files. It is intended to read input files from various
versions of MediaWiki and generate output for the version the user wants.
Testing and feedback is encouraged.

*Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs>*
In January, we had a number of performance and usability improvements.
Three compute nodes were added into the pmtpa zone. Alex Monk added Echo
notification support to labsconsole, passwordless sudo is now the default
for projects, and shell requests are created automatically on account
creation. The sysadmin and netadmin roles have been combined into a single
projectadmin role. Glusterfs was upgraded to handle a memory leak, but
unfortunately a new bug has been introduced that caused some instability in
project storage. Work is ongoing to improve the project storage
situation. Features
Engineering <https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>
Editor
retention: Editing tools

*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>*
In January, the team worked primarily on reviewing and cleaning-up the code
deployed<https://blog.wikimedia.org/2012/12/12/try-out-the-alpha-version-of-the-visualeditor/>in
December. They spent time with their colleagues in the Parsoid team
planning the next phase of development, which is aimed at making the
VisualEditor the default editor for all Wikipedias from July 2013. The
alpha version of the VisualEditor on mediawiki.org and the English
Wikipedia was updated twice
(1.21-wmf7<https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf7#VisualEditor>and
-wmf8 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf8#VisualEditor>),
fixing a number of bugs reported by the community and making some
adjustments to the link inspector's functionality based on feedback.

*Parsoid <https://www.mediawiki.org/wiki/Parsoid>*
In January, the Parsoid <https://www.mediawiki.org/wiki/Parsoid> team did
some Spring cleaning and bug fixing. The serialization subsystem was
overhauled: it now features simpler and more robust separator handling.
Selective serialization was rewritten to deal with content deletions. It
also features DOM diff-based change detection that does not rely on
client-side change marking. Support for non-English wikis and local
configurations was also improved a lot, and will likely stabilize in the
next weeks. The team also discussed and documented the longer-term Parsoid
/ MediaWiki strategy in the Parsoid
roadmap<https://www.mediawiki.org/wiki/Parsoid/Roadmap>.
The performance-oriented C++ port was deprioritized in favor of DOM-based
performance improvements and HTML storage. The basic idea behind storing
(close to) fully processed HTML is to speed things up by doing no
significant parsing on page view at all. In the longer term,
VisualEditor-only wikis can avoid a dependency on Parsoid by switching to
HTML storage exclusively. Overall, the plan is to leverage the
Parsoid-generated HTML/RDFa DOM format inside MediaWiki core to enable
better performance and editing capabilities in the future.
 Editor engagement features

*Notifications <https://www.mediawiki.org/wiki/Echo_%28Notifications%29>*
This month, we stepped up development on the Notifications project
*Echo*and updated our first experimental release on
mediawiki.org. Ryan Kaldari and Benny Situ improved the user experience for
core features such as the badge, fly-out, all-notifications page and email
notifications, and started developing new features such as
bundling<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Bundling>,
dismiss <https://www.mediawiki.org/wiki/Echo/Feature_requirements#Dismiss>and
web
preferences<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Web_Notifications>.
Luke Welling completed work on HTML email and started development of a more
robust job queue<https://www.mediawiki.org/wiki/Echo/Feature_requirements#JobQueue>.
Fabrice Florin led discussions about the Echo product plan, and new
features<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Features_under_consideration>and
notifications<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Sample_Notifications>under
consideration, while Vibha Bamba designed new components of the user
experience. We plan to develop some of these features and notifications in
coming weeks, and are aiming for a first release on the English Wikipedia
by the end of March; in the meantime, you can try the current
version<https://www.mediawiki.org/wiki/Echo/Testing>on
mediawiki.org. We are also recruiting for a software
engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ovvXWfwD&c=qSa9VfwQ>to
join our team and work with us on this and other editor engagement
projects.

*Flow <https://www.mediawiki.org/wiki/Flow>*
Flow entered the product design phase in early January. OPW
intern<https://www.mediawiki.org/wiki/Outreach_Program_for_Women>Kim
Schoonover began user
research <https://www.mediawiki.org/wiki/Flow/User_tests> regarding how
user-to-user talk pages are handled, and collected data about the
difficulties that new (and existing) users have when using them.
Engineering discussions started about potential back-end and scaling
difficulties, the possible use of Wikidata's
ContentHandler<https://www.mediawiki.org/wiki/ContentHandler>,
and the evaluation of Wikia's MessageWall. A plan for community engagement
was proposed and accepted, with a consultation about the problems faced
planned for early February, with experienced and newer users alike.

*Article feedback <https://www.mediawiki.org/wiki/Article_feedback>*
This month, our team updated Article Feedback
v5<https://www.mediawiki.org/wiki/Article_feedback/Version_5>and
discussed its release with communities in the English, French and
German Wikipedias. Developer Matthias Mullie completed a major code
refactoring, which is now being reviewed. He also developed a final set of new
features<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Features_under_consideration>,
such as simpler moderation tools and better filters, to be tested next
month. Dario Taraborelli and Aaron Halfaker posted a feedback evaluation
report<https://meta.wikimedia.org/wiki/Research:Article_feedback/Final_quality_assessment>,
which suggests that about 39% of the feedback collected in their study can
be used to improve articles (see also their other study
results<https://commons.wikimedia.org/wiki/File:AFT5_2012-Q4_report.pdf>).
Oliver Keyes responded to community questions in a request for
comments<https://en.wikipedia.org/wiki/Wikipedia:Requests_for_comment/Article_feedback>about
future deployments on the English Wikipedia, with a final decision
expected next month. Fabrice Florin led product planning and discussed a
possible deployment on the French
Wikipedia<https://en.wikipedia.org/wiki/fr:Wikip%C3%A9dia:Prise_de_d%C3%A9cision/Mise_en_place_de_l%27outil_d%27%C3%A9valuation_des_articles_sur_la_Wikip%C3%A9dia_en_fran%C3%A7ais>and
with the German
Wikipedia <https://en.wikipedia.org/wiki/de:Wikipedia:Artikel-Feedback>,
currently evaluating the tool in an ongoing
pilot<https://en.wikipedia.org/wiki/de:Spezial:Artikelr%C3%BCckmeldungen_v5>with
a vote expected in May. Once our development is complete and
communities reach their decisions for each project, we expect to release
Article Feedback v5 on a range of Wikimedia sites in coming months.
 Editor engagement experiments

*Editor engagement
experiments<https://www.mediawiki.org/wiki/Editor_engagement_experiments>
*
In January, the Editor Engagement Experiments team ("E3") planned its goals
for the quarter<https://www.mediawiki.org/wiki/Editor_engagement_experiments/Quarterly_Planning>,
which ends in March. We also made progress on the following projects which
are included in that plan.

First up, we launched<http://blog.wikimedia.org/2013/02/01/guided-tour-launch/>
guided
tours <https://www.mediawiki.org/wiki/Guided_tours> on the English
Wikipedia, including a test tour to demonstrate the capabilities of the
extension, and a tour associated with the "onboarding new
Wikipedians<https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians>"
(aka GettingStarted) project. In addition to tours created by the team, the
extension <https://www.mediawiki.org/wiki/Extension:GuidedTour> supports
community-created tours. Note that unlike many other projects by the E3
team, guided tours are planned as a permanent addition to Wikipedia, with
each tour implementation considered to be experimental. (For example: the
"getting started" tour will be delivered via a split A/B test.)

While building guided tours, the team also A/B tested the Getting
Started<https://www.mediawiki.org/wiki/Extension:GettingStarted>landing
page and task list, measuring the effect it had on driving new
contributions. Several rounds of analysis were completed and published on
Meta (round 1<https://meta.wikimedia.org/wiki/Research:Onboarding_new_Wikipedians/OB1>,
round 2<https://meta.wikimedia.org/wiki/Research:Onboarding_new_Wikipedians/OB2>),
with the conclusion that the onboarding experience is leading to small but
statistically significant increases in new English Wikipedians attempting
to edit, as well as saving their first edit. In addition to measuring the
effects of the guided tour associated with this project, immediate plans
are to redesign the landing page and add additional task types, to entice
more new contributors.
Work also continued on refining the reliability and precision of the data
collected from EventLogging<https://www.mediawiki.org/wiki/Extension:EventLogging>.
In particular, we migrated EventLogging to a dedicated database, and began
collecting server-side events in addition to client-side, to support work
such as measuring account creations on desktop and mobile. January also saw
the heavy use of the new User Metrics
API<https://meta.wikimedia.org/wiki/Research:Metrics>,
in order to complete cohort analysis of onboarding users and for metrics
reported at the Board
presentation<https://meta.wikimedia.org/wiki/Agenda_February_1-2_2013_Board_Meeting>on
the Foundation's year-to-date progress. Development of the API
continues, and a public announcement is expected for early March. Last but
not least, a call was put out for a part-time Technical
Writer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGH5Wfw8&c=qSa9VfwQ>to
work on documenting both of these pieces of infrastructure.
 Support

*2012 Wikimedia
fundraiser<https://www.mediawiki.org/wiki/2012_Wikimedia_fundraiser>
*
January marks the official end of the 2012 fundraiser. The team spent the
entirety of the month cleaning up and recovering from the very successful
months of November and December, auditing the donations, and writing tools
that will help the team run continuous auditing in the future.
 Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering> Web

*GeoData Storage & API<https://www.mediawiki.org/wiki/GeoData_Storage_%26_API>
*
After its soft launch in December, GeoData was officially
announced<http://blog.wikimedia.org/2013/01/31/geodata-a-new-age-of-geotagging-on-wikipedia/>this
month. Work on improvements and bug fixing continues. The
Special:Nearby page, which has been deployed to an experimental version of
the site, represents the first major use of this feature on mobile
projects. We hope to use it to help contributors identify articles in need
of photos.

*Mobile QA <https://www.mediawiki.org/wiki/Mobile_QA>*
The push to get
MobileFrontend<https://www.mediawiki.org/wiki/Extension:MobileFrontend>up
and running on Beta Labs is well underway. We've also added test
cases for Wikipedia
Zero<https://www.mediawiki.org/wiki/Wikipedia_Zero/Test_cases>and we
are planning a community test event for Mobile Upload and Commons in
February.

*Mobile Web Photo Upload<https://www.mediawiki.org/wiki/Mobile_design/Uploads>
*
This month, the mobile web team finished up work on the watchlist feature
and kicked off a 3-month sprint on photo uploads. The focus in January was
on developing basic uploading infrastructure: uploading images to Commons
under a single Creative Commons license. We also built out the UX/UI design
for a call to action on articles lacking images in the lead section.
Through this workflow, users can upload an image to Commons and add a
thumbnail of the image to the appropriate article on their local Wikipedia
or sister project, in one simple step. We also developed a mobile uploads
page where contributors can see their recent uploads and potentially donate
more images from their mobile device to Commons. These features are
currently live on the Beta mobile site and are set to be released to the
full mobile site in February.
 Apps

*Commons App <https://www.mediawiki.org/wiki/Apps/Commons>*
January marked the first month of the Apps team's existence. Yuvaraj
Pandian has started work with Brion Vibber on iOS and Android-based apps to
upload photos to Commons. Both platforms are being developed concurrently
and will have feature parity. Shankar Narayan joined us and and will be
supporting the team for all design needs. While the first iteration of the
Commons App isn't scheduled to finish until February 8th, the team has
already created two skeleton apps that can upload, share and show the
user's contributions. The team will be spending their next iteration
tweaking workflows and styling the app. We also released new versions of
the Wikipedia app on iOS and Android in order to bring it into compliance
for legal privacy/disclaimer issues.
 Partners

*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>*
During January, Wikimedia was awarded a
grant<https://blog.wikimedia.org/2013/01/17/wikimedia-foundation-winner-of-knight-news-challenge/>in
the Knight News Challenge for our work in expanding Wikimedia mobile
projects. Part of this grant will be used for Wikipedia Zero and the
SMS/USSD <https://www.mediawiki.org/wiki/Wikipedia_over_SMS_%26_USSD>projects
to improve access to knowledge in the developing world. In
addition, we've partnered with
VimpelCom<https://blog.wikimedia.org/2013/01/24/vimpelcom-partnership-grows-wikipedia-zero-330-million/>to
provide Wikipedia Zero to at least 100 million additional customers
this
year.

*J2ME App <https://www.mediawiki.org/wiki/Wikipedia_S40_J2ME_app>*
During January, we've begun to explore ways to reduce the memory and
processor requirements of our J2ME app, to increase the number of phones
that can use this application.

*Wikipedia over SMS &
USSD<https://www.mediawiki.org/wiki/Wikipedia_over_SMS_%26_USSD>
*
We are finishing work on capturing the metrics from the SMS server to learn
usage numbers and determine how many sessions are completed.
 Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>
MediaWiki
Core

*MediaWiki 1.21 <https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap>*
MediaWiki 1.21wmf7 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf7> and
1.21wmf8 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf8> were deployed
in January on a modified schedule, due to holidays and because of the data
center migration. Deployments have returned to their usual fortnightly
schedule.

*Git conversion <https://www.mediawiki.org/wiki/Git/Conversion>*
The ExtensionDistributor<https://www.mediawiki.org/wiki/Extension:ExtensionDistributor>was
rewritten in early January. While this was primarily done to support
the data center
migration<http://wikitech.wikimedia.org/view/Eqiad_Migration_Planning>,
this was the first time ExtensionDistributor had received any signification
attention since the migration to Git. The new version now utilizes the
Github API to generate extension snapshots. We hope that the new version
will be more reliable for users. SVN-based extensions are no longer
supported, but this is not expected to impact many users since these
extensions are largely unmaintained (all popular and active extensions have
long since moved to Gerrit). As always, these extensions will remain in SVN
should anyone still want the code.

*TimedMediaHandler <https://www.mediawiki.org/wiki/TimedMediaHandler>*
Jan Gerber continues bugfixing and refining TimedMediaHandler, mainly
focusing on operational improvements to make more efficient use of our
server infrastructure.

*Wikidata deployment <https://www.mediawiki.org/wiki/Wikidata_deployment>*
Sam Reed helped the Wikidata deployment, deploying the Wikibase Client
extension to Wikipedia in Hungarian, Hebrew, and Italian. Chris Steipp
reviewed the Wikidata team's work to extend AbuseFilter for use with
structured data. Aaron Schulz worked with Daniel Kinzler on job queue
improvements.

*Wikivoyage migration <https://www.mediawiki.org/wiki/Wikivoyage_migration>*
Wikivoyage officially
launched<https://blog.wikimedia.org/2013/01/15/wikimedia-foundation-launches-wikivoyage-a-free-worldwide-travel-guide-that-anyone-can-edit/>on
January 15. Most of the Wikimedia Foundation's involvement was
completed
in November, but some minor bugfixing was done in support of the official
launch.

*SwiftMedia <https://www.mediawiki.org/wiki/SwiftMedia>*
NFS for uploads/thumbnails has been unmounted from all Apache servers and
the NFS back-end configuration was removed from MediaWiki; all files now
only use Swift. A workaround has been added for the Swift back-end class
when used with Ceph, so that temporary URLs can be used (for making video
thumbnails for example). A Python script to copy files into Ceph has been
run and is being worked on. Various issues have been reported in Ceph's bug
tracker and are being looked at by the developers.

*Lua scripting <https://www.mediawiki.org/wiki/Lua_scripting>*
Lua development was put on hold through the Ashburn data center migration.
We've now resumed work on Lua, with Brad Jorsch and Tim Starling making
more functions available in Lua that are currently already available in
template parser functions.

*Site performance <https://www.mediawiki.org/wiki/Site_performance>*
A patch to allow moving the DB job queue to another cluster is under
review. An experimental redis-based job queue patch also exists in gerrit.

*Incremental architectural
improvements<https://www.mediawiki.org/wiki/Incremental_architectural_improvements>
*
Code was merged to support more complex data structures (lists, sets) in
memcached (with atomic updates).

*Admin tools development<https://www.mediawiki.org/wiki/Admin_tools_development>
*
The team mainly focused this month on improving the AbuseFilter extension,
which is now working on the
Wikidata<https://www.mediawiki.org/wiki/Wikidata>site after support
was added for other content types (as defined using
ContentHandler <https://www.mediawiki.org/wiki/ContentHandler>). There was
some significant work done on blocking abusive proxies and abuse limits,
and some additional progress made on global AbuseFilters, user renaming and
the interface for Stewards to mass-lock user
accounts<https://www.mediawiki.org/wiki/Admin_tools_development/CentralAuth_Locking>
.

*Security auditing and
response<https://www.mediawiki.org/wiki/Security_auditing_and_response>
*
The team continued to respond to reported vulnerabilities, began a security
review of fundraising extensions, and continued reviews of Wikidata
features.
 Quality assurance

*Quality Assurance <https://www.mediawiki.org/wiki/QA>*
We started to schedule opportunities for community testing
events<https://www.mediawiki.org/wiki/QA/Weekly_goals>.
Echo, AFTv5, and VisualEditor are all current candidates for testing. A
week-long focus on VisualEditor's support for non-Latin
characters<https://www.mediawiki.org/wiki/VisualEditor/Testing_Non-Latin_Characters_Input_and_Behavior>uncovered
at least one major issue causing data loss, and another one with
tool-assisted Chinese input.

*Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>*
The main use for the Beta Cluster in January was to test
git-deploy<http://wikitech.wikimedia.org/view/git-deploy>.
Zeljko Filipin continues to run regular tests there. Antoine Musso, Max
Semenik, and Andrew Bogott are setting up MobileFrontend to run on Beta for
testing purposes.

*Continuous integration<https://www.mediawiki.org/wiki/Continuous_integration>
*
Antoine Musso worked with several MediaWiki extension authors to ensure
that the unit tests for those extensions are run by Jenkins and that they
work. He hopes to have all extensions that run on the Wikimedia production
cluster fully operational by the end of February. Antoine also integrated PHP
CodeSniffer<https://www.mediawiki.org/wiki/Continuous_integration/PHP_CodeSniffer>into
our automated test runs.

*Browser testing <https://www.mediawiki.org/wiki/QA/Browser_testing>*
Architecture and configuration for browser testing are now stable, and the
focus shifted to increasing test coverage by making existing tests more
extensive and covering new features. An example is the Math
extension<https://www.mediawiki.org/wiki/Extension:Math>,
which was briefly broken after the data center migration. The team has also
instituted a weekly pair-programming session every Friday.
 Analytics

*Limn <https://www.mediawiki.org/wiki/Analytics/Limn>*
The team made performance improvements and added new visualizations. Bar,
line, and geo plots can now be built ad hoc from arbitrary data. Evan
Rosen's Grantmaking and Programs dashboard was migrated to this new version
of Limn. Current work includes a collaboration with the E3 team to provide
visualizations, and development of a MediaWiki extension that will allow
creation and editing of graphs.
 Engineering community team

*Bug management <https://www.mediawiki.org/wiki/Bug_management>*
This month, a first bugday was
held<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065792.html>,
targeting bug reports which had not seen any changes for more than one
year, resulting in about 30 tickets being updated. In addition, some
cleanup work (decreasing the number of unprioritized bug reports and going
through open reports in "ASSIGNED" status for more than a year) took place.
Andre Klapper worked on small Bugzilla code
changes<https://bugzilla.wikimedia.org/buglist.cgi?bug_id=42467,24992>and
published initial information on Bugzilla
usage per development
team<https://www.mediawiki.org/wiki/Bug_management/Development_teams_usage>.
Community members were invited to join the MediaWiki Group Bug
Squad<https://www.mediawiki.org/wiki/Groups/Proposals/Bug_Squad>.
Furthermore, some problems due to data center
migration<https://blog.wikimedia.org/2013/01/19/wikimedia-sites-move-to-primary-data-center-in-ashburn-virginia/>were
investigated, and it was discussed how to improve interaction on
Bugzilla tickets that need handling by the Operations team (who mostly
prefers to use the RT <https://rt.wikimedia.org/> bugtracker instead).

*Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>*
Six Outreach Program for
Women<https://www.mediawiki.org/wiki/Outreach_Program_for_Women>interns
started on January 3rd and will work full time until April.
Mariya <https://www.mediawiki.org/wiki/User:Mitevam> is working on a discussion
among third-party MediaWiki
users<https://www.mediawiki.org/wiki/Third-party_MediaWiki_users_discussion>.
Valerie <https://www.mediawiki.org/wiki/User:Valeriej> has completed the Bug
Squad group proposal<https://www.mediawiki.org/wiki/Groups/Proposals/Bug_Squad>and
a first Bug Day.
Priyanka <https://www.mediawiki.org/wiki/User:Priyanka_Nag> created a
script<https://www.mediawiki.org/wiki/User:Priyanka_Nag/common.js>and
plans to move to
Git <https://www.mediawiki.org/wiki/Git>.
Sucheta<https://www.mediawiki.org/wiki/User:Sucheta_Ghoshal>is on
schedule following her project
plan<https://www.mediawiki.org/wiki/User:Sucheta_Ghoshal/OPW-EtherEditor#Plans_and_Estimated_Timeline>.
Kim <https://www.mediawiki.org/wiki/User:Isarra> is learning about
Flow<https://www.mediawiki.org/wiki/Flow>and the basics of interactive
design as indicated by her mentor.
Teresa <https://www.mediawiki.org/wiki/User:Chot> has completed a
solid base<https://gerrit.wikimedia.org/r/#/q/owner:tcho708%2540gmail.com,n,z>for
her extension and is working on the main functionality. She hit a snag
with her work environment this week, but is still on track with her
proposed timeline. The Google Summer of Code
2013<https://www.mediawiki.org/wiki/Summer_of_Code_2013>page was
created, a pre-planning
discussion<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065771.html>started
on wikitech-l, and
LevelUp <https://www.mediawiki.org/wiki/Mentorship_programs/LevelUp>matchmaking
for the first quarter of 2013 is nearly done.

*Technical communications<https://www.mediawiki.org/wiki/Technical_communications>
*
Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom>
provided communications
support<https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity>to
the engineering team, notably around the data
center migration<https://blog.wikimedia.org/2013/01/19/wikimedia-sites-move-to-primary-data-center-in-ashburn-virginia/>and
associated
banners<https://meta.wikimedia.org/wiki/CentralNotice/Generic_maintenance_notice>,
notices<https://meta.wikimedia.org/w/index.php?title=Global_message_delivery/Spam&oldid=5081391>&
translations<http://lists.wikimedia.org/pipermail/translators-l/2013-January/002199.html>.
He started to organize and clean up the MediaWiki version pages (like MediaWiki
1.21/wmf7 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf7>) to make
them more useful for tech
ambassadors<https://meta.wikimedia.org/wiki/Tech/Ambassadors>,
by highlighting the most important changes, improving translatability and
adding navigation. He also prepared and organized translations for the *How
to report a bug <https://www.mediawiki.org/wiki/How_to_report_a_bug>* and *How
to contribute <https://www.mediawiki.org/wiki/How_to_contribute>* pages, to
facilitate the involvement of volunteers who don't necessarily communicate
in English. Last, he created a
Project:Calendar<https://www.mediawiki.org/wiki/Project:Calendar>to
consolidate and centralize announcements for all
events <https://www.mediawiki.org/wiki/Template:Event>, to make
opportunities for participation more visible. Events around a particular
topic (like QA, testing and
bugs<https://www.mediawiki.org/wiki/QA/Weekly_goals>)
can still be selectively transcluded, using Labeled Section
Transclusion<https://www.mediawiki.org/wiki/Extension:Labeled_Section_Transclusion>
.

*Volunteer coordination and
outreach<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>
*
The MediaWiki groups <https://www.mediawiki.org/wiki/Groups> for
Promotion<https://www.mediawiki.org/wiki/Groups/Promotion>and San
Francisco <https://www.mediawiki.org/wiki/Groups/San_Francisco> were
officially approved by the Wikimedia Affiliations
Committee<https://meta.wikimedia.org/wiki/Affiliations_Committee>,
and are the first Wikimedia User
Groups<https://meta.wikimedia.org/wiki/Wikimedia_User_Groups#Approved_user_groups>created.
We helped the Editor
Engagement <https://www.mediawiki.org/wiki/Editor_Engagement> team organize
a sprint to test Echo <https://www.mediawiki.org/wiki/Echo/Testing>, but
our plans to collaborate further with the Editor Engagement and
Mobile<https://www.mediawiki.org/wiki/Mobile>teams were delayed; Quim
Gil proposed a
different approach<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065758.html>combining
regular, time-based
QA <https://www.mediawiki.org/wiki/QA> and bug
management<https://www.mediawiki.org/wiki/Bug_management>activities,
in the form of QA
weekly goals <https://www.mediawiki.org/wiki/QA/Weekly_goals>. Two such
events (non-Latin character testing in
VisualEditor<https://www.mediawiki.org/wiki/VisualEditor/Testing_Non-Latin_Characters_Input_and_Behavior>and
a
review of old bugs<https://www.mediawiki.org/wiki/Bug_management/Triage#Next_meeting>)
happened in January, and more are scheduled. Heavy work was done with Chris
McMahon to improve the top QA
pages<https://www.mediawiki.org/wiki/Category:QA>,
although some problems
remain<http://lists.wikimedia.org/pipermail/wikitech-l/2013-January/065696.html>.
Template:MediaWiki
News<https://www.mediawiki.org/wiki/Template:MediaWiki_News>is now
manually synced with social
media <https://www.mediawiki.org/wiki/Social_media>, bringing fresh updates
to the mediawiki.org homepage <https://www.mediawiki.org/wiki/MediaWiki>and
News <https://www.mediawiki.org/wiki/News> page. Quim also took the lead on
organizing the Wikipedia Engineering
Meetup<http://www.meetup.com/Wikipedia-Engineering-Meetup/events/89239012/>on
January 17th. He prepared an intro
to MediaWiki & Wikimedia tech
contributions<https://www.mediawiki.org/wiki/File:How_to_hack_on_Wikipedia.pdf>,
which he tested at FOSDEM <https://www.mediawiki.org/wiki/Events/FOSDEM>,
designed to be reused by other presenters. Last, we confirmed that
technical projects are eligible to Individual Engagement
Grants<https://meta.wikimedia.org/wiki/Grants:IEG>
.
 Language engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering>

*Language tools <https://www.mediawiki.org/wiki/Language_tools>*
Development of the new user interface for Translate, as well as the
translation editor functionality, continued throughout the month of
January. Focus was on back-end work and extending the WebAPI to support the
remaining features which are needed to reach feature parity with current
editor. The MediaWiki Language Extension Bundle 2013.01 was released.
Universal Language Selector was deployed with limited features to a
selection of Wikimedia sites projects using the Translate extension.
Collaboration projects also continue with Red Hat's language technologies
teams, with an upcoming work sprint to complete several projects extending
internationalization support for Indic languages. Runa Bhattacharjee kicked
off the Language coverage matrix, an attempt to compile a snapshot of our
internationalization tools coverage per language for 300 languages.

*Milkshake <https://www.mediawiki.org/wiki/Milkshake>*
More input methods were added to jQuery.IME, and bugs were fixed in
jQuery.ULS.
 Kiwix <http://www.kiwix.org>

*The Kiwix project is funded and executed by Wikimedia
CH<https://meta.wikimedia.org/wiki/Wikimedia_CH>
.*
We have adapted the kiwix-plug
<http://www.kiwix.org/index.php/Kiwix-plug>script to
Tonidoplug2 <http://www.tonidoplug.com/>, a device cheaper than the
Dreamplug<https://www.globalscaletechnologies.com/p-54-dreamplug-devkit.aspx>.
Kiwix was elected by Sourceforge users as February's Project of the Month
and an interview of Emmanuel
Engelhart<http://sourceforge.net/blog/potm-201302/>was published. For
the first time, Kiwix has reached 100.000 downloads a
month in January. Beside Kiwix, the openZIM website
<http://www.openzim.org>was revamped and simplified for better
readability. The openZIM bug tracker
and source code management were migrated to the Wikimedia infrastructure
(Bugzilla and Git). Wikidata <https://meta.wikimedia.org/wiki/Wikidata>

*The Wikidata project is funded and executed by Wikimedia
Deutschland<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>
.*
January has been an exciting month for Wikidata. The deployment on the
first Wikipedia sites
(Hungarian<https://blog.wikimedia.de/2013/01/14/first-steps-of-wikidata-in-the-hungarian-wikipedia/>,
Hebrew and Italian<https://blog.wikimedia.de/2013/01/30/wikidata-coming-to-the-next-two-wikipedias/>)
was completed. At the same time, work has continued on the user interface
and back-end for statements, the core part of Wikidata's second phase. This
will enable users to enter information like the children of a given person
or a link to their portrait on Wikimedia Commons. These features can
already be tested on the demo system <http://wikidata-test.wikimedia.de>.
We've also worked on making AbuseFilter work with Wikidata, and wrote a new
mechanism to distribute changes to the clients (Wikipedia) so they can show
Wikidata changes in their RecentChanges. We made progress on using Solr for
search and rewrote the draft for the inclusion
syntax<https://meta.wikimedia.org/wiki/Wikidata/Notes/Inclusion_syntax_v0.3>to
be much simpler. This is the syntax that editors will use to include
data from Wikidata in Wikipedia. A manual for using Pywikipedia on
Wikidata<https://www.mediawiki.org/wiki/Manual:Pywikipediabot/Wikidata>was
written as well. If
you want to code on Wikibase, the software powering Wikidata, have a look
at the outstanding bugs and
tasks<https://bugzilla.wikimedia.org/buglist.cgi?keywords=need-volunteer%2C%20&keywords_type=allwords&emailcc1=1&resolution=---&emailtype1=exact&emailassigned_to1=1&query_format=advanced&email1=wikidata-bugs%40lists.wikimedia.org&list_id=176422>
.  Future The engineering management team continues to update the *
Deployments <http://wikitech.wikimedia.org/view/Deployments>* page weekly,
providing up-to-date information on the upcoming deployments to Wikimedia
sites, as well as the *engineering
roadmap<https://www.mediawiki.org/wiki/Roadmap>
*, listing ongoing and future Wikimedia engineering efforts.

-- 
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation
https://donate.wikimedia.org
-------------- next part --------------
_______________________________________________
Please note: all replies sent to this mailing list will be immediately directed to Wikimedia-l, the public mailing list of the Wikimedia community. For more information about Wikimedia-l:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
_______________________________________________
WikimediaAnnounce-l mailing list
WikimediaAnnounce-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimediaannounce-l

More information about the Wikimedia-l mailing list