[Wikimedia-l] [Wikimedia Announcements] Wikimedia engineering February 2013 report
Sumana Harihareswara
sumanah at wikimedia.org
Wed Mar 13 04:49:37 UTC 2013
Hi,
The report covering Wikimedia engineering activities (including upcoming
events, new hires, and open positions) in February 2013 is
now available. Thanks to Guillaume Paumier, Priyanka Nag, Tilman Bayer
and the engineers who helped put this together.
Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/February
Blog version:
https://blog.wikimedia.org/2013/03/12/engineering-february-2013-report/
We're also providing a shorter, simpler and translatable version of this
report that does not assume specialized technical knowledge:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/February/summary
Below is the full HTML text of the report, as previously requested.
As always, feedback is appreciated about the usefulness of the report
and its summary, and on how to improve them.
------------------------------------------------------------------
Major news in February include:
- The Wikipedia Zero
project<https://blog.wikimedia.org/2013/02/22/getting-wikipedia-to-the-people-who-need-it-most//>got
a Knight News Challenge grant.
- Additional input
methods<https://blog.wikimedia.org/2013/02/20/report-from-the-spring-2013-open-source-language-summit/>were
made available for jQuery.IME.
- The Translate extension introduced a new iteration of the Translation
Editor<https://blog.wikimedia.org/2013/02/15/inching-towards-enabling-our-improvements-to-the-translation-user-experience/>
.
- The Wikimedia mobile web team launched the ability to view or add
pages
to<https://blog.wikimedia.org/2013/02/13/follow-your-favorite-wikipedia-pages-on-the-mobile-web/>watchlist
— all from mobile devices.
- Echo is A new notification system for
Wikipedia<https://blog.wikimedia.org/2013/02/07/echo-a-new-notification-system-for-wikipedia/>
.
- The Technical Operations team found ways to stop
problems<https://blog.wikimedia.org/2013/02/05/how-the-technical-operations-team-stops-problems-in-their-tracks/>in
their tracks.
- Wikipedia Mobile hit 3 billion monthly page
views<https://blog.wikimedia.org/2013/02/01/wikipedia-mobile-hits-3-billion-monthly-page-views/>
.
*Note: We’re also providing a shorter, simpler and translatable version
of this
report<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2013/February/summary>that
does not assume specialized technical knowledge.
*
Upcoming events
There are many opportunities for you to get involved and contribute to
MediaWiki and technical activities to improve Wikimedia sites, both for
coders and contributors with other talents.
For a more complete and up-to-date list, check out the Project:Calendar.
Date Event Contact Mar 7 QA: General MediaWiki reports Bug
Triage<https://www.mediawiki.org/wiki/Bug_management/Triage/20130307>
AKlapper <https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29>,
Valeriej<https://www.mediawiki.org/wiki/User:Valeriej> Mar
13 Increase the
backlog<https://www.mediawiki.org/wiki/QA/Browser_testing/Increase_the_backlog>:
Given/When/Then explained, with examples from Search tests and suggestions
for more
Zeljko.filipin<https://www.mediawiki.org/wiki/User:Zeljko.filipin%28WMF%29>,
Qgil <https://www.mediawiki.org/wiki/User:Qgil>,
Cmcmahon<https://www.mediawiki.org/wiki/User:Cmcmahon%28WMF%29> Mar
18 QA: LiquidThreads (LQT) Bug
Triage<https://www.mediawiki.org/wiki/Bug_management/Triage/20130318>
AKlapper <https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29>,
Valeriej<https://www.mediawiki.org/wiki/User:Valeriej> Mar
19 Office hour about Wikimedia’s issue
tracker<https://www.mediawiki.org/wiki/Bugzilla>and Bug
management <https://www.mediawiki.org/wiki/Bug_management> in
#wikimedia-officeconnect<http://webchat.freenode.net/?channels=#wikimedia-office>
AKlapper <https://www.mediawiki.org/wiki/User:AKlapper_%28WMF%29> Mar
20 SMWCon
Spring 2013 <http://semantic-mediawiki.org/wiki/SMWCon_Spring_2013> (New
York City, USA)
Mar 22 LibrePlanet
<https://www.mediawiki.org/wiki/Events/LibrePlanet2013>(Cambridge, MA,
USA)
Mar 25 QA: Collaborate with Weekend Testers Americas: test new tools for
new users (E3)
Cmcmahon<https://www.mediawiki.org/wiki/User:Cmcmahon%28WMF%29>,
Qgil <https://www.mediawiki.org/wiki/User:Qgil>
Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us>
Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.
- Software Engineer – Editor
Engagement<http://hire.jobvite.com/Jobvite/Job.aspx?j=ovvXWfwD>
- Software Engineer –
Parser<http://hire.jobvite.com/Jobvite/Job.aspx?j=oIsbXfw2>
- Software Engineer –
Apps<http://hire.jobvite.com/Jobvite/Job.aspx?j=oqU0Wfw0>
- Software Engineer –
Mobile<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4cKWfwG>
- Software Engineer – Multimedia
Systems<http://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3>
- Software Engineer – Multimedia User
Interfaces<http://hire.jobvite.com/Jobvite/Job.aspx?j=ohqbXfwz>
- Software Engineer –
Search<http://hire.jobvite.com/Jobvite/Job.aspx?j=ogk1Wfwh>
- Product Manager –
Mobile<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGWJWfw1>
- Director of User
Experience<http://hire.jobvite.com/Jobvite/Job.aspx?j=otv0WfwE>
- Visual Designer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oomJWfw9>
- Partner Solutions
Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=o0FbXfwx>
- Dev-Ops Engineer
(SRE)<http://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf>
- Operations Engineer – Database
Administrator<http://hire.jobvite.com/Jobvite/Job.aspx?j=obMOWfwr>
- Senior Program Manager –
Mobile<http://hire.jobvite.com/Jobvite/Job.aspx?j=o1GbXfwz>
Announcements
- Ed Sanders joined the Features
engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>group
as Software Engineer working on Visual Editor (
announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066421.html>
).
- Christian Aistleitner joined as a contractor specializing in work on
Gerrit
(announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066465.html>
).
- Marc-Andre Pelletier joined the Technical Operations team as
Operations Engineer (contractor), focusing on the Wikimedia Labs
infrastructure and migration of tools
(announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066588.html>
).
- Kirsten Menger-Anderson joined the Features group as a part-time
contractor Technical Writer focusing on Editor Engagement Experiments.
- Greg Grossmeier joined the Platform
engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>group
as Release Manager (
announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066672.html>
).
- Site Performance Engineer and Senior Technical Advisor Patrick
Reilly’s last day with WMF was February 19th
(announcement<https://twitter.com/wikimediaatwork/status/304324931141914624>
).
Technical Operations
*Site infrastructure*
Both Asher Feldman and Peter Youngmeister are proceeding cautiously in
broadening MariaDB deployment in our clusters. We have one MariaDB instance
for each of the database clusters (s1 to s7). The MariaDB support team has
been quick in resolving bugs we encountered along the way. In another
database administration task, Asher reviewed and deployed the Wikidata
schema changes and migrated it from s3 cluster to s5, adding more growth
capacity.We put sixty new application servers into production in each of
the two datacenters. This is in anticipation of expected traffic growth
coming from both our regular and mobile sites in the coming year.Lately we
have been experiencing short time-out failures in the nightly search
indices built with search-pool4. Asher is experimenting with a fix. He
redistributed the search-pool4 indices in the Tampa data center based on
sizes and what seems to be a more acceptable index size-to-ram ratio. We
essentially have a virtual search-pool5 shard, but with the spelling and
highlight indices for pool4 and pool5 sharing the same servers. The pool4
wikis are using the new setup in Tampa, with everything else continuing to
use our Ashburn cluster. We should know soon if it works.The TechOps team
had a in-person team meeting the week of 25th February in WMF’s San
Francisco office.
- The highlights of the meeting were:
- Discuss the upkeep of the “failover” datacenter and capture the
lessons learned from the recent datacenter switchover. For example, we
think we could reduce the switchover “readonly” time from 32
minutes to 10
minutes by automating more of the database and caching failover
procedures.
- Improve and streamline our hiring process and our security access
model.
- Organize a coordinated sprints process to fill the gap between
smaller tasks (for which we use the RT ticketing system) and
larger tasks
(which require department coordination) by collecting some
thoughts. We
started brainstorming and forming teams via the Projects
wikitech page<https://wikitech.wikimedia.org/view/Projects>
.
- Face-to-face meetings with the Engineering teams from Platform,
Mobile, Analytics and Wikidata.
- Short TechOps sprints to reduce cronspams and our RT queue.
- Review budget needs for the 2013-2014 fiscal year.
*Data Dumps <https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps>*
Numerous bug fixes were made to the mwxml2sql tool, and a set of SQL files
bsed on an English language Wikipedia XML dump was published for use by
testers [1]
<http://dumps.wikimedia.org/other/experimental/enwiki/sqlfiles/>.
A tool to convert SQL dumps to escaped tab-delimited format is now
available for use with MySQL’s LOAD DATA INFILE command, much faster than
INSERTs. All SQL fles from the same dump were converted to this format and
also
published<http://dumps.wikimedia.org/other/experimental/enwiki/tabfiles/>
.A new mirror has come on line, initially mirroring historical archives of
XML files as well as MediaWiki releases, page view statistics and other
files
[2]<http://lists.wikimedia.org/pipermail/xmldatadumps-l/2013-February/000706.html>.
Thanks to Robert Smith and Wansecurity.com for providing the resources to
make this happen.
*Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs>*
This month was mostly spent stabilizing Labs components. Labs
Ganglia<http://ganglia.wmflabs.org/>was fixed to report instance
statistics properly. Adminbot was updated to
fix utf8 issues, and to fix package issues when upgrading. A number of
changes were made to the glusterfs support to bring more stability. Gluster
was upgraded to 3.3.1 to fix a memory leak on both the client and server.
Gluster isn’t matching our use case of multitenancy, as the glusterd daemon
isn’t handling the large number of volumes well. To help with this, until
we either fix the issue in gluster, or replace it, we’ve made a change to
not create/manage Gluster volumes for projects unless they opt in. We’ve
also disabled and deleted Gluster volumes for projects that are currently
unused. Work was done to turn Puppet classes for installing MediaWiki in
Labs into modules, so that they can be reused more easily.We merged
wikitech.wikimedia.org (our operations and infrastructure documentation)
and labsconsole.wikimedia.org together into wikitech.wikimedia.org.
wikitech-static.wikimedia.org is available as a backup, in case all access
to our cluster is unavailable. Work was started on supporting saltstack
reactors, to replace the bootstrapping for instance creation. This month we
have new member of the Labs team, Marc-Andre
Pelletier<https://www.mediawiki.org/wiki/User:MPelletier_%28WMF%29>,
also known in the community as Coren. Coren will be working on the new Tool
Labs infrastructure and we’re very excited to have him on-board. Asher and
Peter started work on replicated databases for Tool Labs during the last
week of the month.
Features
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>
Editor
retention: Editing tools
*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>*
In February, the team worked on improving the design, user interface
components and API infrastructure of VisualEditor, preparing for the new
features that will be added in the coming months. The objective is for
VisualEditor to be the default editor for all users, capable of letting
them edit the majority of content without needing to use the wikitext
editor, in July 2013. This will mean adding support for references, (at
least) basic templates, categories and images, each of which is a very
large piece of work. During this time, the team has expanded with the
recruitment of Ed
Sanders<https://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066421.html>,
who will focus on the data infrastructure of VisualEditor’s platform. The
alpha version of VisualEditor on mediawiki.org and the English Wikipedia
was updated twice
(1.21-wmf9<https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf9#VisualEditor>and
-wmf10 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf10#VisualEditor>),
adding support for Microsoft Internet Explorer (version 9 and above),
fixing a number of bugs reported by the community, improving
internationalisation, and restructuring the data model layer so that the
code interfaces are ready for the new features.
*Parsoid <https://www.mediawiki.org/wiki/Parsoid>*
The Parsoid team continued to improve support for non-English wikis. This
involved exposing more configuration information through the MediaWiki API
and using it throughout Parsoid. The support is now reasonably complete,
but needs testing. The round-trip testing framework needs to be adapted to
support running tests on pages from multiple wikis.
A new contributor, C. Scott Ananian, improved Parsoid’s performance by
switching the DOM library from JSDom <https://github.com/tmpvar/jsdom> to
Domino <https://github.com/fgnass/domino>. He also improved image handling
and contributed numerous other
patches<https://gerrit.wikimedia.org/r/#/q/owner:Cscott+status:merged,n,z>
.
The tokenizer was modified to parse one top-level block at a time, which
helps to spread out API requests and minimize the number of tokens in
flight. The serializer is in the process of being rewritten to work on DOM
input to benefit from the context provided by the DOM. This rewrite is
expected to simplify the logic significantly, and help fix some more
selective serialization issues that are blocking a deployment to production.
We also used the ops and core hackathon to discuss and refine our storage
plans<https://www.mediawiki.org/wiki/Parsoid/Roadmap#Performance:_Generate_and_store_HTML_DOM_on_edit>.
Finally, we wrote a blog post about Parsoid on the WMF tech
blog<http://blog.wikimedia.org/2013/03/04/parsoid-how-wikipedia-catches-up-with-the-web/>
.
Editor engagement features
*Notifications <https://www.mediawiki.org/wiki/Echo_%28Notifications%29>*
This month, we continued development on the Notifications project
(code-named *Echo*), which is now being tested on mediawiki.org. Ryan
Kaldari and Benny Situ developed new features such as
bundling<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Bundling>,
dismiss
<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Dismiss>and
web
preferences<https://www.mediawiki.org/wiki/Echo/Feature_requirements#Preferences>,
as well as refactored the code for the fly-out, archive page and email
notifications. Luke Welling continued to develop a more robust job
queue<https://www.mediawiki.org/wiki/Echo/Feature_requirements#JobQueue>.
Fabrice Florin spearheaded discussions about notifications for both new and
experienced users, and updated requirements for this first set
notifications<https://www.mediawiki.org/wiki/Echo/Feature_requirements#First_Notifications>for
our upcoming release. We will develop these notifications and final
features in coming weeks, and are aiming for a first release on the English
Wikipedia next month; in the meantime, you can help us test the current
version <https://www.mediawiki.org/wiki/Echo/Testing> on mediawiki.org. To
learn more, read this project
update<http://blog.wikimedia.org/2013/02/07/echo-a-new-notification-system-for-wikipedia/>on
the Wikimedia blog. If you are a software engineer, check out this job
opening <http://hire.jobvite.com/Jobvite/Job.aspx?j=ovvXWfwD&c=qSa9VfwQ> to
join our team and develop more editor engagement projects like Echo.
*Flow <https://www.mediawiki.org/wiki/Flow>*
In February, we analyzed and collated user research concerning talk pages.
Early designs were shown to members of the Board of Trustees to ask for
their input. Jeff Atwood (from StackOverflow and Discourse) came in to give
us a brain dump of his work. Design work was done on secondary “modules” as
examples for how existing workflows can be rebuilt within the Flow system.
Community engagement strategies saw the beginnings of implementation with
the creation of a “Portal” that will engage discussion about Flow at three
locations (mediawiki.org <https://www.mediawiki.org/wiki/FlowPortal>, meta,
and the English Wikipedia).
*Article feedback <https://www.mediawiki.org/wiki/Article_feedback>*
This month, our team completed feature development for Article
Feedback v5<https://www.mediawiki.org/wiki/Article_feedback/Version_5>(AFT5)
and prepared to release an updated version on English, French and
German Wikipedias. Developer Matthias Mullie developed a final set of new
features, including simpler moderation
tools<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Simpler_moderation_tools>,
better
filters<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Feedback_page_filters>,
a new feedback
link<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Feedback_link_on_article_pages>,
auto-archive<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Auto-archive_comments>and
discussion
on a talk
page<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Discuss_on_talk_page>.
Designer Pau Giner posted a usability study
report<https://meta.wikimedia.org/wiki/Research:Article_feedback/Moderation_Tools_Usability_Study>about
the effectiveness of the new moderation tools. Community liaison
Oliver Keyes contributed to this request for
comments<https://en.wikipedia.org/wiki/Wikipedia:Requests_for_comment/Article_feedback>,
which concluded with a request to remove AFT4 and provide an opt-in version
of AFT5 on the English Wikipedia (going forward, editors who wish to enable
AFT5 for articles they watch can simply add the special Category:Article
Feedback 5 <http://en.wikipedia.org/wiki/Category:Article_Feedback_5> on
those pages). Product manager Fabrice Florin collaborated with members of
the German
Wikipedia<https://en.wikipedia.org/wiki/de:Wikipedia:Artikel-Feedback>(which
is now testing the tool in its
ongoing
pilot<https://en.wikipedia.org/wiki/de:Spezial:Artikelr%C3%BCckmeldungen_v5>,
with a vote expected in May) and the French
Wikipedia<https://en.wikipedia.org/wiki/fr:Wikip%C3%A9dia:Prise_de_d%C3%A9cision/Mise_en_place_de_l%27outil_d%27%C3%A9valuation_des_articles_sur_la_Wikip%C3%A9dia_en_fran%C3%A7ais>(which
just voted to start its own six-month pilot). We plan to deploy
AFT5′s final version on these projects by the end of the month, as
described in this 2013 release
plan<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Release_Plan_2013>
.
Editor engagement experiments
*Editor engagement
experiments<https://www.mediawiki.org/wiki/Editor_engagement_experiments>
*
In February, the Editor Engagement Experiments team (“E3″) continued
working toward completing its goals for the quarter ending in March, which
includes updates on the following projects.
After the intial launch of guided
tours<https://www.mediawiki.org/wiki/Guided_tours>,
Matt Flaschen and other team members worked on A/B testing the
effectiveness of guided tours as part of the onboarding new
Wikipedians<https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians>experiences
currently enabled on English Wikipedia. Results from these
controlled tests are vital to understanding the impact of tours on editor
engagement. In the meantime, the GuidedTour
extension<https://www.mediawiki.org/wiki/Extension:GuidedTour>was
enabled on Wikimedia Commons and six Wikipedias (including French,
German, and Dutch), so that local administrators and volunteer developers
could take advantage of the feature.
In addition to working on polishing and quantifying the effect of guided
tours, significant progress was made on a new landing page for the
onboarding project, with plans to launch early in March. The new Getting
Started page will be expanded to include a wider variety task types offered
to new editors. It will also be generated from a basic recommender system
coupled with the GettingStarted
extension<https://www.mediawiki.org/wiki/Extension:GettingStarted>,
rather than relying on a bot.
Kirsten Menger-Anderson
<https://meta.wikimedia.org/wiki/User:Kmenger>joined the team as
Technical Writer mid-month. She began work with Ori
Livneh, Dario Taraborelli, and others on documenting the
EventLogging<https://www.mediawiki.org/wiki/EventLogging>extension,
with the goal of producing a comprehensive guide for end users
of EventLogging, especially other Wikimedia Engineering teams in need of
data. Future work by Kirsten will include similar documentation of the User
Metrics <https://www.mediawiki.org/wiki/UserMetrics> data analysis API,
which will be opened up for internal use in March.
Support
*2012 Wikimedia
fundraiser<https://www.mediawiki.org/wiki/2012_Wikimedia_fundraiser>
*
The majority of February was spent paying down the more glaring examples of
technical debt we acquired during the 2012 English fundraiser, before
jumping straight into a whole new round of International fundraising that
kicked off on February 27th at approximately 15:00 UTC (7am PST). Due to
unforeseen problems with one of our payment gateways, we were forced to
scrap our plans for a continuous international fundraising effort spanning
March through June, and will instead attempt to raise as much of the
remaining budget in March as we are able. All other plans have been
precluded by the March fundraising efforts.
Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering>
*Commons App <https://www.mediawiki.org/wiki/Apps/Commons>*
February saw numerous beta releases of the Commons iOS and Android app. The
Android app was
published<https://play.google.com/store/apps/details?id=org.wikimedia.commons>to
Google’s Play store with a Beta label. Brion Vibber announced for
testers to get envolved in http://tflig.ht/Zl9Ef7. A significant amount of
time was spent on the visual polish, bug fixing, and internationalization
in prep for a very active series of betas. Extensive work was also done to
log specific user actions using E3′s EventLogging setup to help us make
data driven decisions in the future.
*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>*
During February, we launched with a new partner, Orange Botswana. We’ve
also begun testing with Vimpelcom for an upcoming launch in March. In
addition, we’ve made improvements to the partner dashboard which tracks
Wikipedia Zero usage.
*Open Street Map <https://www.mediawiki.org/wiki/OpenStreetMap>*
Tomasz Finc, Max Semenik and Arthur Richards worked on organising the OSM
hackathon<https://www.mediawiki.org/wiki/Events/Wikimedia_Mapping_Event_2013>which
took place on March 9-10 in Copenhagen. Max Semenik continued working
on OSM in Labs and uploaded initial versions of several OSM-related
packages to Gerrit for Operations to review.
*Mobile Web Photo
Upload<https://www.mediawiki.org/wiki/Mobile_design/Uploads>
*
In February we released the ability to upload and add images to articles
lacking them to the full mobile web. We continued work on improving the
upload and mobile file page views, for full productization in early March.
We also explored two new upload workflows in alpha/beta: adding a call to
action to articles that appear in the Nearby and Watchlist view, allowing
users to quickly see articles near them that may need an image, take a
photo and upload. Lastly, we began collaborating with the Fundraising team
to enable CentralNotice on the mobile web, giving us the ability to deliver
targeted banners to users who might be interested in trying out our new
upload features or Commons apps.
Platform
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>
MediaWiki
Core
*MediaWiki 1.21 <https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap>*
Deployments of 1.21wmf9 and 1.21wmf10 went to production as scheduled with
mininal issue.
*Git conversion <https://www.mediawiki.org/wiki/Git/Conversion>*
Gerrit was upgraded this month to a pre-2.6 snapshot. This enabled the use
of plugins, as well as brought numerous bugfixes and UI improvements. Work
is underway on a plugin to provide Bugzilla integration and to replace
Gitweb with a better repository viewer called Gitblit. All of our git
repositories are now automatically replicated to
GitHub<https://github.com/wikimedia>.
We’ve begun some initial planning into how we can improve the “new
repository request” process, making it much easier for users with a quicker
turnaround time.
*TimedMediaHandler <https://www.mediawiki.org/wiki/TimedMediaHandler>*
Jan Gerber continues to work part-time for the WMF to fix multimedia bugs.
Fixes include better support for FLAC files (bug
43249<https://bugzilla.wikimedia.org/show_bug.cgi?id=43249>)
and better support for metadata display in small embedded players (bug
44272<https://bugzilla.wikimedia.org/show_bug.cgi?id=44272>
).
*Wikidata deployment <https://www.mediawiki.org/wiki/Wikidata_deployment>*
Wikidata has been deployed to Hungarian, Italian, Hebrew, and English
Wikipedias!
*Media storage <https://www.mediawiki.org/wiki/Media_storage>*
Nearly all files have been copied from Swift (in Tampa) to Ceph (in
Ashburn). Further scripts will be run to synchronize the Ceph files to
account for deletions and updates to files. The Varnish configuration to
handle URL rewriting (to take the place of rewrite.py in swift) is already
coded, though not yet in use.
*Lua scripting <https://www.mediawiki.org/wiki/Lua_scripting>*
We deployed Lua/Scribunto to several wikis, including English Wikipedia, on
February 18th. The current plan is to deploy to the remaining wikis on
March
13th<https://blog.wikimedia.org/2013/03/11/lua-templates-faster-more-flexible-pages/>
.
*Site performance <https://www.mediawiki.org/wiki/Site_performance>*
A patch to allow moving the job queue to another DB cluster has been
merged, and another patch to support an alternative Redis-based queue is in
review in gerrit. Currently, job-related operations consume a significant
portion of production database master wall time.
*Admin tools
development<https://www.mediawiki.org/wiki/Admin_tools_development>
*
The team worked on a number of areas this month. The interface for Stewards
to mass-lock user
accounts<https://www.mediawiki.org/wiki/Admin_tools_development/CentralAuth_Locking>was
completed and will be deployed very soon next month. The support for
global AbuseFilters nears completion, with a test deployment to
test.wikipedia.org and mediawiki.org; once internationalisation is more
complete, it will be deployed for all wikis. The team worked to agree a
specification<https://www.mediawiki.org/wiki/Admin_tools_development/Roadmap/Global_CheckUser>for
a global CheckUser tool. Progress was made on a global
account renaming
tool<https://www.mediawiki.org/wiki/Admin_tools_development/Global_Rename>and
XFF-based global and local blocks. The team also worked on finalising
the migration to Single User Login, building some
metrics<https://www.mediawiki.org/wiki/Admin_tools_development/SUL_Audit#Metrics>to
ascertain a sense of the problem.
*REST proposal <https://www.mediawiki.org/wiki/API/REST_proposal>*
Yuri <https://www.mediawiki.org/wiki/User:Yurik> started gathering
requirements for the RESTful
(content-oriented<https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap#Content_API>)
API as part of the overall planning for the API v2.0
roadmap<https://www.mediawiki.org/wiki/Requests_for_comment/API_roadmap>.
Also, Wikia’s focused R&D sprint has led to remove all the remaining
obstacles identified previously and Wikia has identified a new, larger
dedicated Product team to get to a final implementation of the REST API
following the directions set by the existing prototype and internal RFC;
the team (API/Data) is the same one in charge of Search and all the related
API’s, this will ensure a better integration of this product into the new
API strategy. At the moment of writing the knowledge transfer required for
the team to start this new phase has just begun and the team has to first
to complete the current work on another project already in progress. In the
meantime, Wikia will make an RFC public for review and feedback.
*Security auditing and
response<https://www.mediawiki.org/wiki/Security_auditing_and_response>
*
Continued responses to reported vulnerabilities. Preparation for security
releases for 1.19 and 1.20 branches of MediaWiki. Continued review of
Fundraising.
Quality assurance
*Quality Assurance <https://www.mediawiki.org/wiki/QA>*
Concluded mobile app file upload test exercise. Preparing for new test
exercise, possibly for Search. Exploring another collaboration with Weekend
Testers.
*Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>*
We are adding search to the beta cluster following Mobile Frontend tweaks.
We are discussing new ways to use the beta cluster as a result of our San
Francisco gathering in February.
*Continuous
integration<https://www.mediawiki.org/wiki/Continuous_integration>
*
Antoine Musso worked with several MediaWiki extension authors to ensure
that the unit tests for those extensions are run by Jenkins and that they
work. He hopes to have all extensions that run on the Wikimedia production
cluster fully operational by the end of February. Antoine also integrated
PHP CodeSniffer into our automated test runs.
*Browser testing <https://www.mediawiki.org/wiki/QA/Browser_testing>*
Added E3 tests. Preparing for test event to increase the backlog.
Sophisticated tests for Language need tweaking/research.
Analytics
*Kraken (Analytics Cluster)<https://www.mediawiki.org/wiki/Analytics/Kraken>
*
We did two reviews of Kraken: one for security and one for overall
architecture. We’re incorporating the feedback, which includes merging our
puppet modules into the operations puppet repository and the test puppet in
Labs. Work has started to create dashboards for mobile pageviews, Wikipedia
Zero and the mobile alpha and beta sites.
*Limn <https://www.mediawiki.org/wiki/Analytics/Limn>*
Highlights in the past month include
- Stacked charts
- Debianization & Puppetization
- New E3 and Grantmaking dashboards
- Ad-hoc visualization of datasource
*Wikistats <https://www.mediawiki.org/wiki/Wikistats>*
New mobile pageview report is in testing phase but not ready to ship.
*Page view
logging<https://www.mediawiki.org/wiki/Analytics/Pageview_logging>
*
It was a quiet month for the logging infrastructure; things were running
fine. We have been working on a patch to fix bug
45178<https://bugzilla.wikimedia.org/show_bug.cgi?id=45178>,
which we will try to deploy in March.
Engineering community team
*Bug management <https://www.mediawiki.org/wiki/Bug_management>*
As part of the Weekly QA
Goals<https://www.mediawiki.org/wiki/QA/Weekly_goals>,
a Git/Gerrit Bug Triage
day<https://www.mediawiki.org/wiki/Bug_management/Triage/20130219>took
place. About 25 open reports were retested and/or synchronized with
their status in the upstream
bugtracker<https://code.google.com/p/gerrit/issues/>.
The Bug day format will be developed further to make it more attractive to
new contributors.
Valerie published an initial version of a Bug Life Cycle
flowchart<https://commons.wikimedia.org/wiki/File:Bug_Life_Cycle_Diagram.svg>describing
the life of a bug report by its status changes over time,
continued investigating feedback channels and workflows of other bigger
free software projects, and also helped testing the Commons Upload app for
Android and the mobile browser as part of Mobile QA
testing<https://www.mediawiki.org/wiki/Mobile_QA/Commons_uploads>.
A table on Bugzilla use by development
teams<https://www.mediawiki.org/wiki/Bug_management/Development_teams_usage>was
made available.
Furthermore, reachout to several development teams continued to better
understand the different bug management needs, and discussions took place
about a workflow how to mark fixed tickets as backport candidates in the
issue tracker, potentially resulting in the addition of a dropdown menu (“
flag <http://www.bugzilla.org/docs/4.2/en/html/flags-overview.html>“) in
Bugzilla.
*Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>*
The Outreach Program for
Women<https://www.mediawiki.org/wiki/Outreach_Program_for_Women>is
more than half-way. Our six participants are fairly on track; read
February reports from Valerie <http://valmj.wordpress.com/>,
Mariya<http://mitevam.tumblr.com/post/44134682053/february-in-short>,
Priyanka<http://priyankaivy.blogspot.in/2013/02/learning-to-build-gadgets-in-wikipedia.html>and
Sucheta <http://mikipedian.blogspot.in/>. Teresa is working on unit tests
for the Git repository extension and is looking at a request to use this
extension to help to maintain CentralNotice-related content. Isarra
completed her work on Flow/User
tests<https://www.mediawiki.org/wiki/Flow/User_tests>and now is
working with the Editor Engagement team on improvements to the
Watchlist design. Google published the
timeline<https://www.google-melange.com/gsoc/events/google/gsoc2013>for
the Summer
of Code 2013 <https://www.mediawiki.org/wiki/Summer_of_Code_2013> and we
have confirmed our intention to apply as organization. Without big
announcements and more than a month before any deadline, we have already 15
students, 5 mentors and 2 org admins potentially interested.
*Technical
communications<https://www.mediawiki.org/wiki/Technical_communications>
*
Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom> finished
setting up the
Project:Calendar<https://www.mediawiki.org/wiki/Project:Calendar>,
used it to replace content on pages like QA/Weekly
goals<https://www.mediawiki.org/wiki/QA/Weekly_goals>and
Meetings <https://www.mediawiki.org/wiki/Meetings> using selective
transclusion, created an edit
notice<https://www.mediawiki.org/wiki/MediaWiki:Editnotice-4-Calendar>to
make it easier to add and edit events, added a bullet
list display
option<https://www.mediawiki.org/wiki/Template_talk:New_opportunities/Content>,
and added links to icons
credits<https://www.mediawiki.org/wiki/Project:Visual_identity>as part
of an effort to harmonize visual identity for MediaWiki. He updated
the monthly report
how-to<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/How_to>to
reflect the current process, and met with LCA
staff <https://meta.wikimedia.org/wiki/Legal_and_Community_Advocacy> to
discuss possible collaboration between the tech
ambassadors<https://meta.wikimedia.org/wiki/Tech/Ambassadors>and
community advocates programs. While in San Francisco, he met with many
colleagues to discuss engineering project documentation, and ways to
announce to and engage with the rest of the community. Last, he started to
create a Product
development<https://www.mediawiki.org/wiki/Product_development>hub to
facilitate the involvement of contributors, and supported the
engineering team in communicating about their accomplishments on the
Wikimedia Tech blog.
*Volunteer coordination and
outreach<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>
*
- We have consolidated the QA Weekly
goals<https://www.mediawiki.org/wiki/QA/Weekly_goals>as a way to
orchestrate testing and bug management activities with the
wider community. We run two Features
testing<https://www.mediawiki.org/wiki/QA/Features_testing>activities
(Article
Feedback’s new
features<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Features_under_consideration>and
Wikipedia
+ Commons
uploads<https://www.mediawiki.org/wiki/Mobile_QA/Commons_uploads>)
and two Bug Days (Article feedback and
Git/Gerrit<https://www.mediawiki.org/wiki/Bug_management/Triage/20130219>).
So far it has been useful to coordinate better testing activities across
Wikimedia Foundation teams, but we still need better results engaging
volunteers.
- The (newly elected) Affiliations Committee is working on finding an
agreement with MediaWiki Group
Ahmedabad<https://www.mediawiki.org/wiki/Groups/Proposals/Ahmedabad>and
Wikimedia India regarding whether it should be a chapter Special
Interest Groups, a user group, or some other structure.
- Quim Gil went to FOSDEM; as a result, we have now a generic “How to
contribute” presentation and
video<https://www.mediawiki.org/wiki/How_to_contribute/Presentation>
.
- We are helping the organization of the Amsterdam Hackathon
2013<https://www.mediawiki.org/wiki/Amsterdam_Hackathon_2013>and also
helped the Wikimedia Foundation decide what employees would get
travel sponsorship to Amsterdam.
Language
engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering>
*Language tools <https://www.mediawiki.org/wiki/Language_tools>*
- Translate (TUX) enhancements: Development continues full steam ahead
on the new translation editor with proof reading feature by Santhosh and
Amir. Niklas continues to enhance and test backend translate
infrastructure, including Solr integration and other translation aids.
- Plurals support: by Santhosh and Amir to be more consistent with CLDR
standards.
- Technical Font Specification for Indic scripts: We kicked off this
collaborative project between Red Hat and Wikimedia at the Language
Summit<https://www.mediawiki.org/wiki/Pune_LanguageSummit_February_2013/Event_Report>in
February. Santhosh and Runa are contributors to this project.
- Language Coverage
Matrix<https://docs.google.com/a/wikimedia.org/spreadsheet/ccc?key=0Asnprhp63DAtdEQyTnMwV29yVFZFMmlwOXoxQU9SMFE#gid=0>:
This matrix aims to provide an up-to-date status of language support for
all tools that the team is developing and maintaining.
- Mediawiki i18n code review: Team continues to support Mediawiki
release with i18n code reviews across other features and extensions.
- Mediawiki Language Extension Bundle (MLEB): Monthly release of MLEB
completed<https://translatewiki.net/mleb/MediaWikiLanguageExtensionBundle-2013.02.tar.bz2>with
release notes by Amir.
*Milkshake <https://www.mediawiki.org/wiki/Milkshake>*
- jQuery.IME: Continue to merge input methods contributed into
jQuery.IME. We now have 155+ input methods for 75+ languages.
- jQuery.ULS: Continue to maintain jQuery.ULS. Awaiting resolution of
deployment issues.
Kiwix <http://www.kiwix.org>
*The Kiwix project is funded and executed by Wikimedia
CH<https://meta.wikimedia.org/wiki/Wikimedia_CH>
.*
We have migrated our source code repository from Subversion to
Git<http://code.kiwix.org>.
We have have also focused in February on the revamping of the Kiwix
Web site<http://www.kiwix.org>.
The new Web site is really more user friendly. Audience continues to grow
with 120,000 downloads of the software in February.
Wikidata <https://meta.wikimedia.org/wiki/Wikidata>
*The Wikidata project is funded and executed by Wikimedia
Deutschland<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>
.*
In February the first phase of Wikidata (language links) was deployed on
the English-language Wikipedia. Additionally the first parts of phase 2
(infoboxes) went life on wikidata.org. It is now possible to add
statements. For an example see d:159 <https://www.wikidata.org/wiki/159>.
The first tools have already been written on top of this, for example
Geneawiki <http://toolserver.org/%7Emagnus/ts2/geneawiki/?q=Q1339> and
Reasonator <http://toolserver.org/%7Emagnus/ts2/reasonator/?q=Q1339>. In
the meantime more work has been put into additional data-types, like
strings and geocoordinates, as well as the foundations of phase 3 (lists
based on queries).
In other good news: Wikimedia Germany has decided to fund Wikidata
development
<http://blog.wikimedia.de/2013/02/20/the-future-of-wikidata/>after
the end of the first year of development at the end of March.
Future
The engineering management team continues to update the
*Deployments<https://wikitech.wikimedia.org/wiki/Deployments>
* page weekly, providing up-to-date information on the upcoming deployments
to Wikimedia sites, as well as the *engineering
roadmap<https://www.mediawiki.org/wiki/Roadmap>
*, listing ongoing and future Wikimedia engineering efforts.
--
Sumana Harihareswara
Engineering Community Manager
Wikimedia Foundation
_______________________________________________
Please note: all replies sent to this mailing list will be immediately directed to Wikimedia-l, the public mailing list of the Wikimedia community. For more information about Wikimedia-l:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
_______________________________________________
WikimediaAnnounce-l mailing list
WikimediaAnnounce-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimediaannounce-l
More information about the Wikimedia-l
mailing list