[Wikimedia-l] [Wikimedia Announcements] Wikimedia engineering December 2012 report
Guillaume Paumier
gpaumier at wikimedia.org
Thu Jan 10 19:08:19 UTC 2013
Hi,
The report covering Wikimedia engineering activities in December 2012 is
now available.
Wiki version:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/December
Blog version:
https://blog.wikimedia.org/2013/01/10/engineering-december-2012-report/
We're also proposing a shorter, simpler and translatable version of this
report that does not assume specialized technical knowledge:
https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/December/summary
Below is the full HTML text of the report, as previously requested.
As always, feedback is appreciated about the usefulness of the report and
its summary, and on how to improve them.
------------------------------------------------------------------------
Major news in December include:
- The launch of an alpha, opt-in version of the
VisualEditor<https://blog.wikimedia.org/2012/12/12/try-out-the-alpha-version-of-the-visualeditor/>to
the English Wikipedia, a project more
complex than it
appears<https://blog.wikimedia.org/2012/12/07/inventing-as-we-go-building-a-visual-editor-for-mediawiki/>
;
- A research
study<https://blog.wikimedia.org/2012/12/20/article-feedback-new-research-and-next-steps/>on
the use of the Article Feedback feature;
- New metrics for the MediaWiki
community<https://blog.wikimedia.org/2012/12/10/introducing-mediawiki-community-metrics/>
;
- The start of the Outreach Program for
Women<https://blog.wikimedia.org/2012/12/11/welcome-to-floss-outreach-program-for-women-interns/>
;
- Continued work to improve the
workflow<https://blog.wikimedia.org/2012/12/12/translation-interface-makeover-in-progress/>and
interface<https://blog.wikimedia.org/2012/12/31/translation-editor-growing-snazzier/>for
translators.
*Note: We're also proposing a shorter, simpler and translatable version of
this report<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/December/summary>that
does not assume specialized technical knowledge.
*
Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us>
Are you looking to work for Wikimedia? We have a lot of hiring coming up,
and we really love talking to active community members about these roles.
- Software Engineer - Visual
Editor<http://hire.jobvite.com/Jobvite/Job.aspx?j=otYPWfwW>
- Software Engineer - Editor
Engagement<http://hire.jobvite.com/Jobvite/Job.aspx?j=ovvXWfwD>
- Software Engineer
(Partners)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oX2hWfwW>
- Software Engineer
(Apps)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oqU0Wfw0>
- Software Developer General
(Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4cKWfwG>
- Git and Gerrit software development
(Contract)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4gIWfwI>
- Release Manager <http://hire.jobvite.com/Jobvite/Job.aspx?j=oZrQWfwW>
- Software Engineer -
Multimedia<http://hire.jobvite.com/Jobvite/Job.aspx?j=oj40Wfw3>
- Software Engineer
(Search)<http://hire.jobvite.com/Jobvite/Job.aspx?j=ogk1Wfwh>
- Product Manager
(Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGWJWfw1>
- Director of User
Experience<http://hire.jobvite.com/Jobvite/Job.aspx?j=otv0WfwE>
- Visual Designer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oomJWfw9>
- Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf>
- Operations Engineer/Database
Administrator<http://hire.jobvite.com/Jobvite/Job.aspx?j=obMOWfwr>
- Site Reliability
Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=o7k2Wfw9>
Announcements
- Matthew Flaschen joined the Wikimedia Features
engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>team
as Features Engineer (
announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-December/064916.html>
).
- Mike Wang joined the Operations team as part time Labs Ops Engineer
(consultant)
(announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-December/064768.html>
).
Technical Operations
*Production Site
Switchover<http://wikitech.wikimedia.org/view/Eqiad_Migration_Planning>
*
The Technical Operations team continued to work on completing the
outstanding migration tasks, and to ready our Ashburn infrastructure for
the big switchover day, i.e., the complete transition from the Tampa
datacenter to the one in Ashburn, on the week of January 22, 2013.In the
past few months, we've transitioned services from the Tampa datacenter to
the one in Ashburn, which now serves most of our traffic (about 90%).
However, application (MediaWiki), memcached and database systems are all
still running exclusively out of Tampa. We have been working to upgrade the
technologies and set up those systems at Ashburn, and we plan to perform
the switchover of those services from Tampa to Ashburn in the coming weeks.
This will provide us some assurance of a hot standby datacenter, should we
encounter an irrecoverable and lengthy outage in one of the main
datacenters.
*Site Infrastructure*
Because December is when the annual Wikimedia fundraiser happens, the
Operations team usually makes fewer site infrastructure changes to mitigate
the risks of causing outages. Some of the lesser-risk work performed
include deploying the new Parsoid cluster to support the Visual Editor
project, rolling out doc.wikimedia.org (our auto-generated puppet
documentation), using a new and unified SSL certificate for *wikipedia.organd *.
m.wikipedia.org sites, and setting up a monitoring server and service in
Ashburn.Asher Feldman migrated one of the main production slave database
server (db59) for the English Wikipedia (enwiki) to MariaDB 5.5.28. He has
been testing 5.5.27 on the primary research slave, and on the current build
on a slave in Ashburn. Taking the times of 100% of all queries over regular
sample windows, the average query time across all enwiki slave queries is
about 8% faster with MariaDB compared to our production build of MySQL
5.1-fb. Some queries types are 10–15% faster, some are 3% slower, and
nothing looks aberrant beyond those bounds. Overall throughput as measured
by qps has generally been improved by 2–10%. Asher wouldn't draw any
conclusions from this data yet: more testing is needed to filter out noise,
but initial results are positive. The main reason for migrating to MariaDB
is not performance, but rather by the belief that it's in the Wikimedia
Foundation's and the open-source communities' interest to coalesce around
the MariaDB Foundation as the best route to ensuring a truly open and
well-supported future for MySQL-derived database technology.Mark Bergsma
and Faidon Liambotis have made tremendous progress in testing and deploying
Ceph in Ashburn. We are hopeful it will be robust and scalable.Ryan Lane
has been writing a new deployment system using git and Saltstack. Parsoid
is currently being deployed with this system, and MediaWiki is slated to
use it for its next major deployment.
*Fundraising*
There were no major changes on the fundraising infrastructure because of
the fundraiser itself. We ordered and received bastion hosts that we're in
the process of deploying. Monitoring got an overhaul and we're now sending
alerts to the fundraising technical staff or the technical operations team
depending on what triggered the alert.
*Data Dumps <https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps>*
A tool for dump users to set up interwiki links on their local mirrors
is available
in alpha<https://www.mediawiki.org/wiki/Interwiki_cache#Setting_this_up_on_your_own_wiki>,
as well as documentation of the interwiki cdb file. Also, work with
WanSecurity on mirroring is moving forward: they now hold a current copy of
all 'other' files, including page views and Picture of the Year bundles,
among other things.
*Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs>*
Labs came out of beta this month, following the opening of
self-registration. Another major change this month was the migration from
the shared NFS instance to per-project glusterfs volumes. A number of
smaller changes were made, including: the Addition of puppet documentation
links from classes and variables on the instance configuration pages; the
modification of the project filter to act as a table of contents; a split
of LDAP project groups into projects and POSIX groups; and the installation
of Saltstack on all instances to act as a guest agent. Features
Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering>
Editor
retention: Editing tools
*VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>*
[edit<https://www.mediawiki.org/w/index.php?title=VisualEditor/status&action=edit>
]
In December, the team deployed to the English
Wikipedia<https://blog.wikimedia.org/2012/12/12/try-out-the-alpha-version-of-the-visualeditor/>an
alpha version of the VisualEditor for editors to use and give feedback
on issues and priorities. The team's work focussed on ensuring that the
integration was reliable, and providing a dedicated tool for editors to
report problems with editing, and, after deployment, addressing the reports
and ideas from editors. The early version of the VisualEditor on
mediawiki.org was also updated to use the new developments (as part of
1.21-wmf6 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf6#VisualEditor>
).
*Parsoid <https://www.mediawiki.org/wiki/Parsoid>*
[edit<https://www.mediawiki.org/w/index.php?title=Parsoid/status&action=edit>
]
The Parsoid <https://www.mediawiki.org/wiki/Parsoid> project reached a
major milestone with its first deployment to the English Wikipedia along
with the VisualEditor. This was a major test for Parsoid, as it needed to
handle the full range of arbitrary and complex existing wiki content
including templates, tables and extensions for the first time.
As witnessed by the clean edit
diffs<https://en.wikipedia.org/w/index.php?namespace=&tagfilter=visualeditor&title=Special%3ARecentChanges>,
Parsoid passed this test with flying colors. This represents very hard work
by the team (Gabriel Wicke, Subramanya Sastry and Mark Holmquist) on automated
round-trip testing <http://parsoid.wmflabs.org:8001/> and the completion of
a selective serialization strategy just in time for the release.
After catching their breath, the team now has its sights on the next phase
in Parsoid development. This includes a longer-term strategy for the
integration of Parsoid and HTML DOM into MediaWiki, performance
improvements and better support for complex features of wikitext.
Editor engagement features
*Notifications <https://www.mediawiki.org/wiki/Echo_%28Notifications%29>* [
edit<https://www.mediawiki.org/w/index.php?title=Echo_%28Notifications%29/status&action=edit>
]
This month, the team continued to develop key features of the Notifications
project (code-named 'Echo'), and deployed a first experimental release on
mediawiki.org. Fabrice Florin expanded feature
requirements<https://www.mediawiki.org/wiki/Echo/Feature_requirements>for
this release, and Vibha Bamba designed more components of the user
experience <https://www.mediawiki.org/wiki/Echo_User_Experience>. Ryan
Kaldari and Benny Situ developed improved notification flyouts and email
digests, as well as new notifications such as page links. Luke Welling
built an HTML email module, which will soon be available to other projects
as well. We plan to develop more features this month and deploy them for
new editors on the English Wikipedia in early 2013. Please help us
test<https://www.mediawiki.org/wiki/Echo/Testing>these new features to
provide feedback and find bugs. We're also looking to hire
a software engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ovvXWfwD&c=qSa9VfwQ>as
part of this project.
*Article feedback <https://www.mediawiki.org/wiki/Article_feedback>*
[edit<https://www.mediawiki.org/w/index.php?title=Article_feedback/status&action=edit>
]
We made good progress<https://blog.wikimedia.org/2012/12/20/article-feedback-new-research-and-next-steps/>on
Article
Feedback version
5<https://www.mediawiki.org/wiki/Article_feedback/Version_5>this
month. We completed a research
study <https://commons.wikimedia.org/wiki/File:AFT5_2012-Q4_report.pdf> on
the English Wikipedia, confirming that many readers use this feature and a
sizable number of them go on to register and become editors. Based on that
research and editor suggestions, we started development on new
features<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requirements#Features_under_consideration>to
reduce the editor workload through better filters and simpler
moderation
tools. We also continued to refactor our code, to support millions of
comments on a dedicated database cluster to be deployed in coming months.
Once this work is complete, we plan to release Article Feedback v5 to 100%
of the English Wikipedia in March, and to other Wikimedia sites later this
year. The German Wikipedia has already started a
pilot<https://en.wikipedia.org/wiki/de:Spezial:Artikelr%C3%BCckmeldungen_v5>to
evaluate this tool, and a similar initiative is also under
discussion<https://en.wikipedia.org/wiki/fr:Discussion_Wikip%C3%A9dia:Prise_de_d%C3%A9cision/Mise_en_place_de_l%27outil_d%27%C3%A9valuation_des_articles_sur_la_Wikip%C3%A9dia_en_fran%C3%A7ais>on
the French Wikipedia.
*Page Curation <https://www.mediawiki.org/wiki/Page_Curation>*
[edit<https://www.mediawiki.org/w/index.php?title=Page_Curation/status&action=edit>
]
Page Curation <https://en.wikipedia.org/wiki/Wikipedia:Page_Curation> is
now in 'maintenance mode', following its
release<https://en.wikipedia.org/wiki/en:Special:NewPagesFeed>on the
English Wikipedia in September 2012. There was no significant
development activity on this project this month. Oliver Keyes has completed
a project to look at various ways of localizing Page Curation to any and
all wikis that want it: it is currently being reviewed by Howie Fung to
assess its feasibility.
Editor engagement experiments
*Editor engagement
experiments<https://www.mediawiki.org/wiki/Editor_engagement_experiments>
* [edit<https://www.mediawiki.org/w/index.php?title=Editor_engagement_experiments/status&action=edit>
]
In December, the Editor Engagement Experiments team launched a new test
aimed at Onboarding new
Wikipedians<https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians>.
This interface delivers an optimized task list immediately after sign up,
inviting those without an idea of how to get started to choose an article
and try their hand at editing. The related GettingStarted
extension<https://www.mediawiki.org/wiki/Extension:GettingStarted>was
deployed mid-month and continued to evolve throughout the month, as
early quantitative and qualitative research was conducted.
To go along with the launch of GettingStarted and other experimentation,
EventLogging <https://www.mediawiki.org/wiki/Extension:EventLogging>underwent
heavy development, including the launch of a new Schema namespace
on Meta for defining the data collected in a public, collaborative manner.
We created production schemas for
GettingStarted<https://meta.wikimedia.org/wiki/Schema:GettingStarted>,
account creation <https://meta.wikimedia.org/wiki/Schema:AccountCreation>,
mobile, and more<https://meta.wikimedia.org/w/index.php?title=Special%3AAllPages&from=&to=&namespace=470>.
Ori Livneh also reworked the format, transmission, and cleanliness of data
delivered to analysts and product managers, automatically generating
database tables from these schemas for incoming events.
Late in the month, the team collaborated with fundraising to reach out to
donors and readers<https://meta.wikimedia.org/wiki/Research:Donor_engagement>as
part of the annual fundraising campaign via email and a "Thank You"
banner which ran at the end of the year. In addition to introducing
millions of donors and readers to the Wikipedia editor community and
inviting them to join, this campaign helped the team establish an
experimental baseline for what a campaign to convert readers might look
like.
In addition to the above launches, we continued development of the new
account creation experience and Guided
Tours<https://www.mediawiki.org/wiki/Extension:GuidedTour>by Matt
Flaschen, which will be launched in January 2013. Active
development was also begun by Ryan Faulkner and Dario Taraborelli on a user
metrics API <https://meta.wikimedia.org/wiki/Research:Metrics>. The effort
is threefold: to standardize user metrics in data analysis, to build
infrastructure to efficiently compute metrics for a large set of users, and
finally to expose those results via an API.
Support
*2012 Wikimedia
fundraiser<https://www.mediawiki.org/wiki/2012_Wikimedia_fundraiser>
* [edit<https://www.mediawiki.org/w/index.php?title=2012_Wikimedia_fundraiser/status&action=edit>
]
The 2012 annual fundraiser continued in December and was a resounding
success. In addition to the ongoing maintenance required to operate the
fundraiser, the team helped to execute the Thank You
campaign<https://wikimediafoundation.org/wiki/Thank_You_Main>and
started to put into place new tools for auditing the fundraiser after
its completion.
Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering> The
Mobile development and design team worked to finalize contributory and
other experimental editor-focused features on the Beta site (uploads,
editing, and watchlist functionality) in order to clear the way for a full
push on mobile uploads by March 2013. We also worked to improve the reader
and potential editor experience by introducing features geared toward
educating/engaging our users, such as a human-readable last modified
timestamp for articles and watchlist, and thumbnail images to illustrate
the watchlist view. Lastly, because of the huge interest we generated in
our Beta testing site, we created an Alpha site to house very early work on
contributory features, in order not to disrupt the reading experience of
our 100,000+ Beta users.
*GeoData Storage & API<https://www.mediawiki.org/wiki/GeoData_Storage_%26_API>
* [edit<https://www.mediawiki.org/w/index.php?title=GeoData_Storage_%26_API/status&action=edit>
]
During December Max Semenik
<https://www.mediawiki.org/wiki/User:MaxSem>continued work on GeoData,
the extension directly responsible for allowing
us to easily store and retrieve GPS coordinates in our databases. Max
migrated the extension from implementation, to code review, and finally
deployment to the English Wikipedia. It will become 100% production-quality
after a few more tweaks and fixes. After those changes, we'll continue to
roll out to the rest of the wikis. The extension is one of the precursors
to having the "near by" feature on our mobile web site.
*Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>*
[edit<https://www.mediawiki.org/w/index.php?title=Wikipedia_Zero/status&action=edit>
]
During the month of December, Patrick Reilly, Dan Foy and the rest of the
Zero team launched Wikipedia Zero with a new partner, Orange Congo. They
resolved operational issues that prevented the team from accurately
recording traffic from the Opera browser. They also helped on-board Brion
Vibber to help in the interim while the team continues to look for
permanent members. The team is very excited about its upcoming
launches<http://blog.wikimedia.org/2012/12/20/the-countries-in-which-mobile-matters-most/>and
will be announcing them as soon as possible.
*J2ME App <https://www.mediawiki.org/wiki/MobileFrontend/J2ME_app>*
[edit<https://www.mediawiki.org/w/index.php?title=MobileFrontend/J2ME_app/status&action=edit>
]
The J2ME app is ready to launch pending contractual negotiations with
carriers.
*Wikipedia over SMS &
USSD<https://www.mediawiki.org/wiki/Wikipedia_over_SMS_%26_USSD>
* [edit<https://www.mediawiki.org/w/index.php?title=Wikipedia_over_SMS_%26_USSD/status&action=edit>
]
The USSD service is ready to launch pending contractual negotiations.
*Mobile QA <https://www.mediawiki.org/wiki/Mobile_QA>*
[edit<https://www.mediawiki.org/w/index.php?title=Mobile_QA/status&action=edit>
]
The Mobile QA team planned and began several projects in December, in
particular: an upcoming community test event for Mobile features; support
for MobileFrontend in beta labs; and significant new UI-level automated
tests in the gerrit queue.
Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering>
MediaWiki
Core
*MediaWiki 1.21 <https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap>* [
edit<https://www.mediawiki.org/w/index.php?title=MediaWiki_1.21/Roadmap/status&action=edit>
]
We continued the bi-weekly deployment cycle, deploying MediaWiki
1.21wmf5<https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf5>and
1.21wmf6 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf6>. We stopped
deployments at the end of the month due to the holidays, restarting the
1.21wmf7 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf7> cycle on
January 2.
*Git conversion <https://www.mediawiki.org/wiki/Git/Conversion>*
[edit<https://www.mediawiki.org/w/index.php?title=Git/Conversion/status&action=edit>
]
There's not much to report for the month of December so far with Gerrit.
New repositories continue to be created, and the vast majority of active
parts of SVN have been marked read-only by now. Upgrading to a newer
version of Gerrit is still blocked on our LDAP problem with master, but the
patch to fix that is nearly complete. Mid-December, we extended the
Verified category to now allow +2 (in addition to +1 and -1), so Jenkins
has a wider range of statuses it can report.
*TimedMediaHandler <https://www.mediawiki.org/wiki/TimedMediaHandler>*
[edit<https://www.mediawiki.org/w/index.php?title=TimedMediaHandler/status&action=edit>
]
Jan Gerber continued to refine the TimedMediaHandler extension, making the
transcoding steps more robust.
*Wikidata deployment <https://www.mediawiki.org/wiki/Wikidata_deployment>* [
edit<https://www.mediawiki.org/w/index.php?title=Wikidata_deployment/status&action=edit>
]
The Wikibase client extension was deployed to test2 in December. We plan
further deployment work in January, deploying to the Hungarian language
Wikipedia on January 14, 2013.
*SwiftMedia <https://www.mediawiki.org/wiki/SwiftMedia>*
[edit<https://www.mediawiki.org/w/index.php?title=SwiftMedia/status&action=edit>
]
Captchas are ready to be served from Swift. They previously were for
several days, but the configuration had to be reverted to due random errors
from Swift. A new set of captchas are being tweaked for readability and are
served from Swift on the test wikis. Captchas are one of the last NFS
dependencies.
*Site performance <https://www.mediawiki.org/wiki/Site_performance>*
[edit<https://www.mediawiki.org/w/index.php?title=Site_performance/status&action=edit>
]
After an assessment by Asher Feldman, Patrick Reilly and Tim Starling, the
RDB database patch was canceled. Instead, in the short term, a separate
vertically partitioned data cluster will be provided as a temporary storage
until a horizontally scalable architecture can be finalized. Matthias
Mullie is modifying the RDB-dependent ArticleFeedbackToolv5 to remove that
dependency through an abstraction layer. When a sharded or horizontally
scaled solution is provided, AFTv5's abstraction will be migrated. An
initial assessment of various non-MySQL alternatives for using Aaron
Schulz's JobQueue core patch in 1.20 is being done for
Echo<https://www.mediawiki.org/wiki/Echo>.
Because of the time it takes to exhaust the Echo queues, it is written to
bypass the JobQueue through direct calls. Luke Welling is abstracting the
JobQueue for Redis, ZeroMQ, and others.
*Admin tools development<https://www.mediawiki.org/wiki/Admin_tools_development>
* [edit<https://www.mediawiki.org/w/index.php?title=Admin_tools_development/status&action=edit>
]
The initial code was committed for interface for Stewards to mass-lock user
accounts<https://www.mediawiki.org/wiki/Admin_tools_development/CentralAuth_Locking>.
For global AbuseFilters, a permission for global rule-writing was merged
and the initial code for using WikiSets in the rules was written. Initial
code committed for renaming CentralAuth user
accounts<https://www.mediawiki.org/wiki/Admin_tools_development/Global_Rename>
.
*REST proposal <https://www.mediawiki.org/wiki/API/REST_proposal>*
[edit<https://www.mediawiki.org/w/index.php?title=API/REST_proposal/status&action=edit>
]
Wikia has completed a preliminary prototype (deemed to be disposed of after
all the valuable data has been collected) in order to validate the design
and its core concepts, identify and explore possible issues and test limits
imposed by the platform. It will allow be used to explore the usage of PHP
5.4's new features to ease the implementation of a maintainable versioning
system (the prototype abuses PHP's implementation of namespaces in some
cases, this is not meant to persist in the final prototype but was rather a
stress test), test human-readable formatting for responses when called by
specific clients, and measure overhead added by the software abstraction.
As a result, some pain points and alternative routes have been identified
on which research work will be carried on in late January/beginning of
February 2013, leading the team closer to a final implementation and
related RFC. The code will be available for a short time in a dedicated
branch <https://github.com/Wikia/app/tree/lox-REST-prototype> at Wikia's
app repository at Github <https://github.com/Wikia/app>.
*Security auditing and
response<https://www.mediawiki.org/wiki/Security_auditing_and_response>
* [edit<https://www.mediawiki.org/w/index.php?title=Security_auditing_and_response/status&action=edit>
]
The team continued to respond to several reported vulnerabilities. A
follow-up security review for Wikidata phase 2/3 was done.
Quality assurance
*Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>*
[edit<https://www.mediawiki.org/w/index.php?title=Beta_cluster/status&action=edit>
]
The project to support MobileFrontend in Beta labs continues. We intend for
Beta labs to become a test environment for the new git-deploy script from
the Operations team: this should be helpful in ongoing maintenance of the
environment
*Continuous integration<https://www.mediawiki.org/wiki/Continuous_integration>
* [edit<https://www.mediawiki.org/w/index.php?title=Continuous_integration/status&action=edit>
]
The last Jenkins jobs (mostly Analytics ones) that were still using the
Gerrit Trigger plugin have been migrated to being triggered by Zuul. Zuul
now support triggering tests for whitelisted users. This has been deployed
to let trusted users have unit tests run whenever they send a patchset in
mediawiki/core (gerrit change
39310<https://gerrit.wikimedia.org/r/#q,39310,n,z>).
Volunteer Merlijn van Deen built a script to replicate our Jenkins
installation<http://lists.wikimedia.org/pipermail/wikitech-l/2012-December/065088.html>and
worked on having extensions tests run on different MediaWiki branches.
*Browser testing <https://www.mediawiki.org/wiki/Browser_testing>*
[edit<https://www.mediawiki.org/w/index.php?title=Browser_testing/status&action=edit>
]
After its announcement about the state of automated browser
testing<http://lists.wikimedia.org/pipermail/wikitech-l/2012-December/064964.html>on
wikitech-l, the QA team continued to expand test coverage, improve
system and project documentation, and publicize and socialize the project
by means of the "Browser Testing" MediaWiki
Group<https://www.mediawiki.org/wiki/Groups/Proposals/Browser_testing>
.
Analytics
*Kraken (Analytics Cluster)<https://www.mediawiki.org/wiki/Analytics/Kraken>
* [edit<https://www.mediawiki.org/w/index.php?title=Analytics/Kraken/status&action=edit>
]
LDAP Hue/Hadoop authentication works, but group file access still needs to
be worked out. We've puppetized an Apache proxy for internal Kraken and
Hadoop web services, as well as udp2log kafka production and kafka hadoop
consumption. The event.gif log stream is being consumed into Hadoop. We're
attempting to use udp2log to import logs into Kafka and Hadoop without
packet loss, and backing up Hadoop service data files to HDFS (e.g. Hue,
Oozie, Hive, etc.).
*Limn <https://www.mediawiki.org/wiki/Analytics/Limn>*
[edit<https://www.mediawiki.org/w/index.php?title=Analytics/Limn/status&action=edit>
]
A major rework of Limn to use d3.js and Knockout.js is complete and will be
used for the next ReportCard. Dan Andreescu and David Schoonover are
working on graph editing and geospatial data visualization.
Engineering community team
*Bug management <https://www.mediawiki.org/wiki/Bug_management>*
[edit<https://www.mediawiki.org/w/index.php?title=Bug_management/status&action=edit>
]
Daniel Zahn and Andre Klapper upgraded Bugzilla to the latest stable
version (4.2.4) which provides higher flexibility for displaying interface
elements, improved custom search, better JSON-RPC support and a solid base
for future improvements being considered. Andre continued to improve the bug
management documentation <https://www.mediawiki.org/wiki/Bug_management>.
Many bug reports that were previously closed as RESOLVED LATER were
retriaged and RESOLVED LATER was disabled for future use, and a large
number of previously unprioritized bug reports received a priority setting.
Furthermore, Andre looked after reports about CSS issues after the MediaWiki
1.21wmf5 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf5> deployment
and followed up by triaging, creating requested Bugzilla components, etc.
Several smaller regex fixes were deployed in Bugzilla to fix automatic
linking to Gerrit changesets. A "patch in gerrit" bug status was discussed
on wikitech-l<http://lists.wikimedia.org/pipermail/wikitech-l/2012-December/065046.html>with
the conclusion to wait for automatic notifications (comments) from
Gerrit into Bugzilla about patch status changes first (which is being
worked on by the Wikidata team).
*Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>* [
edit<https://www.mediawiki.org/w/index.php?title=Mentorship_programs/status&action=edit>
]
Six MediaWiki candidates have been
announced<http://blog.wikimedia.org/2012/12/11/welcome-to-floss-outreach-program-for-women-interns/>for
the Outreach
Program for Women<https://www.mediawiki.org/wiki/Outreach_Program_for_Women>(OPW).
4 of them are funded by the Wikimedia Foundation and 2 by Google
through an agreement with the GNOME Foundation, organizers of the program.
They will work as full-time interns under the supervision of MediaWiki
mentors between January and March 2013. We got 10 submissions from about 25
people interested. The rather open and participatory selection
process<https://www.mediawiki.org/wiki/Mentorship_programs/Selection_process>we
have defined for OPW will be used as a basis for future mentoring
programs. We've also started matchmaking for the
LevelUp<https://www.mediawiki.org/wiki/Mentorship_programs/LevelUp>mentorships
for the coming quarter.
*Technical communications<https://www.mediawiki.org/wiki/Technical_communications>
* [edit<https://www.mediawiki.org/w/index.php?title=Technical_communications/status&action=edit>
]
Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom>
published a project
plan and timeline<https://www.mediawiki.org/wiki/Technical_communications/Fall_2012_consultation>for
the consultation process started in October about how to improve 2-way
communication between the technical and editing communities. He
summarized<https://www.mediawiki.org/wiki/Technical_communications/Fall_2012_consultation#Phase_2:_Summary_and_wider_outreach>the
results of the first phase and reached out to the wikitech-ambassadors
list<http://lists.wikimedia.org/pipermail/wikitech-ambassadors/2012-December/000105.html>to
widen the consultation process by proxy. After consolidation and
prioritization of the results, the most feasible solution appeared to be to
grow a network of
ambassadors<https://meta.wikimedia.org/wiki/Tech/Ambassadors>,
which he started to organize on meta.
Unrelatedly, Guillaume made a list of 2012 tech blog
posts<https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity>to
map tech blog activity by month & subdepartment (with priority
activities listed separately). Work on setting up a Volunteer product
manager<https://blog.wikimedia.org/2012/11/21/lead-development-process-product-adviser-manager/>program
is also underway.
Quim Gil <https://www.mediawiki.org/wiki/User:Qgil> sorted out Social
media<https://www.mediawiki.org/wiki/Social_media>channels, and we now
have @MediaWiki handles for
identi.ca <http://identi.ca/mediawiki>,
Twitter<https://twitter.com/#%21/MediaWiki>,
Facebook <https://www.facebook.com/MediaWikiProject> and
Google+<https://plus.google.com/u/0/b/103470172168784626509/103470172168784626509/posts>.
He published the community metrics November
report<https://www.mediawiki.org/wiki/Community_metrics/November_2012>and
a blog
post<https://blog.wikimedia.org/2012/12/10/introducing-mediawiki-community-metrics/>introducing
this new activity.
*Volunteer coordination and
outreach<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>
* [edit<https://www.mediawiki.org/w/index.php?title=Volunteer_coordination_and_outreach/status&action=edit>
]
MediaWiki Groups <https://www.mediawiki.org/wiki/Groups> became official
and the first proposals
<https://www.mediawiki.org/wiki/Groups/Proposals>are going through the
approval process. As a side effect, a process for
requesting regional mediawiki-themed mailing
lists<https://www.mediawiki.org/wiki/Groups#Local_vs_regional>has been
created with
mediawiki-india<https://lists.wikimedia.org/mailman/listinfo/mediawiki-india>as
the first case. At least three Wikimedia-related talks have been
accepted at FOSDEM <https://www.mediawiki.org/wiki/Events/FOSDEM>.
Language engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering>
*Language tools <https://www.mediawiki.org/wiki/Language_tools>*
[edit<https://www.mediawiki.org/w/index.php?title=Language_tools/status&action=edit>
]
Development of the new user interface for Translate, as well as the
translation editor functionality, continued at full pace throughout the
month of December, with iterative feature development and user experience
improvements. Santhosh Thottingal and Niklas Laxström are leading
development and Pau Giner is focusing on optimizing user experience
elements. The team also released the latest version of the MediaWiki
Language Extension Bundle. Increased support for language variants,
alternate language codes were added to the Universal Language Selector.
Alolita Sharma continued to work with Red Hat's localization and
internationalization teams to evaluate localization data, translation tools
and internationalization tools and technologies.
*Milkshake <https://www.mediawiki.org/wiki/Milkshake>*
[edit<https://www.mediawiki.org/w/index.php?title=Milkshake/status&action=edit>
]
More language input methods contributed by language communities were added
to the jquery.ime library.
Other newsPau Giner and Amir Aharoni participated in the Open Tech Chat
this month to talk about best practices in multilingual user testing and
internationalization. Amir Aharoni also participated in mentoring
OPW<https://www.mediawiki.org/wiki/Outreach_Program_for_Women>candidate
Priyanka Nag for the new LevelUp program. Srikanth Lakshmanan and
Arun Ganesh’s tenure ended with the Language Engineering team in December.
Kiwix <http://www.kiwix.org>
*The Kiwix project is funded and executed by Wikimedia
CH<https://meta.wikimedia.org/wiki/Wikimedia_CH>
.*
A new Kiwix 0.9rc2 <http://changelog.kiwix.org> was released. This version
embeds our ZIM HTTP server *kiwix-serve* for Windows, OSX and Linux. It is
now integrated in the Kiwix UI, allowing everyone to share Wikipedia on a
LAN in two clicks . We have revamped our audience measurement
tool<http://blog.kiwix.org/post/2012/12/09/Nginx-or-Apache%2C-Mirrorbrain-and-Piwik>,
a solution that could be interesting for other projects using
Mirrorbrain<http://www.mirrorbrain.org>.
We continue at the same time to increase our ZIM production throughput with
8 new Wikipedia ZIM files in December. December was also a month of new
records for Kiwix: for the first time, we have had more than 70.000
downloads a month<http://lists.wikimedia.org/pipermail/offline-l/2013-January/001079.html>and
a Lead
position<https://sourceforge.net/directory/home-education/education/os:linux/freshness:recently-updated/>for
Education software at Sourceforge.
Wikidata <https://meta.wikimedia.org/wiki/Wikidata>
*The Wikidata project is funded and executed by Wikimedia
Deutschland<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>
.*
New code and bugfixes have been deployed (with MediaWiki
1.21wmf5<https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Wikibase.git;a=shortlog;h=refs/heads/mw1.21-wmf5>and
1.21wmf6<https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/extensions/Wikibase.git;a=shortlog;h=refs/heads/mw1.21-wmf6>)
and test2 <http://test2.wikipedia.org> now gets language links from
Wikidata. Changes on Wikidata that concern articles on test2 are shown in
the recent changes of test2 as well. If there are no problems, deployment
on the Hungarian
Wikipedia<https://www.wikidata.org/wiki/Wikidata:Project_chat/Archive/2012/12#next_steps_for_the_first_Wikidata_client_-_the_Hungarian_Wikipedia>will
happen on January 14, 2013. Other Wikipedia sites will follow. For
the second phase of Wikidata, representation of values is the central
focus. We published a
draft<https://meta.wikimedia.org/wiki/Wikidata/Development/Representing_values>and
discussions<http://lists.wikimedia.org/pipermail/wikidata-l/2012-December/001406.html>have
started; we'd appreciate your feedback. Additionally, Denny Vrandečić
and Lydia Pintscher held IRC office hours; logs are available in
English<https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2012-12-18a>and
German<https://meta.wikimedia.org/wiki/IRC_office_hours/Office_hours_2012-12-18b>
. Future The engineering management team continues to update the *
Deployments <http://wikitech.wikimedia.org/view/Deployments>* page weekly,
providing up-to-date information on the upcoming deployments to Wikimedia
sites, as well as the *engineering
roadmap<https://www.mediawiki.org/wiki/Roadmap>
*, listing ongoing and future Wikimedia engineering efforts.
--
Guillaume Paumier
Technical Communications Manager — Wikimedia Foundation
https://donate.wikimedia.org
-------------- next part --------------
_______________________________________________
Please note: all replies sent to this mailing list will be immediately directed to Wikimedia-l, the public mailing list of the Wikimedia community. For more information about Wikimedia-l:
https://lists.wikimedia.org/mailman/listinfo/wikimedia-l
_______________________________________________
WikimediaAnnounce-l mailing list
WikimediaAnnounce-l at lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimediaannounce-l
More information about the Wikimedia-l
mailing list