Wikitech-l November 2012

wikitech-l@lists.wikimedia.org

130 participants
136 discussions

Wikimedia Foundation hiring for several positions
by Sumana Harihareswara 01 Nov '12

01 Nov '12

I know this might be offtopic, but people on this list seem likely to be interested or know people who are. As pulled from the most recent engineering report: Work with us <https://wikimediafoundation.org/wiki/Work_with_us> Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. - Senior Software Engineer - Experimental Features<http://hire.jobvite.com/Jobvite/Job.aspx?j=oHDiWfwi> - Software Engineer - Experimental Features<http://hire.jobvite.com/Jobvite/Job.aspx?j=opDhWfwZ> - Software Engineer - Visual Editor<http://hire.jobvite.com/Jobvite/Job.aspx?j=otYPWfwW> - Software Engineer (Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o1H6Vfwt> - Software Engineer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oX2hWfwW> - Software Developer General (Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4cKWfwG> - Git and Gerrit software development (Contract)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4gIWfwI> - Senior Software Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi> - Release Manager <http://hire.jobvite.com/Jobvite/Job.aspx?j=oZrQWfwW> - Visual Designer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oomJWfw9> - Product Manager (Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGWJWfw1> - RFP-Lucene Search Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oC5fWfwC> - Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf> - IT Technician (Part-time)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o1TKWfwk> - Operations Engineer/Database Administrator<http://hire.jobvite.com/Jobvite/Job.aspx?j=obMOWfwr> Our Job Openings page: http://wikimediafoundation.org/wiki/Job_openings also links to our non-discrimination policy, our pluralism, internationalism, and diversity policy, and our compensation practices. And it mentions a few systems administration jobs open in another WMF department. Those positions are based in our San Francisco headquarters, but in many cases we are open to the possibility of people working remotely, unless otherwise noted in the job posting itself. -- Sumana Harihareswara Engineering Community Manager Wikimedia Foundation

1 0

Wikimedia engineering October 2012 report
by Guillaume Paumier 01 Nov '12

01 Nov '12

Hi, The report covering Wikimedia engineering activities in October 2012 is now available. Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/October Blog version: https://blog.wikimedia.org/2012/11/01/engineering-october-2012-report/ As of last month, we're also proposing a shorter and simpler version of this report for less technically-savvy readers: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/October/su… I'm going to attempt to include the full HTML text of the report below, as requested last month. I usually avoid sending HTML e-mails, especially on mailing lists, so I don't know how this is going to turn out. ====================================== Engineering metrics in October: - 110 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commits<https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z>remained stable around 440. - About 35 shell requests <https://www.mediawiki.org/wiki/Shell_requests>were processed. - About 57 developers got access to Git and Wikimedia Labs<https://www.mediawiki.org/wiki/Developer_access> . - Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs> now hosts 137 projects, 694 users; to date 1268 instances have been created. Major news in October include: - a redesign of the mobile site<https://blog.wikimedia.org/2012/10/24/wikipedia-mobile-gets-a-new-look/>emphasizing readability and navigation; - the launch of a Wikipedia app<https://blog.wikimedia.org/2012/10/26/wikipedia-app-for-windows-8-and-windo…>for Windows RT and Windows 8 tablets; - a test of a redesigned account creation page<https://blog.wikimedia.org/2012/10/05/testing-new-signup-page-for-wikipedia/> ; *Note: As of last month, we're proposing a shorter and simpler version of this report<https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2012/October/su…>for less technically savvy readers. * Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us> Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. - Senior Software Engineer - Experimental Features<http://hire.jobvite.com/Jobvite/Job.aspx?j=oHDiWfwi> - Software Engineer - Experimental Features<http://hire.jobvite.com/Jobvite/Job.aspx?j=opDhWfwZ> - Software Engineer - Visual Editor<http://hire.jobvite.com/Jobvite/Job.aspx?j=otYPWfwW> - Software Engineer (Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o1H6Vfwt> - Software Engineer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oX2hWfwW> - Software Developer General (Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4cKWfwG> - Git and Gerrit software development (Contract)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o4gIWfwI> - Senior Software Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ouLnWfwi> - Release Manager <http://hire.jobvite.com/Jobvite/Job.aspx?j=oZrQWfwW> - Visual Designer <http://hire.jobvite.com/Jobvite/Job.aspx?j=oomJWfw9> - Product Manager (Mobile)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oGWJWfw1> - RFP-Lucene Search Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oC5fWfwC> - Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=ocLCWfwf> - IT Technician (Part-time)<http://hire.jobvite.com/Jobvite/Job.aspx?j=o1TKWfwk> - Operations Engineer/Database Administrator<http://hire.jobvite.com/Jobvite/Job.aspx?j=obMOWfwr> Announcements - Željko Filipin joined the Platform engineering team as QA Engineer ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/063608.html> ). - Andre Klapper joined the Platform engineering team as Bug Wrangler ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/063616.html> ). - Michelle Grover joined the Mobile engineering team as a QA contractor ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/063730.html> ). - Luke Welling joined the Features engineering team as Senior Features Engineer (announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/064025.html> ). - Brad Jorsch joined the Platform engineering team as Software Engineer, working in the MediaWiki Core group (announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2012-October/064120.html> ). - Steven Bernardin joined the Operations team as Data Center Technician, working in our Tampa data center. Technical Operations *Site Infrastructure* Mark Bergsma has successfully implemented range seeking feature in Varnish, fixed several video streaming bugs, and finally redeployed Varnish at Eqiad, replacing the upload Squids in Tampa. Mark is now working on replacing upload Squid at Esams. He has provisioned 8 servers, and based on early testing, less may actually be needed. We are currently using 23 older servers for upload Squid at Esams. In addition, Mark deployed 4 new Varnish servers to serve bits from Esams; The existing 2 are being redeployed for other uses. This will provide higher throughput and added redundancy for the coming Fundraising season. Due to Swift cross-datacenter replication issues, we have moved originals to our nas1 server in Tampa as a stopgap measure, and replicated the contents to nas1001 at Eqiad. Currently, we have only 'originals' in nas1/nas1001. Mark, Aaron Schulz and Faidon Liambotis will work on copying over thumbnails next. Faidon upgraded Swift to 1.74 to address the Swift proxy memory leak issue. Tim Starling has deployed limited php-redis on the Apache servers, and Redis is now capturing session data (mc1). The new Memcached servers at Tampa<http://ganglia.wikimedia.org/2.2.0/?r=day&cs=&ce=&m=load_one&s=by+name&c=Me…>are ready, and Asher Feldman has started testing, and working on putting them into production in the coming weeks. He was originally hoping to use Redis to replace memcached/parser cache, but had to stop due to performance/latency issues with their replication method. Meantime, Asher has put into use the captured session data and identified some areas for performance improvement. The developers have been notified and are working on resolving the issues now. *West Coast caching center* We've started building out a new caching center in the San Francisco area, called *ULSFO*. In October, Leslie Carr and Daniel Zahn racked and stacked networking equipment. Next, we are going to purchase the caching servers once Mark Bergsma confirms the configuration. *Data Dumps <https://www.mediawiki.org/wiki/WMF_Projects/Data_Dumps>* Compressed multistream format files of current articles are now being produced for all dumps; researchers working with content from the larger wikis may find these helpful. We're working with Amazon on hosting the most recent dumps for EC2 users, thanks to Diederik van Liere. We've encountered some performance issues with media bundle generation off-site and are investigating; we're also working on moving that from experimental to production status. *Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs>* Home directories are being migrated to glusterFS: pam_mkhomedir has been enabled, the home directory creation script has been disabled, and /home is now a direct autofs mount, rather than having indirect mounts for each user under /home. Nova, Glance, and Keystone have been upgraded to latest *Essex*stable release. On all instances, Salt has been installed and puppet has been run. Numerous bugs have been fixed in OpenStackManager (project creation bug, removing tiny flavor from interface) and some features have been added as well (add user to bastion when shell group added). A patch was sent into Nova to enforce unique instance names. The Labs team attended the OpenStack summit in San Diego. One takeaway is that we are leading a team to push a DNS service into OpenStack incubation; Andrew Bogott's Nova DNS code will be merged into this project. Features Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering> Editing tools *VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>* [edit<https://www.mediawiki.org/w/index.php?title=VisualEditor/status&action=edit> ] In October, the team worked to finish most of the re-engineering the code design of VisualEditor so that it is more modular and easier to extend. This has involved creating and documenting a number of formal APIs at each point in the architecture, so a developer does not have to understand the entire code base to be able to add new features. The early version of the VisualEditor on mediawiki.org was updated three times (wmf1<https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf1#VisualEditor>, wmf2 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf2#VisualEditor> and wmf3 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf3#VisualEditor>), fixing a number of bugs and replacing the entire browser selection and typing models, and much of how the user interface connects with the rest of the code. *Parsoid <https://www.mediawiki.org/wiki/Parsoid>* [edit<https://www.mediawiki.org/w/index.php?title=Parsoid/status&action=edit> ] The Parsoid <https://www.mediawiki.org/wiki/Parsoid> team focused on testing the JavaScript prototype parser against a corpus of 100,000 randomly-selected articles from the English Wikipedia. A distributed MapReduce <https://en.wikipedia.org/wiki/MapReduce>-like system, which uses several virtual machines on Wikimedia Labs<https://www.mediawiki.org/wiki/Wikimedia_Labs>, constantly converts articles to HTML DOM and back again to wikitext using the latest version of the Parsoid. For a little over 75% of these articles, this results in exactly the same wikitext, as we intend. For another 18% of these articles, there are some differences in the wikitext, but these are so minor that they don't result in any differences in the produced HTML structure when it is re-parsed. In the production version of Parsoid which will attempt to retain original wikitext as far as possible, these minor differences will only show up, if at all, around content that the user edited. Finally, just under 7% of articles still contain errors that change the produced HTML structure. These issues are the focus of the current work in preparation for the December release. Editor engagement *Article feedback <https://www.mediawiki.org/wiki/Article_feedback>* [edit<https://www.mediawiki.org/w/index.php?title=Article_feedback/status&action=…> ] This month, we developed a few final features<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requireme…>for Article Feedback <https://www.mediawiki.org/wiki/Article_feedback/Version_5>, which is being tested on 10% of the English Wikipedia. Though our lead developer was loaned out to the WikiVoyage project for most of the month, we deployed a couple improvements to reduce the moderation workload for editors, including: better abuse filters<https://www.mediawiki.org/wiki/Article_feedback/Version_5/Feature_Requireme…>to automatically disallow swear words; new ways for moderators to filter the feedback page; and a check list to help users request oversight. These and other features can be tested on this sample article feedback page<https://en.wikipedia.org/wiki/Special:ArticleFeedbackv5/Golden-crowned_Spar…>or on the central feedback page <https://en.wikipedia.org/wiki/Special:ArticleFeedbackv5>(please report any bugs on Bugzilla<https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions…>). We also started to collect new research data to track how moderators use the feedback page <http://toolserver.org/%7Edartar/fp/>, as well as measure how many readers who post feedback become editors or registered users. Next month, we plan to make a few final improvements to address these findings, as well as complete code re-factoring to improve database performance. Once these tasks are done in coming weeks, we expect to release Article Feedback v5 to 100% of the English Wikipedia by the end of the year, and to other projects in 2013. For more information about this tool, check our project overview <https://en.wikipedia.org/wiki/Wikipedia:Article_feedback>. *Page Curation <https://www.mediawiki.org/wiki/Page_Curation>* [edit<https://www.mediawiki.org/w/index.php?title=Page_Curation/status&action=edit> ] This month, we made a few more improvements to our first release version of Page Curation <https://en.wikipedia.org/wiki/Wikipedia:Page_Curation>, which we deployed on the English Wikipedia in September 2012, with a very positive response from community editors. New features we developed based on their feedback include: showing the number of list items for selected filters; showing a notice when a page is only a few minutes old; and notifying the last reviewer if a page is 'mark as unreviewed'. We also developed detailed metrics dashboards <http://toolserver.org/%7Edartar/pc/> for tracking the impact of this new tool. To learn more, visit our introduction page<https://en.wikipedia.org/wiki/Wikipedia:Page_Curation/Introduction>, watch this video tour<https://en.wikipedia.org/wiki/Wikipedia:Page_Curation/Video>or read this tutorial <https://en.wikipedia.org/wiki/Wikipedia:Page_Curation/Tutorial>. If you are an experienced editor, try out the final version<https://en.wikipedia.org/wiki/en:Special:NewPagesFeed>on the English Wikipedia, and report any bugs on Bugzilla<https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions…>. We have now ended new feature development for this product and plan to upgrade it again in 2013. *MicroDesign <https://www.mediawiki.org/wiki/Micro_Design_Improvements>* [ edit<https://www.mediawiki.org/w/index.php?title=Micro_Design_Improvements/statu…> ] Several bugfixes were deployed to existing improvements, and work began on a new "Agora" extension to make it easier to productise and deploy the team's work, with Trevor Parscal and Rob Moen taking the lead. Vibha Bamba and Oliver Keyes have begun work on improving the templates that are displayed when the "edit" window is opened. *Editor engagement experiments<https://www.mediawiki.org/wiki/Editor_engagement_experiments> * [edit<https://www.mediawiki.org/w/index.php?title=Editor_engagement_experiments/s…> ] In October, the E3 team permanently deployed a confirmation message<https://www.mediawiki.org/wiki/Confirmation_message>for all editors ( Extension:PostEdit <https://www.mediawiki.org/wiki/Extension:PostEdit>) to 16 Wikipedias, including six of the top 10 projects by size, and worked on associated maintenance of the feature. The team also deployed two iterations of tests for a new registration page<https://www.mediawiki.org/wiki/Account_creation_user_experience>(read more<http://blog.wikimedia.org/2012/10/05/testing-new-signup-page-for-wikipedia/>), including the beginnings of an API for client-side validation of the sign-up form. In support of current and future work, we deployed the beta EventLogging extension <https://www.mediawiki.org/wiki/Extension:EventLogging>, a new architecture to replace the older ClickTracking extension<https://www.mediawiki.org/wiki/Extension:ClickTracking>. Last but not least, work started on redesigning the login<https://www.mediawiki.org/wiki/Wikimedia_Foundation_Design/Login>and new experiments aimed at onboarding new Wikipedians <https://www.mediawiki.org/wiki/Onboarding_new_Wikipedians>. MediaWiki infrastructure & Platform support *Notifications <https://www.mediawiki.org/wiki/Echo_%28Notifications%29>* [ edit<https://www.mediawiki.org/w/index.php?title=Echo_%28Notifications%29/status…> ] This month, we expanded our design and development activities for the Notifications project (code-named 'Echo'), to prepare for a first experimental deployment in early 2013. Fabrice Florin, Howie Fung and Vibha Bamba identified product goals, key features and scope for that first release (see project slides<https://en.wikipedia.org/wiki/File:Echo-Overview-Slides-10-18-2012.pdf>), and discussed them with team members, including our partners at Wikia. We also created new conceptual models and workflows for different use cases, as well as requirements and wireframes for our first features. Ryan Kaldari and Benny Situ started to develop new types of notifications (e.g. edit reversion, new page review), integrating with Andrew Garrett's code (e.g. talkpage message, mention), with support from Alex Monk<https://en.wikipedia.org/wiki/User:Krenair>. Aaron Schulz built a new abstracted version of the JobQueue system to support multiple queuing systems. Support *2012 Wikimedia fundraiser<https://www.mediawiki.org/wiki/2012_Wikimedia_fundraiser> * [edit<https://www.mediawiki.org/w/index.php?title=2012_Wikimedia_fundraiser/statu…> ] Throughout October, the Fundraising Engineering team has been working on the final engineering push before the kickoff of the 2012 fundraiser in November. During testing, performance regressions were noticed across many wikis and geographies. The team, with support from many other groups, has identified and is attempting to resolve these issues to enhance not only fundraising, but the overall user experience on Wikimedia sites. Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering> Contributors *Wiki Loves Monuments App<https://www.mediawiki.org/wiki/Wiki_Loves_Monuments_mobile_application> * [edit<https://www.mediawiki.org/w/index.php?title=Wiki_Loves_Monuments_mobile_app…> ] The mobile team started planning the decommission of the WLM Android app. We'll be retiring both the app and the back-end infrastructure. During the decommissioning process, our product team will analyze data from the WLM contest and prepare for Commons upload next. Readers *Mobile Nav<https://www.mediawiki.org/wiki/Mobile_design/Wikipedia_navigation> * [edit<https://www.mediawiki.org/w/index.php?title=Mobile_design/Wikipedia_navigat…> ] Jon Robson, Brion Vibber, Max Semenik, Arthur Richards, and the product team updated the mobile website<http://blog.wikimedia.org/2012/10/24/wikipedia-mobile-gets-a-new-look>with a new navigation bar, easy beta opt-in settings, new typography, and several bug fixes. The new navigation includes the Main page, a link to a random article, and settings functionality. The article page includes a new 'Read in another language' section. This is the biggest and most ambitious visual change that the mobile team has ever attempted. The new design will open the gates to new functionality by building on a cohesive new navigational infrastructure. *Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>* [edit<https://www.mediawiki.org/w/index.php?title=Wikipedia_Zero/status&action=ed…> ] This month, we've launched Wikipedia Zero with dtac Thailand and STC Saudi. We've also enhanced some diagnostics to inform the users of Zero if they are not using a known partner IP address. We continued to talk with potential new partners and made internal improvements to simplify the configuration process. *J2ME App <https://www.mediawiki.org/wiki/MobileFrontend/J2ME_app>* [edit<https://www.mediawiki.org/w/index.php?title=MobileFrontend/J2ME_app/status&…> ] We're finishing up the final testing of the Wikipedia J2ME application. We moved from testing on a variety of phones to reviewing on the reference hardware devices. Final approval is expected in early November. *Wikipedia over SMS & USSD<https://www.mediawiki.org/wiki/Wikipedia_over_SMS_%26_USSD> * [edit<https://www.mediawiki.org/w/index.php?title=Wikipedia_over_SMS_%26_USSD/sta…> ] The team focused on diagnostics and debugging prior to an initial launch with a partner. We're seeking to hire an engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oX2hWfwW&c=qSa9VfwQ>specifically to work in this area. *GeoData Storage & API<https://www.mediawiki.org/wiki/GeoData_Storage_%26_API> * [edit<https://www.mediawiki.org/w/index.php?title=GeoData_Storage_%26_API/status&…> ] GeoData has been rewritten to support spatial searches via Solr; we are working with the Operations team to prepare for deployment. *Windows 8 app* Brion Vibber released a native Wikipedia application for Windows 8<http://blog.wikimedia.org/2012/10/26/wikipedia-app-for-windows-8-and-window…>. This app was not part of the regular Wikimedia product roadmap; instead, it was the result of a 1-day iteration that each team member gets for research time. Offline *Kiwix <http://www.kiwix.org>* Release 0.9rc2 <http://changelog.kiwix.org> is almost finished. The highlight of this release is kiwix-serve<http://www.kiwix.org/index.php/Kiwix-serve>for MS/Windows, directly available from the Kiwix UI. A first version of kiwix-plug <http://www.kiwix.org/index.php/Kiwix-plug> was installed on 15 devices with the Afripedia<https://blog.wikimedia.org/2012/06/28/afripedia-collaboration-expands-offli…>project. With Wikimedia France, USB sticks containing the French-language Wikipedia<http://blog.wikimedia.fr/cle-framakey-wikipedia-5053>were made available for purchase for the first time in France; they were sold out after only a week. Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering> MediaWiki Core *MediaWiki 1.20 <https://www.mediawiki.org/wiki/MediaWiki_1.20/Roadmap>* [ edit<https://www.mediawiki.org/w/index.php?title=MediaWiki_1.20/Roadmap/status&a…> ] Mark Hershberger has released a release candidate of the 1.20 tarball. *MediaWiki 1.21 <https://www.mediawiki.org/wiki/MediaWiki_1.21/Roadmap>* [ edit<https://www.mediawiki.org/w/index.php?title=MediaWiki_1.21/Roadmap/status&a…> ] We started the MediaWiki 1.21 series this month. We deployed MediaWiki 1.21wmf1 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf1> and 1.21wmf2<https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf2>to all wikis, and started to deploy 1.21wmf3 <https://www.mediawiki.org/wiki/MediaWiki_1.21/wmf3>. 1.21wmf2 contained a number of significant features: ContentHandler code necessary for supporting Wikidata, high-resolution image support, some refactoring of the CologneBlue skin, and a new "Sites" back-end (also used by Wikidata). *Git conversion <https://www.mediawiki.org/wiki/Git/Conversion>* [edit<https://www.mediawiki.org/w/index.php?title=Git/Conversion/status&action=ed…> ] Many more extensions are now replicated from Gerrit to the MediaWiki account on GitHub <https://github.com/mediawiki>. Gerrit 2.5rc1 and rc2 were released over the course of October, and a final 2.5 is expected soon. One particularly exciting feature in Gerrit 2.5 is the new extensibility framework, which will allow us to replace our gitweb based source browser<https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=summary>with GitBlit <http://gitblit.com/> (the latter being a far more usable code browsing option). Assuming a showstopper bug with LDAP propagation gets fixed, we'll be able to deploy Gerrit 2.5 shortly after its release, and GitBlit shortly after that. *Wikidata deployment <https://www.mediawiki.org/wiki/Wikidata_deployment>* [ edit<https://www.mediawiki.org/w/index.php?title=Wikidata_deployment/status&acti…> ] The deployment of MediaWiki 1.21wmf2 was an important prerequisite to the deployment of Wikidata to the Wikimedia cluster. Chad Horohoe, Sam Reed, Daniel Zahn, and Mark Bergsma have been putting together the pieces to deploy a wiki to wikidata.org, which happened on October 30<http://www.wikidata.org> . *Wikivoyage migration <https://www.mediawiki.org/wiki/Wikivoyage_migration>* [edit<https://www.mediawiki.org/w/index.php?title=Wikivoyage_migration/status&act…> ] The extension review<https://www.mediawiki.org/wiki/Wikivoyage_migration/Extensions>is almost finished, and we've got an internal test setup on Wikimedia Labs that we've used to test imports and extensions. We've finalized the account migration strategy<https://www.mediawiki.org/wiki/Wikivoyage_migration/Accounts>and started implementing it. We've developed a draft agreement with Wikivoyage e.V. about the transfer of the domain names. The Wikivoyage e.V. members are currently voting to approve the migration<http://www.wikivoyage.org/assoc/14._Mitgliederversammlung_%28Online%29/TOP_…>. It'll be final on Friday, after which point we can transfer wikivoyage.orgto Wikimedia. Our current goal is to do the actual migration next week. *SwiftMedia <https://www.mediawiki.org/wiki/SwiftMedia>* [edit<https://www.mediawiki.org/w/index.php?title=SwiftMedia/status&action=edit> ] At the beginning of October, we had our new Swift distributed file storage cluster in production, with copies of all original media also being made to an NFS server (ms7) that was destined to fill up in the month of October. Our original plan was to shut off copying to ms7 and rely solely on the Swift cluster. Because of hardware issues with the Swift cluster, we decided we couldn't afford to switch off the copying of files to an NFS backup. We migrated the contents of ms7 over to a much larger NFS server (nas1), and configured nas1 to be the new live backup for images. We plan to remain in this configuration for the foreseeable future as we stabilize our distributed file store. *Site performance <https://www.mediawiki.org/wiki/Site_performance>* [edit<https://www.mediawiki.org/w/index.php?title=Site_performance/status&action=…> ] Tim Starling committed changes to our implementation of libxml to use the PHP memory allocator, rather than using malloc (the C standard for allocating memory directly managed by the operating system). This will allow us to have per-page limits on the complexity of pages in a way that more closely mirrors their impact on our cluster. *Admin tools development<https://www.mediawiki.org/wiki/Admin_tools_development> * [edit<https://www.mediawiki.org/w/index.php?title=Admin_tools_development/status&…> ] The team made some progress on writing an interface for Stewards to mass-lock user accounts<https://www.mediawiki.org/wiki/Admin_tools_development/CentralAuth_Locking>, but spent most of their time on Wikidata<https://www.mediawiki.org/wiki/Wikidata_deployment>ahead of its launch, and the Wikivoyage migration <https://www.mediawiki.org/wiki/Wikivoyage_migration>. *Security auditing and response<https://www.mediawiki.org/wiki/Security_auditing_and_response> * [edit<https://www.mediawiki.org/w/index.php?title=Security_auditing_and_response/…> ] The team continued to respond to and fix reported vulnerabilities. They worked on improving the release process for security updates to supported versions of MediaWiki, and provided significant security reviews of extensions for Wikivoyage and Wikidata. Quality assurance *QA and testing <https://www.mediawiki.org/wiki/QA_and_testing>* [edit<https://www.mediawiki.org/w/index.php?title=QA_and_testing/status&action=ed…> ] Newly hired Michelle Glover and Željko Filipin will be testing software and working on browser-level test automation for both mobile and web platforms. Željko has particular expertise in automated testing and will be joining Antoine Musso and Timo Tijhof in the Netherlands for a "Continuous Integration Summit" in conjunction with Wikimedia Nederland Hackathon 2012<http://nl.wikimedia.org/wiki/Hackathon_2012/en> . *Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>* [edit<https://www.mediawiki.org/w/index.php?title=Beta_cluster/status&action=edit> ] The MediaWiki configuration on the beta cluster has still a few remaining live hacks that prevent it from being upgraded smoothly. The final bits have been tracked down<https://bugzilla.wikimedia.org/show_bug.cgi?id=41132>and will need a final sprint. *Continuous integration<https://www.mediawiki.org/wiki/Continuous_integration> * [edit<https://www.mediawiki.org/w/index.php?title=Continuous_integration/status&a…> ] The continuous integration server has been upgraded to Precise, which will let us install more recent versions of various testing software. This upgrade also made it possible to deploy Zuul in production. *Browser testing <https://www.mediawiki.org/wiki/Browser_testing>* [edit<https://www.mediawiki.org/w/index.php?title=Browser_testing/status&action=e…> ] QA Engineer Željko Filipin has made great improvements to the existing automated browser tests and has created some new Mobile tests as well. Mobile QA Engineer Michelle Grover is creating an automatable regression test suite for MobileFrontend. These tests are currently running as builds under a hosted instance of Jenkins, with the intention of moving them to the WMF Continuous Integration environment pending upgrades to the machines hosting gerrit and Jenkins. Analytics *Kraken (Analytics Cluster)<https://www.mediawiki.org/wiki/Analytics/Kraken> * [edit<https://www.mediawiki.org/w/index.php?title=Analytics/Kraken/status&action=…> ] The Analytics team has been working on: configuring and puppetizing CDH4 (Hue, Sqoop, Oozie, Zookeeper, Hive, Pig), configuring and puppetizing Kafka, benchmarking performance, drafting metadata schemas, setting up ganglia monitoring, setting up prototype pixel service endpoint, and running ad-hoc data queries for fundraising. *Limn <https://www.mediawiki.org/wiki/Analytics/Limn>* [edit<https://www.mediawiki.org/w/index.php?title=Analytics/Limn/status&action=ed…> ] The migration to d3 rendering was completed. The design for the new Edit UI is complete. We are porting over the monthly metrics meeting dashboard to the new Limn and are aiming to show it off in December. *Analytics Infrastructure<https://www.mediawiki.org/wiki/Analytics/Infrastructure> * [edit<https://www.mediawiki.org/w/index.php?title=Analytics/Infrastructure/status…> ] Contractor Stefan Petrea has worked on bug fixes in wikistats, which we also migrated to git. On the udp2log front, we added features in udp-filters and webstatscollector, and deployed a new banner impression filter. Engineering community team *Bug management <https://www.mediawiki.org/wiki/Bug_management>* [edit<https://www.mediawiki.org/w/index.php?title=Bug_management/status&action=ed…> ] New Bug Wrangler Andre Klapper<https://www.mediawiki.org/wiki/User:Malyacko>had many discussions with different stakeholders to get a better impression of how work is done, how people interact with the bug tracker, what the expectations are and what policies might be needed. He investigated the product/component organization within bugzilla, started triaging incoming and older reports, and did maintenance work (creation and partial cleanup of products and components). bugzilla.wikimedia.org was upgraded to 4.0.8 with the help of Daniel Zahn, and investigations started to determine how urgent an upgrade to 4.2 was with regard to functionality improvements. Plans for the next month include improving documentation on bug management and bug triaging, and describing interactions between the bug wrangler and the different teams. *Summer of Code 2012<https://www.mediawiki.org/wiki/Summer_of_Code_2012/management> * [edit<https://www.mediawiki.org/w/index.php?title=Summer_of_Code_2012/management/…> ] The Wikimedia engineering community continues to help the 2012 GSoC students improve their projects towards the goal of release and deployment. Sumana Harihareswara aims to lead a postmortem discussion in November. Rob Lanphier and Sumana attended a GSoC Mentors' Summit in October, and discussed mentor recruitment <http://etherpad.osuosl.org/veFnJ0z05K>, community metrics <http://gsoc-wiki.osuosl.org/index.php/Community_Metrics>, how to be more effective mentors, student selection strategies<http://gsoc-wiki.osuosl.org/index.php/Ideas_an_techniques_to_improves_stude…>, PHP <http://gsoc-wiki.osuosl.org/index.php/PHP> and code review tools, and other related topics. As a follow-up to Summer of Code, the MediaWiki community is discussing whether to participate in Google Code-In<https://www.mediawiki.org/wiki/Google_Code-In> . *Technical communications<https://www.mediawiki.org/wiki/Technical_communications> * [edit<https://www.mediawiki.org/w/index.php?title=Technical_communications/status…> ] This activity was revived as its scope was expanded to include not only on-wiki engineering project documentation, but more generally the improvement of communications between Wikimedia contributors and the technical community (MediaWiki developers, Operations engineers, etc.). Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom> prepared and started a wide and open discussion with editors on some local wikis to identify issues and discussion possible solutions. Management is currently reviewing options to determine the direction this activity will follow in future months. *Volunteer coordination and outreach<https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach> * [edit<https://www.mediawiki.org/w/index.php?title=Volunteer_coordination_and_outr…> ] Sumana Harihareswara <https://www.mediawiki.org/wiki/User:Sumanah>continued to follow up on contacts (such as those gained at October's Grace Hopper Celebration of Women in Computing), recruit new contributors to the Wikimedia tech community, and mentor newer contributors. She granted developer access <https://www.mediawiki.org/wiki/Developer_access> and Gerrit project ownership requests<https://www.mediawiki.org/wiki/Git/Gerrit_project_ownership>, and worked on getting more volunteer developers +2<https://www.mediawiki.org/wiki/%2B2>status in MediaWiki core: 8 volunteers now have MediaWiki core maintainership. Sumana also published a retrospective of the 2012 Berlin Hackathon<https://www.mediawiki.org/wiki/Berlin_Hackathon_2012/Retrospective>and updated the list of towards a better understanding of which parts of the codebase are maintained, and by whom. Hiring for a Volunteer Engineering Coordinator to work on volunteer coordination and outreach is almost finished. Wikidata <https://meta.wikimedia.org/wiki/Wikidata> *The Wikidata project is funded and executed by Wikimedia Deutschland<https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en> .* The Wikidata team has worked on initial parts of Phase 2 of Wikidata (Infoboxes) and worked together with the WMF to get Wikidata deployed on http://www.wikidata.org. A big step towards this deployment was the merge of the content handler branch<https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2012-10-15/Techn…>into MediaWiki core. This allows MediaWiki to handle other content types besides just wikitext. In addition, the team is looking for help with the initial design of the Main Page<http://lists.wikimedia.org/pipermail/wikidata-l/2012-October/001104.html>of wikidata.org. A draft was also published by the team discussing how the propagation of changes from a repository to the clients<https://meta.wikimedia.org/wiki/Wikidata/Notes/Change_propagation>should work. Feedback and questions are welcome during the IRC office hours<https://meta.wikimedia.org/wiki/Wikidata/Events#IRC_office_hours>, on the mailing list<https://lists.wikimedia.org/mailman/listinfo/wikidata-l>and on meta <https://meta.wikimedia.org/wiki/Talk:Wikidata>. Future The engineering management team continues to update the *Software deployments <http://wikitech.wikimedia.org/view/Software_deployments>* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *engineering roadmap<https://www.mediawiki.org/wiki/Roadmap> *, listing ongoing and future Wikimedia engineering efforts. -- Guillaume Paumier Technical Communications Manager — Wikimedia Foundation https://donate.wikimedia.org

2 1

Marking /trunk/extensions as read-only
by Chad 01 Nov '12

01 Nov '12

Hi everyone, I was wondering what people thought of marking /trunk/extensions in SVN as read only. Pretty much every active extension (as far as I can tell) has been moved out already. It doesn't preclude us from migrating any of these extensions to Git at a later date, and nothing is being deleted. Thoughts? -Chad

5 8

Fwd: Seeking feedback on new "Organizations" feature on Ohloh
by Quim Gil 01 Nov '12

01 Nov '12

Hi, what about having Wikimedia features as organization in Ohloh? The proposal is interesting considering the current state of things: MediaWiki seems to be stalled with the SVN to Git migration, and it is close impossible to find out what other projects come from this community. This would help or quest on community metrics, so here goes my humble +1. I also volunteer with some work, basically following the steps of https://github.com/wikimedia and pinging Sumana / here for anything else. -------- Original Message -------- Subject: Seeking feedback on new "Organizations" feature on Ohloh Date: Tue, 16 Oct 2012 18:22:54 +0000 From: Rich Sands <rsands(a)blackducksoftware.com> To: metrics-wg(a)theopensourceway.org <metrics-wg(a)theopensourceway.org> Hi all, As I mentioned here recently, we're rolling out a new feature on Ohloh next week I thought might be interesting to this list. We're adding rollups of projects into "Organizations", so project contributors and others could see for example all the projects in the FooBar Foundation, which ones are most active, the most active contributors, year over year summary statistics, etc. I know many of you or folks in your organizations have rolled your own metrics for watching your foundation's projects and their activity. This feature isn't intended to replace any of that, but rather to provide a view into how organizations are contributing to and influencing FOSS. We've spoken with a number of you who've expressed a need for this, and hope this new feature can be a valuable resource. We're rolling this out as a Beta feature and would love to get your feedback. There are two aspects we're looking at: which organizations steward which projects, and also which organizations contribute to which projects through developers affiliated with those organizations, whether they're for-profit, non-profit, governmental, or educational. Initially we're concentrating on the first aspect - projects in organizations. In a near-term iteration we'll add in the contribution bit, but for now we're not showing stats on which projects an organization contributes to. If you would like a preview of this feature before it is released, let me know and I'll send you a URL and a name/password combo so you can check it out. And if you'd like your organization to be one of the featured ones when we open this up, I'd be thrilled to add you. All I need is an organization name, a short description, a logo (if available), a homepage URL for the organization, and a list of projects to include. If your projects aren't already on Ohloh, I can help you add them as well. We're keen to make this useful, and we're rolling it out in a fairly raw state, so that the FOSS community can help it evolve. Also, please don't post about or publicize this new feature before it comes out next week. Looking forward to hearing from you! -- rms Rich Sands Director of Developer Communities Black Duck Software, Inc. rsands(a)blackducksoftware.com <mailto:rsands@blackducksoftware.com> Cell: +1 617-283-0027 www.ohloh.net <http://www.ohloh.net>

5 18

unsubscibe
by Henry Hu 01 Nov '12

01 Nov '12

3 2

Please claim your Wikimedia commits in Ohloh
by Quim Gil 01 Nov '12

01 Nov '12

Hi, I have been drafting the first community metrics monthly report at http://www.mediawiki.org/wiki/User:Qgil/MediaWiki_Community_Metrics/Metrics… Feedback welcome! The plan was to publish it on November 1st, but as any good first release there have been a couple of unexpected glitches in the last minute. :) 1. Ohloh still hasn't scanned all repos for all of October (we have hundreds). It's better to have all the data retrieved. 2. Sumana found instantly four cases of contributors detected as 'new' that seem to be not new at all: thedj, Ori, Tychay and ankur. This is because they have started contributing commits from a new email address, and Ohloh considers them new contributors. Maybe there are more cases, so why not giving ourselves a chance to fix these obvious cases. FIXING YOUR DATA IN OHLOH TAKES ONLY 5 MINUTES This is of course voluntary: 1. Register or login at http://ohloh.net 2. Go to https://www.ohloh.net/people and search your name, nicks, email addresses. 3. Claim all the contributions that belong to you. 4. Also useful: if you are doing these contributions as an affiliated to an organization (e.g. Wikimedia) make this explicit when claiming your work. That's it. If you want to double check you can go to https://www.ohloh.net/orgs/wikimedia , find the project(s) where you are contributing and there your name or nick should appear under the list of contributors. Thank you for helping us providing more accurate data! -- Quim

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l November 2012