Wikitech-l February 2014

wikitech-l@lists.wikimedia.org

152 participants
162 discussions

Bingle/Bugello broken post-Bugzilla upgrade
by Arthur Richards 14 Feb '14

14 Feb '14

Just wanted to give everyone a head's up that Bingle/Bugello are broken after yesterday's BZ upgrade. At quick glance it appears related to BZ's move to a shared host - the tools are dying with the exception: requests.exceptions.SSLError: hostname 'bugzilla.wikimedia.org' doesn't match either of '*.planet.wikimedia.org', 'planet.wikimedia.org' I'll be digging into this shortly and will update when I have more info and/or resolution. -- Arthur Richards Software Engineer, Mobile [[User:Awjrichards]] IRC: awjr +1-415-839-6885 x6687

4 12

Let's improve our password policy
by Steven Walling 13 Feb '14

13 Feb '14

Hi everyone, For some time now we've had two Requests for Comment floating around related to passwords, neither of them making much progress. One is the older "password strength" RFC which proposed creating a module to tell users about the strength of their passwords. The second, "Password requirements", had some discussion but wasn't reaching consensus and implementation. After proposing it about a month ago, I've merged these two RFCs and refactored them in to https://www.mediawiki.org/wiki/Requests_for_comment/Passwords, partially based on feedback from Chris Steipp. Please comment. I've tried to sharpen the proposals down in to one thing we can do _right now_ which will do the most good for the most users. However, there are several other viable ideas which merit discussion and example implementations. Thanks! Steven

16 38

API-based methods for browser test setup
by Jeff Hall 13 Feb '14

13 Feb '14

For those working with WMF browser tests, we now have test setup methods available via a new Rugygem[1]. The Rubygem uses the Mediawiki API to provide methods for creating and deleting articles, as well as creating users. More methods will likely be added in the near future. Documentation on how to use these methods within the Cucumber/Selenium-based browser test framework is available on mediawiki.org[2], and if you have questions about how to best make use of them, please hit up the QA team on IRC[3] or e-mail[4]. Thanks, Jeff [1] http://rubygems.org/gems/mediawiki_api [2] https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Writing_te… [3] @wikimedia-qa [4] qa(a)lists.wikimedia.org

1 0

Registration Open: Wikimedia Hackathon 2014 in Zürich
by Manuel Schneider 13 Feb '14

13 Feb '14

Dear all, as announced earlier we will have a Hackathon in Zürich May 9th to 11th. Now our registration is open - especially if you come from a country where you need a visa for the Schengen area or if you need a scholarship - register early! Registration Form: https://docs.google.com/forms/d/1nlrQ7cox36xaNK1u9iKP-thogY5TVrilOGJR79DqQ9… Further information on the event can be found on MediaWiki.org: https://www.mediawiki.org/wiki/Z%C3%BCrich_Hackathon_2014 Regards and see you in Zürich! /Manuel -- Wikimedia CH - Verein zur Förderung Freien Wissens Lausanne, +41 (21) 34066-22 - www.wikimedia.ch

2 1

[IRC] deployment of wm-bot 2.x
by Petr Bena 13 Feb '14

13 Feb '14

Hi, When I started working on wm-bot I was designing it for less than 3 channels. I am really very happy that there are so many people who like it, and somehow it happened, that the bot is now being used in more than 170 channels. That is slightly more than I designed it for, and its maintenance became really very complicated. For that reason I decided to redesign it a bit, and I created wm-bot 2.0 which is designed little bit better for this heavy load, but on other hand it's totally incompatible with old bot :-). No worries - i created convertor for configuration files so regular users should see no difference at all. If everything goes well the best scenario is that nobody see any change at all (bot shouldn't even disconnect from freenode, so nobody would literally see any difference), however if something goes wrong, the bot may disappear from some channels and there may be a logging gap. When the deployment is finished, some functions may stop working because I had to rewrite all plugins (around 20 of them now) and some of them can be properly tested only under heavy load. If there is anything broken (I will send another mail when it's all finished) please make a bugzilla ticket https://bugzilla.wikimedia.org/enter_bug.cgi?product=Tools That's all folks TL;DR; There is going to be deployment of new version of wm-bot, which may result in temporary outage (but shouldn't) and some functions may stop working properly after it.

1 1

CfP SEMANTiCS (formerly known as I-SEMANTICS) September 4-5, 2014
by Sebastian Hellmann 13 Feb '14

13 Feb '14

<imap://hellmann@mail.informatik.uni-leipzig.de:143/fetch%3EUID%3E.Drafts%3E14116?part=1.2&filename=cc81ce36d80bcaad222b2fc1c3e9cdf556c5b101><imap://hellmann@mail.informatik.uni-leipzig.de:143/fetch%3EUID%3E.Drafts%3E14116?part=1.2&filename=cc81ce36d80bcaad222b2fc1c3e9cdf556c5b101> Call for Participation SEMANTiCS 2014 Transfer // Engineering // Community 10th International Conference on Semantic Systems (formerly known as I-SEMANTICS) Leipzig, Germany, September 4-5, 2014 http://www.semantics.cc Towards Engineering of Information Machines - We believe that semantic technologies are mature enough to dare the jump from research to engineering. We would like to invite you to Leipzig, Germany to explore with us best practices in engineering semantic systems. *Important Information (to help you prepare):* * The *deadline *for scientific papers will be *May 30th*a separate call for papers will be announced soon. * SEMANTiCS 2014 will be co-located with o September 3: 2nd DBpedia Community Meeting: http://wiki.dbpedia.org/meetings/Leipzig2014 o September 2: Multilingual Linked Open Data for Enterprises (MLODE 2014) o September 1-3: More workshops, meetings, tutorials * Workshop, Community Meetings and Challenges: We have a total of three days reserved for additional meetings, tutorials and workshops for September 1 - 3. If you are interested in a free room (coffee included), please send an email to the Chair of the Organising Committee, Ricardo Usbeck (http://www.semantics.cc/home/committee/organising-committee/). The annual SEMANTiCS conference (formerly known as I-Semantics) is the meeting place for professionals who make semantic computing work and understand the benefits and discuss how to overcome its limitations. Every year, SEMANTiCS attracts information managers, IT-architects, software engineers, and researchers, from organisations ranging from NPOs, public administrations to the largest companies in the world. Attendees learn from industry experts and top researchers about emerging trends and topics in semantic software, enterprise data, linked data & open data strategies and methodologies in knowledge modeling and text & data analytics. The SEMANTiCS community is very diverse; attendees have responsibilities in a range of areas, including knowledge management, technical documentation, e-commerce, big data analytics, enterprise search, document management, business intelligence, and enterprise vocabulary management. SEMANTiCS 2014 in Leipzig continues our long tradition of bringing together colleagues from around the world to present papers, panels, exhibitions and posters, to discuss best practices of semantic systems in birds-of-a-feather sessions and exchange knowledge further in informal settings. SEMANTiCS addresses problems common among information managers, software engineers, IT-architects and various specialist departments working to develop, implement and/or evaluate semantic software systems. The SEMANTiCS program is a rich mix of technical talks, panel discussions of important topics, and presentations by people who are ambitious to make information machines work - just like you. In addition, attendees can network with experts in a variety of fields. These relationships provide great value to organisations as they encounter subtle technical issues. The expertise gained by SEMANTiCS attendees has a long-term impact on their careers and organisations. These factors make SEMANTiCS the premiere event for our community. The following 'horizontals' (topics) and 'verticals' (industries) are mainly addressed: * Business Models, Governance & Data Strategies * Knowledge Discovery & Intelligent Search * Data Integration & Enterprise Linked Data * Big Data & Text Analytics * Data Portals & Knowledge Visualisation * Document Management & Content Management * Terminology, Thesaurus & Ontology Management * Industry & Engineering * Life Sciences & Health Care * Public Administration * Galleries, Libraries, Archives & Museums (GLAM) * Media, Publishing & Advertising * Financial Industry * Telecommunications * Energy We encourage participants to submit proposals for workshops, thematic focus and special programmes and events in contacting anyone from the SEMANTiCS Organising Committee: (http://www.semantics.cc/home/committee/organising-committee/) Call for Industry Presentations and Research Papers as well Posters and Demos will be announced at www.semantics.cc <http://www.semantics.cc> soon. Transfer: Research and experts meet industry, industry meets experts and research. SEMANTiCS is the place to exchange (project) ideas based on best practices and to learn more about the various perspectives one can have on probably the most valuable resources of our days: data & information. Engineering: Software providers, developers and IT-architects meet research and (potential) users from industry. SEMANTiCS is the place to dive deep into semantic web and linked data technologies. Community: Make new friends and renew old acquaintances. SEMANTiCS is the meeting place to discuss and to initiate new projects, to extend ongoing activities, and to showcase the latest developments. On behalf of all the people organizing SEMANTiCS 2014, we hope to see you in September in Leipzig, Sebastian Hellmann (AKSW, DBpedia), Harald Sack (Hasso-Plattner-Institut), Agata Filipowska (Poznan University of Economics), Christian Dirschl (Wolters Kluwer) and Andreas Blumenauer (Semantic Web Company) -- Sebastian Hellmann AKSW/NLP2RDF research group Insitute for Applied Informatics (InfAI) affiliated with DBpedia Events: * *21st March, 2014*: LD4LT Kick-Off <https://www.w3.org/community/ld4lt/wiki/LD4LT_Group_Kick-Off_and_Roadmap_Me…> @European Data Forum * *Sept. 1-5, 2014* Conference Week in Leipzig, including ** *Sept 2nd*, MLODE 2014 ** *Sept 3rd*, 2nd DBpedia Community Meeting ** *Sept 4th-5th*, SEMANTiCS (formerly i-SEMANTICS) <http://semantics.cc/> Venha para a Alemanha como PhD: http://bis.informatik.uni-leipzig.de/csf Projects: http://dbpedia.org, http://nlp2rdf.org, http://linguistics.okfn.org, https://www.w3.org/community/ld4lt <http://www.w3.org/community/ld4lt> Homepage: http://aksw.org/SebastianHellmann Research Group: http://aksw.org Thesis: http://tinyurl.com/sh-thesis-summary http://tinyurl.com/sh-thesis

1 0

Wikimedia Bugzilla upgraded to 4.4.1
by Andre Klapper 13 Feb '14

13 Feb '14

Hi, bugzilla.wikimedia.org has been upgraded from version 4.2.7 to 4.4.1. [For the technical audience: Bugzilla was also moved to a different server in the new datacenter and Bugzilla is puppetized now.] I would really like to thank Daniel Zahn and Sean Pringle for their wonderful work, help and support! For fixed problems in Bugzilla by this upgrade, see: https://bugzilla.wikimedia.org/buglist.cgi?bug_id=47013,42850,32504,49250,5… For new functionality, please see the list at https://bugzilla.wikimedia.org/show_bug.cgi?id=49597#c12 If you run into problems with the Bugzilla software itself after this upgrade, please file a bug report under https://bugzilla.wikimedia.org/enter_bug.cgi?product=Wikimedia&component=Bu… We have one small regression so far: "Saved Searches" in the sidebar look a bit ugly: https://bugzilla.wikimedia.org/show_bug.cgi?id=61288 It is a temporary trade-off for being able to see your Saved Reports (tables and graphs) listed there, similar to "Saved Searches". Cheers, andre -- Andre Klapper | Wikimedia Bugwrangler http://blogs.gnome.org/aklapper/

5 6

Wikimedia engineering report, January 2014
by Guillaume Paumier 13 Feb '14

13 Feb '14

Hi, The report covering Wikimedia engineering activities in January 2014 is now available. Wiki version: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January Blog version: https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/ We're also proposing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge: https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January/su… Below is the HTML text of the report. As always, feedback is appreciated on the usefulness of the report and its summary, and on how to improve them. ------------------------------------------------------------------ Major news in January include: - the transition of our search engines<https://blog.wikimedia.org/2014/01/06/wikimedia-moving-to-elasticsearch/>on Wikimedia sites to ElasticSearch; - a presentation of how the Tech newsletter works<https://blog.wikimedia.org/2014/01/07/tech-news-fighting-technical-informat…>, including a historical perspective; - an invitation to comment on the 2-year vision<https://blog.wikimedia.org/2014/01/09/multimedia-vision-2016/>of the multimedia engineering team; - a request for comments<https://blog.wikimedia.org/2014/01/23/rfc-should-we-support-mp4-video-on-ou…>on whether the MP4 video format should be supported on Wikimedia sites. *Note: We’re also providing a shorter, simpler and translatable version of this report <https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January/su…> that does not assume specialized technical knowledge.* Engineering metrics in January: - 159 unique committers contributed patchsets of code to MediaWiki. - The total number of unresolved commits<https://gerrit.wikimedia.org/r/#q,status:open+project:%255Emediawiki.*,n,z>went from around 1386 to about 1320. - About 22 shell requests <https://www.mediawiki.org/wiki/Shell_requests>were processed. Contents - Personnel<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Pers…> - Work with us<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Work…> - Announcements<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Anno…> - Technical Operations<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Tech…> - Features Engineering<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Feat…> - Editor retention: Editing tools<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Edit…> - Core Features<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Core…> - Growth<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Grow…> - Support<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Supp…> - Mobile<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Mobi…> - Language Engineering<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Lang…> - Platform Engineering<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Plat…> - MediaWiki Core<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Medi…> - Quality assurance<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Qual…> - Multimedia<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Mult…> - Engineering Community Team<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Engi…> - Analytics<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Anal…> - Offline<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Offl…> - Wikidata<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Wiki…> - Future<https://blog.wikimedia.org/2014/02/13/engineering-report-january-2014/#Futu…> Personnel Work with us <https://wikimediafoundation.org/wiki/Work_with_us> Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles. - VP of Engineering <http://hire.jobvite.com/Jobvite/Job.aspx?j=ods8Xfwu> - Software Engineer – Growth<http://hire.jobvite.com/Jobvite/Job.aspx?j=o8NJXfwl> - Software Engineer – VisualEditor (Features)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oqo6XfwB> - Software Engineer – Language Engineering<http://hire.jobvite.com/Jobvite/Job.aspx?j=oH3gXfwH&s> - Software Engineer <http://hire.jobvite.com/Jobvite/Job.aspx?j=o09WXfwM> - Software Engineer – Mobile (Android Apps)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oi6lYfwr> - QA Automation Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oe09Yfw5> - Test Infrastructure Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oFtlYfwb> - Analytics – Product Manager<http://hire.jobvite.com/Jobvite/Job.aspx?j=opkhYfwI> - Director of Community Engagement (Product)<http://hire.jobvite.com/Jobvite/Job.aspx?j=oX0kYfwZ> - Sr. Operations Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=o66gYfwa> - Operations Security Engineer<http://hire.jobvite.com/Jobvite/Job.aspx?j=oT6cYfwT> - User Experience Research Lead <http://grnh.se/3vmjby> Announcements - Shahyar Ghobadpour joined the Wikimedia Core features team as Software Engineer (announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-January/073870.html> ). - David Chan joined the Language Engineering team as Software Engineer ( announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-January/073873.html> ). - Gilles Dubuc joined the Multimedia team as Senior Software Engineer and Tech Lead (announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-January/073884.html> ). - Charles Salvia joined the Analytics team (announcement<http://lists.wikimedia.org/pipermail/analytics/2014-January/001526.html> ). - Sam Smith joined the Features Engineering team as Software Engineer, working with the Growth team (announcement<http://lists.wikimedia.org/pipermail/wikitech-l/2014-January/074118.html> ). - Preteek Saxena joined the UX Design team as UX Prototyping contractor ( announcement<http://lists.wikimedia.org/pipermail/design/2014-January/001441.html> ). Technical Operations *Datacenter RFP <https://wikimediafoundation.org/wiki/RFP/2013_Datacenter>* The Wikimedia Operations team is in the final stages of the selection process. A short list of 4 bids has been created and final negotiations are underway. The winner of the bid will be selected in February based on the technical criteria listed in the RFP, and pricing. Labs metrics in January: - Number of projects: 131 - Number of instances: 441 - Amount of RAM in use (in MBs): 1,734,144 - Amount of allocated storage (in GBs): 23,505 - Number of virtual CPUs in use: 867 - Number of users: 2,595 *Wikimedia Labs <https://www.mediawiki.org/wiki/Wikimedia_Labs>* The Labs Migration team, consisting of Andrew Bogott and Marc-Andre Pelletier, have made good progress with testing the newest version of Openstack (called Havana <https://www.openstack.org/software/havana/>) and with Neutron <https://wiki.openstack.org/wiki/Neutron>, an OpenStack project to provide “networking as a service”. The plan is to upgrade the Openstack software when we migrate the Labs infrastructure out of the Tampa data center. Features Engineering<https://www.mediawiki.org/wiki/Wikimedia_Features_engineering> Editor retention: Editing tools *VisualEditor <https://www.mediawiki.org/wiki/VisualEditor>* In January, the VisualEditor team continued their work on improving the stability and performance of the system, and added some new features. Most of the team’s focus was on major new features and fixing bugs. You can now edit some page settings like whether to display a table of contents or whether to show section edit labels, set the size of a media file manually, see a keyboard shortcuts help screen, and create and edit media galleries using a very basic stand-in editor whilst the final form is being designed. Work also continued on a dialog for quickly adding “citation” references based on templates, more media and page settings, setting content language and right-to-left flags, and equation editing. The deployed version of the code was updated four times (1.23-wmf9<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf9#VisualEditor>, 1.23-wmf10<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf10#VisualEditor>, 1.23-wmf11<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf11#VisualEditor>and 1.23-wmf12<https://www.mediawiki.org/wiki/MediaWiki_1.23/wmf12#VisualEditor> ). *Parsoid <https://www.mediawiki.org/wiki/Parsoid>* In January, the Parsoid team did a lot of bug fixing around images, links, references and various other areas. See the deployment page<https://www.mediawiki.org/wiki/Parsoid/Deployments#Monday.2C_Feb_3.2C_2014_…>for a summary. Part of the team has been mentoring two Outreach Program for Women<https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_7>(OPW) interns. Others are mentoring a group of students in a Facebook Open Academy <https://www.mediawiki.org/wiki/Facebook_Open_Academy> project to build a Cassandra storage back-end for the Parsoid round-trip test server. We also participated in the architecture summit, where our RFCs about embracing a service architecture<https://www.mediawiki.org/wiki/Requests_for_comment/Services_and_narrow_int…>, PHP bindings for services<https://www.mediawiki.org/wiki/Requests_for_comment/PHP_Virtual_REST_Service>, a general-purpose storage service<https://www.mediawiki.org/wiki/Requests_for_comment/Storage_service>based on our Rashomon revision store, and a public content API based on this<https://www.mediawiki.org/wiki/Requests_for_comment/Content_API>were well received. Following up on this, we started Debian packaging for Parsoid, which will soon make the installation of Parsoid as easy as apt-get install parsoid. Core Features *Flow <https://www.mediawiki.org/wiki/Flow/Project_information>* This month, the Core Features team worked on integrating MediaWiki tools for dealing with spam and vandalism (AbuseFilter and Spam Blacklist) into Flow. We also launched an updated visual design and UI, based on the first round of experienced user feedback last month, as well as ongoing user testing with new users. Lastly, we created a script to disable Flow and return Flow discussions back into unstructured wikitext, so that we can begin trialing Flow in production in an extremely safe-to-fail manner. We are set to deploy our first trial on February 3, 2014 to two WikiProjects that volunteered on the English Wikipedia. Growth *Growth <https://www.mediawiki.org/wiki/Growth>* In the first month of the year, the Growth team focused on two projects. First, we enhanced and refactored the GettingStarted<https://www.mediawiki.org/wiki/Extension:GettingStarted>extension, in part to support local configuration for different Wikipedias. The latest version of GettingStarted and GuidedTour will be released in English and 23 other languages in early February. Second, the team wrapped up several iterations of design and data analysis in support of upcoming work on Wikipedia article creation<https://www.mediawiki.org/wiki/Wikipedia_article_creation>. We presented new designs for the Draft namespace, and completed a series of remote usability tests<https://www.mediawiki.org/wiki/Draft_namespace/Usability_testing>(see the results<https://www.mediawiki.org/wiki/Draft_namespace/Usability_testing/Results>). We also finalized and published extensive quantitative analysis<https://meta.wikimedia.org/wiki/Research:Wikipedia_article_creation>of trends in article creation across the largest Wikipedias. Last but not least, the Growth team welcomed its newest member in January, Software Engineer Sam Smith. Support *Wikipedia Education Program <https://www.mediawiki.org/wiki/Wikipedia_Education_Program>* This month, once again we divided our time between the existing Education Program extension and work towards a new version of the software. We thoroughly analyzed database transactions in the current extension and fixed a slew of long-standing database-related bugs. Also on the current extension, we finished adding a notification type and notifications infrastructure, and worked on an improved course editing UX. For the new version, we studied workflow systems and considered how software for the Education Program and other outreach activities might use such a system. Adam Wight started on prototype workflow code. He also went through our code review backlog, bringing a multitude of new features and improvements to production. Mobile <https://www.mediawiki.org/wiki/Wikimedia_Mobile_engineering> *Wikipedia Zero <https://www.mediawiki.org/wiki/Wikipedia_Zero>* During the last month, the team added forward compatibility to Varnish scripting for Wikipedia Zero, and resubmitted a Varnish script patch to support HTTPS for select Wikipedia Zero partners under the new IP address-based zero-rating scheme, after analysis with the Operations team. We also continued proof of concept work on an HTML5 web app for Firefox OS, fixed bugs in the legacy Firefox OS Wikipedia app, and prepared alpha functionality for the integration of Wikipedia Zero with the rebooted Android Wikipedia app. The team also continued work toward a generic JSON configuration extension for use by extensions like ZeroRatedMobileAcces, submitted code for the core MediaWiki API, submitted a ResourceLoader (RL) enhancement and cooperated on alternatives for performance enhancement of RL on non-WMF Redis-backed ResourceLoaders, and submitted a small UX enhancement for the Android rebooted Wikipedia app. January 2014 was also a month of planning: the partners engineering team met for two days with the business development team to plan for partners and Wikipedia Zero-related work at large. The partners engineering team also applied itself to two days of product planning for the Partner Portal. Finally, the team conducted normal tech facilitation to enable partner launches and align approaches with current and future partners. *Mobile web projects <https://www.mediawiki.org/wiki/Mobile_web_projects>* We have been directing much of our attention over the last month at delivering a tablet-friendly MobileFrontend experience. We’ve added support for tables of contents in MobileFrontend for tablets, made some design improvements for tablets, and have worked towards making VisualEditor work with MobileFrontend for tablets (in alpha for now). We’ve hit some roadblocks and are hoping to collaborate more with the VE team in the near future to keep moving forward on the project. Following up from last month, we have also released our overlay UI improvements as well as an improved inline diff view for MobileFrontend into stable. Finally, we have also been working to expand our coverage of browser tests to facilitate quality assurance and help prevent the introduction of bugs and regressions. Language Engineering<https://www.mediawiki.org/wiki/Wikimedia_Language_engineering> *Language tools <https://www.mediawiki.org/wiki/Language_tools>* UniversalLanguageSelector<https://www.mediawiki.org/wiki/UniversalLanguageSelector>was disabled on January 21 2014 in production for all language Wikimedia sites (other than wikidata.org) due to font delivery performance issues. Users can still enable ULS for their language needs by going to their user profile preferences and enabling ULS from the internationalization settings. Development is in progress for a solution to enable ULS when a user logs in and selects their language preferences explicitly to enable webfonts. David Chan continued his work on language support integration for VisualEditor <https://www.mediawiki.org/wiki/VisualEditor> for phase 5 languages <https://www.mediawiki.org/wiki/VisualEditor#Timeline>. Niklas Laxstrom and Santhosh Thottingal participated in the architecture summit in San Francisco in January in RFC discussions and JSONification of i18n support<https://wikimedia.mingle.thoughtworks.com/projects/scrum_of_scrums/cards/58>for VisualEditor. *Language Engineering Communications and Outreach <https://www.mediawiki.org/wiki/Language_engineering_communications_and_outr…>* The team continued their collaborative projects with Google, Twitter, Microsoft internationalization and MT teams on webfonts, input tools and machine translation. *Content translation <https://www.mediawiki.org/wiki/Content_translation>* The language engineering team kicked off development of a prototype version of context translation workflow. This functionality aims to create a workspace for helping editors bootstrap new articles in non-Latin language Wikipedias. In the prototype, Russian and Welsh are being used for initial concept verification. Platform Engineering<https://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering> MediaWiki Core *Site performance and architecture <https://www.mediawiki.org/wiki/Site_performance_and_architecture>* The team worked on performance dashboards for VisualEditor<http://gdash.wikimedia.org/dashboards/ve/>and page load time <http://gdash.wikimedia.org/dashboards/frontend/>, the ProfilerMwprof profiler class for MediaWiki, and draft performance guidelines. *Admin tools development <https://www.mediawiki.org/wiki/Admin_tools_development>* This project is on hold, so there have been no significant developments, although some patches have been contributed by volunteer developers. *Search <https://www.mediawiki.org/wiki/Search>* As of February 3, CirrusSearch is available as a Beta Feature on wikis representing about three quarters of all pages, and serves about 7.5% of our search traffic. Next month, we hope to get the hardware that we need to be a Beta Feature on the remaining wikis. We also hope to be the primary search back-end for more wikis. To that end, we’re working through performance and recall issues as well as trying to save space in the indexes. *Auth systems <https://www.mediawiki.org/wiki/Auth_systems>* The team focused on minor updates to close some of the high priority OAuth bugs. *Deployment Tooling <https://www.mediawiki.org/wiki/Deployment_Tooling>* The Logstash <https://wikitech.wikimedia.org/wiki/Logstash> service was deployed to production at https://logstash.wikimedia.org/. Work has started on an analysis of the current scap process which will be used to draft requirements for further deployment scripting work in the current quarter. *Security auditing and response <https://www.mediawiki.org/wiki/Security_auditing_and_response>* We announced the MediaWiki 1.22.1 and 1.22.2 security releases, and continued to respond to reported vulnerabilities. Quality assurance *Quality Assurance <https://www.mediawiki.org/wiki/Quality_Assurance>* January saw the QA team working closely with the Mobile team in particular to enhance the existing suite of test for MobileFrontend. We also participated in the discussion of the Release Engineering deployment process at the architecture summit. Hiring is underway for two open positions, QA Automation Engineer and Test Infrastructure Engineer. *Beta cluster <https://www.mediawiki.org/wiki/Beta_cluster>* Beta is being used to test the Math extension rewrite. The Parsoid extension is now deploying continuously via a Jenkins job, status can be found on the CI dashboard job “Parsoid update”<https://integration.wikimedia.org/dashboard/> bug 57233 <https://bugzilla.wikimedia.org/show_bug.cgi?id=57233>. The wikis now send updates to the irc.wikimedia.org server bug 60013<https://bugzilla.wikimedia.org/show_bug.cgi?id=60013> . *Continuous integration <https://www.mediawiki.org/wiki/Continuous_integration>* Zuul has been upgraded and uses a Gearman bus to communicate with Jenkins, the l10n-bot is no more triggering change and we enabled a proper gating system to test changes in parallel. The workflow is smoother and faster to provide feedback in Gerrit. Jenkins slave lanthanum does not offer direct access to internet, we configured the jobs to use a web proxy in MediaWiki (web proxy.eqiad.wmnet or webproxy.pmtpa.wmnet. Finally, the Zuul status page <https://integration.wikimedia.org/zuul/> now shows the progress of jobs being run. *Browser testing <https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing>* In January, we had a number of contributions from the students of Google Code-in <https://www.mediawiki.org/wiki/Google_Code-in>, from tests to Jenkins configuration to documentation. We released two entirely new features: one test that monitors the file upload API interface on both production Commons and beta labs Commons, and another test that monitors fatal errors in Beta Labs. We are very close to announcing general availability for two other new features: the ability to run tests headless using Firefox under Xvfb, and the ability to create test data like wiki pages in the target wiki at run time. Multimedia *Multimedia <https://www.mediawiki.org/wiki/Multimedia>* In January, the multimedia team focused on developing the Media Viewer and planning our next projects for the year. Gilles Dubuc, Mark Holmquist, Gergo Tisza and volunteer Aaron Arcos implemented a number of improvements to the beta version of the Media Viewer<https://www.mediawiki.org/wiki/Multimedia/About_Media_Viewer>. Some of the features we created or improved include: faster image load, a full-screen mode, better navigation between files, an expanded meta-data panel with location, categories, permissions and assessments. We invite you to test the new UI features on this beta site<http://en.wikipedia.beta.wmflabs.org/wiki/Lightbox_demo>; faster image load can be tested on this MediaWiki.org page<https://www.mediawiki.org/wiki/Lightbox_demo>(In both cases, you need to create an account, then click on ‘Beta’ in your personal menu and enable Media Viewer.) Pau Giner also designed a new user interface for displaying slides, video and audio files<https://commons.wikimedia.org/w/index.php?title=File:Media_viewer_designs_-…>in the upcoming v0.3 version of Media Viewer, based on team recommendations. Fabrice Florin started a community discussion of our team’s Multimedia Vision for 2016<https://commons.wikimedia.org/wiki/Commons:Multimedia_Features/Vision_2016>, which proposes a range of improvements to help engage users and support productive collaborations in coming years (more comments welcome<https://commons.wikimedia.org/wiki/Commons_talk:Multimedia_Features/Vision_…>). We also planned our work for this quarter’s release<https://wikimedia.mingle.thoughtworks.com/projects/multimedia/cards/grid?ag…>, which focuses on Media Viewer through the end of March, and started planning our next big priorities for the rest of the year: UploadWizard and Structured Data on Commons. Lastly, we started a Request for Comments<https://commons.wikimedia.org/wiki/Commons:Requests_for_comment/MP4_Video>about possible support for the MP4 video standard: we invite you to participate in this discussion, which is due to end in mid-February; we will plan our next steps for video based on community feedback for this RfC. To discuss these projects and keep up with our work, we invite you to join the multimedia mailing list<https://lists.wikimedia.org/mailman/listinfo/multimedia> . Engineering Community Team<https://www.mediawiki.org/wiki/Engineering_Community_Team> *Bug management <https://www.mediawiki.org/wiki/Bug_management>* Valhallasw wrote a script to import tickets from JIRA<https://github.com/wikimedia/pywikibot-sf-export/blob/master/jira.py>(used by Toolserver) to Wikimedia Bugzilla, added a “Browse projects” link <https://gerrit.wikimedia.org/r/#/c/104342/> to Bugzilla’s sidebar, and added an “Upload to Gerrit” button<https://gerrit.wikimedia.org/r/#/c/110702/>for Bugzilla attachments to the Greasemonkey triagescripts<https://git.wikimedia.org/tree/wikimedia%2Fbugzilla%2Ftriagescripts>. Inline displaying of image attachments<https://bugzilla.wikimedia.org/show_bug.cgi?id=54181#c44>in Bugzilla was re-enabled and the default bug assignee was renamed from “Nobody” to “Nobody – You can work on this!”<https://bugzilla.wikimedia.org/show_bug.cgi?id=57648>to be more descriptive. The link to the guided bug entry form at the top of the standard bug entry form now sets the already chosen product directly<https://bugzilla.wikimedia.org/show_bug.cgi?id=59886>, saving you two clicks when you switch to the guided form. Work continued on preparing the Bugzilla 4.4 upgrade: Andre Klapper’s patches for porting custom changes from 4.2 to 4.4 were deployed on the Bugzilla test instance on Zirconium and tested, and Daniel Zahn fixed a problem with Bugzilla’s collectstats.pl <https://bugzilla.wikimedia.org/show_bug.cgi?id=29203>, so the 4.4 upgrade and server move will take place in February. Bug management documentation related, Andre added a “Situation specific information” section to the Triage guide<https://www.mediawiki.org/wiki/Bug_management/How_to_triage>documentation about purging and profiling. *Project management tools review <https://www.mediawiki.org/wiki/Project_management_tools/Review>* Andre Klapper reached out to the teampractices<https://lists.wikimedia.org/mailman/listinfo/teampractices>mailing list as well as individual stakeholders, asking users to share their workflow and needs<https://www.mediawiki.org/wiki/Talk:Project_management_tools/Review>regarding project management and tracking tools. Guillaume Paumier summarized all that content into consolidated requirements<https://www.mediawiki.org/wiki/Project_management_tools/Review/Requirements>; those are now in the process of being compared to features offered by available tools, in order to assemble a shortlist of candidates for community discussion. *Mentorship programs <https://www.mediawiki.org/wiki/Mentorship_programs>* [image: Google Code-in 2013, the Wikimedia debut.pdf]<https://commons.wikimedia.org/wiki/File:Google_Code-in_2013,_the_Wikimedia_…> <https://commons.wikimedia.org/wiki/File:Google_Code-in_2013,_the_Wikimedia_…> Wikimedia’s first participation in the Google Code-In<https://www.mediawiki.org/wiki/Google_Code-In>program ended up with great success: 273 tasks completed by 46 students with the help of about 30 mentors. Theo Patt and Mateusz Maćkowski were selected winners for Wikimedia<http://google-opensource.blogspot.com/2014/01/google-code-in-2013-drumroll-…>, and we sent a special mention to Mayank Madan. Round 7 of the FOSS Outreach Program for Women<https://www.mediawiki.org/wiki/Outreach_Program_for_Women/Round_7>started and all projects are on track so far: - Compacting interlanguage links<https://www.mediawiki.org/wiki/User:Niharika/Project_Progress_Report#Januar…> - MediaWiki Homepage Redesign<https://www.mediawiki.org/wiki/User:Monteirobrena/MediaWiki_Homepage_Redesi…> - Complete the MediaWiki API development course on Codecademy<https://www.mediawiki.org/wiki/User:Diwanshipandey/OPW_January_Report> - Clean up Parsoid round-trip testing UI<https://www.mediawiki.org/wiki/User:5xbe/OPW_Monthly_Progress_Reports#Janua…> - Clean up tracing/debugging/logging inside Parsoid<https://www.mediawiki.org/wiki/User:Mariapacana/OPW_Progress_Report#January…> - UploadWizard: OSM Embedding<https://www.mediawiki.org/wiki/User:Inchikutty/OPW_Internship_Report#January> Facebook Open Academy <https://www.mediawiki.org/wiki/Facebook_Open_Academy>‘s warm-up period saw a slow progress in the beginning of the projects. At the end it seemed that everybody was waiting for the official start at the kick-off in Facebook headquarters on February 7−9. *Technical communications <https://www.mediawiki.org/wiki/Technical_communications>* In January, Guillaume Paumier <https://www.mediawiki.org/wiki/User:Guillom>wrapped up work on mentoring Google Code-in <https://www.mediawiki.org/wiki/Google_Code-in> students and continued to provide ongoing communications support<https://www.mediawiki.org/wiki/Technical_communications/Tech_blog_activity>for the engineering staff. He contributed to writing, simplifying, publishing and distributing the weekly technical newsletter<https://meta.wikimedia.org/wiki/Tech/News>, and published an in-depth article<https://blog.wikimedia.org/2014/01/07/tech-news-fighting-technical-informat…>explaining the process by which the newsletter is put together every week. *Volunteer coordination and outreach <https://www.mediawiki.org/wiki/Volunteer_coordination_and_outreach>* We helped organizing the Architecture Summit 2014<https://www.mediawiki.org/wiki/Architecture_Summit_2014>in San Francisco (January 23−24) and we got everything ready for FOSDEM <https://www.mediawiki.org/wiki/Events/FOSDEM> in Brussels (February 1−2). We continued working with the tech community metrics<https://www.mediawiki.org/wiki/Community_metrics>around two key performance indicators: who contributes code<http://korma.wmflabs.org/browser/who_contributes_code.html>, and the Gerrit review queue<http://korma.wmflabs.org/browser/gerrit_review_queue.html> . Analytics <https://www.mediawiki.org/wiki/Analytics> *Kraken <https://www.mediawiki.org/wiki/Analytics/Kraken>* The team has been monitoring the mobile stream and adding additional load to Kafka which has exposed some scaling issues. These have been resolved. In addition, work has been done with the Operations team on designing and implementing a Java deployment system for use with Hadoop and other systems. Finally, work has been initiated to use the data in the warehouse on mobile browser distribution and session length. *Limn <https://www.mediawiki.org/wiki/Analytics/Limn>* Usability issues continue to be addressed while the team explores options around other visualization frameworks. This month we implemented a feature that simplifies the creation of dashboard by automatically inferring metadata from the data source. *Wikimetrics <https://www.mediawiki.org/wiki/Analytics/Wikimetrics>* We are adding features to Wikimetrics to support scheduled jobs and data access via evergreen URLs. This will support dashboarding and other services that are will be built on top of the service. In addition, we are preparing a Wikimetrics-Vagrant image to help getting started with Wikimetrics development. *Kafka <https://www.mediawiki.org/wiki/Analytics/Logging_infrastructure>* We’ve increased the throughput on Kafka from 6K/RPS to 50K/RPS to test stability under higher loads. *Data Quality <https://www.mediawiki.org/wiki/Analytics/Data_Quality>* <https://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdf> <https://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdf> Review of 2013 traffic trends by the Wikimedia Analytics Team. The team has spent an intense month analyzing data to explain the page view issues identified in December. The team’s report<https://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdf>was shared at the February metrics meeting. *Research and Data <https://www.mediawiki.org/wiki/Analytics/Research_and_Data>* We conducted a thorough review of traffic data and trends<https://commons.wikimedia.org/wiki/File:2013_Wikimedia_traffic_trends.pdf>and confirmed a downward trend in desktop pageviews in 2013. This trend is not reflected in desktop unique visitors or mobile traffic. We are working on complementing pageviews with other traffic metrics that will help us better monitor readership trends. We engaged with external parties (Google and comScore) to obtain data about referral and mobile traffic respectively. We completed research on article creation trends<https://meta.wikimedia.org/wiki/Research:Wikipedia_article_creation>on the largest Wikipedias and found substantial differences between different language Wikipedias; specifically, where anonymous editors are allowed to create articles, their success rate (% of articles kept) is substantially higher than that of newly registered editors. We also found that articles that started as Articles for Creation (AfC) and userspace drafts have a near 100% success rate, but the transition that English Wikipedia made toward directing newcomers to start AfC drafts appears to have substantially reduced the amount of successful articles created by newcomers, presumably due to the large review backlog. We published an update<https://meta.wikimedia.org/wiki/Research:Visual_editor_usage_%28January,_20…>on Visual Editor usage on Wikipedia projects where the editor is enabled by default. We continued work on metrics standardization for the editor engagement vital signs<https://www.mediawiki.org/wiki/Analytics/Epics/Editor_Engagement_Vital_Signs>project and published supportive analysis on definitions and parameter exploration for two proposed standardized user classes: new editor<https://meta.wikimedia.org/wiki/Research:New_editor>and productive new editor <https://meta.wikimedia.org/wiki/Research:Productive_new_editor>. We worked with the Analytics Development and Legal teams to articulate use cases and the retention and anonymization strategy<https://meta.wikimedia.org/wiki/Talk:Data_retention_guidelines>for data subject to the retention guidelines, in particular with respect to user agents <https://www.mediawiki.org/wiki/EventLogging/UserAgentSanitization>. We welcomed Sahar Massachi as a research contractor supporting the team with data analysis for fundraising tests and iterated on new modeling strategies for estimating test success (such as the number of dollars per banner impression). Before he joined us, Sahar worked with the fundraising team, where most recently he focused on writing tools to help the team easily and quickly understand the results of each test. Offline *Kiwix <http://www.kiwix.org>* *The Kiwix project is funded and executed by Wikimedia CH <https://meta.wikimedia.org/wiki/Wikimedia_CH>.* Much time this month was spent planning for 2014. We mainly worked on mwoffliner<https://sourceforge.net/p/kiwix/other/ci/master/tree/mwoffliner/>and almost managed to create a full English Wikipedia ZIM file with thumbnails. The upgrade of our main storage platform allowed us to start our automatic ZIM file generation system. Wikidata <https://meta.wikimedia.org/wiki/Wikidata> *The Wikidata project is funded and executed by Wikimedia Deutschland <https://meta.wikimedia.org/wiki/Wikimedia_Deutschland/en>.* In January, the team worked mainly on performance improvements around Wikidata. The Quantities datatype was deployed so it is now possible to enter data like the number of inhabitants of a country. Wikisource can now manage its language links via Wikidata as well just like Wikipedia, Wikivoyage and Commons could already. Two new front-end developers, Adrian and Thiemo joined the team to help improve Wikidata’s user interface. Last but not least, the team released their plan for the development of Wikidata in 2014 and beyond <https://www.wikidata.org/wiki/Wikidata:Development_plan> . Future The engineering management team continues to update the *Deployments <https://wikitech.wikimedia.org/wiki/Deployments>* page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the *annual goals <https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals>*, listing ongoing and future Wikimedia engineering efforts. ------------------------------ *This article was written collaboratively by Wikimedia engineers and managers. See revision history <https://www.mediawiki.org/w/index.php?title=Wikimedia_engineering_report/20…> and associated status pages. A wiki version <https://www.mediawiki.org/wiki/Wikimedia_engineering_report/2014/January> is also available.* -- Guillaume Paumier Technical Communications Manager — Wikimedia Foundation https://donate.wikimedia.org

1 0

Tip for newcomers and experts: MediaWiki Virtual Libray (MVL) "books": Developer's Guide, Security Guide...
by Thomas Gries 12 Feb '14

12 Feb '14

https://www.mediawiki.org/wiki/MVL offers ready-for download PDF and ebook versions of up-to-date collections of articles like MediaWiki Developer's Guide (MDG) MediaWiki Security Guide (MSG) and more.

1 0

Presenting inline diffs
by Max Semenik 12 Feb '14

12 Feb '14

Hi, I'd like to present you another RFC: https://www.mediawiki.org/wiki/Requests_for_comment/Inline_diffs It proposes to integrate into MediaWiki inline diffs from MobileFrontend. -- Best regards, Max Semenik ([[User:MaxSem]])

1 0

← Newer
1
...
9
10
11
12
13
14
15
16
17
Older →

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l February 2014