Wikitech-l July 2014

wikitech-l@lists.wikimedia.org

144 participants
151 discussions

MediaWiki's manual of style for occasional patch contributors
by Rainer Rillke 20 Jul '14

20 Jul '14

We have a number of contributors who do prefer contributing through web-interfaces or minimal installations to MediaWiki's code base. Often they run -1 with MediaWiki's coding conventions and do not have the time to set up commit hooks [1]. To save them some work re-indenting or adding a space between braces, I have decided to expose the excellent stylize.php from wikimedia tools-code-utils and js-beautifier as a web service at WMF labs. The result is a web interface: http://tools.wmflabs.org/stylize/ If you find a security vulnerable in the source code[2], you get a barnstar by me personally. Happy coding and don't forget to recommend it when -1ing for whitespace reasons! -- Rillke [1] https://www.mediawiki.org/wiki/Chemical_Markup_support_for_Wikimedia_Common… [2] https://github.com/Rillke/stylize

2 1

[GSOC 2014] Demo - Parsoid Based Linter
by hardik juneja 19 Jul '14

19 Jul '14

Hi Everyone, I am a GSOC student working with Parsoid Team [1] < https://www.mediawiki.org/wiki/Parsoid> to build a Parsoid based linter (Linttrap) [2] < https://www.mediawiki.org/wiki/Parsoid/Linting/GSoC_2014_Application>. Lintrap will detect broken wikitext found on the wiki pages and will also collect stats about certain wikitext usage patterns. Currently, for this demo, Lintrap can detect 4 types of broken wikitext, But, other kinds of issues could be detected in the coming weeks and months : * Fostered Wikitext : Eg [3] <http://lintbridge.wmflabs.org/_html/issues/53a2fe1e94641d9101f8e8b2> * Missing End Tag : Eg [4] <http://lintbridge.wmflabs.org/_html/issues/53a2fca994641d9101f8e72a> * Missing Start Tag : Eg [5] <http://lintbridge.wmflabs.org/_html/issues/53a2fdc394641d9101f8e850> * Stripped Tags : Eg [6] <http://lintbridge.wmflabs.org/_html/issues/53a2fd4394641d9101f8e819> Linttrap also collect information about transclusion usages where multiple templates are used to construct a DOM structure [7] < http://lintbridge.wmflabs.org/_html/issues/53a3026794641d9101f8ea81>. Here's our stats page [8] <http://lintbridge.wmflabs.org/stats>. Once a page is parsed, Lintrap uses parsoid based logger facility to log them to a web service. We call it Lintbridge [9] < http://lintbridge.wmflabs.org/>. Currently Lintbridge is hosted on Wikimedia Labs and use mongodb to store all the issues. Lintbridge offers a REST api which can be used by bots and other applications to fix the broken wikitext. Linttrap uses this REST api to store issues into Lintbridge. We have also built a HTML app on top of Lintbridge [10] < http://lintbridge.wmflabs.org/_html/issues>. This is a basic app for now which is used to demonstrate linttrap abilities. But, It is quite useful as it is today. Feel free to browse over the issues. * You can use the links in the table to filter the issues. * Click on issue type to filter issue by issue type. * You can filter issue by page too. * You can use Fix All to fix all issue for that page. * You can even use filters on the top bar to filter by Wiki and Type. * Each issue contain a info about wiki, page, revision on the left and the wikitext snippet on the right. Just for the demo of this working prototype, we have collected issues by parsing 1000 picked from http://parsoid-tests.wikimedia.org/topfails <http://parsoid-tests.wikimedia.org/topfails/0>. If you want to try the JSON API you can use the following routes. GET /_api/issues : Show all issues ( http://lintbridge.wmflabs.org/_api/issues) GET /_api/issues/type/issue-type : Filter by issue-type ( http://lintbridge.wmflabs.org/_api/issues/type/fostered) GET /_api/enwiki/issues : Filter by enwiki ( http://lintbridge.wmflabs.org/_api/enwiki/issues) POST /_api/add : Add a issue to the Lintbridge Inviting feedback. Thank you Hardik Juneja -- [1] Parsoid: https://www.mediawiki.org/wiki/Parsoid [2] Linttrap: https://www.mediawiki.org/wiki/Parsoid/Linting/GSoC_2014_Application [3] Fostered Ex : http://lintbridge.wmflabs.org/_html/issues/53a2fe1e94641d9101f8e8b2 [4] Missing End Tag eg : http://lintbridge.wmflabs.org/_html/issues/53a2fca994641d9101f8e72a [5] Missing Start Tag eg : http://lintbridge.wmflabs.org/_html/issues/53a2fdc394641d9101f8e85 <http://lintbridge.wmflabs.org/_html/issues/53a2fdc394641d9101f8e850>0 <http://lintbridge.wmflabs.org/_html/issues/53a2fdc394641d9101f8e850> [6] Stripped Tag eg : http://lintbridge.wmflabs.org/_html/issues/53a2fd4394641d9101f8e819 [7] Mixed Template eg : http://lintbridge.wmflabs.org/_html/issues/53a3026794641d9101f8ea81 [8] Stats Page : http://lintbridge.wmflabs.org/stats [9] Lintbridge: http://lintbridge.wmflabs.org/ [10] HTML App: http://lintbridge.wmflabs.org/_html/issues

3 4

image metadata not shown by MediaWiki
by Ricordisamoa 19 Jul '14

19 Jul '14

How does MediaWiki decide when to show the mw_metadata table in file description pages? The imageinfo API lists 3 different metadata types: metadata, commonmetadata and extmetadata. While commons:File:Gaivota - Portosín-5.jpg <https://commons.wikimedia.org/wiki/File:Gaivota_-_Portos%EDn-5.jpg> displays the EXIF info, commons:File:Chess klt45.svg <https://commons.wikimedia.org/wiki/File:Chess_klt45.svg> does not. However, the imageinfo query for the latter <https://commons.wikimedia.org/w/api.php?action=query&format=jsonfm&titles=F…> shows width and height. Are they just ignored by MediaWiki when rendering the final HTML? I am trying to replace an old function in pywikibot that searches for "mw_metadata".

2 1

Re: [Wikitech-l] [Wikimedia-l] Quarterly reviews of high priority WMF initiatives
by Tilman Bayer 19 Jul '14

19 Jul '14

Minutes and slides from Wednesday's quarterly review of the Foundation's MediaWiki Core team are now available at https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Quarterly_revi… (agenda/overview page: https://www.mediawiki.org/wiki/Wikimedia_MediaWiki_Core_Team/Quarterly_revi… ) On Wed, Dec 19, 2012 at 6:49 PM, Erik Moeller <erik(a)wikimedia.org> wrote: > Hi folks, > > to increase accountability and create more opportunities for course > corrections and resourcing adjustments as necessary, Sue's asked me > and Howie Fung to set up a quarterly project evaluation process, > starting with our highest priority initiatives. These are, according > to Sue's narrowing focus recommendations which were approved by the > Board [1]: > > - Visual Editor > - Mobile (mobile contributions + Wikipedia Zero) > - Editor Engagement (also known as the E2 and E3 teams) > - Funds Dissemination Committe and expanded grant-making capacity > > I'm proposing the following initial schedule: > > January: > - Editor Engagement Experiments > > February: > - Visual Editor > - Mobile (Contribs + Zero) > > March: > - Editor Engagement Features (Echo, Flow projects) > - Funds Dissemination Committee > > We’ll try doing this on the same day or adjacent to the monthly > metrics meetings [2], since the team(s) will give a presentation on > their recent progress, which will help set some context that would > otherwise need to be covered in the quarterly review itself. This will > also create open opportunities for feedback and questions. > > My goal is to do this in a manner where even though the quarterly > review meetings themselves are internal, the outcomes are captured as > meeting minutes and shared publicly, which is why I'm starting this > discussion on a public list as well. I've created a wiki page here > which we can use to discuss the concept further: > > https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings/Quarterly_r… > > The internal review will, at minimum, include: > > Sue Gardner > myself > Howie Fung > Team members and relevant director(s) > Designated minute-taker > > So for example, for Visual Editor, the review team would be the Visual > Editor / Parsoid teams, Sue, me, Howie, Terry, and a minute-taker. > > I imagine the structure of the review roughly as follows, with a > duration of about 2 1/2 hours divided into 25-30 minute blocks: > > - Brief team intro and recap of team's activities through the quarter, > compared with goals > - Drill into goals and targets: Did we achieve what we said we would? > - Review of challenges, blockers and successes > - Discussion of proposed changes (e.g. resourcing, targets) and other > action items > - Buffer time, debriefing > > Once again, the primary purpose of these reviews is to create improved > structures for internal accountability, escalation points in cases > where serious changes are necessary, and transparency to the world. > > In addition to these priority initiatives, my recommendation would be > to conduct quarterly reviews for any activity that requires more than > a set amount of resources (people/dollars). These additional reviews > may however be conducted in a more lightweight manner and internally > to the departments. We’re slowly getting into that habit in > engineering. > > As we pilot this process, the format of the high priority reviews can > help inform and support reviews across the organization. > > Feedback and questions are appreciated. > > All best, > Erik > > [1] https://wikimediafoundation.org/wiki/Vote:Narrowing_Focus > [2] https://meta.wikimedia.org/wiki/Metrics_and_activities_meetings > -- > Erik Möller > VP of Engineering and Product Development, Wikimedia Foundation > > Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate > > _______________________________________________ > Wikimedia-l mailing list > Wikimedia-l(a)lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l -- Tilman Bayer Senior Operations Analyst (Movement Communications) Wikimedia Foundation IRC (Freenode): HaeB

1 0

Fwd: HHVM deployment update
by Rob Lanphier 18 Jul '14

18 Jul '14

Hi everyone, The email below was written with an internal audience in mind, but Krenair pointed out that there would generally be a lot of general interest in this. Rob ---------- Forwarded message ---------- From: Rob Lanphier <robla(a)wikimedia.org> Date: Fri, Jul 18, 2014 at 5:20 PM Subject: HHVM deployment update To: WMF Engineering List, Operations Engineers Hi everyone, I'm writing to give you a quick update about where we are with HHVM deployment, so you know what to expect. Ori, Aaron, Tim, Giuseppe, Brett, Antoine and probably others I'm forgetting have been hard at work getting HHVM ready, and are about to make changes that may affect your work in production. The good news is that the way it'll most affect you is that your code will run a lot faster. The bad news is that there is some risk of breakage due to the number and nature of things we're changing. We've started referring to the stack that we're migrating to as "HAT" (HHVM, Apache 2.4, Trusty), as a nod to LAMP and as a useful tag on Gerrit changes. The point being: this is not merely a change in our PHP implementation, but a full stack upgrade that may have implications beyond just the problems that might be introduced by HHVM. The team is deploying this one piece at a time, with the first bit of the deployment happening very soon. Here's our rough timeline: * Now: limited deployments of production job runners to osmium, which the team only leaves on when they are monitoring it for errors * Week of July 21: Deployment to Beta Cluster. The timing on this may slip, since it might be a surprise to a few people who are deeply affected by it (/me waves to Chris McMahon), but we think it's generally ready from an engineering perspective. * Week of July 21: Deployment to a few job runners in production. You'll know the first job runner was deployed when you see this patch[1] get its +2. We didn't get a chance to coordinate this with Greg today, so exact timing is TBD. * Sometime later: Deployment to test.wikipedia.org application server * Sometime later: Deploy Varnish module allowing partial deployment to a fraction of application servers * Sometime later: Limited deployment to small number of application servers * Sometime later: Ramp up deployment to more application servers until most servers use HHVM * Sometime later: Deploy to remainder of services How to test your extension with HHVM: ------------------------------------- Historically, we've treated HHVM-related bugs in MediaWiki extensions as the sole responsibility of HHVM team, because we could not reasonably expect developers to test their code on HHVM while it was still difficult to build and configure. As we head toward full deployment, however, we are going to progressively shift responsibility onto you to be proactive about testing your code with HHVM and reporting any issues you encounter. If you're not sure how to test your code with HHVM, ask! The options that are currently available to you are: * On your machine, using MediaWiki-Vagrant (HHVM is the default PHP runtime) * On Labs, using Labs-Vagrant (<https://wikitech.wikimedia.org/wiki/Labs-vagrant>). The Flow team is doing this; ask them how. :) * Sometime next week: on the Beta cluster, when we switch it over to HHVM. Use the "hiphop" keyword in Bugzilla to catch our attentions. Some things that will change, and the associated challenges: * Lots of C++ code that is generally high-quality but doesn't have quite as many flight-hours logged in production as PHP. All that is entailed by that. * We expect the performance profile to improve substantially, but we can't rule out the possibility that specific operations will suffer performance regressions * Distribution-upgrade risks: there are many utilities we rely on besides MediaWiki itself, and many of those utilities will see upgrades as well. For example, a lot of the utilities on our image scalers (e.g. imagemagick, avconf, etc) will be upgraded. What we're doing to mitigate/minimize risk: test, test, test. A lot of the work that's been going on has been to improve the state of our unit tests such that we can have a clean test run before deploying all the way; a task made trickier by the fact that our current codebase doesn't meet that bar[2] That's all for now. More information about HHVM and our deployment to it can be found at https://www.mediawiki.org/wiki/HHVM . Anything that isn't there, come talk to me, and I'll turn around and ask Ori. :-) Thanks! Rob [1] Puppet repo patch "jobrunner: create hhvm-only jobrunners" https://gerrit.wikimedia.org/r/#/c/147086/ [2] Tracking bugs for unit tests that fail in HHVM (and unfortunately, our current production setup too): https://bugzilla.wikimedia.org/show_bug.cgi?id=67216

1 0

Include an updater
by Thomas Mulhall 18 Jul '14

18 Jul '14

Hi could we develop an in wiki upgrades meaning we could ether put the upgraded in the special version page or create its own special page and then add a notification to the special version page that tell them about upgrades for extensions and Mediawiki. It will help people migrate and know when updates are avalible instead of keep checking so for example a stable version would check for ref1_xx and the alpha one would look for the master branch.

2 2

Re: [Wikitech-l] [Wikimedia-l] [Wikitech-ambassadors] Deprecating print-on-demand functionality
by Matthew Walker 18 Jul '14

18 Jul '14

The new renderer should already be working in Hebrew and other RTLs. ~Matt Walker Wikimedia Foundation On Mon, Jul 14, 2014 at 2:16 PM, Itzik Edri <itzik(a)infra.co.il> wrote: > Any plans also to improve this module and make it work well also in Hebrew > (and maybe other RTL languages)? > > > On Fri, Jul 11, 2014 at 6:48 PM, Erik Moeller <erik(a)wikimedia.org> wrote: > > > On Fri, Jul 11, 2014 at 8:45 AM, Luca Martinelli > > <martinelliluca(a)gmail.com> wrote: > > > so the Book Creator will still be active, maybe under another name, > > > maybe with another engine, but still active? > > > > Same name and functionality, just the "Order a printed book" feature > > will disappear. > > > > Erik > > -- > > Erik Möller > > VP of Engineering and Product Development, Wikimedia Foundation > > > > _______________________________________________ > > Wikimedia-l mailing list, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > Wikimedia-l(a)lists.wikimedia.org > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe> > > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > Wikimedia-l(a)lists.wikimedia.org > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe> >

2 1

Hotmail rejecting mailing list messages
by Andrew "FastLizard4" Adams 18 Jul '14

18 Jul '14

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, devs! It appears that Hotmail may be blocking emails from Wikimedia, at least the mailing lists. Two users were unsubscribed forcibly from the accounts-enwiki-l mailing list due to excessive fatal bounces (username portion of addresses redacted): > [user1](a)hotmail.com > SMTP error from remote mail server after MAIL FROM:<accounts-enwiki-l-bounces(a)lists.wikimedia.org> SIZE=6776: > host mx2.hotmail.com [65.54.188.126]: 550 SC-004 (BAY004-MC4F5) Unfortunately, messages from 208.80.154.4 weren't sent. We recommend that you contact your Internet service provider. The problem is that too many unwanted messages have been sent from the following IP address above. You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. > [user2](a)live.com > SMTP error from remote mail server after MAIL FROM:<accounts-enwiki-l-bounces(a)lists.wikimedia.org> SIZE=6776: > host mx3.hotmail.com [65.55.33.135]: 550 SC-004 (COL004-MC6F21) Unfortunately, messages from 208.80.154.4 weren't sent. We recommend that you contact your Internet service provider. The problem is that too many unwanted messages have been sent from the following IP address above. You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. According to Microsoft's troubleshooting site: > Mail rejected by Outlook.com for policy reasons. A block has been placed against your IP address because we have received complaints concerning mail coming from that IP address. We recommend enrolling in our Junk Email Reporting Program (JMRP), a free program intended to help senders remove unwanted recipients from their email list. If you are not an email/network admin please contact your Email/Internet Service Provider for help. Is there anything that can be done about this, or should I just advise the users to switch to an alternate email provider? Thanks in advance for your help! - -- Sincerely, Andrew "FastLizard4" Adams <https://en.wikipedia.org/wiki/User:FastLizard4> <FastLizard4(a)gmail.com> GPG Key ID: 0x221A627DD76E2616 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTyNIhAAoJECIaYn3XbiYWtYQP/AwtzrF656wsZm/UvFx8pZBY Hf9UGSlG1jPsuMMWCPhK/3+5HkFXWb7fIn1BIJjfRlJmUeUkuuOCrtwAGbAaKEWb jWd+r5QMEt6kgj57TxEz/kJnrqAPafmLcb1N6KR2AealWkYBWGzqy+c9XcnyhDBa 5ZcceYsIjEEiOLtV6KVgiooPlKSeuH3CMQayE/2qYdKCRFkstok2V/FEAHfPB7uZ KJWP4jSVsRnYQwP4xVHK40lT8LydxgcjVY5IJ7i+wAX0RmzIN8IgCTVoLi+mJpI9 Ev/K1+6au5Yju9G+PHellTIPQMmgN84BotJNVSBlVit5zWhpcvhAricaGrmoNq6D kh+v8OOSyb9ZY3ocEe/K7KrW0BTIfFbE5mGX+kEwcpohw8349sl0JxnWEpFvq+Ri vObxcKav0dGDK2WgEyRmdZUbhL+PCfc5Yn5fAHQOanDSnu3HWBCvzQJGrrRnLW6Y u05615mdqa+QshEh3gFx4bxPjo4qR7wrmvXBW38h8MxoKADxaaxgbWWhz+G2G9cW /J1z6yxn2ZZSe8Yr34KxQ3T2+k97ueusi9YcV7TIRSQk/XE4OLiArg3HHVxN5vuf MlL5p8izYsQLSBpZDbaMiqzRMbw6Gws99eq8nXlCadI0rHcUABSZKWLWYKvAk41/ 7Tgv05ml0wuruTbvExq5 =wEyo -----END PGP SIGNATURE-----

3 2

Fwd: Tech Talk: Hadoop and Beyond. An overview of Analytics infrastructure, Tuesday!
by Rachel Farrand 18 Jul '14

18 Jul '14

Hello! Please join Nuria Ruiz and Andrew Otto next Tuesday, July 15th at 10am SF time/5pm UTC <http://www.timeanddate.com/worldclock/fixedtime.html?msg=Analytics+Tech+Tal…> for a 30 min tech talk. You can join our hangout or follow along on youtube: https://plus.google.com/u/0/b/103470172168784626509/events/c53ho5esd0luccd0… (please note that a link to join the hangout will be posted in the comments of this event just as it starts). You can follow ask questions on IRC during the talk in #wikimedia-dev. If you are not able to follow along live, a video recording will be posted here <https://plus.google.com/u/0/b/103470172168784626509/103470172168784626509/v…>, to the MediaWiki YouTube channel immediately following the tech talk for you to view at any time. More information about the tech talk: *Hadoop and Beyond. An overview of Analytics infrastructure*In this tech talk we will be presenting the analytics infrastructure that we have recently rolled out in production. By now probably everybody knows that wikimedia hosts an instance of hadoop from which we are going to extract pageview data in the near future. But .. how exactly does the data get there? We will go over the path that webrequest log data takes from varnish to kafka (a distributed log buffer) to hadoop and the challenges of deploying this java-based infrastructure in production. We will also talk about how can we query the data with hive, an SQL-like interface. How can you set up this stack on vagrant to play with and, last but not least, how we used hive recently to provide GLAM folks with image view stats: https://commons.wikimedia.org/wiki/Commons:GLAMwiki_Toolset_Project/NARA_an… Thanks!

2 2

Naigos and Icinga
by Pine W 18 Jul '14

18 Jul '14

Is there a plan to switch from Naigos to Icinga, or is that merely under discussion? Thanks, Pine

4 4

← Newer
1
...
5
6
7
8
9
10
11
...
16
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2014