Wikitech-l July 2015

wikitech-l@lists.wikimedia.org

97 participants
75 discussions

MediaWiki, as seen on TV
by Steven Walling 13 Sep '15

13 Sep '15

There's a pretty hilarious American police procedural TV show in 2015 called "CSI: Cyber", featuring mostly cybercrime. Obviously they have to dredge up snippets of code from places for screenshots on the show. Episode 4 happened to include a tidbit from MediaWiki 1.25/wmf3. Supposedly the code was a hack to make your printer blow up. Original lulz and screenshots via http://moviecode.tumblr.com/post/114815574587/this-is-from-csi-cyber-s01e04…

6 5

MediaWiki user survey and un-reachable users
by Mark A. Hershberger 17 Aug '15

17 Aug '15

We haven't announced this survey on wikitech-l yet, so if you run a wiki outside of those run by the WMF, please take the time to fill out the survey at <http://hexm.de/MWSurvey>. More information about the survey and its purpose can be found at <https://www.mediawiki.org/wiki/2015_MediaWiki_User_Survey>. That said, we received a report (T104010) from the Analytics team today that most downloads of MediaWiki are coming from China. The report indicated there were twice as many downloads from China as the U.S. in June. I don't know of a good way to reach these users -- I wasn't even aware of them till today. Through our efforts at outreach so far we've uncovered a number of private wikis that we wouldn't have been able to discover otherwise, but I'd like to extend our reach even farther. Can we add a link to the survey to the top of https://www.mediawiki.org/wiki/Download until the end of July? -- Mark A. Hershberger NicheWork LLC 717-271-1084

3 4

Bikeshedding a good name for "the api.php API"
by Brad Jorsch (Anomie) 11 Aug '15

11 Aug '15

When api.php was basically the only API in MediaWiki, calling it "the API" worked well. But now we've got a Parsoid API, Gabriel's work on a REST content API, Gabriel's work on an internal storage API, and more on the way. So just saying "the API" is getting confusing. So let's bikeshed a reasonably short name for it that isn't something awful like "the api.php API". I'm horrible at naming. -- Brad Jorsch (Anomie) Software Engineer Wikimedia Foundation

30 38

[recommended reading] do terrible things to your code
by Brian Gerstle 31 Jul '15

31 Jul '15

...or your users will! "Doing Terrible Things To Your Code <http://blog.codinghorror.com/doing-terrible-things-to-your-code/>" is a good read on testing by Jeff Atwood on his blog, Coding Horror <http://blog.codinghorror.com/>. I also found the "falsehood" snippets poignant—maybe we should come up with some for Wikipedia ;-). Here are a couple off the top of my head, at least for "official" Wikipedia instances: 1. Wikipedia sites all have standard ISO/BFC prefixes (see sitematrix <https://www.mediawiki.org/wiki/Special:SiteMatrix>) 2. A site's main page is always titled "Main Page" (also see sitematrix) 3. A page's revision is a reliable snapshot of its content (nope: transclusions [and images?] can change independent of a page revision) 4. API error messages are plain text (nope, can contain HTML <https://phabricator.wikimedia.org/T107082>) Interested in hearing "falsehoods" you've encountered. Cheers, Brian -- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle

3 2

Measuring performance
by Jon Robson 30 Jul '15

30 Jul '15

The reading team are currently focusing energy on speeding up the site for all our users (https://phabricator.wikimedia.org/T98986 is the tracking bug where this work can be followed) Off the back of https://phabricator.wikimedia.org/T105361 I had a quick chat with Ori to document how the performance team is currently identifying problems with MediaWiki's code. I'm sharing here, so anyone who is interested in helping us improve the time our users can load our content can analyse our data, raise tasks, and submit patches. I'm hoping this will be useful for anyone who wants to get involved in an effort to make our site faster for our users (this is not desktop specific). If you have anything useful to add please do, after some discussion or nods I'd love to share some best practices on mediawiki.org Tool 1) Use http://webpagetest.org (no credentials necessary) * Use https://en.wikipedia.org/wiki/Facebook as an example wiki page * Choose a region of the world and browser * Select first view only since this is what we are currently interested in (repeat view is when they load again - and it should be quicker as it is from cache). * Capture video can be turned off - I personally find the screenshots more useful To shout out some of the advanced settings, the more interesting/useful features include: *Chrome > capture dev tools timeline * Setting speed 3G or 2G * Script can be used to conditionally turn on things which are not yet available to everyone e.g. VisualEditor You can do a lot of this in your Chrome browser locally, but different browsers may have different behaviours and are in a fixed location so this does not get captured in this tool. The visual screenshots also make it easier to see where things get blocked. With the timeline from advanced tools you can match up white screens with blocking scripts/styles Tool 2) Add http://performance.wikimedia.org to your browser bookmarks. Navigation timing section is probably the most interesting right now. It points to https://grafana.wikimedia.org (no credentials needed) which is powered by http://graphite.wikimedia.org (Access graphite with your wikitech credentials). This data is sourced from our users, so is a good representation of how we are doing. If a graph is missing you can create a new one from data in graphite by clicking "add row" or editing an existing graph. Clicking edit on https://grafana.wikimedia.org/#/dashboard/db/navigation-timing?panelId=12&f… you'll be able to understand where the data comes from on graphite e.g. metrics/frontend/navtiming/totalPageLoadTime Note for graphs median data is less sensitive to edge cases so best to use this as a more realistic indicator. Folders in graphite, are populated by scripts that live in: https://github.com/wikimedia/operations-puppet/tree/production/modules/webp… To create a graph, simply go to an existing workboard, save it under a different name (this clones it) - don't worry you can't mess up and delete existing workboards. Tool 3) Speedcurve requires you to setup an account but it gives you an opiniated view about things you care about and is nicely presented so could be a good source of inspiration for your own grafana dashboard. To oversimplify what it does: each day it will access a page, store result, allow you to see historic data. Note the performance team has plans to setup infrastructure to automate this. Tool 4) is one we are not using - http://sitespeed.io. We might want to use it for performance regressions test. In the grand scheme of things it would be great to get to a place where Jenkins complains if you cause a regression in firstPaint time but we are a long way from that but let's work in that direction :-) Let's live up to the Hawaiian word after which we are named! Apologies if this is oversimplified, please take this as an opportunity to share how you/your team/your company test page performance. I see this mailing list as a good place to share these sort of things!

2 1

compressed old_text
by wp mirror 29 Jul '15

29 Jul '15

0) Context In the XML dump files, I get <text ...>plaintext</text>. When I build a mirror using XML dump files, I get: old_text: plaintext old_flags: utf-8 However, when I then create a new page on my mirror, I get: old_text: ciphertext old_flags: utf-8.gzip 1) Objective When I build a mirror, I would like to compress the <text ...>plaintext</text> to get: old_text: ciphertext old_flags: utf-8,gzip I would like this done for every text revision, so as to save both disk space and communication bandwidth between web server and browser. 2) Problem There is little relevant documentation on <https://www.mediawiki.org>. So I have run a few experiments. exp1) I pipe the plaintext through gzip, escape for MySQL, and build the mirror. However, when I browse, I get the message: ``The revision #165770 of the page named "Main Page" does not exist'' When I look in the database, some kind of ciphertext does indeed exist. 3) Variants Many utilities compress plaintext using LZ77 and Huffman encoding, but each differs as to the file header and tail. Some versions of deflate have no header at all. So I try four more experiments: exp2) gzip, but throw away the 10 byte header (to simulate deflate) /bin/gzip | tail -c +11 exp3) perl compress /usr/bin/perl -MCompress::Zlib -e 'undef $/; print compress(<>)' exp4) python compress, then throw away the single-quotes /usr/bin/python -c \"import zlib,sys;print repr(zlib.compress(sys.stdin.read()))\" | /bin/sed 's/^.//; s/.$//' exp5) zlib-flate from the qpdf DEB package /usr/bin/zlib-flate -compress For all experiments, the browser gives the same error message. 4) Reading compressed old_text It should be possible to read the old_text ciphertext using command-line tools. I created a user page which mediawiki stored compressed. It is displayed correctly by the browser. But when I tried to read it directly from the database, there were problems. (shell) mysql --host=localhost --user=root --password simplewiki --skip-column-names --silent --execute 'select old_text from simplewiki.text where old_id=5146705' | zlib-flate -uncompress Enter password: flate: inflate: data: incorrect data check 5) Request Please provide documentation as to how mediawiki handles compressed old_text. a) How is plaintext compressed? b) Is the ciphertext escaped for MySQL after compression? c) How does mediawiki handle old_flags=utf-8,gzip? d) How are the contents of old_text unescaped and decompressed for rendering? e) Where in the mediawiki code should I be looking to understand this better? SIncerely Yours, Kent

2 1

2015-07-29 Scrum of Scrums notes
by Grace Gellerman 29 Jul '15

29 Jul '15

https://www.mediawiki.org/wiki/Scrum_of_scrums/2015-07-29

3 2

RFC meeting this week
by Tim Starling 29 Jul '15

29 Jul '15

In the next RFC meeting, we will discuss the following RFC: * Content model storage <https://www.mediawiki.org/wiki/Requests_for_comment/Content_model_storage> The meeting will be on the IRC channel #wikimedia-office on chat.freenode.net at the following time: * UTC: Wednesday 21:00 * US PDT: Wednesday 14:00 * Europe CEST: Wednesday 23:00 * Australia AEST: Thursday 07:00 -- Tim Starling

1 0

Phabricator filter by language
by Petr Bena 29 Jul '15

29 Jul '15

Hi, What if we added extra projects to phabricator for programming languages (such as language-php, language-c) which could be optionally added to some tickets if help of people who know these languages would be needed. So that it would be possible for example to c++ experts to filter out open tasks that need c++ expert to look in them and so on? Currently I have few of such tasks that I would like to have experts in some language to look at, but there isn't really an easy way to do that. What you think? Should we add these meta-projects?

6 8

Wikipedia iOS app moving to GH
by Brian Gerstle 28 Jul '15

28 Jul '15

Hey everyone, I'm writing with plans for the Wikimedia iOS engineering team to move its workflow to GitHub with Travis CI, much like RESTbase. The Wikimedia iOS engineers have been maintaining their own CI and build server and using Gerrit for code review. The more time efficient and commonplace approach for open source iOS software development leans heavily on GitHub with Travis CI instead (e.g., WordPress[0][1] and Firefox[2][3]). By using GitHub with Travis CI, the team believes it will work faster, improve testing, grow developer confidence in making code changes, and, most importantly, deploy fewer bugs to production. For builds requiring sensitive information (e.g., prod certs), will continue to run on WMF's Mac Mini. As is done for Android, when betas are pushed, the team will notify mobile-l. Feel free to reply or email me directly with any questions or comments. Regards, Brian 0: https://github.com/wordpress-mobile/WordPress-iOS 1: https://travis-ci.org/wordpress-mobile/WordPress-iOS 2: https://github.com/mozilla/firefox-ios 3: https://travis-ci.org/mozilla/firefox-ios -- EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle IRC: bgerstle

11 27

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l July 2015