Wikitech-l June 2014

wikitech-l@lists.wikimedia.org

139 participants
147 discussions

Roadmap and deployment updates - week of June 9th
by Greg Grossmeier 06 Jun '14

06 Jun '14

Hello and welcome to the latest edition of the WMF Engineering Roadmap and Deployment update. The full log of planned deployments next week can be found at: <https://wikitech.wikimedia.org/wiki/Deployments#Week_of_June_9th> A quick list of notable items... == Monday == * ElasticSearch ** There will be an upgrade of the backend/server software of the new search system (to ElasticSearch 1.2.1). This should not have any user facing impact. == Tuesday == * MediaWiki deploy ** group1 to 1.24wmf6: All non-Wikipedia sites (Wiktionary, Wikisource, Wikinews, Wikibooks, Wikiquote, Wikiversity, and a few other sites) ** <https://www.mediawiki.org/wiki/MediaWiki_1.24/wmf8> * WikiQuote/WikiData: WikiQuote will now have access to data in WikiData. == Thursday == * MediaWiki deploy ** group2 to 1.24wmf8 (all Wikipedias) ** group0 to 1.24wmf9 (test/test2/testwikidata/mediawiki) * MediaViewer ** Will be enabled on by default on all Wikis ** <https://www.mediawiki.org/wiki/Multimedia/Media_Viewer/Release_Plan#Timeline> Thanks and as always, questions and comments welcome, Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |

1 0

Re: [Wikitech-l] [Wikitech-ambassadors] Hovercards - Request for feedback and volunteer-wikis
by Jared Zimmerman 06 Jun '14

06 Jun '14

One of Dan's next steps is coordinating with a few language wikis for a ~month long public rollout test period, similar to the way mediaviewer did. I agree with you Erik, that there should be a way to disable for readers, the team is thinking about possible alternatives currently. As well as making sure that logged in users have full control over using hovercards, navigation popups, or turning off link previews all together. Jan, we can possibly plan on including sverige wiki in the pilot program if that works for Dan and the community there, however the beta feature is really more geared toward content wikis (wikipedia, wiktionary) rather than other wiki types. *Jared Zimmerman * \\ Director of User Experience \\ Wikimedia Foundation M +1 415 609 4043 \\ @jaredzimmerman <http://loo.ms/g0> On Fri, Jun 6, 2014 at 6:44 AM, Jan Ainali <jan.ainali(a)wikimedia.se> wrote: > Very nice! > > You can turn it on the Swedish chapter wiki se.wikimedia.org > > > *Best regards, Jan Ainali* > > CEO, Wikimedia Sverige <http://se.wikimedia.org/wiki/Huvudsida> > +46 729 67 29 48 > > > *Tänk dig en värld där varje människa har fri tillgång till mänsklighetens > samlade kunskap. Det är det vi gör.* > Bli medlem. <http://blimedlem.wikimedia.se> > > > > 2014-06-06 0:10 GMT+02:00 Nick Wilson (Quiddity) <nwilson(a)wikimedia.org>: > >> The Hovercards Beta Feature [1] is inspired by the Navigation Popups >> gadget. The goal of Hovercards is to make the reading experience better for >> our readers. When readers hover over links to other articles, they are >> provided a short summary and image. They can decide whether they need to >> visit that subject more fully before continuing the current subject. >> >> We are looking for local project wikis who are interested in turning on >> the Hovercards feature for all readers, ahead of the future Beta Feature >> graduation. Please let us know, if your wiki might be interested in this. >> >> We're also looking for additional feedback on all aspects of the >> extension.[2] >> >> - >> >> [1] https://www.mediawiki.org/wiki/Beta_Features/Hovercards >> [2] https://www.mediawiki.org/wiki/Talk:Beta_Features/Hovercards >> >> Sidenotes: >> * The recent flickering-bug (primarily in Firefox) was fixed today. >> * There is a gerrit patch which is examining the conflict of Navigation >> Popups and Hovercards. https://gerrit.wikimedia.org/r/#/c/120188/ >> * We also plan to suggest some potential updates to the styles in the >> original Navpopups gadget, so that it has better usability and is visually >> consistent with Hovercard and Reference-tooltip styles. >> >> -- >> >> Quiddity >> Community Liaison >> >> _______________________________________________ >> Wikitech-ambassadors mailing list >> Wikitech-ambassadors(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors >> > > > _______________________________________________ > Wikitech-ambassadors mailing list > Wikitech-ambassadors(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikitech-ambassadors > >

1 0

Hardening WP/WM against traffic analysis (take two)
by Zack Weinberg 06 Jun '14

06 Jun '14

I'd like to restart the conversation about hardening Wikipedia (or possibly Wikimedia in general) against traffic analysis. I brought this up ... last November, I think, give or take a month? but it got lost in a larger discussion about HTTPS. For background, the type of attack that it would be nice to be able to prevent is described in this paper: http://sysseclab.informatics.indiana.edu/projects/sidebuster/sidebuster-fin… Someone is eavesdropping on an encrypted connection to LANG.wikipedia.org. (It's not possible to prevent the attacker from learning the DNS name and therefore the language the target reads, short of Tor or similar. It's also not possible to prevent them from noticing accesses to ancillary servers, e.g. Commons for media.) The attacker's goal is to figure out things like * what page is the target reading? * what _sequence of pages_ is the target reading? (This is actually easier, assuming the attacker knows the internal link graph.) * is the target a logged-in user, and if so, which user? * did the target just edit a page, and if so, which page? * (... y'all are probably better at thinking up these hypotheticals than me ...) Wikipedia is different from a tax-preparation website (the case study in the above paper) in that all of the content is public, and edit actions are also public. The attacker can therefore correlate their eavesdropping data with observations of Special:RecentChanges and the like. This may mean it is impossible to prevent the attacker from detecting edits. I think it's worth the experiment, though. What I would like to do, in the short term, is perform a large-scale crawl of one or more of the encyclopedias and measure what the above eavesdropper would observe. I would do this over regular HTTPS, from a documented IP address, both as a logged-in user and an anonymous user. This would capture only the reading experience; I would also like to work with prolific editors to take measurements of the traffic patterns generated by that activity. (Bot edits go via the API, as I understand it, and so are not reflective of "naturalistic" editing by human users.) With that data in hand, the next phase would be to develop some sort of algorithm for automatically padding HTTP responses to maximize eavesdropper confusion while minimizing overhead. I don't yet know exactly how this would work. I imagine that it would be based on clustering the database into sets of pages with similar length but radically different contents. The output of this would be some combination of changes to MediaWiki core (for instance, to ensure that the overall length of the HTTP response headers does not change when one logs in) and an extension module that actually performs the bulk of the padding. I am not at all a PHP developer, so I would need help from someone who is with this part. What do you think? I know some of this is vague and handwavey but I hope it is at least a place to start a discussion. zw

8 7

Hovercards - Request for feedback and volunteer-wikis
by Nick Wilson (Quiddity) 05 Jun '14

05 Jun '14

The Hovercards Beta Feature [1] is inspired by the Navigation Popups gadget. The goal of Hovercards is to make the reading experience better for our readers. When readers hover over links to other articles, they are provided a short summary and image. They can decide whether they need to visit that subject more fully before continuing the current subject. We are looking for local project wikis who are interested in turning on the Hovercards feature for all readers, ahead of the future Beta Feature graduation. Please let us know, if your wiki might be interested in this. We're also looking for additional feedback on all aspects of the extension.[2] - [1] https://www.mediawiki.org/wiki/Beta_Features/Hovercards [2] https://www.mediawiki.org/wiki/Talk:Beta_Features/Hovercards Sidenotes: * The recent flickering-bug (primarily in Firefox) was fixed today. * There is a gerrit patch which is examining the conflict of Navigation Popups and Hovercards. https://gerrit.wikimedia.org/r/#/c/120188/ * We also plan to suggest some potential updates to the styles in the original Navpopups gadget, so that it has better usability and is visually consistent with Hovercard and Reference-tooltip styles. -- Quiddity Community Liaison

1 0

Help: Needed in OAuth
by Amanpreet Singh 05 Jun '14

05 Jun '14

Hi, I need some help regarding my GSoC project in which I need to implement an OAuth login system for a browser based plugin, so we can identify users. But I am stuck and not able to get anything here > https://github.com/apsdehal/Mediawiki-login-api/blob/master/lib/OAuth/MWOAu… . Kindly help me, and tell me if further information is needed. -- Amanpreet Singh, IIT Roorkee

4 9

MediaWiki RC3 & 1.23.0 (Special:SpecialPages seg fault)
by Thomas Fellows 05 Jun '14

05 Jun '14

Hey All, Upgraded from 1.20 to 1.23rc3 yesterday, ran update.php, everything worked fine -- except for Special:SpecialPages. Disabled all extensions to see if that was doing it, but no change. It comes back with a Connection Reset error. I went to Apache's error logs and came across this for each failed attempt: [Thu Jun 05 09:10:51 2014] [notice] child pid 18983 exit signal Segmentation fault (11) Uh oh. Thought that maybe 1.23.0 would fix the issue, woke up this morning, updated again, and the page loaded!! However, with warnings: PHP Warning: Illegal offset type in isset or empty in /var/www/ops_dev/includes/User.php on line 1390 PHP Warning: Illegal offset type in isset or empty in /var/www/ops_dev/includes/User.php on line 1390 PHP Warning: Illegal offset type in isset or empty in /var/www/ops_dev/includes/User.php on line 1390 PHP Warning: Illegal offset type in isset or empty in /var/www/ops_dev/includes/User.php on line 1390 PHP Warning: Illegal offset type in isset or empty in /var/www/ops_dev/includes/User.php on line 1390 I thought, great! Maybe one of the extensions is causing the error. So I went and disabled all extensions, and reloaded the page and got the 'Connection reset'. [Thu Jun 05 09:15:11 2014] [notice] child pid 19200 exit signal Segmentation fault (11) Re-enabled extensions, Seg fault still. Tried again a few seconds ago, Warning: Illegal offset type in isset or empty in /var/www/ops_dev/includes/User.php on line 1390 (x10) Any help would be greatly appreciated! Thanks, Tom ---------------------------------------------------------------more info MediaWiki 1.23.0 <https://www.mediawiki.org/wiki/MediaWiki_1.23> (1346cdb) <https://git.wikimedia.org/commit/mediawiki%2Fcore.git/1346cdbba7e7f4560b572…> 16:57, 4 June 2014 PHP 5.3.6-13ubuntu3.10 (apache2handler) MySQL 5.1.52 Ubuntu 11.10 Server version: Apache/2.2.20 (Ubuntu) Server built: Mar 8 2013 15:58:04 Server's Module Magic Number: 20051115:28 Server loaded: APR 1.4.5, APR-Util 1.3.12 Compiled using: APR 1.4.5, APR-Util 1.3.12 Architecture: 64-bit Server MPM: Prefork threaded: no forked: yes (variable process count)

1 1

Using Composer to manage libraries for mediawiki/core on Jenkins and Foundation cluster
by Bryan Davis 04 Jun '14

04 Jun '14

My logging changes [0][1][2][3] are getting closer to being mergeable (the first has already been merged). Tony Thomas' Swift Mailer change [4] is also progressing. Both sets of changes introduce the concept of specifying external library dependencies, both required and suggested, to mediawiki/core.git via composer.json. Composer can be used by people directly consuming the git repository to install and manage these dependencies. I gave a example set of usage instructions in the commit message for my patch that introduced the dependency on PSR-3 [0]. In the production cluster, on Jenkins job runners and in the tarball releases we will want a different solution. My idea of how to deal with this is to create a new gerrit repository (mediawiki/core/vendor.git?) that contains a composer.json file similar to the one I had in patch set 7 of my first logging patch [5]. This composer.json file would be used to tell Composer the exact versions of libraries to download. Someone would manually run Composer in a checkout of this repository and then commit the downloaded content, composer.lock file and generated autoloader.php to the repository for review. We would then be able to branch and use this repository as git submodule in the wmf/1.2XwmfY branches that are deployed to production and ensure that it is checked out along with mw-core on the Jenkins nodes. By placing this submodule at $IP/vendor in mw-core we would be mimicking the configuration that direct users of Composer will experience. WebStart.php already includes $IP/vendor/autoload.php when present so integration with the rest of wm-core should follow from that. It would also be possible to add this repo to the tarballs for distribution. There will probably need to be some adjustments for that process however and the final result may be that release branches update the mediawiki/core composer.json and provide a composer.lock along with a pre-populated vendor directory. I would be glad to participate in discussions of that use case, but we will have about 6 months before we need to solve it (and a new release management RFC to resolve between now and then). There are several use cases to consider for the general solution: == Adding/updating a library == * Update composer.json in mediawiki/core/vendor.git * Run `composer update` locally to download library (and dependencies) * Run `composer dump-autoload --optimize` to make an optimized autoloader.php * Commit changes * Push changes for review in gerrit == Hotfix for an external library == At some point we will run into a bug or missing feature in a Composer managed library that we need to work around with a patch. Obviously we will attempt to upstream any such fixes (otherwise what's the point of this whole exercise?). To keep from blocking things for our production cluster we would want to fork the upstream, add our patch for local use and upstream the patch. During the time that the patch was pending review in the upstream we would want to use our locally patched version in production and Jenkins. Composer provides a solution for this with its "repository" package source. The Composer documentation actually gives this exact example in their discussion of the "vcs" repository type [6]. We would create a gerrit repository tracking the external library, add our patch(es), adjust the composer.json file in mediawiki/core/vendor.git to reference our fork, and finally run Composer in mediawiki/core/vendor.git to pull in our patched version. == Adding a locally developed library == The Platform Core team has been talking about extracting libraries from mw-core and/or extensions to be published externally. This may be done for any and all of the current $IP/includes/libs classes and possibly other content from core such as FormatJson. My idea for this would be to create a new gerrit repository for each exported project. The project repo would contain a composer.json manifest describing the project correctly to be published at packagist.org like most Composer installable libraries. In the mediawiki/core/vendor.git composer.json file we would pull these libraries just like any third-party developed library. This isn't functionally much different than the way that we use git submodules today. There is one extra level of indirection when a library is changed. The mediawiki/core/vendor.git will have to be updated with the new library version before the hash for the git submodule of mediawiki/core/vendor.git is updated in a deploy or release branch. == wmf/1.XwmfY branches == The make-wmf-branch script (found in mediawiki/tools/release.git) is used to create the weekly release branches that are deployed by the "train" on each Thursday. This script would be updated to branch the new mediawiki/core/vendor.git repository and add the version appropriate branch as a submodule of mediawiki/core.git on the wmf/* branch. This is functionally exactly what we do for extensions today. == Updating a deployment branch == SWAT deploys often deploy bug fixes for extensions and core that can't wait for the next train release. It is a near certainty that mediawiki/core/vendor.git will have the same need. The process for updating mediawiki/core/vendor.git will be almost the same as updating an extension. * Follow the adding/updating library or hotfix instructions to get the changes merged into the mediawiki/core/vendor.git master branch. * Cherry-pick the change into the proper deployment branch * Merge the cherry-pick * Update the git submodule for mediawiki/core/vendor.git in the appropriate deployed branch * Pull update to tin * sync-dir to deploy to cluster == Security fixes == This is a special case of upstreaming a patch. A security patch would be applied directly on the deployed branch of mediawiki/core/vendor.git as we would do for any extension. The vulnerability and patch must then be submitted upstream in a responsible manner and tracked for resolution. == Jenkins == The Jenkins jobs that checkout and run tests involving mediawiki/core would need to be amended to also checkout the mediawiki/core/vendor.git in the appropriate location before running tests. What use cases did I miss? What other concerns do we have for this process? [0]: https://gerrit.wikimedia.org/r/#/c/119939/ [1]: https://gerrit.wikimedia.org/r/#/c/119940/ [2]: https://gerrit.wikimedia.org/r/#/c/119941/ [3]: https://gerrit.wikimedia.org/r/#/c/119942/ [4]: https://gerrit.wikimedia.org/r/#/c/135290/ [5]: https://gerrit.wikimedia.org/r/#/c/119939/7/libs/composer.json,unified [6]: https://getcomposer.org/doc/05-repositories.md#vcs Bryan -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

7 15

Re: [Wikitech-l] enwiki display issues
by Helder . 04 Jun '14

04 Jun '14

Shouldn't we have an incident report[1] about this? Helder [1] https://wikitech.wikimedia.org/wiki/Incident_documentation#Incident_reports

3 3

Zuul upgraded
by Antoine Musso 04 Jun '14

04 Jun '14

Hello, I have upgraded Zuul (our gateway between Jenkins and Gerrit) a few minutes a go. The main change is that the merge of patches with tip of the target branch is now done in a different process (zuul-merge). Beside that new feature, I don't expect any trouble. But we never know hence this short announcement =) -- Antoine "hashar" Musso

1 0

New tabs for images
by ENWP Pine 04 Jun '14

04 Jun '14

So here I am working late night / early morning trying to get the Signpost published, and I see something new at the top of image pages on English Wikipedia such as https://en.wikipedia.org/wiki/File:Wikidata.png We now have "View on Wikimedia Commons" and "Add local description" tabs. Cool! Can we get the local description tab to appear on Commons also? Also, because the local description tab opens a free-text entry space, can that tab be split into "Add location" and "Add description" with the former opening up places to enter geolocation data, nearby landmarks, or an address? Pine

4 3

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l June 2014