Wikitech-l March 2014

wikitech-l@lists.wikimedia.org

188 participants
234 discussions

Re: [Wikitech-l] MediaWiki to Latex Converter
by Fred Bauder 17 Jul '14

17 Jul '14

I have a log of what happens on when the commands: sudo apt-get install mediawiki2latex mediawiki2latex -u https://en.wikipedia.org/wiki/Adam_Ries -o AdamRies.pdf are entered on the command line of ubuntu (13.10) Better than TV... Happy to send it to anyone. Fred

6 11

Changes in API action=parse&mobileformat=...
by Max Semenik 09 Jun '14

09 Jun '14

Hi, in response to bug 54607 [1], we've changed the semantics of the mobileformat parameter to action=parse == Summary == Previously, it used to accept strings 'html' or 'wml', later just 'html' and modify the structure of output (see below). This was problematic because you needed to retrieve the HTML from output in different ways, depending on whether mobileformat is specified or not. Now, mobileformat is a boolean parameter, that is if there's a 'mobileformat' parameter in request, it will be treated as "the output should be mobile-friendly", regardless of value. And the output structure will be the same. For compatibility with older callers, mobileformat=(html|wml) will be special-cased to return the older structure at least for 6 month from now. These changes will start being rolled out to the WMF sites starting from tomorrow, Tuesday October 24th and this process will be complete by October 31st. == Examples == === Non-mobile parse === api.php?action=parse&format=xml { "parse": { "title": "...", "text": { "*": "foo" } } } api.php?action=parse&format=json <?xml version="1.0"?> <api> <parse title="..." displaytitle="..."> <text xml:space="preserve">foo</text> </parse> </api> === Parse that outputs mobile HTML, old style === api.php?action=parse&format=json&mobileformat=html { "parse": { "title": "API", "text": "foo" } } api.php?action=parse&format=xml&mobileformat=html <?xml version="1.0"?> <api> <parse title="..." text="foo" displaytitle="..."> </parse> </api> === Parse that outputs mobile HTML, new style === api.php?action=parse&format=...&mobileformat Same as for non-mobile parses. == FAQ == Q: I didn't use mobileformat before, does anything change for me? A: No. Q: I use mobileformat=html, will my bot/tool be broken now? A: No, you will have 6 months to switch to new style. Q: I'm only planning to use mobileformat, what should I do? A: Just use the new style. Q: How did this format discrepancy appear in the first place? A: To err is human. ----- [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=54607 -- Best regards, Max Semenik ([[User:MaxSem]])

1 1

workflow to add multiple patches to gerrit.wikimedia.org:29418/operations/puppet.git
by Petr Bena 14 May '14

14 May '14

Currently I do: * clone a repo * setup git & hooks # patch 1: * apply my changes * commit * execute git-review # patch 2: * apply my changes * commit FAIL - the new commit it depending on previous commit - I can't push What am I supposed to do in order to push multiple separate patches? GIT-IDIOT way please, no long explanations, just commands and examples. Thanks

6 11

HTML templating systems & MediaWiki - is this summary right?
by Sumana Harihareswara 13 May '14

13 May '14

I'm trying to understand what our current situation is and what our choices are around HTML templating systems and MediaWiki, so I'm gonna note what I think I understand so far in this mail and then would love for people to correct me. TL;DR - did we already consense on a templating system and I just missed it? Description: An HTML templates system (also known as a templating engine) lets you (the programmer) write something that looks more like a document than it looks like code, then has hooks/entry points/macro substitution points (for user input and whatnot) that then invoke code, then emits finished HTML for the browser to render. Examples: PHP itself is kinda a templating language. In the PHP world, Smarty is a somewhat more mature/old-school choice. Mustache.js is a popular modern choice. And in other languages, you'd pick a lot of the MVC frameworks that are popular, e.g. Django or Jinja in Python. Spectrum of approaches: One approach treats HTML as a string ("here's a bunch of bytes to interpolate"). From a security perspective, this is dangerously easy to have vulnerabilities in, because you just naively insert strings. Then on the other end of the spectrum, you have code that always keeps the document object model (DOM) in memory, so the programmer is abstractly manipulating that data model and passing around an object. Sure, it spits out HTML in the end, but inherent in the method for turning those objects into HTML is a sanitization step, so that's inherently more secure. There's some discussion at https://www.mediawiki.org/wiki/Parsoid/Round-trip_testing/Templates . I presume we want the latter, but that the former model is more performant? We talked about this stuff in https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-02-21 and https://www.mediawiki.org/wiki/Talk:Architecture_Summit_2014/HTML_templatin… . Based on that plus https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_clusters#HTML_t… it seems like we are supposed to get consensus on which system(s) to use, and we kind of have four things we could choose: * oojs - https://www.mediawiki.org/wiki/OOjs_UI -- could use this toolkit with one of the template approaches below, or maybe this is enough by itself! Currently used inside VisualEditor and I am not sure whether any other MediaWiki extensions or teams are using it? This is a DOM-based templating system. Template approaches which are competing?: * MVC framework - Wikia has written their own templating library that Wikia uses (Nirvana). Owen Davis is talking about this tomorrow in the RFC review meeting. https://www.mediawiki.org/wiki/Requests_for_comment/MVC_framework * mustache.js stuff - Ryan Kaldari and Chris Steipp mentioned this I think? * Knockout-compatible implementation in Node.js & PHP https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library… and https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library… , being worked on by Gabriel Wicke, Matt Walker, and others. DOM-based. There's also an OutputPage refactor suggested in https://www.mediawiki.org/wiki/Requests_for_comment/OutputPage_refactor that's part of the HTML Templating RFC Cluster https://www.mediawiki.org/wiki/Architecture_Summit_2014/RFC_clusters#HTML_t… . I guess my biggest question right now is whether I have all the big moving parts right in my summary above. Thanks. -- Sumana Harihareswara Senior Technical Writer Wikimedia Foundation

12 37

Fate of hierarchical list on Special:Allpages
by Ori Livneh 11 May '14

11 May '14

Following change I189ba71de[0], the hierarchical list in Special:Allpages becomes a simple alphabetic pager if the total number of pages exceeds a safety threshold. The threshold is designed to protect wikis on which the load generated by the process of generating the hierarchical list would be prohibitively expensive (bug 56840[1]). I189ba71de resolved the immediate operational issue, but there is a further question of whether we want to keep the hierarchical list at all, especially given that it cannot be enabled (in its current implementation, at least) on larger installations. >From my perspective, the ideal outcome of this discussion would be that we agree that the hierarchical list is a poor fit for the MediaWiki of today, and we resolve to remove it from core. According to stats.grok.se, enwiki's Special:Allpages receives approximately 158 hits a day.[2] [0]: https://gerrit.wikimedia.org/r/#/c/94690/ [1]: https://bugzilla.wikimedia.org/show_bug.cgi?id=56840 [2]: http://stats.grok.se/en/latest90/Special:Allpages --- Ori Livneh ori(a)wikimedia.org

6 7

MediaWiki 1.19.0beta2
by Sam Reed 09 May '14

09 May '14

I'm happy to announce the availability of the second beta release of the new MediaWiki 1.19 release series. Please try it out and let us know what you think. Don't run it on any wikis that you really care about, unless you are both very brave and very confident in your MediaWiki administration skills. MediaWiki 1.19 is a large release that contains many new features and bug fixes. This is a summary of the major changes of interest to users. You can consult the RELEASE-NOTES-1.19 file for the full list of changes in this version. Five security issues were discovered. It was discovered that the api had a cross-site request forgery (CSRF) vulnerability in the block/unblock modules. It was possible for a user account with the block privileges to block or unblock another user without providing a token. For more details, see https://bugzilla.wikimedia.org/show_bug.cgi?id=34212 It was discovered that the resource loader can leak certain kinds of private data across domain origin boundaries, by providing the data as an executable JavaScript file. In MediaWiki 1.18 and later, this includes the leaking of CSRF protection tokens. This allows compromise of the wiki's user accounts, say by changing the user's email address and then requesting a password reset. For more details, see https://bugzilla.wikimedia.org/show_bug.cgi?id=34907 Jan Schejbal of Hatforce.com discovered a cross-site request forgery (CSRF) vulnerability in Special:Upload. Modern browsers (since at least as early as December 2010) are able to post file uploads without user interaction, violating previous security assumptions within MediaWiki. Depending on the wiki's configuration, this vulnerability could lead to further compromise, especially on private wikis where the set of allowed file types is broader than on public wikis. Note that CSRF allows compromise of a wiki from an external website even if the wiki is behind a firewall. For more details, see https://bugzilla.wikimedia.org/show_bug.cgi?id=35317 George Argyros and Aggelos Kiayias reported that the method used to generate password reset tokens is not sufficiently secure. Instead we use various more secure random number generators, depending on what is available on the platform. Windows users are strongly advised to install either the openssl extension or the mcrypt extension for PHP so that MediaWiki can take advantage of the cryptographic random number facility provided by Windows. Any extension developers using mt_rand() to generate random numbers in contexts where security is required are encouraged to instead make use of the MWCryptRand class introduced with this release. For more details, see https://bugzilla.wikimedia.org/show_bug.cgi?id=35078 A long-standing bug in the wikitext parser (bug 22555) was discovered to have security implications. In the presence of the popular CharInsert extension, it leads to cross-site scripting (XSS). XSS may be possible with other extensions or perhaps even the MediaWiki core alone, although this is not confirmed at this time. A denial-of-service attack (infinite loop) is also possible regardless of configuration. For more details, see https://bugzilla.wikimedia.org/show_bug.cgi?id=35315 ********************************************************************* What's new? ********************************************************************* MediaWiki 1.19 brings the usual host of various bugfixes and new features. Comprehensive list of what's new is in the release notes. * Bumped MySQL version requirement to 5.0.2. * Disable the partial HTML and MathML rendering options for Math, and render as PNG by default. * MathML mode was so incomplete most people thought it simply didn't work. * New skins/common/*.css files usable by skins instead of having to copy piles of generic styles from MonoBook or Vector's css. * The default user signature now contains a talk link in addition to the user link. * Searching blocked usernames in block log is now clearer. * Better timezone recognition in user preferences. * Extensions can now participate in the extraction of titles from URL paths. * The command-line installer supports various RDBMSes better. * The interwiki links table can now be accessed also when the interwiki cache is used (used in the API and the Interwiki extension). Internationalization - -------------------- * More gender support (for instance in user lists). * Add languages: Canadian English. * Language converter improved, e.g. it now works depending on the page content language. * Time and number-formatting magic words also now depend on the page content language. * Bidirectional support further improved after 1.18. Release notes - ------------- Full release notes: https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=blob_plain;f=RE LEASE-NOTES-1.19;hb=1.19.0beta2 https://www.mediawiki.org/wiki/Release_notes/1.19 Co-inciding with these security releases, the MediaWiki source code repository has moved from SVN (at https://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3) to Git (https://gerrit.wikimedia.org/gitweb/mediawiki/core.git). So the relevant commits for these releases will not be appearing in our SVN repository. If you use SVN checkouts of MediaWiki for version control, you need to migrate these to Git. If you up are using tarballs, there should be no change in the process for you. Please note that any WMF-deployed extensions have also been migrated to Git also, along with some other non WMF-maintained ones. Please bear with us, some of the Git related links for this release may not work instantly, but should later on. To do a simple Git clone, the command is: git clone https://gerrit.wikimedia.org/r/p/mediawiki/core.git More information is available at https://www.mediawiki.org/wiki/Git For more help, please visit the #mediawiki IRC channel on freenode.net irc://irc.freenode.net/mediawiki or email The MediaWiki-l mailing list at mediawiki-l(a)lists.wikimedia.org. ********************************************************************** Download: http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.0beta2.tar.gz Patch to previous version (1.19.0beta1), without interface text: http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.0beta2.patch.gz Interface text changes: http://download.wikimedia.org/mediawiki/1.19/mediawiki-i18n-1.19.0beta2.patc h.gz GPG signatures: http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.0beta2.tar.gz.si g http://download.wikimedia.org/mediawiki/1.19/mediawiki-1.19.0beta2.patch.gz. sig http://download.wikimedia.org/mediawiki/1.19/mediawiki-i18n-1.19.0beta2.patc h.gz.sig Public keys: https://secure.wikimedia.org/keys.html

5 5

Engineering Community team updates
by Quim Gil 25 Apr '14

25 Apr '14

Hi, Just like every first Tuesday of the month [1], the Engineering Community team has reviewed and fine tuned the monthly and quarterly plans: https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings With Google Code-in and Outreach Program for Women starting soon, this month is expected to be very busy. We also have our team's quarterly review, which we expect will be useful to sync our longer term plans with our actual activities - see https://www.mediawiki.org/wiki/Wikimedia_Engineering/2013-14_Goals#Wikimedi… Last week we had the monthly ECT Showcase, just like every last Tuesday of the month [1]. There we presented a selection of tasks completed in October. You can watch the video at http://www.youtube.com/watch?v=Y3Gl-oR2ucU (copy to Commons pending, help welcome). Next week we will have our monthly IRC meeting, just like every second Tuesday of the month. You can propose topics to be discussed at https://www.mediawiki.org/wiki/Engineering_Community_Team/Meetings#2013-11-… That is all for now. We will use this thread for future updates. Questions? Just ask. [1] Just joking, we only completed a first iteration. -- Quim Gil Technical Contributor Coordinator @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

2 9

TitleValue reloaded
by Daniel Kinzler 23 Apr '14

23 Apr '14

I have just pushed a new version of the TitleValue patch to Gerrit: <https://gerrit.wikimedia.org/r/106517>. I have also updated the RDF to reflect the latest changes: <https://www.mediawiki.org/wiki/Requests_for_comment/TitleValue>. Please have a look. I have tried to address several issues with the previous proposal, and reduce the complexity of the proposal. I have also tried to adjust the service interfaces to make migration easier. Any feedback would be very welcome! -- daniel

2 2

Structured logging RFC discussion on 2014-03-19
by Bryan Davis 11 Apr '14

11 Apr '14

The structured logging RFC [0] will be up for discussion again next Wednesday [1]. There is a strawman implementation in gerrit [2] that is likely to be the focus of discussion unless there are other issues that the reviewers find more pressing. At this point the most controversial aspect of my proposed implementation seems to be importing third-party libraries into mw-core and/or the use of composer to manage that activity. I would welcome discussion of alternatives or consensus that this is a reasonable approach for the immediate future that should be revisited if and when a better idea is found for the general problem. [0]: https://www.mediawiki.org/wiki/Requests_for_comment/Structured_logging [1]: https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-03-19 [2]: https://gerrit.wikimedia.org/r/#/c/112699/ Bryan -- Bryan Davis Wikimedia Foundation <bd808(a)wikimedia.org> [[m:User:BDavis_(WMF)]] Sr Software Engineer Boise, ID USA irc: bd808 v:415.839.6885 x6855

1 1

GSoC & FOSS OPW selection process
by Quim Gil 09 Apr '14

09 Apr '14

The application deadline for GSoC and FOSS OPW has passed. The pool of candidates to choose from includes * 43 GSoC proposals from 42 candidates * 18 OPW proposals from 18 candidates, one of them a GSoC candidate as well There are many projects that have received several proposals. In a few hours the dust will settle in the wiki pages, where many candidates are currently syncing their information. https://www.mediawiki.org/wiki/Google_Summer_of_Code_2014 https://www.mediawiki.org/wiki/FOSS_Outreach_Program_for_Women/Round_8 Everybody is invited to join the selection process. https://www.mediawiki.org/wiki/Mentorship_programs/Selection_process All mentors must join the GSoC or OPW review websites in order to join the private evaluation of candidates. The rest of the community is encouraged to help reviewing proposals and asking hard questions to candidates and mentors in their respective talk pages or Bugzilla reports. By April 7 we need to decide how many slots we want to request in each program. In GSoC we request a number of slots to Google, which we might or might not get. In OPW we fund some slots, we might request more, and then we might get them or not. Questions? Just ask. -- Quim Gil Engineering Community Manager @ Wikimedia Foundation http://www.mediawiki.org/wiki/User:Qgil

1 4

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Wikitech-l March 2014