As MediaWiki 1.19 is getting ready, I'd like to offer information on how MediaWiki 1.18.0 was the most difficult MW upgrade I've ever been through.
Some background: my team administers an internal wiki at a major company with ~2000 users, over 100 extensions (many of them custom/unreleased), and 100K articles. I've been upgrading MW regularly since 1.11 - every release and patch - and have never had this much trouble before, mainly because of extensions that broke in 1.18. The typical MW upgrade takes me a day or two including regression-testing our extensions. But 1.18 has taken me weeks and I'm still not done.
This message is meant to be constructive & helpful, not blameful: it's quite possible that every issue was "our fault" for not keeping up on exactly which functions & globals were being deprecated, etc. I'd just like to describe what kinds of things broke for a reasonably active wiki run by well-meaning people, and to document how we fixed them.
So, here's the list of what we had trouble with, and what we did. I welcome any improvements to our fixes!
1. The global variable $action disappeared, breaking a bunch of our extensions. I switched to $wgRequest->getVal('action').
2. The removal of Xml::hidden() caused one of our extensions to break. I switched to Xml::input(..., array('type', 'hidden'))
3. A few of our older extensions were not ported to ResourceLoader yet and were adding JS and CSS via $wgOut->add... calls. They worked in 1.17 and all broke in 1.18. I ported them to use ResourceLoader, but this is not a good solution yet because of bug 31676 (the 32-stylesheet limit of IE, https://bugzilla.wikimedia.org/show_bug.cgi?id=31676) which IMHO is a very serious time-bomb waiting to explode. I hope it makes it into "1.19wmf deployment" as planned.
4. Some of our parser tag extensions had a bug, in that they didn't return a value in the tag callback. (These tags had no visual display.) This didn't cause problems in 1.17 and earlier, but in 1.18.0 it caused a UNIQ.....QINU string to render on the page. I fixed our extensions to return the empty string, and the problem went away.
5. The removal of $wgMessageCache->addMessage() broke many extensions, some ours and some from mediawiki.org like SimpleForms. Some fixes just required use of the i18n file. Our more difficult issue was that we were injecting system messages into articles to add tracking categories. On advice from this list (thanks!), we used code patterned after Parser::addTrackingCategory() to inject categories and it works fine, actually much better than what we had.
6. The removal of ts_makeSortable() from wikibits.js threw off a bunch of our JavaScript: we were using the function to sort on a different column than the first one on render, and in extensions that create tables within dialogs. We left the problem unfixed until I can understand the new jQuery UI way of doing things (jquery.ui.sortable.js).
7. Nearly 100% of our customizations to WikiEditor 1.17 broke in 1.18. We had followed the documented rules on mediawiki.org, using extensions, ResourceLoader, etc., and everything worked in 1.17. Nevertheless in 1.18, toolbars and menus disappeared in IE. Menus appeared multiple times instead of once in Firefox. JavaScript objects in one module became undefined in others, even with proper dependencies. Some of these issues are still not worked out, but most were fixed by a variety of changes.
8. Our MediaWiki:common.js stopped running on the login page. I realize this was a security fix; it just took me by surprise. Fixed by writing a custom extension using the hook UserLoginForm to inject the few lines of JS we needed, and I'm evaluating other non-JS solutions for more security.
9. The addHandler() function in JavaScript does not seem to work in IE8 anymore. We worked around this by using jQuery's "bind" function.
At this point, our test wiki is stable and I am not anticipating any further large issues, so we should roll out in the next two weeks or so.
Thanks for reading, and I hope this helps someone, DanB
On 01/12/2012 11:31 AM, Daniel Barrett wrote:
So, here's the list of what we had trouble with, and what we did. I welcome any improvements to our fixes!
Daniel, thank you so much for taking the time to list these out. And my sympathies that your upgrade was so troublesome!
On 12 January 2012 16:31, Daniel Barrett danb@vistaprint.com wrote:
So, here's the list of what we had trouble with, and what we did. I welcome any improvements to our fixes!
This deserves a page on mediawiki.org.
- d.
Skipping most of this, but there's a few I at least have an answer for.
On Thu, Jan 12, 2012 at 11:31 AM, Daniel Barrett danb@vistaprint.com wrote:
- The global variable $action disappeared, breaking a bunch of our extensions. I switched to $wgRequest->getVal('action').
This may have been available in the global context, but as far as I know it was never documented as a stable global that should be used.
- The removal of Xml::hidden() caused one of our extensions to break. I switched to Xml::input(..., array('type', 'hidden'))
Xml::hidden() was removed entirely? There *is* Html::hidden() which should be functionally similar.
- The removal of $wgMessageCache->addMessage() broke many extensions, some ours and some from mediawiki.org like SimpleForms. Some fixes just required use of the i18n file. Our more difficult issue was that we were injecting system messages into articles to add tracking categories. On advice from this list (thanks!), we used code patterned after Parser::addTrackingCategory() to inject categories and it works fine, actually much better than what we had.
MessageCache::addMessages() has been on a clear deprecation path for awhile now and should've been well-documented. Even before it was removed entirely, it should've already been a no-op.
- Our MediaWiki:common.js stopped running on the login page. I realize this was a security fix; it just took me by surprise. Fixed by writing a custom extension using the hook UserLoginForm to inject the few lines of JS we needed, and I'm evaluating other non-JS solutions for more security.
This hasn't changed any time recently as far as I can tell...we've had this in place for quite awhile.
-Chad
Me:
- Our MediaWiki:common.js stopped running on the login page. I realize this was a security fix; it just took me by surprise. Fixed by writing a custom extension using the hook UserLoginForm to inject the few lines of JS we needed, and I'm evaluating other non-JS solutions for more security.
Chad writes:
This hasn't changed any time recently as far as I can tell...we've had this in place for quite awhile.
Thanks Chad. FYI, MediaWiki:common.js definitely runs on Special:UserLogin in 1.17.1, the immediately previous release. DanB
On Thu, Jan 12, 2012 at 11:51 AM, Daniel Barrett danb@vistaprint.com wrote:
Me:
- Our MediaWiki:common.js stopped running on the login page. I realize this was a security fix; it just took me by surprise. Fixed by writing a custom extension using the hook UserLoginForm to inject the few lines of JS we needed, and I'm evaluating other non-JS solutions for more security.
Chad writes:
This hasn't changed any time recently as far as I can tell...we've had this in place for quite awhile.
Thanks Chad. FYI, MediaWiki:common.js definitely runs on Special:UserLogin in 1.17.1, the immediately previous release. DanB
Hrm...I distinctly remember user's personal JS was disabled on that page. I wonder if ResourceLoader by grouping the JS also ends up disabling it. In either case, it is a security issue and there's not much we can do about it right now.
-Chad
On Thu, Jan 12, 2012 at 5:57 PM, Chad innocentkiller@gmail.com wrote:
On Thu, Jan 12, 2012 at 11:51 AM, Daniel Barrett danb@vistaprint.com wrote:
Me:
Our MediaWiki:common.js stopped running on the login page. I
realize this was a security fix; it just took me by surprise. Fixed by writing a custom extension using the hook UserLoginForm to inject the few lines of JS we needed, and I'm evaluating other non-JS solutions for more security.
Chad writes:
This hasn't changed any time recently as far as I can tell...we've had
this
in place for quite awhile.
Thanks Chad. FYI, MediaWiki:common.js definitely runs on
Special:UserLogin in 1.17.1, the immediately previous release.
DanB
Hrm...I distinctly remember user's personal JS was disabled on that page. I wonder if ResourceLoader by grouping the JS also ends up disabling it. In either case, it is a security issue and there's not much we can do about it right now.
-Chad
You're both right. It's basically for ever that those special pages call OutputPage::disallowUserJs().
In 1.18 Happy-melon implemented something I think we should've had a long time ago, proper origin recognizition on a module/script level. So it is known about each module where it comes from and to what extend it should be trusted or not.
When rewriting security implementation from the basic "OutputPage::disallowUserJs" to this more elaborate way (using ORIGIN constants defined in the ResourceLoader class) it was probably (unconsciously?) switched from just "JS by users", to "modules (js/css) by origin <= site" (which also matches user JS).
I'm not sure if that's how it happened, but that what I remember and it was kept.
Krinkle
On Thu, Jan 12, 2012 at 5:39 PM, Chad innocentkiller@gmail.com wrote:
- The removal of Xml::hidden() caused one of our extensions to break. I switched to Xml::input(..., array('type', 'hidden'))
Xml::hidden() was removed entirely? There *is* Html::hidden() which should be functionally similar.
Someone moved random methods from Xml to Html class without leaving proper redirects for b/c behind. Whoever did this, he is surely loved by the extension writers all around the world.
--vvv
On 12.01.2012, 20:31 Daniel wrote:
The removal of $wgMessageCache->addMessage() broke many
extensions, some ours and some from mediawiki.org like SimpleForms. Some fixes just required use of the i18n file. Our more difficult issue was that we were injecting system messages into articles to add tracking categories. On advice from this list (thanks!), we used code patterned after Parser::addTrackingCategory() to inject categories and it works fine, actually much better than what we had.
I made addTrackingCategory() public a day before your email to make life easier for extension developers.
wikitech-l@lists.wikimedia.org