deploying the most recent MediaWiki code: which branch?

List overview All Threads
Download

newer

older

February '14 appreciation thread

How to install MediaWiki via...

Sumana Harihareswara

20 Feb 2014 20 Feb '14

9:20 p.m.

I got permission from Reuben Smith of wikihow and WMF release manager Greg Grossmeier to re-post this exchange. Sumana Harihareswara Engineering Community Manager Wikimedia Foundation Reuben Smith of wikiHow asked: "We're having a hard time figuring out whether we should be basing our wikiHow code off Mediawiki's external releases (such as the latest 1.22.2), or off the branches that WMF uses for their internal infrastructure (latest looks to be wmf/1.23wmf14). Do you have any thoughts of guidance on that? We're leaning towards moving to using the WMF internal branches, since we use MySQL as well, but I wanted to hear from different people about the drawbacks." Greg Grossmeier responded: "The quick answer is: Feel free to base it off of either. There shouldn't be any WMF-specific things in those wmfXX branches. If there is, it is a commit called something like "Commit of various WMF live hacks". That one commit can be safely reverted. The wmfXX branches are made every week on Thursday morning (Pacific) before we deploy. As we get closer to the next release (1.23) the MediaWiki Release Managers (our outside contractors, Mark and Markus, not myself) will pick a wmfXX to call a Release Candidate. Going with a 1.22.x would give you stability at the loss of getting fixes faster and it means a bigger upgrade task when 1.23 is out. Summary: If you want to keep closer to WMF, pick a wmfXX branch (this assumes you'll follow at some pace behind WMF). If you don't want to be that bleeding edge, stick with 1.22.x. Hope that helps, Greg PS: To learn more about our deploy cycle, see: https://wikitech.wikimedia.org/wiki/Deployments/One_week PPS: To see where we are at any given point, wrt MediaWiki, see: https://www.mediawiki.org/wiki/MediaWiki_1.23/Roadmap#Schedule_for_the_depl…

Show replies by date

Bartosz Dziewoński

20 Feb 20 Feb

9:32 p.m.

It's worth noting that WMF branches also include temporary hacks to keep current JS/CSS and cached HTML output compatible (for at least 30 days), while release branches never contain them (and thus require HTML caches to be purged during the upgrade process). This usually only affects changes to the Vector skin which is not the default there (I'm not sure if it's even available), but nevertheless wikiHow might prefer one of these depending on how their caching layer is set up. -- Matma Rex

Isarra Yos

9:53 p.m.

On 20/02/14 21:32, Bartosz Dziewoński wrote:

...

If wikiHow's setup is sufficiently different, it might be worth considering making their own branches from the master - test and fix things from what was the current master, deploy it, move the test up to the new master, and repeat. This would do the same as with the wmf branches to avoid a lot of the usual difficulty involved in upgrading, but could perhaps also be tailored to be more specific to the systems wikiHow uses. It would require pretty consistent maintenance of their own, but it could be worth it. -I

Jeremy Baron

9:55 p.m.

New subject: deploying the most recent MediaWiki code: which branch?

On Thu, Feb 20, 2014 at 9:53 PM, Isarra Yos <zhorishna(a)gmail.com> wrote:

...

It would require pretty consistent maintenance of their own, but it could be worth it.

IOW, you need to hire a Greg Grossmeier :) -Jeremy

Greg Grossmeier

10:21 p.m.

...

Isn't that:

...

> Feel free to base it off of either. There shouldn't be any WMF-specific > things in those wmfXX branches. If there is, it is a commit called > something like "Commit of various WMF live hacks". That one commit can > be safely reverted.

eg: https://git.wikimedia.org/commit/mediawiki%2Fcore.git/a868d086b68f05e7f9372… Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |

Bartosz Dziewoński

10:50 p.m.

On Thu, 20 Feb 2014 23:21:37 +0100, Greg Grossmeier <greg(a)wikimedia.org> wrote:

...

Isn't that:

eg: https://git.wikimedia.org/commit/mediawiki%2Fcore.git/a868d086b68f05e7f9372…

Nope, I mean different hacks (that people generally don't bother ops/deployers with), like the ones being removed by https://gerrit.wikimedia.org/r/#/c/61075/ or https://gerrit.wikimedia.org/r/#/c/72151/ or https://gerrit.wikimedia.org/r/#/c/102492/ or https://gerrit.wikimedia.org/r/#/c/82102/ . They are required because (to simplify) generated page HTML (which is cached for up to 30 days on our cluster) includes links to "autoupdating" JS and CSS code. Thus any JS/CSS changes need to be compatible with older generated HTML. -- Matma Rex

Daniel Friesen

11:10 p.m.

On 2014-02-20 2:50 PM, Bartosz Dziewoński wrote:

...

On Thu, 20 Feb 2014 23:21:37 +0100, Greg Grossmeier <greg(a)wikimedia.org> wrote:

Isn't that:

> Feel free to base it off of either. There shouldn't be any > WMF-specific > things in those wmfXX branches. If there is, it is a commit called > something like "Commit of various WMF live hacks". That one commit can > be safely reverted.

eg: https://git.wikimedia.org/commit/mediawiki%2Fcore.git/a868d086b68f05e7f9372…

I've thought for awhile that there are better ways to deal with the cache than this. My thought was a sort of snapshot feature for ResourceLoader: https://www.mediawiki.org/wiki/User:Dantman/Code_Ideas#rl-snapshot - We start outputting &version= on the load.php requests for skin resources. - WMF runs a script that dumps the current state of just about all modules into a snapshot folder. - The MediaWiki upgrade is performed. - Some configuration makes reference to the location of the snapshot. - When load.php sees an old &version= it serves resources from the snapshot instead of MediaWiki, this way even if the styles, scripts, etc... have been modified, the module has been renamed, or even entirely deleted the module is still served to old pages. This way we don't have to create or revert any hacks at all, we are unrestricted in the types of CSS/JS and HTML changes we make, and WMF doesn't get any special treatment since now any wiki large enough to have issues resetting its entire cache can take advantage of it. ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://danielfriesen.name/]

Ryan Lane

10:37 p.m.

New subject: deploying the most recent MediaWiki code: which branch?

On Thu, Feb 20, 2014 at 1:20 PM, Sumana Harihareswara <sumanah(a)wikimedia.org

...

wrote:

...

Reuben Smith of wikiHow asked: "We're having a hard time figuring out whether we should be basing our wikiHow code off Mediawiki's external releases (such as the latest 1.22.2), or off the branches that WMF uses for their internal infrastructure (latest looks to be wmf/1.23wmf14). Do you have any thoughts of guidance on that? We're leaning towards moving to using the WMF internal branches, since we use MySQL as well, but I wanted to hear from different people about the drawbacks." Greg Grossmeier responded: "The quick answer is: Feel free to base it off of either. There shouldn't be any WMF-specific things in those wmfXX branches. If there is, it is a commit called something like "Commit of various WMF live hacks". That one commit can be safely reverted. The wmfXX branches are made every week on Thursday morning (Pacific) before we deploy. As we get closer to the next release (1.23) the MediaWiki Release Managers (our outside contractors, Mark and Markus, not myself) will pick a wmfXX to call a Release Candidate. Going with a 1.22.x would give you stability at the loss of getting fixes faster and it means a bigger upgrade task when 1.23 is out. Summary: If you want to keep closer to WMF, pick a wmfXX branch (this assumes you'll follow at some pace behind WMF). If you don't want to be that bleeding edge, stick with 1.22.x.

Note that unless you're willing to keep up to date with WMF's relatively fast pace of branching, you're going to miss security updates. No matter what, if you use git you're going to get security updates slower, since they are released into the tarballs first, then merged into master, then branches (is this accurate?). Sometimes the current WMF branch won't even get the security updates since they are already merged locally onto Wikimedia's deployment server. That said, due to the poor state of extension management in MediaWiki, the only reasonable way to manage MediaWiki is to use the WMF branches since they handle dependencies for the most popular extensions. I was hoping that composer would make managing extensions easier, but I've been tracking it in SMW and it's actually making things more difficult. - Ryan

Greg Grossmeier

10:57 p.m.

...

That's a good point, with one small clarification/rewording: Someone who's following wmfXX branches will get the security fixes the next branch after the tarball is released. That's usually with in the working week (we tend to release tarballs on Mon/Tues, with new branches on Thursday). So, yes, if you're pacing behind on the wmfXX branches, you'll want to take note of security releases and backport patches as appropriate (all security bugs have single patches attached to the Bugzilla report, and those are made public after the tarball is released). Greg -- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |

Chris Steipp

21 Feb 21 Feb

1:19 a.m.

New subject: deploying the most recent MediaWiki code: which branch?

On Thu, Feb 20, 2014 at 2:37 PM, Ryan Lane <rlane32(a)gmail.com> wrote:

...

I've been releasing tarballs, then pushing the fixes into the release branches and master in gerrit. It all happens within a couple of hours, but the tarballs have a slightly narrower timeframe. I rarely push to wmfXX branches, since those already have the patches applied on the cluster, and the next branch cut from master will contain the fix from master. We're potentially moving to pushing them into gerrit and having jenkins build the tarballs, so this process might be flipped in the near future.

3718

days inactive

3719

days old

wikitech-l@lists.wikimedia.org

Manage subscription

9 comments

8 participants

tags (0)

participants (8)

Bartosz Dziewoński
Chris Steipp
Daniel Friesen
Greg Grossmeier
Isarra Yos
Jeremy Baron
Ryan Lane
Sumana Harihareswara