Is there any interest in having nightly snapshots of MediaWiki available?
I realize people could just use git, but this poses a problem for users who are familiar with extracting, say, a .xip file, and making MediaWiki work, but are stymied by the esoteric nature of git.
This would be similar to Mozilla's nightlies: (http://nightly.mozilla.org/) and may also be a stepping stone for people to get into development, or at least patch submission.
This all came up because I had the chance to provide a snapshot to help solve a problem in 1.19 (http://thread.gmane.org/gmane.org.wikimedia.mediawiki/40011, shortened: http://hexm.de/kt).
I used the make-release script (http://svn.wikimedia.org/viewvc/mediawiki/trunk/tools/make-release/, shortened: http://hexm.de/ku) and put the snapshot up at http://mah.everybody.org/snapshots/.
I'm willing to set this up to run on wmflabs.org or on my own server if there is interest. This may also be a good way to measure the "need" for a point release -- for example, if the nightly starts including fixes for annoying bugs that affect a lot of people, then a point release is probably needed. (I'm looking at you, Bug #24985.)
I believe there is already a process of this happening on the TS (don't remember the address), or it did used to happen.
On 08/01/2012 03:48 PM, K. Peachey wrote:
I believe there is already a process of this happening on the TS (don't remember the address), or it did used to happen.
I believe it. Now, to find it and publicize it. (And maybe get some stats on the usage of it.)
On 08/01/2012 03:48 PM, K. Peachey wrote:
I believe there is already a process of this happening on the TS (don't remember the address), or it did used to happen.
Found it: http://toolserver.org/~vvv/mw-nightly/
On 1 August 2012 21:55, Mark A. Hershberger mah@everybody.org wrote:
Found it: http://toolserver.org/~vvv/mw-nightly/
Please note this is hosted in vvv's personal space. Due to the nature of toolserver accounts, which are locked if the account is not renewed, the address has a very low 'bus factor'[1] - so it it's better not to promote this address publicly.
Best, Merlijn
On Wed, 01 Aug 2012 15:23:37 -0700, Merlijn van Deen valhallasw@arctus.nl wrote:
On 1 August 2012 21:55, Mark A. Hershberger mah@everybody.org wrote:
Found it: http://toolserver.org/~vvv/mw-nightly/
Please note this is hosted in vvv's personal space. Due to the nature of toolserver accounts, which are locked if the account is not renewed, the address has a very low 'bus factor'[1] - so it it's better not to promote this address publicly.
Best, Merlijn
Also, gitweb has a snapshot feature so you don't exactly even need a nightly: https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=snapshot;h=HEAD...
On 02/08/12 00:39, Daniel Friesen wrote:
Also, gitweb has a snapshot feature so you don't exactly even need a nightly: https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=snapshot;h=HEAD...
I don't think we need a separate nightly page. If someone wants a "nightly", he can use this git download. Differences with release mediawiki are minimal. We could make a page with the steps to download it and providing the url (which I agree is not pretty), but that's all.
On 08/01/2012 06:50 PM, Platonides wrote:
I don't think we need a separate nightly page. If someone wants a "nightly", he can use this git download. Differences with release mediawiki are minimal.
The release *does* include some standard extensions that are not in the gitweb link.
But, yes, having the gitweb link makes this less of an issue.
We could make a page with the steps to download it and providing the url (which I agree is not pretty), but that's all.
Right. I'll do that soon if no one else does. If someone else does it, then I would ask that they post a followup here.
Mark.
On Wed, Aug 1, 2012 at 7:17 PM, Mark A. Hershberger mah@everybody.org wrote:
On 08/01/2012 06:50 PM, Platonides wrote:
I don't think we need a separate nightly page. If someone wants a "nightly", he can use this git download. Differences with release mediawiki are minimal.
The release *does* include some standard extensions that are not in the gitweb link.
But, yes, having the gitweb link makes this less of an issue.
Let's please not link to the gitweb tars. They're not cached, so each one would be generated on demand.
-Chad
On Wed, Aug 1, 2012 at 4:26 PM, Chad innocentkiller@gmail.com wrote:
On Wed, Aug 1, 2012 at 7:17 PM, Mark A. Hershberger mah@everybody.org wrote:
On 08/01/2012 06:50 PM, Platonides wrote:
I don't think we need a separate nightly page. If someone wants a "nightly", he can use this git download. Differences with release mediawiki are minimal.
The release *does* include some standard extensions that are not in the gitweb link.
But, yes, having the gitweb link makes this less of an issue.
Let's please not link to the gitweb tars. They're not cached, so each one would be generated on demand.
-Chad
That's why I created mwSnapshots are a replacement for vvv's mw-nightly (which, before it was taken down, afaik still only worked with SVN).
https://toolserver.org/~krinkle/mwSnapshots/#!/mediawiki-core/master
And it does cache :)
-- Krinkle
On 08/01/2012 06:39 PM, Daniel Friesen wrote:
Also, gitweb has a snapshot feature so you don't exactly even need a nightly: https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=snapshot;h=HEAD...
Excellent. Glad the solution here was just fixing my ignorance.
Mark.
On Thu, Aug 2, 2012 at 2:39 AM, Daniel Friesen lists@nadir-seen-fire.com wrote:
On Wed, 01 Aug 2012 15:23:37 -0700, Merlijn van Deen valhallasw@arctus.nl wrote:
On 1 August 2012 21:55, Mark A. Hershberger mah@everybody.org wrote:
Found it: http://toolserver.org/~vvv/mw-nightly/
Please note this is hosted in vvv's personal space. Due to the nature of toolserver accounts, which are locked if the account is not renewed, the address has a very low 'bus factor'[1] - so it it's better not to promote this address publicly.
Best, Merlijn
Also, gitweb has a snapshot feature so you don't exactly even need a nightly: https://gerrit.wikimedia.org/r/gitweb?p=mediawiki/core.git;a=snapshot;h=HEAD...
Also, the tool linked works as a redirect to the exact URL you specified :)
— Victor.
We already have hourly snapshots of the stable master though:
https://toolserver.org/~krinkle/mwSnapshots/#!/mediawiki-core/master
(and it includes release branches, feature branches and wmf branches).
That could be expanded to keep old version (right now it only keeps the latest one).
-- KJrinkle
On Wed, Aug 1, 2012 at 12:29 PM, Mark A. Hershberger mah@everybody.orgwrote:
Is there any interest in having nightly snapshots of MediaWiki available?
I realize people could just use git, but this poses a problem for users who are familiar with extracting, say, a .xip file, and making MediaWiki work, but are stymied by the esoteric nature of git.
This would be similar to Mozilla's nightlies: (http://nightly.mozilla.org/) and may also be a stepping stone for people to get into development, or at least patch submission.
This all came up because I had the chance to provide a snapshot to help solve a problem in 1.19 (http://thread.gmane.org/gmane.org.wikimedia.mediawiki/40011, shortened: http://hexm.de/kt).
I used the make-release script (http://svn.wikimedia.org/viewvc/mediawiki/trunk/tools/make-release/, shortened: http://hexm.de/ku) and put the snapshot up at http://mah.everybody.org/snapshots/.
I'm willing to set this up to run on wmflabs.org or on my own server if there is interest. This may also be a good way to measure the "need" for a point release -- for example, if the nightly starts including fixes for annoying bugs that affect a lot of people, then a point release is probably needed. (I'm looking at you, Bug #24985.)
Human evil is not a problem. It is a mystery. It cannot be solved. -- When Atheism Becomes Religion, Chris Hedges
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Expanding it might be a good idea. Though you should probably rethink the tool and do things differently.
When you wrote that we discussed Git's built in `git archive` which can create a tarball of an arbitrary repo. But besides that what we didn't think of was Gitweb. Which already has snapshot capabilities. The hourly snapshot tool doesn't really serve much purpose: - Gitweb HEAD snapshots are always completely up to date while the snapshot lags by an hour - Gitweb can give snapshots of every branch in the repo so the hourly snapshot doesn't add anything there - Gitweb can give snapshots of every repo instead of just core. So we already essentially have snapshots of extensions built into Gitweb while the hourly tool only does core. - Gitweb can make a snapshot of any commit, not just the latest commit
So the only thing that a tool really has to offer is a clean UI and some information like the checksum. Those are probably what we should focus on.
- Drop the notion of hourly snapshots; Either use Gitweb snapshot links directly or package snapshots dynamically when a new one is requested. - Support getting snapshots from old commits in the repo. - Consider expanding the tool to support other repos.
Because this will basically be a UI we shouldn't be half-hearted about the UI. The tool is pointless if we don't go all the way. Don't leave 'Branches' out in the open like that. This is a user tool rather than a Git tool. So instead of listing things like "master" and "REL1_19" use a mapping to give them proper labels like "Development" or "Alpha" and "MediaWiki 1.19.x". Also cleanly group things together. ie: Take all the REL#_## branches and put them in a "Release snapshots" group. wmf/ in "WMF branch snapshots". And things like Wikidata and iwtransclusion in a "Experimental project snapshots" group. Also drop the select field and instead make it a list of links that users can actually see.
I wonder if we should turn this into a special page on MW.org.
Daniel Friesen wrote:
I wonder if we should turn this into a special page on MW.org.
I'm not sure about a Special page, but I created a very short page on MediaWiki.org: https://www.mediawiki.org/wiki/Nightlies.
MZMcBride
On Wed, Aug 1, 2012 at 12:29 PM, Mark A. Hershberger mah@everybody.org wrote:
I'm willing to set this up to run on wmflabs.org or on my own server if there is interest. This may also be a good way to measure the "need" for a point release -- for example, if the nightly starts including fixes for annoying bugs that affect a lot of people, then a point release is probably needed. (I'm looking at you, Bug #24985.)
For testing reasons, I think this would be a great thing to have. I would love to have a process where Jenkins kicks off a nightly tarball, running make-release and then uploading it to Swift somehow. I asked Ben offlist if it would be possible to set up a non-wiki public front end, and he acknowledged it would be and that we'll probably need it for other stuff (though nothing is planned, and no one has explicitly asked before now).
The reason I bring Swift into this is because I'm pretty sure we don't have any place else that would have the capacity we need to feel comfortable hosting something like this. Even then, we'd still probably want to do pruning (keep a couple weeks of tarballs, then nuke the old stuff).
Don't read "I think this would be great" to mean that this is a high priority thing, since I can't promise this won't get stalled behind other priorities, but it's an interesting idea. The next step, I think, is to figure out exactly how hosting on Swift would work.
Rob
Le 01/08/12 21:29, Mark A. Hershberger a écrit :
Is there any interest in having nightly snapshots of MediaWiki available?
<snip>
I have created a new Jenkins job to generate nightly snapshots of mediawiki/core.git :
https://integration.mediawiki.org/ci/job/Nightly%20-%20MediaWiki%20Core/
Files are available at: https://integration.mediawiki.org/nightly/mediawiki/core/
mediawiki-latest.zip being a symbolic link to the latest build.
The job is triggered everyday at 3am UTC which has been a low activity hour for the last 4 months or so.
There is no process to cleanup the old snapshot, something we will have to think about later.
The build script needed a git-archive macro: https://gerrit.wikimedia.org/r/#/c/17416/ And the Jenkins + ant target is at: https://gerrit.wikimedia.org/r/#/c/17417/
cheers,
wikitech-l@lists.wikimedia.org