Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
On Jun 29, 2012, at 1:04 PM, Chad wrote:
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Note that one can also use git-review in labs (if not, lets install it then).
I'm not sure if this sounds crazy, but you could do git review -d 1234 and test it that way.
-- Krinkle
Yes but that would probably overwrite any previous tests
On Fri, Jun 29, 2012 at 1:23 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 1:04 PM, Chad wrote:
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Note that one can also use git-review in labs (if not, lets install it then).
I'm not sure if this sounds crazy, but you could do git review -d 1234 and test it that way.
-- Krinkle
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Current idea:
someone submit a config change
this change is merged to testing branch
we git pull on labs
people test if change works ok and submit review to gerrit
we merge to master branch or reject it
On Fri, Jun 29, 2012 at 2:07 PM, Petr Bena benapetr@gmail.com wrote:
Yes but that would probably overwrite any previous tests
On Fri, Jun 29, 2012 at 1:23 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 1:04 PM, Chad wrote:
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Note that one can also use git-review in labs (if not, lets install it then).
I'm not sure if this sounds crazy, but you could do git review -d 1234 and test it that way.
-- Krinkle
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Jun 29, 2012, at 2:11 PM, Petr Bena wrote:
Current idea:
someone submit a config change this change is merged to testing branch we git pull on labs people test if change works ok and submit review to gerrit we merge to master branch or reject it
On Fri, Jun 29, 2012 at 2:07 PM, Petr Bena benapetr@gmail.com wrote:
Yes but that would probably overwrite any previous tests
No, git-review checks out the the remote ref/changeset in a local branch, preserving the proper context/history of that change set. These branches are not removed or overwritten.
It does mean, however, that you can't randomly stack different tests upon eachother, but that's a good thing and avoids common errors.
Herein lies also immediately the advantage: If a series of commits is submitted that depend on eachother, git-review'ing the top one will give you all, just like in mediawiki/core.git. Because we'd checkout a commit pointer with its parent tree, not a single commit on top of the previous test state (which what a testing branch would enforce).
And it saves maintenance by not having to continously reset test to master – after (re-)submission and merging it for the second time.
I'm not (yet) very active in the "beta" project on labs, but I imagine it would make sense not to introduce an neccecary double-review process here. They are in gerrit pending merge to master, and they can be checked out on labs as it is, that's a main feature of git-review. And afterwards checkout master again and leave your comments on gerrit and/or review, merge it.
I'm not a big fan of Gerrit's overal design, but one the great aspects is that each change proposal is a branch by design. So in a way, every time a merge request is pushed to gerrit, you've got a branch new "testing" branch ready to be checked out.
I agree with Chad, there is no problem with an actual "testing" branch, but if we can avoid it with an (what could be) a better workflow with less overhead of rebasing, dependency manipulation and double-merging etc. then..
-- Krinkle
On Fri, Jun 29, 2012 at 1:23 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 1:04 PM, Chad wrote:
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
Note that one can also use git-review in labs (if not, lets install it then).
I'm not sure if this sounds crazy, but you could do git review -d 1234 and test it that way.
-- Krinkle
If we are testing multiple items in same moment, replacing one test with another is actually problem. That's what we need to somehow merge right now. I hoped that gerrit could do it for us.
And we test multiple items there almost all time :)
On Fri, Jun 29, 2012 at 2:22 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 2:11 PM, Petr Bena wrote:
Current idea:
someone submit a config change this change is merged to testing branch we git pull on labs people test if change works ok and submit review to gerrit we merge to master branch or reject it
On Fri, Jun 29, 2012 at 2:07 PM, Petr Bena benapetr@gmail.com wrote:
Yes but that would probably overwrite any previous tests
No, git-review checks out the the remote ref/changeset in a local branch, preserving the proper context/history of that change set. These branches are not removed or overwritten.
It does mean, however, that you can't randomly stack different tests upon eachother, but that's a good thing and avoids common errors.
Herein lies also immediately the advantage: If a series of commits is submitted that depend on eachother, git-review'ing the top one will give you all, just like in mediawiki/core.git. Because we'd checkout a commit pointer with its parent tree, not a single commit on top of the previous test state (which what a testing branch would enforce).
And it saves maintenance by not having to continously reset test to master – after (re-)submission and merging it for the second time.
I'm not (yet) very active in the "beta" project on labs, but I imagine it would make sense not to introduce an neccecary double-review process here. They are in gerrit pending merge to master, and they can be checked out on labs as it is, that's a main feature of git-review. And afterwards checkout master again and leave your comments on gerrit and/or review, merge it.
I'm not a big fan of Gerrit's overal design, but one the great aspects is that each change proposal is a branch by design. So in a way, every time a merge request is pushed to gerrit, you've got a branch new "testing" branch ready to be checked out.
I agree with Chad, there is no problem with an actual "testing" branch, but if we can avoid it with an (what could be) a better workflow with less overhead of rebasing, dependency manipulation and double-merging etc. then..
-- Krinkle
On Fri, Jun 29, 2012 at 1:23 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 1:04 PM, Chad wrote:
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
Note that one can also use git-review in labs (if not, lets install it then).
I'm not sure if this sounds crazy, but you could do git review -d 1234 and test it that way.
-- Krinkle
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Imagine:
Some guy wants to deploy extension to Jamaican wikiversity and test it on labs for 2 months before deploying to production. He create a patch of config files and submit to gerrit. We merge it to test and extension is immediately deployed on beta.
Another guy wants to test configuration change of Korean wikisource, he submit a patch, we merge it to test and he can test it on labs. Both tests are running together.
Once any of tests are over, we can merge them to master. This isn't possible with gerrit-review, or I don't know how
On Fri, Jun 29, 2012 at 2:28 PM, Petr Bena benapetr@gmail.com wrote:
If we are testing multiple items in same moment, replacing one test with another is actually problem. That's what we need to somehow merge right now. I hoped that gerrit could do it for us.
And we test multiple items there almost all time :)
On Fri, Jun 29, 2012 at 2:22 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 2:11 PM, Petr Bena wrote:
Current idea:
someone submit a config change this change is merged to testing branch we git pull on labs people test if change works ok and submit review to gerrit we merge to master branch or reject it
On Fri, Jun 29, 2012 at 2:07 PM, Petr Bena benapetr@gmail.com wrote:
Yes but that would probably overwrite any previous tests
No, git-review checks out the the remote ref/changeset in a local branch, preserving the proper context/history of that change set. These branches are not removed or overwritten.
It does mean, however, that you can't randomly stack different tests upon eachother, but that's a good thing and avoids common errors.
Herein lies also immediately the advantage: If a series of commits is submitted that depend on eachother, git-review'ing the top one will give you all, just like in mediawiki/core.git. Because we'd checkout a commit pointer with its parent tree, not a single commit on top of the previous test state (which what a testing branch would enforce).
And it saves maintenance by not having to continously reset test to master – after (re-)submission and merging it for the second time.
I'm not (yet) very active in the "beta" project on labs, but I imagine it would make sense not to introduce an neccecary double-review process here. They are in gerrit pending merge to master, and they can be checked out on labs as it is, that's a main feature of git-review. And afterwards checkout master again and leave your comments on gerrit and/or review, merge it.
I'm not a big fan of Gerrit's overal design, but one the great aspects is that each change proposal is a branch by design. So in a way, every time a merge request is pushed to gerrit, you've got a branch new "testing" branch ready to be checked out.
I agree with Chad, there is no problem with an actual "testing" branch, but if we can avoid it with an (what could be) a better workflow with less overhead of rebasing, dependency manipulation and double-merging etc. then..
-- Krinkle
On Fri, Jun 29, 2012 at 1:23 PM, Krinkle krinklemail@gmail.com wrote:
On Jun 29, 2012, at 1:04 PM, Chad wrote:
On Fri, Jun 29, 2012 at 6:58 AM, Petr Bena benapetr@gmail.com wrote: > Can we create a new branch which would be speedily merged when changes > were done to it, so that we could check out on labs and apply the > change there in order to test if patches submitted by devs works ok? > Thanks to Antoine we use the same repository on beta project, but > right now it's really hard to test stuff submitted to gerrit because > we need to merge stuff by hand. >
I don't see any problem with this really, as long as the branches don't get wildly out of sync like the puppet repo did.
-Chad
Note that one can also use git-review in labs (if not, lets install it then).
I'm not sure if this sounds crazy, but you could do git review -d 1234 and test it that way.
-- Krinkle
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Petr Bena wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I will really prefer we avoid using a test branch. That is mainly based out of the experience of operations/puppet.git and I am not volunteering to maintain them in sync (will do if the community wants though).
The current way is to put labs stuff in the -wmflabs.php files. We might want to rewrite our actual system to avoid the multiple conditional statement which are cluetering the files and just have one or two statement in the entry file (CommonSettings.php).
Timo proposed a system where we would have a common configuration directory, one for production and another one for labs. Much like how /etc/php5/ is organized on Debian systems:
/etc/php5/conf.d/ <-- common conf /etc/php5/apache2 <-- conf while running under web server /etc/php5/cli <-- conf for CLI scripts
Then each directory contains an ini file per extension.
Regarding branches, I am fine having people maintain their own branch which they will be individually responsible for keeping it up to date. The thing is that Gerrit does not work that way and people are supposed to send patches. So they can maintain their own fork in a local branch which can be submitted to Gerrit. Their local branch will become a topic branch of our master. Then we can review / merge.
But that's a lot of hand work, if we really do this, we won't use gerrit at all, we just copy paste code from diffs by hand and insert it to some extra files. I would rather volunteer to sync branches rather than this creep work.
On Fri, Jun 29, 2012 at 4:22 PM, Antoine Musso hashar+wmf@free.fr wrote:
Petr Bena wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I will really prefer we avoid using a test branch. That is mainly based out of the experience of operations/puppet.git and I am not volunteering to maintain them in sync (will do if the community wants though).
The current way is to put labs stuff in the -wmflabs.php files. We might want to rewrite our actual system to avoid the multiple conditional statement which are cluetering the files and just have one or two statement in the entry file (CommonSettings.php).
Timo proposed a system where we would have a common configuration directory, one for production and another one for labs. Much like how /etc/php5/ is organized on Debian systems:
/etc/php5/conf.d/ <-- common conf /etc/php5/apache2 <-- conf while running under web server /etc/php5/cli <-- conf for CLI scripts
Then each directory contains an ini file per extension.
Regarding branches, I am fine having people maintain their own branch which they will be individually responsible for keeping it up to date. The thing is that Gerrit does not work that way and people are supposed to send patches. So they can maintain their own fork in a local branch which can be submitted to Gerrit. Their local branch will become a topic branch of our master. Then we can review / merge.
-- Antoine "hashar" Musso
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Le 29/06/12 18:36, Petr Bena wrote:
But that's a lot of hand work, if we really do this, we won't use gerrit at all, we just copy paste code from diffs by hand and insert it to some extra files. I would rather volunteer to sync branches rather than this creep work.
What hand work are you referring to? copy/paste in git world is something like:
# Copy: git fetch <repo> <ref> # Paste: git merge FETCH_HEAD
Anyway, anyone can clone the repository and do their stuff on their own. That is defacto a branch (your master is not the WMF master, it is a copy). A local commit can be distributed to over people via Gerrit.
There are several config change requests which are usually merged within few hours. The current process is that someone submit a patch which is directly merged to production, ignoring any staging platform at all.
What we can do now, is to copy paste the diff and merge it by hand into wmf-config files in labs, just to test if configuration change is ok.
What I want to do: Making a new branch would allow us to use gerrit to do this hand work for us. We would approve the change to production config on test branch, so that gerrit would merge it. Then we would checkout on labs and we could test the configuration change there, before putting it to production.
In case it is ok, then we could give it green on gerrit and merge it on production.
On Sat, Jun 30, 2012 at 1:51 AM, Antoine Musso hashar+wmf@free.fr wrote:
Le 29/06/12 18:36, Petr Bena wrote:
But that's a lot of hand work, if we really do this, we won't use gerrit at all, we just copy paste code from diffs by hand and insert it to some extra files. I would rather volunteer to sync branches rather than this creep work.
What hand work are you referring to? copy/paste in git world is something like:
# Copy: git fetch <repo> <ref> # Paste: git merge FETCH_HEAD
Anyway, anyone can clone the repository and do their stuff on their own. That is defacto a branch (your master is not the WMF master, it is a copy). A local commit can be distributed to over people via Gerrit.
-- Antoine "hashar" Musso
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Jun 29, 2012, at 6:36 PM, Petr Bena wrote:
On Fri, Jun 29, 2012 at 4:22 PM, Antoine Musso hashar+wmf@free.fr wrote:
Timo proposed a system where we would have a common configuration directory, and one for production and another one for labs. Much like how /etc/php5/ is organized on Debian systems:
/etc/php5/conf.d/ <-- common conf /etc/php5/apache2 <-- conf while running under web server /etc/php5/cli <-- conf for CLI scripts
Then each directory contains an ini file per extension.
But that's a lot of hand work, if we really do this, we won't use gerrit at all, we just copy paste code from diffs by hand and insert it to some extra files. I would rather volunteer to sync branches rather than this creep work.
Misunderstanding :)
This directory structure would be inside the repository, not locally. So no hard word or manual diffing and stuff.
-- Krinkle
On Jun 29, 2012, at 6:55 PM, Rob Lanphier wrote:
Petr, I'm a little alarmed by this statement: "we need to merge stuff by hand". What do you mean by "merging by hand", and what sorts of things are you merging?
Rob
On Fri, Jun 29, 2012 at 3:58 AM, Petr Bena benapetr@gmail.com wrote:
Can we create a new branch which would be speedily merged when changes were done to it, so that we could check out on labs and apply the change there in order to test if patches submitted by devs works ok? Thanks to Antoine we use the same repository on beta project, but right now it's really hard to test stuff submitted to gerrit because we need to merge stuff by hand.
I think we need to keep beta labs reasonably close to "near production" state. We made an exception for Timed Media Handler, but generally speaking, we want to have an environment that doesn't have a lot of experimental stuff. If we mix up a lot of different people's experiemental stuff there, then it will diminish the value of the bug reports that come out there.
Here's my thoughts on what we should be doing there: * Always stay up to the very latest version of master, preferably automatically * Only deploy extensions which have a scheduled release window[1] * For items that are a little more experimental in nature, we should deploy new feature wikis in the beta cluster. (e.g. crazyfeature.beta.wmflabs.org ). Note, these still need to run on the same codebase, so anything needed by crazyfeature needs to be in master, guarded by a runtime feature flag if necessary
Given how hard it has been to keep beta labs in any sort of functional state, encouraging unreviewed and experimental code moves this from "beta" to "alpha", and makes it pretty much useless for the type of integration testing that we need to do. Note, even as we speak, the beta.wmflabs.org redirect points to a non-existent instance.
Petr, I'm a little alarmed by this statement: "we need to merge stuff by hand". What do you mean by "merging by hand", and what sorts of things are you merging?
Rob
wikitech-l@lists.wikimedia.org