Hi,
I would like to propose the following idea
We already started working on a new virtual cluster known as labs (wmflabs.org) which purpose is to allow people develop stuff and later move it to some production, some time ago. I believe it would be nice to have exactly same environment (probably we could just extend wmflabs for that) running probably on same platform (virtual cluster managed through some site, using nova extension) which would have exactly same possibilities but it would be supposed to run final products (not a testing environment as labs, but "production" where the stable version would live)
Why do we need this?
Wikimedia labs will offer cloned db of production in future which would allow it to run community managed tools like http://toolserver.org/~quentinv57/tools/sulinfo.php and similar. I think it would be best if such tools were developed using labs as a testing platform and stable version pushed to this "production" which should only run the stable code. In fact it doesn't even need to be physically another cluster, just another set of virtual instances isolated from testing environment on labs. The environment would have restrictions which we don't have on labs. People would need to use puppet and gerrit mostly for everything, and root would not be given to everyone in this environment (some projects might be restricted to wmf ops only), so that we could even move all stable bots, we currently host on wmflabs there, without being afraid of leaking the bot credentials and such (that's a reason why bots project is restricted atm). Also the applications which ask for wikimedia credentials could be allowed there, since the code living on this "production" would be subject of review, and such projects which could mean security risk could be managed by wmf ops only (the changes could be done by volunteers but would need to be submitted to gerrit).
We could also move some parts of current production to this "community managed" environment. I talked to Roan Kattouw in past regarding moving the configuration of wikimedia sites to some git repository so that volunteers could submit some patches to gerrit or handle bugzilla reports without needing shell access. Changes to production config would be merged by operation enginners, so that it would be completely secure.
In a nutshell:
This environment could be set up on same platform as wmf labs (no extra costs, just hard work :)), stable products (bots, user scripts) would be living there, while labs would serve only for development and nothing else.
The production version would live on another domain, like wikimedia-tools.org or wmtools.org
Thanks for your comments and responses
Inserted labs list to copy, and clarify:
I only propose to split current labs to two parts: testing and production (I don't propose to purchase whole new virtualization cluster) and these parts should be completely separated (by firewall at least)
On Mon, Mar 26, 2012 at 2:19 PM, Petr Bena benapetr@gmail.com wrote:
Hi,
I would like to propose the following idea
We already started working on a new virtual cluster known as labs (wmflabs.org) which purpose is to allow people develop stuff and later move it to some production, some time ago. I believe it would be nice to have exactly same environment (probably we could just extend wmflabs for that) running probably on same platform (virtual cluster managed through some site, using nova extension) which would have exactly same possibilities but it would be supposed to run final products (not a testing environment as labs, but "production" where the stable version would live)
Why do we need this?
Wikimedia labs will offer cloned db of production in future which would allow it to run community managed tools like http://toolserver.org/~quentinv57/tools/sulinfo.php and similar. I think it would be best if such tools were developed using labs as a testing platform and stable version pushed to this "production" which should only run the stable code. In fact it doesn't even need to be physically another cluster, just another set of virtual instances isolated from testing environment on labs. The environment would have restrictions which we don't have on labs. People would need to use puppet and gerrit mostly for everything, and root would not be given to everyone in this environment (some projects might be restricted to wmf ops only), so that we could even move all stable bots, we currently host on wmflabs there, without being afraid of leaking the bot credentials and such (that's a reason why bots project is restricted atm). Also the applications which ask for wikimedia credentials could be allowed there, since the code living on this "production" would be subject of review, and such projects which could mean security risk could be managed by wmf ops only (the changes could be done by volunteers but would need to be submitted to gerrit).
We could also move some parts of current production to this "community managed" environment. I talked to Roan Kattouw in past regarding moving the configuration of wikimedia sites to some git repository so that volunteers could submit some patches to gerrit or handle bugzilla reports without needing shell access. Changes to production config would be merged by operation enginners, so that it would be completely secure.
In a nutshell:
This environment could be set up on same platform as wmf labs (no extra costs, just hard work :)), stable products (bots, user scripts) would be living there, while labs would serve only for development and nothing else.
The production version would live on another domain, like wikimedia-tools.org or wmtools.org
Thanks for your comments and responses
We already started working on a new virtual cluster known as labs (wmflabs.org) which purpose is to allow people develop stuff and later move it to some production, some time ago. I believe it would be nice to have exactly same environment (probably we could just extend wmflabs for that) running probably on same platform (virtual cluster managed through some site, using nova extension) which would have exactly same possibilities but it would be supposed to run final products (not a testing environment as labs, but "production" where the stable version would live)
Why do we need this?
Wikimedia labs will offer cloned db of production in future which would allow it to run community managed tools like http://toolserver.org/~quentinv57/tools/sulinfo.php and similar. I think it would be best if such tools were developed using labs as a testing platform and stable version pushed to this "production" which should only run the stable code. In fact it doesn't even need to be physically another cluster, just another set of virtual instances isolated from testing environment on labs. The environment would have restrictions which we don't have on labs. People would need to use puppet and gerrit mostly for everything, and root would not be given to everyone in this environment (some projects might be restricted to wmf ops only), so that we could even move all stable bots, we currently host on wmflabs there, without being afraid of leaking the bot credentials and such (that's a reason why bots project is restricted atm). Also the applications which ask for wikimedia credentials could be allowed there, since the code living on this "production" would be subject of review, and such projects which could mean security risk could be managed by wmf ops only (the changes could be done by volunteers but would need to be submitted to gerrit).
We could also move some parts of current production to this "community managed" environment. I talked to Roan Kattouw in past regarding moving the configuration of wikimedia sites to some git repository so that volunteers could submit some patches to gerrit or handle bugzilla reports without needing shell access. Changes to production config would be merged by operation enginners, so that it would be completely secure.
In a nutshell:
This environment could be set up on same platform as wmf labs (no extra costs, just hard work :)), stable products (bots, user scripts) would be living there, while labs would serve only for development and nothing else.
The production version would live on another domain, like wikimedia-tools.org or wmtools.org
Thanks for your comments and responses
I don't see the need for a different domain name. tools.wmflabs.org should suffice.
Also, this can be accomplished by having multiple projects.
- Ryan
I thought that labs are not going to be used to run some final products, rather just to develop them? Also is it secure to use same domain for testing and stable version? The stable version would be better if it was more restricted.
On Tue, Mar 27, 2012 at 5:36 AM, Ryan Lane rlane32@gmail.com wrote:
We already started working on a new virtual cluster known as labs (wmflabs.org) which purpose is to allow people develop stuff and later move it to some production, some time ago. I believe it would be nice to have exactly same environment (probably we could just extend wmflabs for that) running probably on same platform (virtual cluster managed through some site, using nova extension) which would have exactly same possibilities but it would be supposed to run final products (not a testing environment as labs, but "production" where the stable version would live)
Why do we need this?
Wikimedia labs will offer cloned db of production in future which would allow it to run community managed tools like http://toolserver.org/~quentinv57/tools/sulinfo.php and similar. I think it would be best if such tools were developed using labs as a testing platform and stable version pushed to this "production" which should only run the stable code. In fact it doesn't even need to be physically another cluster, just another set of virtual instances isolated from testing environment on labs. The environment would have restrictions which we don't have on labs. People would need to use puppet and gerrit mostly for everything, and root would not be given to everyone in this environment (some projects might be restricted to wmf ops only), so that we could even move all stable bots, we currently host on wmflabs there, without being afraid of leaking the bot credentials and such (that's a reason why bots project is restricted atm). Also the applications which ask for wikimedia credentials could be allowed there, since the code living on this "production" would be subject of review, and such projects which could mean security risk could be managed by wmf ops only (the changes could be done by volunteers but would need to be submitted to gerrit).
We could also move some parts of current production to this "community managed" environment. I talked to Roan Kattouw in past regarding moving the configuration of wikimedia sites to some git repository so that volunteers could submit some patches to gerrit or handle bugzilla reports without needing shell access. Changes to production config would be merged by operation enginners, so that it would be completely secure.
In a nutshell:
This environment could be set up on same platform as wmf labs (no extra costs, just hard work :)), stable products (bots, user scripts) would be living there, while labs would serve only for development and nothing else.
The production version would live on another domain, like wikimedia-tools.org or wmtools.org
Thanks for your comments and responses
I don't see the need for a different domain name. tools.wmflabs.org should suffice.
Also, this can be accomplished by having multiple projects.
- Ryan
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
tools-dev.wmflabs.org and tools.wmflabs.org? I'd prefer to not have to register and maintain a ton of top levels for this. It makes management of things way harder in a bunch of ways (cost, maintenance, legal, documentation, etc).
Labs is meant for semi-production things. So, tools, bots, etc. We'll never provide a production-level of support for it, though.
On Tue, Mar 27, 2012 at 4:47 PM, Petr Bena benapetr@gmail.com wrote:
I thought that labs are not going to be used to run some final products, rather just to develop them? Also is it secure to use same domain for testing and stable version? The stable version would be better if it was more restricted.
On Tue, Mar 27, 2012 at 5:36 AM, Ryan Lane rlane32@gmail.com wrote:
We already started working on a new virtual cluster known as labs (wmflabs.org) which purpose is to allow people develop stuff and later move it to some production, some time ago. I believe it would be nice to have exactly same environment (probably we could just extend wmflabs for that) running probably on same platform (virtual cluster managed through some site, using nova extension) which would have exactly same possibilities but it would be supposed to run final products (not a testing environment as labs, but "production" where the stable version would live)
Why do we need this?
Wikimedia labs will offer cloned db of production in future which would allow it to run community managed tools like http://toolserver.org/~quentinv57/tools/sulinfo.php and similar. I think it would be best if such tools were developed using labs as a testing platform and stable version pushed to this "production" which should only run the stable code. In fact it doesn't even need to be physically another cluster, just another set of virtual instances isolated from testing environment on labs. The environment would have restrictions which we don't have on labs. People would need to use puppet and gerrit mostly for everything, and root would not be given to everyone in this environment (some projects might be restricted to wmf ops only), so that we could even move all stable bots, we currently host on wmflabs there, without being afraid of leaking the bot credentials and such (that's a reason why bots project is restricted atm). Also the applications which ask for wikimedia credentials could be allowed there, since the code living on this "production" would be subject of review, and such projects which could mean security risk could be managed by wmf ops only (the changes could be done by volunteers but would need to be submitted to gerrit).
We could also move some parts of current production to this "community managed" environment. I talked to Roan Kattouw in past regarding moving the configuration of wikimedia sites to some git repository so that volunteers could submit some patches to gerrit or handle bugzilla reports without needing shell access. Changes to production config would be merged by operation enginners, so that it would be completely secure.
In a nutshell:
This environment could be set up on same platform as wmf labs (no extra costs, just hard work :)), stable products (bots, user scripts) would be living there, while labs would serve only for development and nothing else.
The production version would live on another domain, like wikimedia-tools.org or wmtools.org
Thanks for your comments and responses
I don't see the need for a different domain name. tools.wmflabs.org should suffice.
Also, this can be accomplished by having multiple projects.
- Ryan
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org