On Mon, Oct 15, 2012 at 1:44 AM, S Page spage@wikimedia.org wrote:
In the promised land, developers use vagrant to run local VM instances on laptops that puppet configures to run a production-ish MediaWiki. At Etsy and Facebook, the day a developer walks in she can make changes in her personal VM and push them to production (or so they claim in blog posts and meetups ;-) ).
The new Mozilla Kuma project is a good example for this as well, and unlike the aforementioned ones, you can download the VM yourself. See:
https://github.com/mozilla/kuma/ https://github.com/mozilla/kuma/blob/master/docs/installation-vagrant.rst
Relevant blog post about their Vagrant setup: http://decafbad.com/blog/2011/10/02/putting-clouds-in-boxes
Some design notes: https://wiki.mozilla.org/Webdev:DevBoxVMImages
We have an awesome and unique infrastructure with Labs for testing and staging, but for local development, having a pre-packaged dev environment (probably slightly less ambitious than beta.wmflabs) would indeed seem very useful. How feasible/useful would it be to build on the existing work, e.g. Andrew Bogott's MediaWiki class, to provide a first iteration of such an environment? [1]
Erik
[1] https://gerrit.wikimedia.org/r/gitweb?p=operations/puppet.git;a=blob_plain;f...
On 10/15/12 4:37 PM, Erik Moeller wrote:
On Mon, Oct 15, 2012 at 1:44 AM, S Page spage@wikimedia.org wrote:
In the promised land, developers use vagrant to run local VM instances on laptops that puppet configures to run a production-ish MediaWiki. At Etsy and Facebook, the day a developer walks in she can make changes in her personal VM and push them to production (or so they claim in blog posts and meetups ;-) ).
The new Mozilla Kuma project is a good example for this as well, and unlike the aforementioned ones, you can download the VM yourself. See:
https://github.com/mozilla/kuma/ https://github.com/mozilla/kuma/blob/master/docs/installation-vagrant.rst
Relevant blog post about their Vagrant setup: http://decafbad.com/blog/2011/10/02/putting-clouds-in-boxes
Some design notes: https://wiki.mozilla.org/Webdev:DevBoxVMImages
We have an awesome and unique infrastructure with Labs for testing and staging, but for local development, having a pre-packaged dev environment (probably slightly less ambitious than beta.wmflabs) would indeed seem very useful. How feasible/useful would it be to build on the existing work, e.g. Andrew Bogott's MediaWiki class, to provide a first iteration of such an environment? [1]
Does a single-instance install provide a sufficient test platform for 80% of the likely patches, or does all of the interesting stuff require a full-blown cluster?
If single-system servers are truly useful in most cases, then a prepackaged VM image seems straightforward and handy. Presuming that the devs are willing/able to run a few git commands before they start coding, we could potentially leave puppet and Vagrant out of the equation and just build a one-off image by hand and include strict instructions to fetch and rebase immediately after opening. It looks like that's roughly what Mozilla is doing at the moment.
Of course, depending on how easy Vagrant is it might be quicker to build that image using puppet in the first place... It's not obvious to me that we would ever want actual consumers to have to deal with puppet.
-Andrew
On Mon, Oct 15, 2012 at 10:28 PM, Andrew Bogott abogott@wikimedia.org wrote:
Does a single-instance install provide a sufficient test platform for 80% of the likely patches, or does all of the interesting stuff require a full-blown cluster?
Define "sufficient" :). Most development is done on "single instance" configs (developer machines) with varying degrees of sophistication before it hits the cluster. The environment defined in labsmediawiki.pp is very simplistic indeed, but I'd venture a guess that some devs (both inside and outside WMF) develop even with similarly minimal developments _right now_; maybe we can do some informal experience sharing on that at the Thursday tech chat to see just how far most local dev environments deviate from production, how many folks run what types of tests, etc.
A dev-environment-in-a-box is emphatically not a replacement for a staging cluster like Beta Labs. What you'd do is:
1) Develop locally against VM environment, run tests in VM, rinse, repeat 2) Submit changeset for review; upon test pass and review, change gets merged auto-deployed to Labs 3) Deploy changeset into prod
We already have 2) in a working state, and 3) is a hybrid of scheduled deployment windows + some ad hoc deployment right now. But we don't have 1) at all, so we basically tell every new dev to make up their dev environment from whole cloth, meaning that after 1) is when the nasty surprises happen.
I think there's a lot that can be done with a single instance setup to ease that pain. To take the Kuma example, the VM not only comes with the core software, but also with fun dependencies like NodeJS, RabbitMQ, Sphinx Search, etc. all configured to work correctly, as well as the whole test suite runnable right away. The equivalent for us would be to have stuff like Lucene search, the full unit test suite, or a Parsoid NodeJS service running "out of the box" -- not to mention nice debug settings by default.
If single-system servers are truly useful in most cases, then a prepackaged VM image seems straightforward and handy. Presuming that the devs are willing/able to run a few git commands before they start coding, we could potentially leave puppet and Vagrant out of the equation and just build a one-off image by hand and include strict instructions to fetch and rebase immediately after opening.
A Kuma-style Vagrant config lets you develop on the host machine (using your favorite IDE, dev tools, etc.) with the dev dir mounted via NFS and accessible in the VM. Vagrant makes that pretty painless. So you can do:
me@mylaptop:/var/www/mediawiki$ git commit -m 'Awesome stuff' -a
on your host machine, while you're logged in via
me@mylaptop:/var/www/mediawiki$ vagrant ssh
to your guest VM to administer/restart/configure services. I've found this pretty performant on a mid-level PC, although obviously there's a penalty for using a VM.
The Vagrant folks say using NFS exports is the best way to overcome massive performance issues with VirtualBox shared folders when dealing with larger numbers of files:
http://vagrantup.com/v1/docs/nfs.html
I tested this in practice and indeed, without NFS, the Kuma VM was pretty much unusable. Again, Vagrant eliminates the need to wrestle with these details as a dev end user -- as long as you have NFS installed and enabled in the Vagrant config, it'll take care of it for you.
It looks like that's roughly what Mozilla is doing at the moment.
The Kuma VM comes with detailed Puppet manifests: https://github.com/mozilla/kuma/tree/master/puppet/manifests
Whenever you run "vagrant up" to start the VM, it runs the provisioners set up in the Vagrantfile. This means that if the PP files change in the repo, as a dev end user, you'll more or less automatically get updated service definitions.
The VM image you can download already has most of the required stuff pre-installed to speed things up, but for example, RabbitMQ was installed after-the-fact via Puppet.
Vagrant shields developer end users from most of the details of Puppet in practice, while providing its benefits by making it relatively straightforward to stay up to date with whatever the recommended "standard dev environment" is without downloading new box images (and thereby losing whatever local customizations you've made).
Erik
On 10/16/12 3:11 AM, Erik Moeller wrote:
On Mon, Oct 15, 2012 at 10:28 PM, Andrew Bogott abogott@wikimedia.org wrote:
Does a single-instance install provide a sufficient test platform for 80% of the likely patches, or does all of the interesting stuff require a full-blown cluster?
Define "sufficient" :). Most development is done on "single instance" configs (developer machines) with varying degrees of sophistication before it hits the cluster.
'Most' is a perfectly compelling statistic.
A dev-environment-in-a-box is emphatically not a replacement for a staging cluster like Beta Labs. What you'd do is:
- Develop locally against VM environment, run tests in VM, rinse, repeat
- Submit changeset for review; upon test pass and review, change gets
merged auto-deployed to Labs 3) Deploy changeset into prod
Yep, that makes perfect sense. I'm at a conference this week with limited time and network access, but I want to learn all about vagrant when I get back.
-A
On Tuesday, October 16, 2012 at 8:10 AM, Andrew Bogott wrote:
Yep, that makes perfect sense. I'm at a conference this week with limited time and network access, but I want to learn all about vagrant when I get back.
-A
I created a project skeleton: https://github.com/atdt/wmf-vagrant
If you clone the repository and run "vagrant up" from its root, you'll get an Ubuntu Precise box. The mediawiki/ subfolder in the repository will be mounted as /srv/mediawiki, and port 8080 on the host will be forwarded to port 80 on the guest. It'll also load the Puppet manifest base.pp from the manifests/ subfolder. base.pp is currently blank.
The next step would be to look at manifests/base.pp, /manifests/role/labsmediawiki.pp in the operations/puppet repository in Gerrit, and abstract away things like network configurations, etc.
PS This is on GitHub because I wasn't sure where it should live in Gerrit -- it could plausibly go in operations/puppet, mediawiki/core, or a separate repository altogether. If you want to move it, please do!
-- Ori Livneh ori@wikimedia.org
On Tuesday, October 16, 2012 at 2:10 PM, Ori Livneh wrote:
I created a project skeleton: https://github.com/atdt/wmf-vagrant
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
It'll take some time, because it'll need to fetch the base precise32 box if you don't already have it. Once it's done, you should be able to browse to http://127.0.0.1/w/ and see a vanilla MediaWiki install, served by the guest VM. If you need to destroy and recreate the VM in the future, it should go much faster, because vagrant will save the base image.
It'd be nice to integrate this with a build tool (Makefile, rake, grunt, etc.) to give command-line shortcuts for running unit tests.
Andrew, I borrowed from your labs MediaWiki manifests heavily, but did not use them directly, because I was worried about getting tangled up in a web of Puppet requirements. It'd be nicer by far to add operations/puppet as a submodule and simply refer Vagrant to that.
Again, I am a bit stretched at the moment, so I'd be very glad for someone to ownership of this, if it proves useful.
-- Ori Livneh ori@wikimedia.org
(Corrections: MediaWiki will be available at http://127.0.0.1:8080/w/ -- note the port number. And you're going to need to install Vagrant and VirtualBox. Both are free and open-source.)
-- Ori Livneh ori@wikimedia.org
On Wednesday, October 17, 2012 at 3:25 AM, Ori Livneh wrote:
On Tuesday, October 16, 2012 at 2:10 PM, Ori Livneh wrote:
I created a project skeleton: https://github.com/atdt/wmf-vagrant
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
It'll take some time, because it'll need to fetch the base precise32 box if you don't already have it. Once it's done, you should be able to browse to http://127.0.0.1/w/ and see a vanilla MediaWiki install, served by the guest VM. If you need to destroy and recreate the VM in the future, it should go much faster, because vagrant will save the base image.
It'd be nice to integrate this with a build tool (Makefile, rake, grunt, etc.) to give command-line shortcuts for running unit tests.
Andrew, I borrowed from your labs MediaWiki manifests heavily, but did not use them directly, because I was worried about getting tangled up in a web of Puppet requirements. It'd be nicer by far to add operations/puppet as a submodule and simply refer Vagrant to that.
Again, I am a bit stretched at the moment, so I'd be very glad for someone to ownership of this, if it proves useful.
-- Ori Livneh ori@wikimedia.org (mailto:ori@wikimedia.org)
On Wed, Oct 17, 2012 at 3:25 AM, Ori Livneh ori@wikimedia.org wrote:
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
And indeed, it works like magic. This is an awesome beginning, Ori - thanks so much for pulling this off. I really think this is a potentially great path to getting pre-built and optimized dev environments into people's hands.
As for a permanent home, this isn't really "operations" and probably lots of folks should have merge rights on it, so perhaps a mediawiki/vagrant repo with a broad permission set would make sense?
Erik
Since were getting serious about it let's move it to the Wikimedia repot On Oct 17, 2012 11:39 PM, "Erik Moeller" erik@wikimedia.org wrote:
On Wed, Oct 17, 2012 at 3:25 AM, Ori Livneh ori@wikimedia.org wrote:
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
And indeed, it works like magic. This is an awesome beginning, Ori - thanks so much for pulling this off. I really think this is a potentially great path to getting pre-built and optimized dev environments into people's hands.
As for a permanent home, this isn't really "operations" and probably lots of folks should have merge rights on it, so perhaps a mediawiki/vagrant repo with a broad permission set would make sense?
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Done and done... https://github.com/wikimedia/wmf-vagrant
Staff team has the following permissions push & pull granted.
— Patrick
On Thu, Oct 18, 2012 at 12:52 PM, Tomasz Finc tfinc@wikimedia.org wrote:
Since were getting serious about it let's move it to the Wikimedia repot On Oct 17, 2012 11:39 PM, "Erik Moeller" erik@wikimedia.org wrote:
On Wed, Oct 17, 2012 at 3:25 AM, Ori Livneh ori@wikimedia.org wrote:
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
And indeed, it works like magic. This is an awesome beginning, Ori - thanks so much for pulling this off. I really think this is a potentially great path to getting pre-built and optimized dev environments into people's hands.
As for a permanent home, this isn't really "operations" and probably lots of folks should have merge rights on it, so perhaps a mediawiki/vagrant repo with a broad permission set would make sense?
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Woot! Thanks guys. I'll use my five-minute slot at the meet up tonight to demo its use.
-- Ori Livneh ori@wikimedia.org
On Thursday, October 18, 2012 at 10:05 AM, Patrick Reilly wrote:
Done and done... https://github.com/wikimedia/wmf-vagrant
Staff team has the following permissions push & pull granted.
— Patrick
On Thu, Oct 18, 2012 at 12:52 PM, Tomasz Finc <tfinc@wikimedia.org (mailto:tfinc@wikimedia.org)> wrote:
Since were getting serious about it let's move it to the Wikimedia repot On Oct 17, 2012 11:39 PM, "Erik Moeller" <erik@wikimedia.org (mailto:erik@wikimedia.org)> wrote:
On Wed, Oct 17, 2012 at 3:25 AM, Ori Livneh <ori@wikimedia.org (mailto:ori@wikimedia.org)> wrote:
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
And indeed, it works like magic. This is an awesome beginning, Ori - thanks so much for pulling this off. I really think this is a potentially great path to getting pre-built and optimized dev environments into people's hands.
As for a permanent home, this isn't really "operations" and probably lots of folks should have merge rights on it, so perhaps a mediawiki/vagrant repo with a broad permission set would make sense?
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
That's awesome! Will it be recorded tonight?
— Patrick
On Thu, Oct 18, 2012 at 2:28 PM, Ori Livneh ori@wikimedia.org wrote:
Woot! Thanks guys. I'll use my five-minute slot at the meet up tonight to demo its use.
-- Ori Livneh ori@wikimedia.org
On Thursday, October 18, 2012 at 10:05 AM, Patrick Reilly wrote:
Done and done... https://github.com/wikimedia/wmf-vagrant
Staff team has the following permissions push & pull granted.
— Patrick
On Thu, Oct 18, 2012 at 12:52 PM, Tomasz Finc <tfinc@wikimedia.org(mailto:
tfinc@wikimedia.org)> wrote:
Since were getting serious about it let's move it to the Wikimedia
repot
On Oct 17, 2012 11:39 PM, "Erik Moeller" <erik@wikimedia.org (mailto:
erik@wikimedia.org)> wrote:
On Wed, Oct 17, 2012 at 3:25 AM, Ori Livneh <ori@wikimedia.org(mailto:
ori@wikimedia.org)> wrote:
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
And indeed, it works like magic. This is an awesome beginning, Ori - thanks so much for pulling this off. I really think this is a potentially great path to getting pre-built and optimized dev environments into people's hands.
As for a permanent home, this isn't really "operations" and probably lots of folks should have merge rights on it, so perhaps a mediawiki/vagrant repo with a broad permission set would make sense?
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:
Wikitech-l@lists.wikimedia.org)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Chip is trying to use Hangout/Youtube to make it happen.
--tomasz
On Thu, Oct 18, 2012 at 11:31 AM, Patrick Reilly preilly@wikimedia.org wrote:
That's awesome! Will it be recorded tonight?
— Patrick
On Thu, Oct 18, 2012 at 2:28 PM, Ori Livneh ori@wikimedia.org wrote:
Woot! Thanks guys. I'll use my five-minute slot at the meet up tonight to demo its use.
-- Ori Livneh ori@wikimedia.org
On Thursday, October 18, 2012 at 10:05 AM, Patrick Reilly wrote:
Done and done... https://github.com/wikimedia/wmf-vagrant
Staff team has the following permissions push & pull granted.
— Patrick
On Thu, Oct 18, 2012 at 12:52 PM, Tomasz Finc <tfinc@wikimedia.org(mailto:
tfinc@wikimedia.org)> wrote:
Since were getting serious about it let's move it to the Wikimedia
repot
On Oct 17, 2012 11:39 PM, "Erik Moeller" <erik@wikimedia.org (mailto:
erik@wikimedia.org)> wrote:
On Wed, Oct 17, 2012 at 3:25 AM, Ori Livneh <ori@wikimedia.org(mailto:
ori@wikimedia.org)> wrote:
Ok, I made it work, I think
git clone https://github.com/atdt/wmf-vagrant.git cd ./wmf-vagrant: git submodule update --init vagrant up
And indeed, it works like magic. This is an awesome beginning, Ori - thanks so much for pulling this off. I really think this is a potentially great path to getting pre-built and optimized dev environments into people's hands.
As for a permanent home, this isn't really "operations" and probably lots of folks should have merge rights on it, so perhaps a mediawiki/vagrant repo with a broad permission set would make sense?
Erik
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:
Wikitech-l@lists.wikimedia.org)
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org (mailto:Wikitech-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Thu, Oct 18, 2012 at 7:05 PM, Patrick Reilly preilly@wikimedia.org wrote:
Done and done... https://github.com/wikimedia/wmf-vagrant
Hi,
I am following the instructions, but it does not want to install mysql.
Virtualbox 4.2.4
$ vagrant -v Vagrant version 1.0.5
$ git clone https://github.com/atdt/wmf-vagrant.git (...)
$ cd wmf-vagrant/ $ git submodule update --init (...)
$ vagrant up [default] Importing base box 'precise32'... [default] The guest additions on this VM do not match the install version of VirtualBox! This may cause things such as forwarded ports, shared folders, and more to not work properly. If any of those things fail on this machine, please update the guest additions and repackage the box.
Guest Additions Version: 4.2.0 VirtualBox Version: 4.2.4 [default] Matching MAC address for NAT networking... [default] Clearing any previously set forwarded ports... [default] Forwarding ports... [default] -- 22 => 2222 (adapter 1) [default] -- 80 => 8080 (adapter 1) [default] Creating shared folders metadata... [default] Clearing any previously set network interfaces... [default] Available bridged network interfaces: 1) en1: Wi-Fi (AirPort) 2) en0: Ethernet 3) vmnet1 4) vmnet8 What interface should the network bridge to? 1 [default] Preparing network interfaces based on configuration... [default] Booting VM... [default] Waiting for VM to boot. This can take a few minutes. [default] VM booted and ready for use! [default] Configuring and enabling network interfaces... [default] Mounting shared folders... [default] -- vagrant: /srv [default] -- v-root: /vagrant [default] -- manifests: /tmp/vagrant-puppet/manifests [default] -- v-pp-m0: /tmp/vagrant-puppet/modules-0 [default] Running provisioner: Vagrant::Provisioners::Puppet... [default] Running Puppet with /tmp/vagrant-puppet/manifests/base.pp... stdin: is not a tty debug: Failed to load library 'ldap' for feature 'ldap'
(...)
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/dpkg-query -W --showformat ${Status} ${Package} ${Version}\n mysql-server'
debug: Puppet::Type::Package::ProviderApt: Executing '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install mysql-server'
err: /Stage[main]/Mysql/Package[mysql-server]/ensure: change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install mysql-server' returned 100: Reading package lists... Building dependency tree... Reading state information... The following extra packages will be installed: libdbd-mysql-perl libdbi-perl libhtml-template-perl libmysqlclient18 libnet-daemon-perl libplrpc-perl mysql-client-5.5 mysql-client-core-5.5 mysql-common mysql-server-5.5 mysql-server-core-5.5 Suggested packages: libipc-sharedcache-perl libterm-readkey-perl tinyca mailx The following NEW packages will be installed: libdbd-mysql-perl libdbi-perl libhtml-template-perl libmysqlclient18 libnet-daemon-perl libplrpc-perl mysql-client-5.5 mysql-client-core-5.5 mysql-common mysql-server mysql-server-5.5 mysql-server-core-5.5 0 upgraded, 12 newly installed, 0 to remove and 66 not upgraded. Need to get 26.5 MB of archives. After this operation, 91.9 MB of additional disk space will be used. Err http://us.archive.ubuntu.com/ubuntu/ precise-updates/main mysql-common all 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.91.13 80] Get:1 http://us.archive.ubuntu.com/ubuntu/ precise/main libnet-daemon-perl all 0.48-1 [43.1 kB] Get:2 http://us.archive.ubuntu.com/ubuntu/ precise/main libplrpc-perl all 0.2020-2 [36.0 kB] Err http://security.ubuntu.com/ubuntu/ precise-security/main mysql-common all 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Err http://security.ubuntu.com/ubuntu/ precise-security/main libmysqlclient18 i386 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Get:3 http://us.archive.ubuntu.com/ubuntu/ precise/main libdbi-perl i386 1.616-1build2 [849 kB] Get:4 http://us.archive.ubuntu.com/ubuntu/ precise/main libdbd-mysql-perl i386 4.020-1build2 [104 kB] Err http://us.archive.ubuntu.com/ubuntu/ precise-updates/main mysql-client-core-5.5 i386 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.91.13 80] Err http://security.ubuntu.com/ubuntu/ precise-security/main mysql-client-core-5.5 i386 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Get:5 http://us.archive.ubuntu.com/ubuntu/ precise/main libhtml-template-perl all 2.10-1 [65.0 kB] Err http://security.ubuntu.com/ubuntu/ precise-security/main mysql-client-5.5 i386 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Err http://security.ubuntu.com/ubuntu/ precise-security/main mysql-server-core-5.5 i386 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Err http://security.ubuntu.com/ubuntu/ precise-security/main mysql-server-5.5 i386 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Err http://us.archive.ubuntu.com/ubuntu/ precise-updates/main mysql-server all 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.91.13 80] Err http://security.ubuntu.com/ubuntu/ precise-security/main mysql-server all 5.5.24-0ubuntu0.12.04.1 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/mysql-common_5.5.24-... 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/libmysqlclient18_5.5... 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/mysql-client-core-5.... 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/mysql-client-5.5_5.5... 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/mysql-server-core-5.... 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/mysql-server-5.5_5.5... 404 Not Found [IP: 91.189.92.190 80] Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/m/mysql-5.5/mysql-server_5.5.24-... 404 Not Found [IP: 91.189.92.190 80] Fetched 1097 kB in 2s (427 kB/s) E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
notice: /Stage[main]/Mysql/Service[mysql]: Dependency Package[mysql-server] has failures: true
warning: /Stage[main]/Mysql/Service[mysql]: Skipping because of failed dependencies
(...)
notice: /Stage[main]/Mediawiki/File[/var/www/srv/mediawiki]: Dependency Package[mysql-server] has failures: true warning: /Stage[main]/Mediawiki/File[/var/www/srv/mediawiki]: Skipping because of failed dependencies
(...)
Not surprisingly, mysql is not installed:
$ vagrant ssh Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic-pae i686) * Documentation: https://help.ubuntu.com/ Welcome to your Vagrant-built virtual machine. Last login: Fri Sep 14 06:22:31 2012 from 10.0.2.2
$ mysql --version The program 'mysql' is currently not installed. You can install it by typing: sudo apt-get install mysql-client-core-5.5
Am I doing something wrong?
Please let me know if I should provide the entire output from terminal, or if this is not the place for this kind of support.
Željko
On Saturday, November 10, 2012 at 8:01 AM, Željko Filipin wrote:
I am following the instructions, but it does not want to install mysql.
The solution was to make all required packages depend on the apt-get update task. There's a pull request on GitHub with the fix if anyone wants to review it :)
https://github.com/wikimedia/wmf-vagrant/pull/3
Thanks for the report!
-- Ori Livneh ori@wikimedia.org
On Sun, Nov 11, 2012 at 3:04 AM, Ori Livneh ori@wikimedia.org wrote:
The solution was to make all required packages depend on the apt-get
update task. There's a pull request on GitHub with the fix if anyone wants to review it :)
Looks like it is merged. I have tested the fix using your fork a few days ago and now mediawiki is installed with no problems. Thanks! :)
Željko
The Wikidata folks have used Vagrant (forked from Ori's repo) to package a dev environment for the Wikidata-related extensions. I gave it a try yesterday and it seemed to work pretty nicely:
https://github.com/SilkeMeyer/wikidata-vagrant
Cheers, Erik
On Monday, October 15, 2012 at 10:28 PM, Andrew Bogott wrote:
Does a single-instance install provide a sufficient test platform for 80% of the likely patches, or does all of the interesting stuff require a full-blown cluster?
I don't want to guess percentages, but a significant amount of development work -- especially front-end -- can be adequately tested on a VM.
If single-system servers are truly useful in most cases, then a prepackaged VM image seems straightforward and handy. Presuming that the devs are willing/able to run a few git commands before they start coding, we could potentially leave puppet and Vagrant out of the equation and just build a one-off image by hand and include strict instructions to fetch and rebase immediately after opening. It looks like that's roughly what Mozilla is doing at the moment.
Vagrant is a means for doing precisely that. If you don't want to use Puppet or Chef, you can just configure an instance by hand (by SSHing into it, usually) and regenerate a Vagrant box from the result. Vagrant can set up shared folders between the host and guest VM, so what we might want to do is simply tell Vagrant to mount its project directory on the host machine as /srv/mediawiki (or whatever) in the guest VM, and have apache serve that. That makes it very easy to track head in git: you keep a clone of the repository up-to-date on your local disk, and leave it for the VM to serve it. That would spare people the trouble of having to set up a LAMP stack.
Is anyone interested in taking this on? I've done it before and found it useful, so I'd be happy to provide some assistance. Otherwise I'll work on it myself when I get the chance.
-- Ori Livneh ori@wikimedia.org
If you don't want to use Puppet or Chef, you can just configure an instance by hand (by SSHing into it, usually) and regenerate a Vagrant box from the result.
Actually, even if you do want to use Puppet, a VM can be useful. I have a local VM (not Vagrant) set up that I use to test new puppet manifests. I'm using our operations/puppet production branch directly. As is, it wasn't completely straightforward to set this up with the way our puppet repository is now, but it is possible. Labs is good for final testing of manifests, but it is hard to use it to test changes as you make them.
So ummmm, yeahhhhhh! This could be good for (some) ops people too.
On Oct 16, 2012, at 6:24 AM, Ori Livneh ori@wikimedia.org wrote:
On Monday, October 15, 2012 at 10:28 PM, Andrew Bogott wrote:
Does a single-instance install provide a sufficient test platform for 80% of the likely patches, or does all of the interesting stuff require a full-blown cluster?
I don't want to guess percentages, but a significant amount of development work -- especially front-end -- can be adequately tested on a VM.
If single-system servers are truly useful in most cases, then a prepackaged VM image seems straightforward and handy. Presuming that the devs are willing/able to run a few git commands before they start coding, we could potentially leave puppet and Vagrant out of the equation and just build a one-off image by hand and include strict instructions to fetch and rebase immediately after opening. It looks like that's roughly what Mozilla is doing at the moment.
Vagrant is a means for doing precisely that. If you don't want to use Puppet or Chef, you can just configure an instance by hand (by SSHing into it, usually) and regenerate a Vagrant box from the result. Vagrant can set up shared folders between the host and guest VM, so what we might want to do is simply tell Vagrant to mount its project directory on the host machine as /srv/mediawiki (or whatever) in the guest VM, and have apache serve that. That makes it very easy to track head in git: you keep a clone of the repository up-to-date on your local disk, and leave it for the VM to serve it. That would spare people the trouble of having to set up a LAMP stack.
Is anyone interested in taking this on? I've done it before and found it useful, so I'd be happy to provide some assistance. Otherwise I'll work on it myself when I get the chance.
-- Ori Livneh ori@wikimedia.org
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
wikitech-l@lists.wikimedia.org