[Labs-l] Problems when setting up an instance of Wikidata

[Ryan Lane]

> We have today tried to set up an instance for wikidata, namely "wikidata-dev-1"
> in the "wikidata" project. We have run into several issues - some are merely
> things that appear unclear, others seem genuinely misswing or broken. I'll try
> to be brief in my descriptions in order to provide an overview. You can ask me
> or reedy for details.
>
> So, here goes:
>
> * UNCLEAR: Which image (ubuntu version) should be used (when trying to be close
> to the live environment)? Why are the different versions there? And why is there
> both "oneric" and "oneiric"?
>

I need to remove oneric, it's a typo. That said, you should only use
LTS releases, and only Lucid is available right now. We'll be adding
precise when it's released. Note that lucid is the default selected
image.

> * MISSING: apparently, the security groups (firewall rule sets) for an instance
> can not be changed after it was created.
>

As Ben mentioned, this is due to EC2's design. Hopefully Nova's API,
when we switch to it, will allow us to change an instance's security
groups after creation. Again as others have said, you can modify the
rules in a group, so you can always add the rules to the "default"
group.

The docs are pretty clear that you should make your security groups
before creating instances.

> * MISSING/UNCLEAR: We will want to create our own puppet scripts for setting up
> wikidata repos for testing, etc. Where do we need to submit them? How would we
> apply them to instances?
>

As Daniel mentioned, you submit the code to the test branch of the
puppet repo. After doing so, you can use "Manage puppet groups" to add
your puppet classes and variables to your project, so they'll be
available in your instance's configuration.

> * MISSING: it would be very convenient to be save "profiles" for setting up new
> instances - profiles would just be the puppet config for that instance.
> Alternatively, allow a new instances to be created "configured just like that
> one over there".
>

Agreed, this would be convenient. That said, we tend to use "role"
classes, that do most things via a single class.

> * MISSING: foll VM snapshots. But that's just "nice to have". Puppet templates
> would be much more useful.
>

We'll eventually have this, but we always recommend using puppet over this.

> * BROKEN: We were unable to install a mysql server. Trying to apply db:core
> failed with an error. The error ocurrs when trying ot restart the ganglia
> monitoring daemon (gmon or whatever). It complains about a syntax error
> (unexpected token "}") in the config file.
>
> * BROKEN: Actually, installing mysql (db:core) initially failed with a different
> error: it's missing the directory /a. If that dir is needed, the puppet script
> should create it, no?
>
> * UNCLEAR: after changing the puppet config for an instance, when are the
> changes applied? We ended up running puppetd -tv manually from the shell...
> shouldn't the console just trigger that whenever the config is changes?
>

This would be ideal, but it isn't easy. Puppet runs can often fail, so
it's usually best to force a run. I guess when creating instances with
known working configuration it would be useful. I'll have to think
some about sane ways to implement this.

> * MISSING/UNCLEAR: some puppet bundles are incompatible with each other. Trying
> to install all of apache2 and apache2:php5 and apache2:php5-mysql fails with an
> unhelpful error message about conflicting declarations. So, dependencies and
> conflicts between bundles should somehow be indicated (or at least there should
> be meaningful messages on the console).
>
> * UNCLEAR: which are the bundles needed to install a standard LAMP stack?
>

We really need docs for our puppet repo. Honestly, though, our answer
for this right now is "it kind of depends on what you are trying".

> * MISSING: we were not able to find out reliably when an instance has finished
> booting. It would be helpful to at least have a big message in the console
> output. Just put a big "READY TO ROLL" in there. Well, ideally, the status in
> the instance list should be "booting", not "running", until it's fully up.
>

Yep, this would be nice. You're supposed to get an email when the
instance is finished building itself, but this isn't working at this
moment.

> We'd be very grateful for any input on the above issues. As it stands, we'll be
> using labs as a plain VM hoster, and install everything by hand. Which kind of
> sucks...
>

Well, at its most basic level, that's what Labs is. The strength of
Labs is the ability to change things about it, and letting us know
your issues like this is definitely one way to help the community do
so. I hope our responses helped you some.

- Ryan