[Labs-l] A (21) day in the Labs

[Ryan Lane]

On Sun, Apr 28, 2013 at 7:28 AM, Tim Landscheidt <tim at tim-landscheidt.de>wrote:

> "Marc A. Pelletier" <marc at uberbox.org> wrote:
>
> >> I'm all for the "lazy sysadmin" paradigm, but I think that
> >> shouldn't preclude usable databases.  River's trainwreck is
> >> freely available
> >> (https://svn.wikimedia.org/svnroot/mediawiki/trunk/tools/trainwreck)
> >> and open source, and the effort to port it to Ubuntu and set
> >> it up is a valuable investment (or the setup of any other
> >> replication engine that provides the *needed* functionality;
> >> many-to-one/many isn't something only Tool Labs requires).
>
> > There's no reason that can't be looked into, and it seems like a good
> > idea if we can swing it.
>
> > Another possibility, given suitably designed queries, would be to add
> > federated tables to the appropriate places.
>
> I don't know how they are implemented in MySQL, but commonly
> there is no real caching which means that not only likely
> millions of rows have to be passed through the network per
> query, but that this is repeated again and again for every
> query.  It's not a question whether "you" can swing it - the
> Toolserver (and other DBs) demonstrates that it can be done.
>
> > That said, the priority is to first get sanitized replication /up/, in a
> > form that is suitable for extant tools.  Having iterative improvements
> > after that is definitely a Good Thing.
>
> The problem there is the snail's speed at which this trek is
> moving.  Replicated databases were originally scheduled for
> February/March, and the allocated time for that project was
> *very* generous (several months IIRC).  If we need to wait
> another, say, six months till we have a working environment
> for non-trivial stuff, it'll kill any excitement there might
> be about Tool Labs.  Having the financial and organizational
> force of WMF behind it should have accelerated the process.
>
>
Things take time. Having the WMF behind Labs doesn't mean things will
happen faster, but that it'll be properly resourced from a hardware
perspective, will have adequate levels of funded staff and will fit into
our infrastructure in a way that's manageable. It also means things will be
puppetized, documented, and open to be run by the community.

Our current goal is to have the replicated databases completely ready for
the tools labs project by the Amsterdam hackathon. Ideally it'll be
accessible by all projects by that time.

If folks work with us rather than fight with us, we'll get things done
faster, together.

- Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20130428/4a5180fc/attachment.html>