[Labs-l] swift available from within labs

[Ben Hartshorne]

On Thu, Mar 29, 2012 at 2:42 PM, Platonides <platonides at gmail.com> wrote:

> On 28/03/12 02:07, Ben Hartshorne wrote:
> > Hi everyone,
> >
> > Swift (running in labs) is now available to all labs projects to use for
> > integration testing with mediawiki.  It is only available from within
> > labs; it does not have a public IP address.
> >
> > Swift in labs treats upload.wikimedia.org <http://upload.wikimedia.org>
> > as its back end, so if you request an image that doesn't exist in swift
> > (but does exist in upload), it will use the 404 handler and fetch the
> > real data.  Because of this it will appear to have all the data you
> > need, though will respond slowly at first.
> >
> > I have only created the commons container, so right now it will only
> > fetch images that exist in commons.  It will return 401 or 500 for all
> > other wikis.  If there is a short list of wikis for which you would like
> > it to work, let me know and I will create them for you.  Alternately,
> > you can join the 'swift' project and follow the documentation
> > <http://wikitech.wikimedia.org/view/Swift/Setup_New_Swift_Cluster> and
> > make them yourself!  :)
>
> Great!
>
> What will happen when someone decides to iterate all commons files from
> labs? Will swift-labs begin discarding non-owned files in some LRU kind?
> Or will it stubbornly try to keep all of them until it fills its disks
> and is no longer able to operate?
>

Swift won't do any sort of auto-discarding; it will be very stubborn and
try and take it all.  I don't know what Swift's failure mode is when the
disks fill.  It'd be worth a test to create a cluster with an artificially
tiny amount of backend storage and test it out and see what happens.  The
current labs swift cluster has 4 backend hosts with 20G each, which means
80 / 3 total G of available space (because swift writes every object 3
times).

My guess - some sort of 500 error.

-ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wikimedia.org/pipermail/labs-l/attachments/20120329/355dfc09/attachment.html>