On 3/18/11 9:22 PM, Daniel Friesen wrote:
Take a look at riak too. Wikia's sysadmin Artur
(crucially) built file
storage for Wikia using riak and a custom fuse module.
This is way past the planning stages -- Russell has been working on this
since late last year, along with a handful of WMF people.
Early on we decided we wanted a REST interface more than a FUSE
interface, so we had a different set of choices. Swift not only seems to
perform well, but it also has some nice management consoles built-in,
and it's backed by some large companies.
I don't think Riak ever came up in the discussions -- it might have been
eliminated early due to being Erlang based. While Erlang is a wicked
cool language, we have zero experience with it in production.
I had a look at Artur's RiakFuse just now. As far as I can tell it works
like this:
MediaWiki (PHP) --> FUSE --> RiakFuse (Perl) --> HTTP REST/JSON -->
Riak (Erlang)
Artur's a lot smarter than I am... but I look at this and am already a
bit alarmed. ;)
Also I wonder why not go directly from a MediaWiki FileRepo using REST
calls straight into Riak. That's more or less what Russell is building
for us except the backend will be Swift.
Also, in the README Artur mentions his solution isn't good for big
files, and the whole point of this project was a backend for Commons.
I can read Perl fine but everybody else hated the idea of using Perl
solutions, so perhaps Artur's idea was doubly doomed from our
perspective. Still I feel bad that we didn't at least consider this
earlier, unless Russell did at some point and rejected it early on. (?)
--
Neil Kandalgaonkar (| <neilk(a)wikimedia.org>