Hi Faidon,
We share your concerns regarding Scribe, the reason why we looked at it:
1) It's not Java, like Flume, Kafka and the rest which means we do not have
to install the JVM on all the squid/varnish/nginx/apache servers
2) Team members have experience running it.
But if you could help us convince ops to go the Java way then that would be
a huuuge help!
Best,
Diederik
On Thu, Jul 26, 2012 at 11:14 AM, Faidon Liambotis <faidon(a)wikimedia.org>wrote;wrote:
On Thu, Jul 26, 2012 at 09:43:48AM -0400, Andrew Otto
wrote:
Here's his opinion on Scribe:
I personally gave up on Scribe, I'd recommend
that you consider
Flume as a better replacement, that is more supported and developed.
Scribe has never been really well written or maintained, it's just
one of the many hacks that Facebook released.
In general, it does seem to be pretty given up on. There have been a
couple of pull requests merged in the last year, but beyond that there
isn't much activity:
https://github.com/facebook/scribe/commits/master
That's what I said to you the other day when I looked a bit at Scribe
and read about how it doesn't work with a newer Thrift and the patches
to make it so haven't been merged.
It seems abandonded, or at least its public releases are. My 2 cents is
that this is quite dangerous for us: if it just works we'll never get
new features; if it fails, we're basically on our own to fix it, same as
udp2log. I thought the whole point was to to move something that has
active developers working on it :-)
It would be really interesting to know how (and
if?) Facebook still
uses Scribe internally. I'm pretty sure they've done a lot more with
Hadoop since 2008-2010 when Scribe was being more actively promoted.
Maybe they're using Flume instead now? We need a Facebook insider,
anyone know one?
Domas works at Facebook and has been on of the first Wikimedia
(volunteer) techops, being our main DBA among other things for years. He
does less things nowadays but still hangs around on IRC channels,
mailing lists, was at the Berlin Hackathon etc. You should catch-up with
him on IRC; his nickname is "domas" and he doesn't have a predictable
timezone :)
Regards,
Faidon
_______________________________________________
Analytics mailing list
Analytics(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/analytics