Re: [Analytics] Scribe Packaging Effort

But it seems that we have to choose between two evils: non-maintained package vs. java-based package. D On Thu, Jul 26, 2012 at 11:29 AM, Andrew Otto <otto(a)wikimedia.org> wrote: If the solution we want to use happens to be Java, then yeah, we'll have to convince ops to go the Java way. But! I think more importantly is to figure out which of the solutions is best, irrelevant of the language. When I say best, I am including ease of configuration and maintenance. If a JVM based solution is not easy to configure and maintain, then that will be big ol' minus points for it. We talked briefly with Robla about this yesterday, and he thinks that ops is going to be resistant to Java no matter what. I understand why they would be. I had to set up Jira for CouchSurfing, and it was the most annoying thing to configure and install that I had yet done, simply because Java does everything their own way. It was difficult to puppetize and required a lot of hacking around to make it modular in puppet. But! We'll see, eh? -A On Jul 26, 2012, at 11:22 AM, Diederik van Liere <dvanliere(a)wikimedia.org> wrote: > Hi Faidon, > > We share your concerns regarding Scribe, the reason why we looked at it: > 1) It's not Java, like Flume, Kafka and the rest which means we do not have to install the JVM on all the squid/varnish/nginx/apache servers > 2) Team members have experience running it. > > > > Best, > Diederik > > > On Thu, Jul 26, 2012 at 11:14 AM, Faidon Liambotis <faidon(a)wikimedia.org> wrote: > On Thu, Jul 26, 2012 at 09:43:48AM -0400, Andrew Otto wrote: > > Here's his opinion on Scribe: > > > I personally gave up on Scribe, I'd recommend that you consider > > > Flume as a better replacement, that is more supported and developed. > > > Scribe has never been really well written or maintained, it's just > > > one of the many hacks that Facebook released. > > > > In general, it does seem to be pretty given up on. There have been a > > couple of pull requests merged in the last year, but beyond that there > > isn't much activity: > > https://github.com/facebook/scribe/commits/master > > That's what I said to you the other day when I looked a bit at Scribe > and read about how it doesn't work with a newer Thrift and the patches > to make it so haven't been merged. > > It seems abandonded, or at least its public releases are. My 2 cents is > that this is quite dangerous for us: if it just works we'll never get > new features; if it fails, we're basically on our own to fix it, same as > udp2log. I thought the whole point was to to move something that has > active developers working on it :-) > > > It would be really interesting to know how (and if?) Facebook still > > uses Scribe internally. I'm pretty sure they've done a lot more with > > Hadoop since 2008-2010 when Scribe was being more actively promoted. > > Maybe they're using Flume instead now? We need a Facebook insider, > > anyone know one? > > Domas works at Facebook and has been on of the first Wikimedia > (volunteer) techops, being our main DBA among other things for years. He > does less things nowadays but still hangs around on IRC channels, > mailing lists, was at the Berlin Hackathon etc. You should catch-up with > him on IRC; his nickname is "domas" and he doesn't have a predictable > timezone :) > > Regards, > Faidon > > _______________________________________________ > Analytics mailing list > Analytics(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/analytics >