On Jul 26, 2012, at 11:30 AM, Diederik van Liere <dvanliere(a)wikimedia.org> wrote:
But it seems that we have to choose between two evils:
non-maintained package vs. java-based package.
D
On Thu, Jul 26, 2012 at 11:29 AM, Andrew Otto <otto(a)wikimedia.org> wrote:
But if you could help us convince ops to go the
Java way then that would be a huuuge help!
If the solution we want to use happens to be Java, then yeah, we'll have to convince
ops to go the Java way. But! I think more importantly is to figure out which of the
solutions is best, irrelevant of the language. When I say best, I am including ease of
configuration and maintenance. If a JVM based solution is not easy to configure and
maintain, then that will be big ol' minus points for it.
We talked briefly with Robla about this yesterday, and he thinks that ops is going to be
resistant to Java no matter what. I understand why they would be. I had to set up Jira
for CouchSurfing, and it was the most annoying thing to configure and install that I had
yet done, simply because Java does everything their own way. It was difficult to
puppetize and required a lot of hacking around to make it modular in puppet.
But! We'll see, eh?
-A
On Jul 26, 2012, at 11:22 AM, Diederik van Liere <dvanliere(a)wikimedia.org> wrote:
> Hi Faidon,
>
> We share your concerns regarding Scribe, the reason why we looked at it:
> 1) It's not Java, like Flume, Kafka and the rest which means we do not have to
install the JVM on all the squid/varnish/nginx/apache servers
> 2) Team members have experience running it.
>
>
But if you could help us convince ops to go the
Java way then that would be a huuuge help!
>
> Best,
> Diederik
>
>
> On Thu, Jul 26, 2012 at 11:14 AM, Faidon Liambotis <faidon(a)wikimedia.org>
wrote:
> On Thu, Jul 26, 2012 at 09:43:48AM -0400, Andrew Otto wrote:
> > Here's his opinion on Scribe:
> > > I personally gave up on Scribe, I'd recommend that you consider
> > > Flume as a better replacement, that is more supported and developed.
> > > Scribe has never been really well written or maintained, it's just
> > > one of the many hacks that Facebook released.
> >
> > In general, it does seem to be pretty given up on. There have been a
> > couple of pull requests merged in the last year, but beyond that there
> > isn't much activity:
> >
https://github.com/facebook/scribe/commits/master
>
> That's what I said to you the other day when I looked a bit at Scribe
> and read about how it doesn't work with a newer Thrift and the patches
> to make it so haven't been merged.
>
> It seems abandonded, or at least its public releases are. My 2 cents is
> that this is quite dangerous for us: if it just works we'll never get
> new features; if it fails, we're basically on our own to fix it, same as
> udp2log. I thought the whole point was to to move something that has
> active developers working on it :-)
>
> > It would be really interesting to know how (and if?) Facebook still
> > uses Scribe internally. I'm pretty sure they've done a lot more with
> > Hadoop since 2008-2010 when Scribe was being more actively promoted.
> > Maybe they're using Flume instead now? We need a Facebook insider,
> > anyone know one?
>
> Domas works at Facebook and has been on of the first Wikimedia
> (volunteer) techops, being our main DBA among other things for years. He
> does less things nowadays but still hangs around on IRC channels,
> mailing lists, was at the Berlin Hackathon etc. You should catch-up with
> him on IRC; his nickname is "domas" and he doesn't have a predictable
> timezone :)
>
> Regards,
> Faidon
>
> _______________________________________________
> Analytics mailing list
> Analytics(a)lists.wikimedia.org
>
https://lists.wikimedia.org/mailman/listinfo/analytics
>