Now that the stream is blasting in, I plan to start regularly (hourly?) importing it into
Hadoop via Kafka. Once it is there, it can be poked and prodded in any way you like :)
On Nov 29, 2012, at 7:05 PM, Diederik van Liere <dvanliere(a)wikimedia.org> wrote:
My main questions right now:
- Would vanadium or kraken be better suited for building this sooner than later (within a
few weeks)
I definitely think that this is something for Kraken but would like David and Ottomata to
chime in as well.
- Would anyone like to help? (David, your guidance around coding the frontend
visualization would be highly valued even if you don't have a day or two to personally
throw at it)
I would say, let's use Etsy's statsd & graphite visualization solution, see
http://github.com/etsy/statsd
statsd is network daemon that runs on the Node.js platform and listens for statistics,
like counters and timers, sent over UDP and sends aggregates to one or more pluggable
backend services like Graphite and posssibly also Ganglia. Very little coding involved :)
just my 2 cents.
D
Ops(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ops
_______________________________________________
Ops mailing list
Ops(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/ops