Hey folks,
I just finished a blog post about how I'm incorporating hadoop streaming into my workflow. http://socio-technologist.blogspot.com/2014/11/fitting-hadoop-streaming-into...
TL;DR: I have strong opinions about Good Ways(TM) to process large datafiles in interesting ways and hadoop streaming will support them nicely. :)
Props to ottomata for spending a bunch of time helping me get up to speed with our cluster and to gage for making it easier to find hadoop's error messages.
-Aaron
AwesoOOOOOOME!
On Nov 30, 2014, at 16:06, Aaron Halfaker ahalfaker@wikimedia.org wrote:
Hey folks,
I just finished a blog post about how I'm incorporating hadoop streaming into my workflow. http://socio-technologist.blogspot.com/2014/11/fitting-hadoop-streaming-into... http://socio-technologist.blogspot.com/2014/11/fitting-hadoop-streaming-into-my-python.html
TL;DR: I have strong opinions about Good Ways(TM) to process large datafiles in interesting ways and hadoop streaming will support them nicely. :)
Props to ottomata for spending a bunch of time helping me get up to speed with our cluster and to gage for making it easier to find hadoop's error messages.
-Aaron _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
This is indeed awesome -- thanks for trying out the new tools (and all the pain involved) and also for documenting your work.
-Toby
On Mon, Dec 1, 2014 at 8:07 AM, Andrew Otto aotto@wikimedia.org wrote:
AwesoOOOOOOME!
On Nov 30, 2014, at 16:06, Aaron Halfaker ahalfaker@wikimedia.org wrote:
Hey folks,
I just finished a blog post about how I'm incorporating hadoop streaming into my workflow. http://socio-technologist.blogspot.com/2014/11/fitting-hadoop-streaming-into...
TL;DR: I have strong opinions about Good Ways(TM) to process large datafiles in interesting ways and hadoop streaming will support them nicely. :)
Props to ottomata for spending a bunch of time helping me get up to speed with our cluster and to gage for making it easier to find hadoop's error messages.
-Aaron _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics