On Sat, Aug 1, 2009 at 12:05 PM, Brian Brian.Mingus@colorado.edu wrote:
On Sat, Aug 1, 2009 at 11:47 AM, Gregory Maxwell gmaxwell@gmail.comwrote:
As a server distribution infrastructure [snip]
It had occured to me that wiki@home might be better generalized to an heterogeneous compute cloud for foundation trusted code. The idea would be qemu sandboxes distributed via boinc. So the foundation could distribute transcoder sandboxes to a certain number of clients, and sandboxes specific to the needs of researchers using datasets such as the dumps which are often easily parellelized using map/reduce. The head node would sit on the tool server. The qemu instances would run ubuntu. The researcher submits a job, which consists of a directory containing his code, his data, and a file describing the map/reduce partitioning of the data. The head node compiles the code into a qemu instance and uses boinc to map it to a client that is running win/linux/mac. Crazy, right? ;-)
Various obvious effeciency improvements occured to me. If the clients are already running an Ubuntu qemu instance then they can simply be shipped the code and the data. They compile the code and run their portion of the data. The transcoder clients sit idle with a transcoder instance ready, process the data and send it back. Obviously, it is not very optimal to ship out an entire os for every job..:)