2009/2/25 Brian <Brian.Mingus(a)colorado.edu>du>:
Ahh ok. Anyone who wants to do processing on the full
history (and there are
a lot of these people who exist!) by definition *has* to be willing to throw
some money at it. It simply doesn't fit on commercial drives. In fact, it
would hardly fit on either of the two raid clusters I have access to. Making
it available on Amazon means that, for a fair market rate, you don't have to
download or uncompress the data. You can just start your data crunching. I
can only speak for academics but there is generally funding available for
Amazon EC2 etc... for specific projects. Professors are even known to pay
for a fixed amount of processing for ambitious student projects, and these
kinds of earmarks are easily fit into grants.
Academics usually have access to the necessary computers (or clusters
thereof) to do such processing directly. I think Amazon hosting of
dumps would appeal mainly to non-academics who only have access to
home PCs.