[Foundation-l] dumps

[Felipe Ortega]

--- El jue, 26/2/09, Brian <Brian.Mingus at colorado.edu> escribió:

> De: Brian <Brian.Mingus at colorado.edu>
> Asunto: Re: [Foundation-l] dumps
> Para: "Wikimedia Foundation Mailing List" <foundation-l at lists.wikimedia.org>
> Fecha: jueves, 26 febrero, 2009 12:33
> Ahh ok. Anyone who wants to do processing on the full
> history (and there are
> a lot of these people who exist!) by definition *has* to be
> willing to throw
> some money at it. It simply doesn't fit on commercial
> drives. 

Not necessarily. For instance, WikiXRay is capable of parsing the dump file on the fly, so you don't need to uncompress the whole file if you don't want to, and the result tipically fits in a 6-8 GB DB (depending on the amount of data your recover), which fits perfectly in commodity hw.

On the other hand, I completely agree with you in that working with the huge XML file requires specific hw (we bought a couple of servers for that).

> People *just want
> the data*.  Many people would be willing to pay a fee.
> 

Probably, but anyway, I would like to avoid paying a fee to access what should be publicly available (at least, until the dump process broke, it was).

Some universities (including ourselves) has offered storage capacity and some bandwith to distribute mirrors and improve the dump availability, at no cost at all :).

> I have a rare copy of the last available full text dump.
> Perhaps I should
> initiate the process myself.
> 

Nothing prevents you to do that (I think) and it could be a stimulus for thinking on subsequent solutions.

Best,

F.

> 
> On Wed, Feb 25, 2009 at 2:20 PM, Thomas Dalton
> <thomas.dalton at gmail.com>wrote:
> 
> > 2009/2/25 Brian <Brian.Mingus at colorado.edu>:
> > > What has led you to believe there is no demand
> for a full dump of the
> > > english wikipedia?
> >
> > He didn't say there was no demand, he said there
> was no demand for
> > having it on Amazon.
> >
> > _______________________________________________
> > foundation-l mailing list
> > foundation-l at lists.wikimedia.org
> > Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l
> >
> _______________________________________________
> foundation-l mailing list
> foundation-l at lists.wikimedia.org
> Unsubscribe:
> https://lists.wikimedia.org/mailman/listinfo/foundation-l