On 8/14/07, David Gerard <dgerard(a)gmail.com> wrote: bzip2recover. Genius. I've been wanting to do something like this for a long time, and the one thing standing in my way was that I couldn't figure out how to do the random access bit. For those too lazy to read, here's the stroke of genius (IMHO): "We would certainly prefer not to use MySQL or any other database, since we are only *reading* Wikipedia, not writing into it. [....] we can use the bzip2recover tool (part of bzip2 distribution) to "recover" the individual parts of this compressed file: Basically, BZIP splits its input into 900K (by default) size blocks, [....] What this means, in plain English, is that we can convert the huge downloaded .bz2 file to a large set of small (smaller than 1MB) files, each one individually decompressible!" Now, what's the applicable command to do this with the .7zip file?

David Gerard wrote: It would have been simpler if he used the static HTML dump instead of the XML. It's not hard to make a desktop reader out of it. I wrote a proof of concept a while back. -- Tim Starling

On 8/14/07, David Gerard <dgerard(a)gmail.com> wrote: FWIW, I mailed him privately and suggested to use a zero-install server running PHP with MediaWiki, accessing the bzip dump through a modified Database.php file. I did that once with a sqlite database, but I ran into size/access speed trouble when trying a "real-size" wikipedia instead of a small demo. He seems to agree, but doesn't know enough PHP to hack it. Magnus