Hello all,<br>I am working on this project from past few months <a href="http://code.google.com/p/offline-wikipedia/" target="_blank">http://code.google.com/p/offline-wikipedia/</a>, i have presented a talk related to this in <a href="http://freed.in/" target="_blank">freed.in</a> 09 too. <br>
<br>My aim with this project is:<br><ul><li>To create DVD distribution for English wikipedia up to the standards that it can make match to <a href="http://download.wikimedia.org/dvd.html" target="_blank">http://download.wikimedia.org/dvd.html</a>.</li>
<li>Making it easy to install and usable straight from DVD.</li></ul>Target Audience are:<br><ul><li>Those who don't have Internet access.</li><li>Those who want to access content to wikipedia irrespective of Internet connection.</li>
<li>Those who use existing proprietary encyclopedias available in market.</li></ul>Present status:<br><ul><li>Apart from source code hosted at google, whole setup is also available at <a href="http://92.243.5.147/offline-wiki" target="_blank">http://92.243.5.147/offline-wiki</a>
there are two parts, for complete English wikipedia you have to get
blocks.tgz and offline-wikipedia.tgz, there are instructions in README
file available there. There is also also a small prototype
sample.tar.bz2 available, in case one wants to check the quality of
work.</li><li>I am following approach taken by <a href="http://users.softlab.ece.ntua.gr/%7Ettsiod/buildWikipediaOffline.html" target="_blank">http://users.softlab.ece.ntua.gr/~ttsiod/buildWikipediaOffline.html</a>, but with some difference, i am using python to convert wiki-text to html and django for server.</li>
<li>As of now, with XML dumps provided by media-wiki, last year's
October dump was 4.1G, i have csv files to locate articles inside those
dumps of size ~300M, and small django configuration to access and
convert the articles to html, and all this fits into a DVD.</li></ul>Issues at hand:<br><ul><li>My python parser to create html out of
wiki-text if not perfect, i can replace it with something which is
better and existing, but am yet to find that.</li><li>To access
articles faster i am breaking single bz2 using bz2recover, and it gives
me 20k odd files for English content, i am trying to avoid those many
files and not compromising with the speed of browsing the articles.</li><li>We can replace django server with something more light and simple
given they don't have dependency cycles and making it hard to
access/use/install.</li><li>March 09 English content is 4.6G making things more tight</li><li>It is only text content excluding multimedia, pictures(which are improtant part and cant be neglected).</li></ul>Target:<br><ul>
<li>Make it updatebale.</li><li>To make it editable.</li><li><a name="120231c0b959d0be_120230f26a8242e1_12023071c1769227_12022c7a617dfca2_Future_targets:">To manage different categories of articles, and segregation based on that to make refined and better education/learning tool.</a></li>
</ul>There are other issues too, i know of other attempts like
wiki-taxi, wikipedia-dumpreader, and am trying to patch things to get
better results. But don't know why those present parser/attempts never
made to that DVD distribution list. I am working to improve it, in the
meanwhile any suggestion, feedback, contribution are most welcome.<br>
-- <br>Regards<br><font color="#888888">Shantanu Choudhary</font>