Offline-l June 2009

offline-l@lists.wikimedia.org

4 participants
14 discussions

Re: [openZIM dev-l] Postgresql help in creating ZIM files
by emmanuel＠engelhart.org 30 Jun '09

30 Jun '09

Hi Asaf, it looks like the script has difficulties with the welcome page. Few things: - Update your code against the SVN if necessary. - Are you sure /home/asaf/dev/hewiki_dump/index.html exists ? - I will review this part of the code this evening an add checks and logs to help finding the issue. Regards Emmanuel Le mar 30/06/09 11:20, "Asaf Bartov" asaf.bartov(a)gmail.com a écrit: > Hello, Emmanuel, and everyone. > > Im still trying to create a ZIM file of the Hebrew Wikipedia dump. > Ive made some progress -- the dump is complete, but the > buildZimFileFromDirectory.pl script is still failing: > > The command line I used was: > > asaf@abartov-deb:~/dev/kiwix/dumping_tools/scripts$ > ./BUILDZIMFILEFROMDIRECTORY.PL --HTMLPATH=/HOME/ASAF/DEV/HEWIKI_DUMP > --WELCOMEPAGE=./INDEX.HTML > > The output is: > > NOTICE: CREATE TABLE will create implicit sequence > "article_aid_seq" for serial column "article.aid" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "article_pkey" for table "article" > NOTICE: CREATE TABLE will create implicit sequence > "category_cid_seq" for serial column "category.cid" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "category_pkey" for table "category" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "categoryarticles_pkey" for table "categoryarticles" > NOTICE: CREATE TABLE will create implicit sequence > "zimfile_zid_seq" for serial column "zimfile.zid" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "zimfile_pkey" for table "zimfile" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "zimdata_pkey" for table "zimdata" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "zimarticles_pkey" for table "zimarticles" > NOTICE: CREATE TABLE will create implicit sequence > "indexarticle_xid_seq" for serial column "indexarticle.xid" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "indexarticle_pkey" for table "indexarticle" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "words_pkey" for table "words" > NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index > "trivialwords_pkey" for table "trivialwords" > Use of uninitialized value in concatenation (.) or string at > ../classes//Kiwix/ZimIndexer.pm line 805. > Use of uninitialized value $welcomePage in concatenation (.) or > string at ../classes//Kiwix/ZimIndexer.pm line 746. > DBD::Pg::db do failed: ERROR: invalid input syntax for integer: "" > at ../classes//Kiwix/ZimIndexer.pm line 385. > ERROR: invalid input syntax for integer: "" > > The last few lines in all.log are: > > [2009-06-30 07:05:43,997] builZimFileFromDirectory.pl - Rewriting url > in > /home/asaf/dev/hewiki_dump/articles/�×/� /22/Portal~�×� _�_� > ��¢�×__79ba.html[2009-06-30 07:05:44,010] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/�×/� /22/Portal~�×� _�_� > ��¢�×__79ba.html[2009-06-30 07:05:44,243] builZimFileFromDirectory.pl - Rewriting url > in > /home/asaf/dev/hewiki_dump/articles/�/�¨/� /��¨� ��_ > ��.html[2009-06-30 07:05:44,305] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/�/�¨/� /��¨� ��_ > ��.html[2009-06-30 07:05:44,547] builZimFileFromDirectory.pl - Rewriting url > in > /home/asaf/dev/hewiki_dump/articles/a/c/h/�§��¥~Achdut.JPG_bd38. > html[2009-06-30 07:05:44,567] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/a/c/h/�§��¥~Achdut.JPG_bd38. > html[2009-06-30 07:05:44,797] builZimFileFromDirectory.pl - Rewriting url > in > /home/asaf/dev/hewiki_dump/articles/�/�/�/�×�� ×~�� > ��¤26��.html[2009-06-30 07:05:44,812] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/�/�/�/�×�� ×~�� > ��¤26��.html[2009-06-30 07:05:45,007] builZimFileFromDirectory.pl - Rewriting url > in > /home/asaf/dev/hewiki_dump/articles/t/r/i/�§��¥~Triumph_of_the_W > ill_-_Congress_Hall.jpg_2aad.html[2009-06-30 07:05:45,024] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/t/r/i/�§��¥~Triumph_of_the_W > ill_-_Congress_Hall.jpg_2aad.html[2009-06-30 07:05:45,249] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/�×/�/�×/�×��×��×_� > ��×.html[2009-06-30 07:05:45,509] builZimFileFromDirectory.pl - Rewriting > url in > /home/asaf/dev/hewiki_dump/articles/�/�/�/�� > [2009-06-30 07:05:45,578] builZimFileFromDirectory.pl - Adding to DB > /home/asaf/dev/hewiki_dump/articles/�/�/�/�� > > Any idea what is wrong? Any more diagnostics I can provide? > > Thanks in advance, > > Asaf > > On Mon, Apr 20, 2009 at 11:11 PM, Emmanuel Engelhart > wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Asaf Bartov a écrit : > > So far, Ive been able to compile and run all the tools, but am > having some > > trouble creating ZIM files: Ive dumped a locally-loaded > Mediawiki > > > installation to HTML using these instructions > , > > but when I try to run the builZimFileFromDirectory.pl script, I > get silly > > postgresql errors about failing to connect using "kiwix" user. > > I have commited to Kiwix svn a new version of > builZimFileFromDirectory.pl allowing to specify the dbUser and > dbPassword on the command line. > > Emmanuel > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org [2] > > iEYEARECAAYFAkns1vEACgkQn3IpJRpNWtNliwCeKyaPwXWdCr5rclkuYvqgCa2m > wdMAn2F+sgUByhQT5g4o6XoD7WJeIeIL > =USFE > -----END PGP SIGNATURE----- > > -- > -- > Asaf Bartov > > > > Links: > ------ > [1] http://openzim.org/Wiki2html > [2] http://enigmail.mozdev.org > >

3 2

[openZIM dev-l] Postgresql help in creating ZIM files
by Asaf Bartov 30 Jun '09

30 Jun '09

Hello, everyone. As Manuel mentioned some time ago, I'm from Wikimedia Israel, and we're interested in trying the OpenZIM tools for a static version of the Hebrew Wikipedia we are working on, for inclusion in the Israeli government's One Laptop Per Child (OLPC) project. So far, I've been able to compile and run all the tools, but am having some trouble creating ZIM files: I've dumped a locally-loaded Mediawiki installation to HTML using these instructions <http://openzim.org/Wiki2html>, but when I try to run the builZimFileFromDirectory.pl script, I get silly postgresql errors about failing to connect using "kiwix" user. I should probably create a user, or a database, or both, but I've never used postgresql (I use mysql), and despite trying some combinations of commands to create a 'kiwix' user, with and without a password, couldn't get the DBI::connect call to succeed. I am sure you are all familiar with postgresql and can give me a quick pointer on how to proceed. I'm using a modern unstable Debian box, with a default packager-supplied installation of postgresql and the related Perl DBD module. Thanks in advance, Asaf Bartov -- Asaf Bartov <asaf(a)forum2.org>

3 4

[openZIM dev-l] Wikipedia-DVD LinuxTag 2009
by Manuel Schneider 27 Jun '09

27 Jun '09

Hallo Kurt, vielen Dank für Deinen Einsatz die Wikipedia-DVDs an Schulen zu verteilen. Auf der DVD befinden sich folgende Daten (leider wurden beim Erstellen der ISO-Datei die Dateinamen auf acht Zeichen verkürzt): * /bin/ ** zimreade - kleiner Webserver, der die Inhalte auf http://localhost:8080 ausliefert (statisch gelink) ** kiwix - der XUL-basierende Kiwix-Reader * /zim/ ** wikip000.zim - 1,4 GB, Artikeldaten (alle ~900'000 Artikel der deutschsprachigen Wikipedia) ** wikipedi.zim - 1,0 GB, Volltextsuchindex Kommandozeilenparameter des zimreaders: usage: bin/zimreade [options] zim-file options: -l <ip> listen ip (default 0.0.0.0) -p <port> listen port (default 8080) -x <indexfile> full text index file name Dh. auf unserer DVD mit den beiden og. ZIM-Dateien: $ bin/zimreade zim/wikip000.zim -x zim/wikipedi.zim Unsere Webseite: http://openzim.org/ Kiwix (mit weiteren ZIM-Dateien): http://www.kiwix.org/ Viele Grüsse, Manuel -- Regards Manuel Schneider Wikimedia CH - Verein zur Förderung Freien Wissens Wikimedia CH - Association for the advancement of free knowledge www.wikimedia.ch

1 0

[openZIM dev-l] Wikipedia DVD on Heise
by Manuel Schneider 26 Jun '09

26 Jun '09

http://www.heise.de/ix/LinuxTag-Aktualisierte-Wikipedia-DVD--/news/meldung/… -- Regards Manuel Schneider Wikimedia CH - Verein zur Förderung Freien Wissens Wikimedia CH - Association for the advancement of free knowledge www.wikimedia.ch

1 0

[openZIM dev-l] how the ZIM file format / openZIM software works and looks like
by Manuel Schneider 22 Jun '09

22 Jun '09

Dear all, ever wondered * how ZIM works and looks like internal? * how the process to make ZIM file looks? We have now posters on our website: http://openzim.org/images/2/20/Poster_File_Format.pdf http://openzim.org/images/1/1d/Poster_ZIM_Creation.pdf I just printed them in A1 paper size for our booth at LinuxTag this week (June 24th to 27th) in Berlin. Get there and see / get our new Wikipedia DVD with all german Wikipedia articles! Cheers, Manuel -- Regards Manuel Schneider Wikimedia CH - Verein zur Förderung Freien Wissens Wikimedia CH - Association for the advancement of free knowledge www.wikimedia.ch

1 0

Re: [openZIM dev-l] LinuxTag - booth applied
by Manuel Schneider 17 Jun '09

17 Jun '09

Dear Friends! LinuxTag will take place on June 24th to 27th. As I have informed you earlier on this list we have successfully applied for a booth there. Now we have to plan our attendance there. Please see our planning page: http://openzim.org/LinuxTag_2009 Put yourself on the list, indicating when you plan to be there. I will have to order exhibitor tickets in advance for you and organise accommodation. Please also state if you agree with sharing a room or if you want to have a single room. I will try to book at Sunflower Hostel which is quite nice and cheap (near S Wahrschauer Strasse / Ostbahnhof). But we will have to book asap as the rooms might already be occupied (remember: A lot of young people are getting to Berlin during LinuxTag, looking for cheap accommodation). If you want to bring something for the booth (notebook, something to show...) please list it in the Material List and help us defining contents we want to see on our posters and Flyers. Also give your vote if we should get T-Shirts or not. Thank you for your quick response, Manuel -- Regards Manuel Schneider Wikimedia CH - Verein zur Förderung Freien Wissens Wikimedia CH - Association for the advancement of free knowledge www.wikimedia.ch -- Regards Manuel Schneider Wikimedia CH - Verein zur Förderung Freien Wissens Wikimedia CH - Association for the advancement of free knowledge www.wikimedia.ch

1 1

Re: [openZIM dev-l] packages sources for DVD
by emmanuel＠engelhart.org 15 Jun '09

15 Jun '09

Hi, I will be a little bit late at home this evening, but I will do it today. Otherwise, simply copy the stuff I have done and integrate Tommi's work. The last version I have of the DVD cover was uploaded to the wiki. Regards Emmanuel Le lun 15/06/09 17:38, "Manuel Schneider" manuel.schneider(a)wikimedia.ch a écrit: > That looks really good. > > Can someone please make an ISO from it and put it on the server? The DVD > production company needs the download link *TODAY* to start with the > production. > > Otherwise we miss the time. > > Emmanuel, do you please send me the latest version of the cover art > please? > Thanks and greets, > > > Manuel > > Am Sonntag, 14. Juni 2009 13:54:41 schrieb Tommi Mäkitalo: > > Hi, > > > > I packaged the sources for the DVD. It is better to > make a dist package > instead of just exporting the svn. To build from > dist package you do not > need autoconf and automake installed. The configure > script is part of the > package. > > > > Here are the urls for downloading the source > packages: > http://www.tntnet.org/download/cxxtools-1.99.4.tar.gz >> http://www.tntnet.org/download/tntdb-1.0.99.tar.gz >> http://www.tntnet.org/download/tntnet-1.6.99.tar.gz >> http://www.openzim.org/download/zimlib-0.90.tar.gz >> http://www.openzim.org/download/zimreader-0.90.tar.gz >> http://www.openzim.org/download/zimwriter-0.90.tar.gz >> > > And the statically build ZimReader: > > http://www.openzim.org/download/ZimReader >> > > I write a README file how to build the sources today > evening. > > > > > Tommi > > > _______________________________________________ > dev-l mailing list > > dev-l@openz > im.org > https://intern.openzim.org/mailman/listinfo/dev-l > > > > > -- > Regards > Manuel Schneider > > Wikimedia CH - Verein zur Förderung Freien Wissens > Wikimedia CH - Association for the advancement of free knowledge > www.wikimedia.ch _______________________________________________ > dev-l mailing list > dev-l@openz > im.org https://intern.openzim.org/mailman/listinfo/dev-l > >

3 2

[openZIM dev-l] packages sources for DVD
by Tommi Mäkitalo 15 Jun '09

15 Jun '09

Hi, I packaged the sources for the DVD. It is better to make a dist package instead of just exporting the svn. To build from dist package you do not need autoconf and automake installed. The configure script is part of the package. Here are the urls for downloading the source packages: http://www.tntnet.org/download/cxxtools-1.99.4.tar.gz http://www.tntnet.org/download/tntdb-1.0.99.tar.gz http://www.tntnet.org/download/tntnet-1.6.99.tar.gz http://www.openzim.org/download/zimlib-0.90.tar.gz http://www.openzim.org/download/zimreader-0.90.tar.gz http://www.openzim.org/download/zimwriter-0.90.tar.gz And the statically build ZimReader: http://www.openzim.org/download/ZimReader I write a README file how to build the sources today evening. Tommi

2 2

[openZIM dev-l] [NEW ZIM] ~30.000 articles from Wikipedia in Italian
by emmanuel＠engelhart.org 12 Jun '09

12 Jun '09

Hi I have released a new ZIM file with ~30.000 articles (text+thumbs) in Italian. http://tmp.kiwix.org/zim/wikipedia_it_kiwix_30000+_06_2009_alpha2.zim This should allow Frederico to test with a little bit more articles as 1000 (unlike the last time). I have in addition put a footer to each article about license, history, etc... A few bugs and feature request are still open and part of them will be again fixed in the next release with this time ~80.000 articles. I have also released the beta3 of Kiwix 0.8 fixing a few issues and integrating the UI Italian translation (translation by Frederico). http://tmp.kiwix.org/bin/kiwix-0.8-beta3+xulrunner.tar.bz2 I have also started to work on the Windows porting and it looks until now pretty good for the Kiwix part (everything but not libzim). About libzim porting, we will have in two weeks a dev. meeting in Berlin, and I think this will be a topic we should discuss. Enjoy testing! Emmanuel

1 0

Re: [openZIM dev-l] Alpha of the DVD tree
by emmanuel＠engelhart.org 10 Jun '09

10 Jun '09

Le mer 10/06/09 17:26, "Manuel Schneider" manuel.schneider(a)wikimedia.ch a écrit: > Can someone of you please compile an ISO file, as soon as Tommi added > the zimreader and place it there for download? I will do it, but we have to be agree about the contents before we can start to test the ISO. Emmanel

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Offline-l June 2009