Hi Asaf,
it looks like the script has difficulties with the welcome page.
Few things:
- Update your code against the SVN if necessary.
- Are you sure /home/asaf/dev/hewiki_dump/index.html exists ?
- I will review this part of the code this evening an add checks and logs to help finding the issue.
Regards
Emmanuel
Le mar 30/06/09 11:20, "Asaf Bartov" asaf.bartov(a)gmail.com a écrit:
> Hello, Emmanuel, and everyone.
>
> Im still trying to create a ZIM file of the Hebrew Wikipedia dump.
> Ive made some progress -- the dump is complete, but the
> buildZimFileFromDirectory.pl script is still failing:
>
> The command line I used was:
>
> asaf@abartov-deb:~/dev/kiwix/dumping_tools/scripts$
> ./BUILDZIMFILEFROMDIRECTORY.PL --HTMLPATH=/HOME/ASAF/DEV/HEWIKI_DUMP
> --WELCOMEPAGE=./INDEX.HTML
>
> The output is:
>
> NOTICE: CREATE TABLE will create implicit sequence
> "article_aid_seq" for serial column "article.aid"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "article_pkey" for table "article"
> NOTICE: CREATE TABLE will create implicit sequence
> "category_cid_seq" for serial column "category.cid"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "category_pkey" for table "category"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "categoryarticles_pkey" for table "categoryarticles"
> NOTICE: CREATE TABLE will create implicit sequence
> "zimfile_zid_seq" for serial column "zimfile.zid"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "zimfile_pkey" for table "zimfile"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "zimdata_pkey" for table "zimdata"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "zimarticles_pkey" for table "zimarticles"
> NOTICE: CREATE TABLE will create implicit sequence
> "indexarticle_xid_seq" for serial column "indexarticle.xid"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "indexarticle_pkey" for table "indexarticle"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "words_pkey" for table "words"
> NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
> "trivialwords_pkey" for table "trivialwords"
> Use of uninitialized value in concatenation (.) or string at
> ../classes//Kiwix/ZimIndexer.pm line 805.
> Use of uninitialized value $welcomePage in concatenation (.) or
> string at ../classes//Kiwix/ZimIndexer.pm line 746.
> DBD::Pg::db do failed: ERROR: invalid input syntax for integer: ""
> at ../classes//Kiwix/ZimIndexer.pm line 385.
> ERROR: invalid input syntax for integer: ""
>
> The last few lines in all.log are:
>
> [2009-06-30 07:05:43,997] builZimFileFromDirectory.pl - Rewriting url
> in
> /home/asaf/dev/hewiki_dump/articles/�×/� /22/Portal~�×� _�_�
> ����¢�×__79ba.html[2009-06-30 07:05:44,010] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/�×/� /22/Portal~�×� _�_�
> ����¢�×__79ba.html[2009-06-30 07:05:44,243] builZimFileFromDirectory.pl - Rewriting url
> in
> /home/asaf/dev/hewiki_dump/articles/�/�¨/� /��¨� ���_
> ���.html[2009-06-30 07:05:44,305] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/�/�¨/� /��¨� ���_
> ���.html[2009-06-30 07:05:44,547] builZimFileFromDirectory.pl - Rewriting url
> in
> /home/asaf/dev/hewiki_dump/articles/a/c/h/�§���¥~Achdut.JPG_bd38.
> html[2009-06-30 07:05:44,567] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/a/c/h/�§���¥~Achdut.JPG_bd38.
> html[2009-06-30 07:05:44,797] builZimFileFromDirectory.pl - Rewriting url
> in
> /home/asaf/dev/hewiki_dump/articles/�/�/�/�×�� ��×~��
> �����¤26����.html[2009-06-30 07:05:44,812] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/�/�/�/�×�� ��×~��
> �����¤26����.html[2009-06-30 07:05:45,007] builZimFileFromDirectory.pl - Rewriting url
> in
> /home/asaf/dev/hewiki_dump/articles/t/r/i/�§���¥~Triumph_of_the_W
> ill_-_Congress_Hall.jpg_2aad.html[2009-06-30 07:05:45,024] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/t/r/i/�§���¥~Triumph_of_the_W
> ill_-_Congress_Hall.jpg_2aad.html[2009-06-30 07:05:45,249] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/�×/�/�×/�×��×��×_�
> �������×.html[2009-06-30 07:05:45,509] builZimFileFromDirectory.pl - Rewriting
> url in
> /home/asaf/dev/hewiki_dump/articles/�/�/�/������
> [2009-06-30 07:05:45,578] builZimFileFromDirectory.pl - Adding to DB
> /home/asaf/dev/hewiki_dump/articles/�/�/�/������
>
> Any idea what is wrong? Any more diagnostics I can provide?
>
> Thanks in advance,
>
> Asaf
>
> On Mon, Apr 20, 2009 at 11:11 PM, Emmanuel Engelhart
> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Asaf Bartov a écrit :
> > So far, Ive been able to compile and run all the tools, but am
> having some
> > trouble creating ZIM files: Ive dumped a locally-loaded
> Mediawiki
>
> > installation to HTML using these instructions
> ,
> > but when I try to run the builZimFileFromDirectory.pl script, I
> get silly
> > postgresql errors about failing to connect using "kiwix" user.
>
> I have commited to Kiwix svn a new version of
> builZimFileFromDirectory.pl allowing to specify the dbUser and
> dbPassword on the command line.
>
> Emmanuel
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org [2]
>
> iEYEARECAAYFAkns1vEACgkQn3IpJRpNWtNliwCeKyaPwXWdCr5rclkuYvqgCa2m
> wdMAn2F+sgUByhQT5g4o6XoD7WJeIeIL
> =USFE
> -----END PGP SIGNATURE-----
>
> --
> --
> Asaf Bartov
>
>
>
> Links:
> ------
> [1] http://openzim.org/Wiki2html
> [2] http://enigmail.mozdev.org
>
>
Hello, everyone.
As Manuel mentioned some time ago, I'm from Wikimedia Israel, and we're
interested in trying the OpenZIM tools for a static version of the Hebrew
Wikipedia we are working on, for inclusion in the Israeli government's One
Laptop Per Child (OLPC) project.
So far, I've been able to compile and run all the tools, but am having some
trouble creating ZIM files: I've dumped a locally-loaded Mediawiki
installation to HTML using these instructions <http://openzim.org/Wiki2html>,
but when I try to run the builZimFileFromDirectory.pl script, I get silly
postgresql errors about failing to connect using "kiwix" user. I should
probably create a user, or a database, or both, but I've never used
postgresql (I use mysql), and despite trying some combinations of commands
to create a 'kiwix' user, with and without a password, couldn't get the
DBI::connect call to succeed. I am sure you are all familiar with
postgresql and can give me a quick pointer on how to proceed. I'm using a
modern unstable Debian box, with a default packager-supplied installation of
postgresql and the related Perl DBD module.
Thanks in advance,
Asaf Bartov
--
Asaf Bartov <asaf(a)forum2.org>
Hallo Kurt,
vielen Dank für Deinen Einsatz die Wikipedia-DVDs an Schulen zu verteilen.
Auf der DVD befinden sich folgende Daten (leider wurden beim Erstellen der
ISO-Datei die Dateinamen auf acht Zeichen verkürzt):
* /bin/
** zimreade - kleiner Webserver, der die Inhalte auf http://localhost:8080
ausliefert (statisch gelink)
** kiwix - der XUL-basierende Kiwix-Reader
* /zim/
** wikip000.zim - 1,4 GB, Artikeldaten (alle ~900'000 Artikel der
deutschsprachigen Wikipedia)
** wikipedi.zim - 1,0 GB, Volltextsuchindex
Kommandozeilenparameter des zimreaders:
usage: bin/zimreade [options] zim-file
options:
-l <ip> listen ip (default 0.0.0.0)
-p <port> listen port (default 8080)
-x <indexfile> full text index file name
Dh. auf unserer DVD mit den beiden og. ZIM-Dateien:
$ bin/zimreade zim/wikip000.zim -x zim/wikipedi.zim
Unsere Webseite: http://openzim.org/
Kiwix (mit weiteren ZIM-Dateien): http://www.kiwix.org/
Viele Grüsse,
Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
Dear all,
ever wondered
* how ZIM works and looks like internal?
* how the process to make ZIM file looks?
We have now posters on our website:
http://openzim.org/images/2/20/Poster_File_Format.pdfhttp://openzim.org/images/1/1d/Poster_ZIM_Creation.pdf
I just printed them in A1 paper size for our booth at LinuxTag this week (June
24th to 27th) in Berlin.
Get there and see / get our new Wikipedia DVD with all german Wikipedia
articles!
Cheers,
Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
Dear Friends!
LinuxTag will take place on June 24th to 27th. As I have informed you earlier
on this list we have successfully applied for a booth there.
Now we have to plan our attendance there.
Please see our planning page:
http://openzim.org/LinuxTag_2009
Put yourself on the list, indicating when you plan to be there. I will have to
order exhibitor tickets in advance for you and organise accommodation.
Please also state if you agree with sharing a room or if you want to have a
single room. I will try to book at Sunflower Hostel which is quite nice and
cheap (near S Wahrschauer Strasse / Ostbahnhof). But we will have to book
asap as the rooms might already be occupied (remember: A lot of young people
are getting to Berlin during LinuxTag, looking for cheap accommodation).
If you want to bring something for the booth (notebook, something to show...)
please list it in the Material List and help us defining contents we want to
see on our posters and Flyers.
Also give your vote if we should get T-Shirts or not.
Thank you for your quick response,
Manuel
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
--
Regards
Manuel Schneider
Wikimedia CH - Verein zur Förderung Freien Wissens
Wikimedia CH - Association for the advancement of free knowledge
www.wikimedia.ch
Hi,
I will be a little bit late at home this evening, but I will do it today.
Otherwise, simply copy the stuff I have done and integrate Tommi's work.
The last version I have of the DVD cover was uploaded to the wiki.
Regards
Emmanuel
Le lun 15/06/09 17:38, "Manuel Schneider" manuel.schneider(a)wikimedia.ch a écrit:
> That looks really good.
>
> Can someone please make an ISO from it and put it on the server? The DVD
>
production company needs the download link *TODAY* to start with the
> production.
>
> Otherwise we miss the time.
>
> Emmanuel, do you please send me the latest version of the cover art
> please?
> Thanks and greets,
>
>
> Manuel
>
> Am Sonntag, 14. Juni 2009 13:54:41 schrieb Tommi Mäkitalo:
> > Hi,
> >
> > I packaged the sources for the DVD. It is better to
> make a dist package
> instead of just exporting the svn. To build from
> dist package you do not
> need autoconf and automake installed. The configure
> script is part of the
> package.
> >
> > Here are the urls for downloading the source
> packages:
> http://www.tntnet.org/download/cxxtools-1.99.4.tar.gz
>> http://www.tntnet.org/download/tntdb-1.0.99.tar.gz
>> http://www.tntnet.org/download/tntnet-1.6.99.tar.gz
>> http://www.openzim.org/download/zimlib-0.90.tar.gz
>> http://www.openzim.org/download/zimreader-0.90.tar.gz
>> http://www.openzim.org/download/zimwriter-0.90.tar.gz
>>
> > And the statically build ZimReader:
> > http://www.openzim.org/download/ZimReader
>>
> > I write a README file how to build the sources today
> evening.
>
> >
> > Tommi
> >
> _______________________________________________
> dev-l mailing list
> > dev-l@openz
> im.org
> https://intern.openzim.org/mailman/listinfo/dev-l
> >
>
>
> --
> Regards
> Manuel Schneider
>
> Wikimedia CH - Verein zur Förderung Freien Wissens
> Wikimedia CH - Association for the advancement of free knowledge
> www.wikimedia.ch
_______________________________________________
> dev-l mailing list
> dev-l@openz
> im.orghttps://intern.openzim.org/mailman/listinfo/dev-l
>
>
Hi
I have released a new ZIM file with ~30.000 articles (text+thumbs) in Italian.
http://tmp.kiwix.org/zim/wikipedia_it_kiwix_30000+_06_2009_alpha2.zim
This should allow Frederico to test with a little bit more articles as 1000 (unlike the last time).
I have in addition put a footer to each article about license, history, etc...
A few bugs and feature request are still open and part of them will be again fixed in the next release with this time ~80.000 articles.
I have also released the beta3 of Kiwix 0.8 fixing a few issues and integrating the UI Italian translation (translation by Frederico).
http://tmp.kiwix.org/bin/kiwix-0.8-beta3+xulrunner.tar.bz2
I have also started to work on the Windows porting and it looks until now pretty good for the Kiwix part (everything but not libzim).
About libzim porting, we will have in two weeks a dev. meeting in Berlin, and I think this will be a topic we should discuss.
Enjoy testing!
Emmanuel
Le mer 10/06/09 17:26, "Manuel Schneider" manuel.schneider(a)wikimedia.ch a écrit:
> Can someone of you please compile an ISO file, as soon as Tommi added
> the zimreader and place it there for download?
I will do it, but we have to be agree about the contents before we can start to test the ISO.
Emmanel