I've been trying to set up the Collection extension and my own render farm so I can generate zim files from my wiki. So far no luck.
Here's where I am:
I followed this guide:
http://edutechwiki.unige.ch/en/Mediawiki_collection_extension_installation
And I can create zim files from wikipedia using:
mw-zip -c :en -o test.zip Acdc Number mw-render -c test.zip -o test.pdf -w zim
mw-zip works just as one would expect.
------------------------- mw-zip -c :en -o test.zip Acdc Number creating nuwiki in u'tmpuIdHyY/nuwiki' 2013-09-07T10:10:58 mwlib.utils.info >> fetching 'http://en.wikipedia.org/w/index.php?title=Help:Books/License&action=raw&...' removing tmpdir u'tmpuIdHyY' memory used: res=25.0 virt=816.4 --------------------------
I can read those files using kiwix, so I know my basic render farm setup is solid.
The problem is that I cannot get it to work with anything other than wikipedia.
If I try the URL in the guide:
------------------ mw-zip -c http://edutechwiki.unige.ch/mediawiki/ -o test2.zip Mediawiki_collection_extension_installation creating nuwiki in u'tmpRnDvRH/nuwiki' Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 328, in run result = self._run(*self.args, **self.kwargs) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 747, in refcall_fun fun(*args, **kw) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 632, in handle_new_basepath api = self._get_mwapi_for_path(path) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 684, in _get_mwapi_for_path raise RuntimeError("cannot guess api url for %r" % (path,)) RuntimeError: cannot guess api url for 'http://edutechwiki.unige.ch/en' <Greenlet at 0x24d2cd0: refcall_fun> failed with RuntimeError
WARNING: (u'Mediawiki_collection_extension_installation', None) could not be fetched removing tmpdir u'tmpRnDvRH' memory used: res=19.3 virt=226.7 -------------------
and if I try my own:
------------------- mw-zip --username=uuu --password=ppp -c http://newmoon.seiner.lan/mediawiki/index.php/ -o test2.zip Test creating nuwiki in u'tmpG82RPH/nuwiki' Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 328, in run result = self._run(*self.args, **self.kwargs) File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 114, in run api = self.get_api() File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 28, in get_api api.login(self.username, self.password, self.domain) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/sapi.py", line 186, in login res = self._post(**args) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/sapi.py", line 106, in _post res = loads(self._fetch(req)) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/sapi.py", line 23, in loads return json.loads(s) File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 413, in loads return _default_decoder.decode(s) File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 420, in raw_decode raise JSONDecodeError("No JSON object could be decoded", s, idx) JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) <Greenlet at 0x1a7b870: <bound method start_fetcher.run of <mwlib.apps.make_nuwiki.start_fetcher object at 0x1acf790>>> failed with JSONDecodeError
removing tmpdir u'tmpG82RPH' memory used: res=16.8 virt=152.5 Traceback (most recent call last): File "/usr/local/bin/mw-zip", line 9, in <module> load_entry_point('mwlib==0.15.11', 'console_scripts', 'mw-zip')() File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 155, in main make_zip(output, options, env.metabook, podclient=podclient, status=status) File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 50, in make_zip make_nuwiki(fsdir, metabook=metabook, options=options, podclient=podclient, status=status) File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 189, in make_nuwiki pool.join(raise_error=True) File "/usr/local/lib/python2.7/dist-packages/gevent/pool.py", line 98, in join raise greenlet.exception simplejson.decoder.JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) -----------------
I can't even begin to work on the actual extension interface until I have this working..... Any suggestions? Where do I go next?
Thanks!
I wish I could say what I did to fix this... But it seems to work, at least from within mediawiki. I got started on a different problem, restarted apache a few times, and reloaded the page and the Print/Export menu just showed and and it creates zim files... :headscratch:
Yan Seiner wrote:
I've been trying to set up the Collection extension and my own render farm so I can generate zim files from my wiki. So far no luck.
Here's where I am:
I followed this guide:
http://edutechwiki.unige.ch/en/Mediawiki_collection_extension_installation
And I can create zim files from wikipedia using:
mw-zip -c :en -o test.zip Acdc Number mw-render -c test.zip -o test.pdf -w zim
mw-zip works just as one would expect.
mw-zip -c :en -o test.zip Acdc Number creating nuwiki in u'tmpuIdHyY/nuwiki' 2013-09-07T10:10:58 mwlib.utils.info >> fetching 'http://en.wikipedia.org/w/index.php?title=Help:Books/License&action=raw&...'
removing tmpdir u'tmpuIdHyY' memory used: res=25.0 virt=816.4
I can read those files using kiwix, so I know my basic render farm setup is solid.
The problem is that I cannot get it to work with anything other than wikipedia.
If I try the URL in the guide:
mw-zip -c http://edutechwiki.unige.ch/mediawiki/ -o test2.zip Mediawiki_collection_extension_installation creating nuwiki in u'tmpRnDvRH/nuwiki' Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 328, in run result = self._run(*self.args, **self.kwargs) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 747, in refcall_fun fun(*args, **kw) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 632, in handle_new_basepath api = self._get_mwapi_for_path(path) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/fetch.py", line 684, in _get_mwapi_for_path raise RuntimeError("cannot guess api url for %r" % (path,)) RuntimeError: cannot guess api url for 'http://edutechwiki.unige.ch/en' <Greenlet at 0x24d2cd0: refcall_fun> failed with RuntimeError
WARNING: (u'Mediawiki_collection_extension_installation', None) could not be fetched removing tmpdir u'tmpRnDvRH' memory used: res=19.3 virt=226.7
and if I try my own:
mw-zip --username=uuu --password=ppp -c http://newmoon.seiner.lan/mediawiki/index.php/ -o test2.zip Test creating nuwiki in u'tmpG82RPH/nuwiki' Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/gevent/greenlet.py", line 328, in run result = self._run(*self.args, **self.kwargs) File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 114, in run api = self.get_api() File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 28, in get_api api.login(self.username, self.password, self.domain) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/sapi.py", line 186, in login res = self._post(**args) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/sapi.py", line 106, in _post res = loads(self._fetch(req)) File "/usr/local/lib/python2.7/dist-packages/mwlib/net/sapi.py", line 23, in loads return json.loads(s) File "/usr/lib/python2.7/dist-packages/simplejson/__init__.py", line 413, in loads return _default_decoder.decode(s) File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 402, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python2.7/dist-packages/simplejson/decoder.py", line 420, in raw_decode raise JSONDecodeError("No JSON object could be decoded", s, idx) JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0) <Greenlet at 0x1a7b870: <bound method start_fetcher.run of <mwlib.apps.make_nuwiki.start_fetcher object at 0x1acf790>>> failed with JSONDecodeError
removing tmpdir u'tmpG82RPH' memory used: res=16.8 virt=152.5 Traceback (most recent call last): File "/usr/local/bin/mw-zip", line 9, in <module> load_entry_point('mwlib==0.15.11', 'console_scripts', 'mw-zip')() File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 155, in main make_zip(output, options, env.metabook, podclient=podclient, status=status) File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/buildzip.py", line 50, in make_zip make_nuwiki(fsdir, metabook=metabook, options=options, podclient=podclient, status=status) File "/usr/local/lib/python2.7/dist-packages/mwlib/apps/make_nuwiki.py", line 189, in make_nuwiki pool.join(raise_error=True) File "/usr/local/lib/python2.7/dist-packages/gevent/pool.py", line 98, in join raise greenlet.exception simplejson.decoder.JSONDecodeError: No JSON object could be decoded: line 1 column 0 (char 0)
I can't even begin to work on the actual extension interface until I have this working..... Any suggestions? Where do I go next?
Thanks!
Yan Seiner wrote:
I wish I could say what I did to fix this... But it seems to work, at least from within mediawiki. I got started on a different problem, restarted apache a few times, and reloaded the page and the Print/Export menu just showed and and it creates zim files... :headscratch:
Figured it out... Was a strange DNS caching issue, nothing to do with mwlib. Sometimes the render server would resolve correctly, at other times it would not.
mediawiki-l@lists.wikimedia.org