I am not an expert, but from what I understand it looks like a good thing for those who
cannot or will not edit. It will not help build the encyclopaedias, but at least could
make what exists more accessible.
Cheers,
Peter
-----Original Message-----
From: WikimediaZA [mailto:wikimediaza-bounces@lists.wikimedia.org] On Behalf Of Michael
Graaf
Sent: 24 June 2020 23:57
To: wikimediaza(a)lists.wikimedia.org
Subject: [Wikimedia ZA] Fwd: Offline-l Digest, Vol 100, Issue 10
Dear Wikimedians,
I'm sharing with all of you a digest email from the "offline-l list" to
which I subscribe since I believe it has two items of wider interest than
the usual ones on that list.
The first one concerns a new facility called Open ZIMfarm which automates
the curation of offline archives of web material (not just wikis). I
believe this is bringing a step-change in the possibilities of
decentralised content hosting.
The second describes the recent deployment of the Kiwix-serve application
in a commercial telecoms network in West Africa (Kiwix stores and presents
the very same ZIM files created by ZIMfarm). Given that the Wikimedia
Foundation no longer funds zero-rating of its products, this represents a
new way to bring content to people free of charge. Not only cellular
networks but local-government-supported WiFi providers such as Project
Isizwe as well as community-owned and -operated networks can do this.
I will be interested to know the feelings of the community on these.
Regards,
---------- Forwarded message ---------
From: <offline-l-request(a)lists.wikimedia.org>
Date: Wed, Jun 24, 2020 at 6:01 PM
Subject: Offline-l Digest, Vol 100, Issue 10
To: <offline-l(a)lists.wikimedia.org>
Send Offline-l mailing list submissions to
offline-l(a)lists.wikimedia.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.wikimedia.org/mailman/listinfo/offline-l
or, via email, send a message with subject or body 'help' to
offline-l-request(a)lists.wikimedia.org
You can reach the person managing the list at
offline-l-owner(a)lists.wikimedia.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Offline-l digest..."
Today's Topics:
1. Re: [KIWIX] Our openZIM farm... (Samuel Klein)
2. Re: [KIWIX] Our openZIM farm... (Emmanuel Engelhart)
3. [AAR] Interesting online/offline use case across West Africa
(Stephane Coillet-Matillon)
4. Re: [AAR] Interesting online/offline use case across West
Africa (Federico Leva (Nemo))
----------------------------------------------------------------------
Message: 1
Date: Wed, 24 Jun 2020 10:09:04 -0400
From: Samuel Klein <meta.sj(a)gmail.com>
To: Using Wikimedia projects and MediaWiki offline
<offline-l(a)lists.wikimedia.org>
Subject: Re: [Offline-l] [KIWIX] Our openZIM farm...
Message-ID:
<CAAtU9WJ14yPAJqikYqownk2Nh1mBnStC5TpROU8h+iJcDfuGMQ(a)mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
Wow, this is fabulous. If a new zimfarm starts up, can it coordinate with
existing ones?
On Tue, Jun 23, 2020 at 3:23 AM Emmanuel Engelhart <kelson(a)kiwix.org> wrote:
Hi
There is a topic I wanted to talk about here for a long time and for
which I never have achieved to take the time to write something. A few
recent events have been a healthy remember that I should present one our
most recent and most useful tool: Zimfarm.
The Zimfarm is the online tool which is in charge of building and
publishing all our ZIM files. After years of creating ZIM files by
launching scrapers more or less manually, we had to automatise the
process to just be able to scale the operations, ie. publishing more and
more often ZIM files.
The effort started 3 years ago with the support of the WMF but we use it
only since Spring 2019 in production. The tool is now perfectly running
and we fully rely on it now. If we can publish an update of all our
wikis one time a month, this is thanks to this piece of software too.
The Zimfarm is a half-decentralized solution which has a central node
(called "dispatcher") in charge of orchestrating the work to do and
multiple decentralized nodes (called "workers") which run the scraping
tasks.
The dispatcher provides an API to manage the ZIM recipes and tasks, have
a look to
https://api.farm.openzim.org/. We have setup a Web frontend on
this API to allow easy mgmt through a Web browser. For a better
transparency, even anonymous users can have a look and monitor what is
going on. Look at
https://farm.openzim.org/.
One important point is that, like all the rest of our infrastructure,
the whole system is Dockerized. Which means, this is really easy to
install a Zimfarm worker and we invite anybody having a spare server to
help us to provide offline snapshots of the best of the Web. The
procedure is documented and a few volunteers have already joined in.
Look at
https://farm.openzim.org/about for more details.
The development is fully transparent at
https://github.com/openzim/zimfarm. We have a few things which are on
the roadmap which would welcome volunteer Python developers. Look at the
good first issues and make your first PR!
Regards
Emmanuel
--
Kiwix - Wikipedia Offline & more
* Web:
https://kiwix.org/
* Twitter:
https://twitter.com/KiwixOffline
* Wiki:
https://wiki.kiwix.org/
_______________________________________________
Offline-l mailing list
Offline-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/offline-l
--
Samuel Klein @metasj w:user:sj +1 617 529 4266