Offline-l

offline-l@lists.wikimedia.org

7 participants
692 discussions

WM chat
by Michael Graaf 31 Aug '20

31 Aug '20

Hi all, just letting you know that the Foundation is now hosting a Mattermost chat server, which already has a "wiki-offline" channel. See https://chat.wmcloud.org/signup_email Cheers, -- Michael Graaf, M.Phil (UCT) Researcher, Editor & Community Informatics Practitioner Mob +27795487242 WhatsApp +27647754342 ORCID 0000-0002-1951-5739

1 0

[KIWIX][GUTENBERG] 60, 000 public domain books can fit on a flashdrive
by Emmanuel Engelhart 28 Aug '20

28 Aug '20

60,000 public domain books can fit on a flashdrive There are still parts of the world where access to the global internet is limited. Rural parts of world from Alaska to Africa to Asia have bandwidth to that is unevenly distributed. Governments regularly cut off website and apps access to their residents, with education often a collateral damage. There is and will always be a need for offline access to online resources. Project Gutenberg <https://gutenberg.org/> is a 48 year old organization whose mission is to make public domain works available for free on the internet. For the past two years, the Free Ebook Foundation has been working with Project Gutenberg to improve their website and ebook-building infrastructure. Kiwix has been developing content modules for some time now to facilitate their distribution effort. It was only a matter of time for these two orgs to meet and plan mischief together. Together with a team of computer-science seniors from Steven Institute of Technology <https://www.stevens.edu/>, and building on our development work with Project Gutenberg, the Free Ebook Foundation undertook to improve the usability and currency of Kiwix-developed Gutenberg content modules. The students added bookshelf functionality to the modules, and helped to streamline the process of module creation. These enhanced content modules are available today on the Kiwix website – a list is available <https://ebookfoundation.org/openzim.html> on the Free Ebook Foundation website, along with links to a demo site and the software used to create the content modules. USB flash drives that can store 128GB of data are can be found for as low as $20 on most online stores – that’s more than enough storage for all 60,000+ books. About the Free Ebook Foundation The Free Ebook Foundation <https://ebookfoundation.org> is a 501(c)3 New Jersey non-profit corporation. The Free Ebook Foundation envisions a world where ebooks will be funded, distributed and maintained for the benefit of all, by coordinating the efforts and resources of many. About Project Gutenberg Project Gutenberg <http://www.gutenberg.org/> was the first provider of free electronic books. Its mission is to encourage the creation and distribution of eBooks. Thousands of volunteers digitized and diligently proofread the eBooks, for enjoyment and education. About Kiwix Kiwix is an offline reader for online content like Wikipedia, Project Gutenberg, or TED Talks. It makes knowledge available to people with no or limited internet access. The software as well as the content is free to use for anyone. Kiwix is a non-profit organisation and is based in Switzerland. Our software is open-source available for free. The work we do is partially provided by volunteers, partially funded by individual donations, and grants from foundations. -- Kiwix - Wikipedia Offline & more * Web: https://kiwix.org/ * Twitter: https://twitter.com/KiwixOffline * Wiki: https://wiki.kiwix.org/

1 0

Invitation to affiliate members to complete a survey about WMF wide Universal Code of Conduct
by Florence Devouard 26 Aug '20

26 Aug '20

Transfered from Mervat She is inviting you to answer this survey, to help the Trust and Safety team to create a safter environment for all on our projects with the design of a global set of conduct rules. Please weight in Florence ==Message to Members== Hello; My name is Mervat, and I am helping the Trust and Safety team to reach out to affiliates in order to discuss the Universal Code of Conduct. There has been talks about the need for a global set of conduct rules in different communities over time, and finally, Wikimedia Foundation Board announced a Community Culture Statement, enacting new standards to address harassment and promote inclusivity across projects. Since the universal code of conduct that will be a binding minimum set of standards across all Wikimedia projects, will apply to all of us, staff and volunteers alike, all around the globe, and will impact our work as groups, individuals and projects, it’s of great importance that we all participate in expressing our opinions and thoughts about UCoC, its nature, what we think it should cover or include and what it shouldn’t include; how it may develop, drawback or help our groups. This is the time to talk about it. As you are a valuable contributor to the Wikimedia movement, your voice counts. Before starting to draft the code of conduct, we would like to hear from you; We invite you to devote some minutes to take this survey; your answers will help us create a safer environment for all on Wikimedia: (English): https://docs.google.com/forms/d/e/1FAIpQLSd36dNdU3C5shXEkKp9itJOhuCTx9hZE5A… <https://docs.google.com/forms/d/e/1FAIpQLSd36dNdU3C5shXEkKp9itJOhuCTx9hZE5A…> It’s possible that you are a member of more than one affiliate; hence you may receive this survey more than once, and you may have participated previously in the discussions or filled out a previous survey during the first round of consultations about UCoC which targeted wikipedia/wikimedia communities. We apologize for this; it is really difficult to identify if a wikimedianbelongs to multiple groups that work to spread free knowledge. Looking forward to your thoughts and opinions and hoping that you can respond within the next 2 weeks. If you have any questions about the surveys or difficulties accessing the link, please contact me by email (msalman-ctr(a)wikimedia.org <mailto:msalman-ctr@wikimedia.org>) Results will be considered during the drafting process for the UCoC Thank you for your participation Mervat Salman Trust & Safety (Policy) Facilitator Wikimedia Foundation <https://wikimediafoundation.org/> "Imagine a world in which every single human being can freely share in the sum of all knowledge. That's our commitment."

1 0

[WIKIMEDIA TECH TALK] In 3 HOURS: openZIM/Kiwix ETL toolchain for Wikipedia dumping
by Emmanuel Engelhart 26 Aug '20

26 Aug '20

Hi In 3 hours I will make a Wikimedia Tech Talk to present our toolchain to publish offline content: ==== Enjoying Wikipedia offline wherever, whenever is easy with Kiwix. But behind the scenes, a bunch of tools are needed to make it work. From article selection to dump publishing through scraping, optimisation and packaging: here is a quick overview of how we do it. ==== Join at: https://www.youtube.com/watch?v=9rFWaPSP3AM Regards Emmanuel -- Kiwix - Wikipedia Offline & more * Web: https://kiwix.org/ * Twitter: https://twitter.com/KiwixOffline * Wiki: https://wiki.kiwix.org/

1 0

UserGroup Report :)
by Florence Devouard 26 Aug '20

26 Aug '20

Hello everyone This is this time of the year where... report is back ! Please take a few minutes to summarize the last few months * Achievements * Challenges * Next steps * Anything you think relevant... https://meta.wikimedia.org/wiki/Offline_Projects/Reports/June_2020 Thanks in advance Flo

1 1

[OPENZIM] Introducing Python-libzim
by Emmanuel Engelhart 28 Jul '20

28 Jul '20

Hi I'm happy to introduce you to Python-libzim. Python-libzim package allows you to read/write ZIM files in Python. It provides a shallow Python interface on top of the libzim C++ library. It supports out-of-the-box macOS and GNU/Linux. For the other OSes you will have to compile the libzim manually. After Node.js, this is the second scripting language for which openZIM proposes a binding of its famous reference implementation of the ZIM open specification. This move is really important to allow more people to benefit of the file format and ZIM files already published. On our side, Python-libzim was critical for a few other projects which are currently running. In the next months a few critical scrapers will be migrated from zimwriterfs to python-libzim and benefit of a sensitive code simplification and speed-up. Install easily python-libzim with pip and give it a try: https://pypi.org/project/libzim/ Happy coding! Regards Emmanuel -- Kiwix - Wikipedia Offline & more * Web: https://kiwix.org/ * Twitter: https://twitter.com/KiwixOffline * Wiki: https://wiki.kiwix.org/

5 6

[KIWIX] Release 2.8.0 of Kiwix JS + Microsoft Edge Extension
by Emmanuel Engelhart 16 Jul '20

16 Jul '20

Hi We just have released version 2.8.0 of Kiwix JS browser extensions for Firefox & Chrome and Kiwix JS for Windows. You can find the new browser extensions: * Chrome: https://chrome.google.com/webstore/detail/kiwix/donaljnlmapmngakoipdmehbfci… * Firefox: https://addons.mozilla.org/en-US/firefox/addon/kiwix-offline/ From now we publish as well a version of Microsoft Edge Browser: https://microsoftedge.microsoft.com/addons/detail/kiwix-js/jlepddlenlljlnnh… Kiwix JS 2.8.0 Changelog: * NEW: Intelligently select the best Storage API for storing settings between sessions (cookie or localStorage) * NEW: Display title search results progressively instead of waiting for search to complete before displaying them * NEW: User can change depth of title search from Configuration * UPDATE: Better handling of case in title search, returning more accurate search results * UPDATE: Use const to declare constants, enabling better app memory management * UPDATE: Use SameSite=Strict for any residual cookie usage to conform to new security standards * BUGFIX: Major blocking bug causing high memory usage and slow load times when reading MIME type table of recent ZIM archives All our Windows apps based on Kiwix JS have been updated in Microsoft App Store. You can enjoy our apps on all Windows based devices (mobiles, tablets, PC, Xbox, ...): * Wikivoyage: https://www.microsoft.com/fr-ch/p/wikivoyage-by-kiwix/9n5sb90q4jbj * Wikimed: https://www.microsoft.com/fr-ch/p/wikimed/9phjsnp1cz8j * Kiwix: https://www.microsoft.com/fr-ch/p/kiwix-js/9p8slz4j979j Please report any problem. Work continues on our forge at https://github.com/kiwix/kiwix-js and https://github.com/kiwix/kiwix-js-windows. Regards Emmanuel -- Kiwix - Wikipedia Offline & more * Web: http://www.kiwix.org * Twitter: https://twitter.com/KiwixOffline * more: http://www.kiwix.org/wiki/Communication

1 0

Re: [Offline-l] Offline-l Digest, Vol 101, Issue 3
by Tim Moody 04 Jul '20

04 Jul '20

Congratulations on completing the huge task of producing the EN Wikip. I also look forward to pyzim. Looking at the notes I see an example of reading an article. However, I would like to be able to read the zim metadata. Is this possible? Even further afield would it be possible to extract the search index so as to merge it with another index, even one prepared from another source? As always great work, Tim On Sat, Jul 4, 2020 at 8:00 AM <offline-l-request(a)lists.wikimedia.org> wrote: > Send Offline-l mailing list submissions to > offline-l(a)lists.wikimedia.org > > > Today's Topics: > > 1. At long last, a new version of offline enwp > (Stephane Coillet-Matillon) > 2. Re: At long last, a new version of offline enwp > (Emmanuel Engelhart) > 3. [OPENZIM] Introducing Python-libzim (Emmanuel Engelhart) > 4. Re: [OPENZIM] Introducing Python-libzim (Wilfredo Rodríguez) > > >

2 1

At long last, a new version of offline enwp
by Stephane Coillet-Matillon 03 Jul '20

03 Jul '20

Hi everyone, quick announcement for a major success on our side: we finally released late last night an updated version of the English Wikipedia[1]. The last full version (ie. with images) we had was from October 2018 (!), and since then we had been plagued by regressions, bugs, resource limitations and probably some very dark magic. The new .zim file adds 900,000 articles (6.1 vs. 5.2 millions) and a healthy 11 Gb in size (89 vs. 78 Gb). The numbers are somewhat misleading because we need to include internal links and redirects, which brings the total to 100+ million interlinked items. Emmanuel will have more details on the hurdles that he had to deal with. Updates will now run on a monthly basis, which is another major improvement: we had initially planned on bimonthly updates as a single run used to take up to three weeks. It can now be done in 5-6 days \o/ Congrats to everyone involved or who supported us one way or another, including the Foundation with the new & bigger servers they recently gave us access to. Hopefully now we can move on to newer problems. Stephane [1] http://download.kiwix.org/zim/wikipedia/wikipedia_en_all_maxi_2020-06.zim <http://download.kiwix.org/zim/wikipedia/wikipedia_en_all_maxi_2020-06.zim> You can also torrent it by adding .torrent at the end (seeders welcome, as a matter of fact)

2 1

HTML dumps
by Samuel Klein 01 Jul '20

01 Jul '20

A recent discussion around high-volume APIs led to a ticket about generating HTML dumps on the foundation servers: https://phabricator.wikimedia.org/T254275 Kelson weighed in there, highlighting that it isn't clear everyone working on this is aware of the existing kiwix pipeline. Seemed worth mentioning here. SJ --

3 3

← Newer
1
...
5
6
7
8
9
10
11
...
70
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Offline-l