Offline-l

offline-l@lists.wikimedia.org

696 discussions

How libzim and zimwriterfs handle the bigger than memory nature of Wikipedia dumps?
by Amirouche Boubekki 21 Jun '20

21 Jun '20

Hello, I am new to the mailing list. I used to work on sotoki. My question is somewhat related to my failed attempt to store stackoverflow dumps inside wiredtiger. Eventually, I figured that wiredtiger could not keep up with the write load and that it is a pain point at least with wiredtiger (but also with sqlite lsm extension). The workaround is to have as much as RAM as the data (which is in my opinion not acceptable), and fine-tune eviction triggers et al. My questions are about libzim, zimwriterfs and how full-text search is implemented: 1) Why zimwriterfs or libzim succeed at putting together all the html dumps of wikipedia? Is it because they're a lot of RAM? Or is it a particular algorithm? 2) Follow up question, how the full-text search is put together? Is the index written document by document then packed into the zim file? I am working on my free time on a search engine [1], my goal is to have my own search engine that I can use locally. That is why, I was thinking about kiwix, because kiwix via .zim files provide readily available dumps of many useful resources. The last question is: 3) how can I read the content of .zim from C code? Are there C bindings of libzim? Thanks! [0] It does not work anymore but the code is at https://github.com/amirouche/babelia

2 1

Fwd: Movement Strategy: transition to implementation begins
by Florence Devouard 10 Jun '20

10 Jun '20

Hello everyone, I hope all of you are safe. I am forwarding this email from Ryan from wikimedia-l, about the next steps of the implementation process of the Strategy. The team is looking for a set of diverse people to prepare the virtual transition discussions. The placeholder page on meta : https://meta.wikimedia.org/wiki/Strategy/2030 There is an office meeting with the Movement Strategy core team on Thursday. June 11 @ 08:00 UTC (Google Meet <http://meet.google.com/rva-yqaq-zdk>) [7] to share any comments and ask questions. I think that if you are interested in joining this group, attending the office meeting or getting in touch with a regional group might be a good idea. Florence -------- Message transféré -------- Sujet : [Wikimedia-l] Movement Strategy: transition to implementation begins Date : Tue, 9 Jun 2020 17:27:30 -0400 De : Ryan Merkley <rmerkley(a)wikimedia.org> Répondre à : Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org> Pour : Wikimedia Mailing List <wikimedia-l(a)lists.wikimedia.org> Dear all, It’s my pleasure to be writing to you about the Movement Strategy. We have come together over the past 3 years to develop our Strategic Direction [1] and then the recommendations [2] to guide our collaborations and future. So many of you contributed to the process — some for only a part, and others throughout the entire process. With the recommendations complete, our focus now shifts to implementing the recommendations in a collaborative, open, and transparent way. Transition To make the transition from the publication of the recommendations to their implementation, we need to do some planning. It’s a 10-year strategy, with dozens of initiatives, dependencies, and connected goals, to be delivered in a more distributed, deliberative, and open collaborative model than ever before. We’ll need to work together to define how we prioritize, sequence, resource, and support each initiative. So much of our work is done online, but a lot of strategic work also happens in person. We can’t do that now, and so we’ve had to adapt to engage broadly, and in inclusive ways. To create this plan, movement-wide virtual events will kick off in September. We will use the Movement Strategy principles [3] as a guiding framework to ensure the planning will be inclusive and empowering for our diverse range of communities, without leaving anyone behind. As a result of the pandemic, we lost the chance to work together in-person on the transition to implementation at the Wikimedia Summit in Berlin [4]. Yet we gained an opportunity to include a higher number and a more diverse profile of participants. Engaging with online contributors, technical developer communities, and smaller user groups throughout the process will be a key priority. Successful virtual engagements with a high number of diverse participants are difficult to do well. Therefore, a Design Group will collaborate to prepare for the virtual transition discussions. This group will consist of community members reflecting different parts and perspectives of the movement, including representatives of regional collaboratives (CEE, ESEAP, Indaba, Iberocoop, North America, South Asia, WikiArabia, WikiFranca), the EDs and chairpersons groups, and WMF staff. Anyone who is interested can contribute. Regular summaries of the preparation work and design discussions will be published on meta so that anyone interested will be able to share insights and help improve the process, even if not part of the Design Group itself. Participation I look forward to: Working with many of you at the virtual transition events. Ways to participate and the schedule of events will be determined by the Design Group. The current plan is to start the virtual transition discussions with the movement in September. The virtual events is where major discussions will take place on sequencing, prioritizing, and resourcing the recommendations across the movement. Seeing those of you interested participate in the open review of the transition preparations. The task will be to review the work of the Design Group and share your perspective, enriching the thinking to improve the events. Open review will happen in parallel to the work of the Design Group from the end of June to the end of July / beginning of August. Having nominations from different movement groups and collaboratives (mentioned above) for the Design Group. The task will be to design as a group how the transition process of online events will be set up. Want to know more? We have put together a placeholder meta page [5] and will keep updating it as more information becomes available. Join office hours with the Movement Strategy core team on Wednesday. June 10 @ 17:00 UTC (Google Meet <http://meet.google.com/uun-pzmb-kti>) [6] or Thursday. June 11 @ 08:00 UTC (Google Meet <http://meet.google.com/rva-yqaq-zdk>) [7] to share any comments and ask questions. Our email channel is always open: strategy2030(a)wikimedia.org <mailto:strategy2030@wikimedia.org>. Best, Ryan Merkley Chief of Staff, Wikimedia Foundation [1] https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2017 <https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2017> [2] https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recomme… <https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recomme…> [3] https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recomme… <https://meta.wikimedia.org/wiki/Strategy/Wikimedia_movement/2018-20/Recomme…> [4] https://meta.wikimedia.org/wiki/Wikimedia_Summit_2020/Report <https://meta.wikimedia.org/wiki/Wikimedia_Summit_2020/Report> [5] https://meta.wikimedia.org/wiki/Strategy/2030 <https://meta.wikimedia.org/wiki/Strategy/2030> [6] https://meet.google.com/uun-pzmb-kti <https://meet.google.com/uun-pzmb-kti>[7] https://meet.google.com/rva-yqaq-zdk <https://meet.google.com/rva-yqaq-zdk> _____________________________ Ryan Merkley (he/him) Chief of Staff, Wikimedia Foundation <https://wikimediafoundation.org/> rmerkley(a)wikimedia.org <mailto:rmerkley@wikimedia.org> @ryanmerkley <https://twitter.com/ryanmerkley> _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and https://meta.wikimedia.org/wiki/Wikimedia-l New messages to: Wikimedia-l(a)lists.wikimedia.org Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:wikimedia-l-request@lists.wikimedia.org?subject=unsubscribe>

3 3

Wikimedia Brand Project LIVE Unified Concept presentation Thursday 16 April from 1500 to 1600 UTC
by Florence Devouard 16 Apr '20

16 Apr '20

WMF Brand Project team will be holding the LIVE Unified Concept presentation Thursday 16 April from 1500 to 1600 UTC. You can join live on Youtube using this link https://www.youtube.com/watch?v=xS72O6Si94Q Join Snøhetta and the Brand Project team to review, celebrate, and conclude the first phase of the 2030 Movement Brand Project; with a unified “concept”. Listen in as Snøhetta explains how they used movement members' ideas, feedback and conceptual models to arrive at a final, unified concept. This unified concept will be the foundation for the upcoming work around the naming conventions, visual decisions, and the overall design system. This session will also be recorded and made available for viewing after. Please note: A concept is not a visible part of brand and design projects, but a guiding idea or spirit, summarized in one word. This is not a presentation of a proposal for a naming convention or design. The topic is currently discussed on wikimedia-l And a few additional links.... *https://meta.wikimedia.org/wiki/Communications/Wikimedia_brands/2030_movement_brand_project * https://meta.wikimedia.org/wiki/Requests_for_comment/Should_the_Foundation_… Florence

1 0

Wikimedia Summit cancelled
by Florence Devouard 29 Feb '20

29 Feb '20

Dear all, It’s with regret that we have to inform you that due to the global health situation (COVID-19), the physical meeting of the Wikimedia Summit 2020 and related side events in Berlin have been cancelled. Since we currently look into hosting the Summit in a series of virtual meetings, starting at the dates of the Summit, please keep the dates blocked in your calendar. We are aware that this announcement will cause many questions (not only in relation to the Summit). Please be patient, more information will follow from our teams in the next few days. Best regards, Abraham Taherivand, Executive Director Wikimedia Deutschland Katherine Maher, Executive Director Wikimedia Foundation

1 0

wikifundi synchronisation
by Manfred Gruening 11 Feb '20

11 Feb '20

Somewhere I read that wikifundi is working towards synchronizing with the online wikipedias. Is this something for the near future or very far out??? That would be REALLY exciting! Also, can wikifundi be used for a wikipedia / wiktionary in a newly setup language - or only for the few languages currently available for download??? Thanks, Manfred

3 2

(no subject)
by Samuel Klein 14 Jan '20

14 Jan '20

WMF is inviting members of our group to fill out a survey[1] -- to help with strategic planning. If you have a moment, please fill it out, if only briefly.. Link to survey: [1] https://wikimedia.qualtrics.com/jfe/form/SV_6z2rDBS5Iom0NKt 🌍🌏🌎🌑

1 0

Kiwix wins Mozilla MOSS award
by Stephane Coillet-Matillon 06 Dec '19

06 Dec '19

Hey all, We got a bunch of prizes this year but the one award that really matters is the one that was officially confirmed this morning: the Mozilla Foundation has selected Kiwix to build ZIMit, a universal scraper to turn online content to the offline-compatible ZIM format. The plan at this stage is to start work early next year and have the project completed by end of 2020. Ultimately we hope that everyone will be able to create their own zim files to distribute in offline environments :-) Cheers, Stephane

1 0

2-for-1 release announcement, aka 32 and 64-bit goodness
by Stephane Coillet-Matillon 02 Dec '19

02 Dec '19

Hello all, we have published our latest release candidate for Kiwix-dekstop (Windows & GNU/Linux)! And if you are running deployments on old machines (like, XP/Vista/7 old, ie 32-bits), then stay tuned until the end of this mail - Kiwix-desktop 2.0 RC4: nothing major, but still a few improvements. Here's the log of new features and fixes for this RC4: * New "Settings" page * Better aria2c error message * Fix Reading list with bookmarks from missing books * Support video fullscreen mode * Support the default zoom level setting * Support zoom level per content * Remove page view source feature * Stop crashing if a download fails - Now for the interesting bit: Kiwix-JS has been ported to work on all flavours of Windows 32-bit systems. It actually runs on all architectures, but that'll be especially relevant to anyone having to deal with older machines (Windows 7 and below, there's still plenty of XP and Vista around). Please note that just as with the (Firefox/Chrome) browser extensions, there might be some limitations compared to the real thing. But hey, the reference point here is having nothing instead since the old Kiwix-desktop 0.9 doesn't handle so well the new flavours of our zim files. But even then, Jaifroid and Mossroy have done nothing less than an Amazing Job*. You can download it all here: * Windows x86 64 bits: https://download.kiwix.org/release/kiwix-desktop/kiwix-desktop_windows_x64.… [https://download.kiwix.org/release/kiwix-desktop/kiwix-desktop_windows_x64.…] * Linux x86 64 bits: https://download.kiwix.org/release/kiwix-desktop/kiwix-desktop_x86_64.appim… [https://download.kiwix.org/release/kiwix-desktop/kiwix-desktop_x86_64.appim…] or https://download.kiwix.org/release/kiwix-desktop/org.kiwix.desktop.flatpak [https://download.kiwix.org/release/kiwix-desktop/org.kiwix.desktop.flatpak]. Also available on Flathub https://flathub.org/apps/details/org.kiwix.desktop [https://flathub.org/apps/details/org.kiwix.desktop] * Kiwix-JS for Windows x86 32 bits https://download.kiwix.org/release/kiwix-js-windows/kiwix-js-windows_0.9.9.… [https://download.kiwix.org/release/kiwix-js-windows/kiwix-js-windows_0.9.9.…] * Kiwix-JS Chrome extension https://chrome.google.com/webstore/detail/kiwix/donaljnlmapmngakoipdmehbfci… [https://chrome.google.com/webstore/detail/kiwix/donaljnlmapmngakoipdmehbfci…] * Kiwix-JS Firefox extension https://addons.mozilla.org/fr/firefox/addon/kiwix-offline/ [https://addons.mozilla.org/fr/firefox/addon/kiwix-offline/] Thanks to everyone who made these possible. Enjoy, Stephane * alternative wording would be Stellar, Excellent, Great, YUGE.

2 2

Knowledge equity advent calendar
by Stephane Coillet-Matillon 01 Dec '19

01 Dec '19

Hi everyone, I see that Wikimedia Deutschland is running an advent calendar on knowledge equity (ie access to knowledge for, you know, people without access).[1] They have a couple of stories lined up already on famously sidelined places like Norway and Taiwan. I'd expect most of the 20+ others to be written up already, yet to my knowlege they haven't reached out to Kiwix so we could point them to people doing deployments in Africa, India, South America, US & European prisons und so weiter. By the look of it they seem very much focused on people building knowledge rather than distributing it, and I can see a couple of good reasons for that. But I'm still half hoping that they did, in fact, reach out to someone else on this list and that offline access will be mentioned somehow. Can anyone confirm they spoke with our German friends? Cheers, Stephane [1] https://meta.wikimedia.org/wiki/Knowledge_Equity_Calendar [https://meta.wikimedia.org/wiki/Knowledge_Equity_Calendar]

1 0

Summarizing offline work this past year
by Samuel Klein 09 Nov '19

09 Nov '19

Dear WOW, We should compile a report on offline work this past year. If you have run an offline event, project, or partnership, please add it to the December report here: https://meta.wikimedia.org/wiki/Offline_Projects/Reports Warmly, SJ -- Samuel Klein @metasj w:user:sj +1 617 529 4266

1 0

← Newer
1
...
7
8
9
10
11
12
13
...
70
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Offline-l