Hayley:
We have opened a task for this: T437454 Implement 7z compression for v2
history dumps <https://phabricator.wikimedia.org/T437454>.
It looks like we will go ahead with supporting it now that we have numbers
(see task), but do please chime in to further make the case.
Thanks,
-xabriel
On Tue, Aug 18, 2026 at 10:40 AM Hayley <hayleybale2(a)gmail.com> wrote:
> > We went with bzip2 for compatibility with existing dumps
>
> Well the old dumps provided 7z as well as bz2, and I was using 7z. So this
> is an incompatibility issue as well.
>
> On Mon, Aug 17, 2026 at 2:35 PM Xabriel Collazo Mojica <
> xcollazo(a)wikimedia.org> wrote:
>
>> Hayley:
>>
>> Apologies for the delay, I was on leave.
>>
>> The new infrastructure is built to support multiple compressions, and
>> hopefully better containers other than XML, but as you pointed out, today
>> we only support XML compressed via bzip2.
>>
>> We went with bzip2 for compatibility with existing dumps, and because it
>> is easy to split in systems like Apache Spark, but I hear you on the
>> decompression cost.
>>
>> We currently have no plans to reintroduce 7z, but feel free to open a
>> ticket on phabricator to make your case. There is a related ticket ( T324954
>> Provide zstd compression algorithm for dumps
>> <https://phabricator.wikimedia.org/T324954> ), which makes a similar
>> case on decompression cost.
>>
>> -xabriel
>>
>> On Sat, Jul 25, 2026 at 1:12 PM Hayley via Xmldatadumps-l <
>> xmldatadumps-l(a)lists.wikimedia.org> wrote:
>>
>>> Hello, I am wondering with the transition to the new data dumps
>>> infrastructure, will 7z-compressed dumps be provided at some point? The old
>>> dumps pipeline provided both bz2 and 7z. 7z is much preferable to bz2, as
>>> bz2 is several times larger, and at the same time, also several times more
>>> computationally expensive to decompress.
>>> _______________________________________________
>>> Xmldatadumps-l mailing list -- xmldatadumps-l(a)lists.wikimedia.org
>>> To unsubscribe send an email to xmldatadumps-l-leave(a)lists.wikimedia.org
>>>
>>
>>
>> --
>> Xabriel J. Collazo Mojica (he/him, pronunciation
>> <https://commons.wikimedia.org/wiki/File:Xabriel_Collazo_Mojica_-_pronunciat…>
>> )
>> Staff Data Engineer
>> Wikimedia Foundation
>>
>
--
Xabriel J. Collazo Mojica (he/him, pronunciation
<https://commons.wikimedia.org/wiki/File:Xabriel_Collazo_Mojica_-_pronunciat…>
)
Staff Data Engineer
Wikimedia Foundation