Hi!
The underlying filesystem (ZFS) uses block-level deduplication, so unique
chunks of 128KiB (default value) are only stored once. The 128KB chunks
making up dumps are mostly unique since there is no alignment so
deduplication will not help as far as I can see.
Best regards,
Count Count
On Tue, Jul 28, 2020 at 3:51 AM griffin tucker <gtucker4.une(a)hotmail.com>
wrote:
I’ve tried using freenas/truenas with a data
deduplication volume to store
multiple sequential dumps, however it doesn’t seem to save much space at
all – I was hoping someone could point me in the right direction so that I
can download multiple dumps and not have it take up so much room
(uncompressed).
Has anyone tried anything similar and had success with data deduplication?
Is there a guide?
_______________________________________________
Xmldatadumps-l mailing list
Xmldatadumps-l(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l