Tim Starling wrote:
About 40% of our text storage has been recompressed into DiffHistoryBlob format, which uses a combination of binary diffs and gzip to reduce storage space.
Approximately 1.9TB of text storage, mostly revisions compressed individually with gzip, was recompressed to about 140GB, a saving of 93%.
-- Tim Starling
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Many thanks to Tim for making this happen.
This has been super helpful in making the XML snapshots run faster.
Is the re-compression in an automated enough state to do the next chunks on its own? Curious to see if you have to do all the shepherding for this.
--tomasz