if you use one of the utilities listed here:
https://phabricator.wikimedia.org/T239866
I'd like you to download one of the 'multistream' dumps and see if your
utility decompresses it fully or not (you can compare the md5sum of the
decompressed content to the regular file's decompressed content and see if
they are the same). Then note the results and the version of the utility on
this task.
Alternatively, if you use some other utility to work with the bz2 files,
please test using that, and add that on the task too.
Here are two files for download and comparison of decompressed content:
https://dumps.wikimedia.org/cewiki/20191201/cewiki-20191201-pages-articles.…
and
https://dumps.wikimedia.org/cewiki/20191201/cewiki-20191201-pages-articles-…
Both are around 50 megabytes.
Thank you in advance to whomever participates!
Ariel
Greetings XML Dump users and contributors!
This is your automatic monthly Dumps FAQ update email. This update
contains figures for the 20191101 full revision history content run.
We are currently dumping 910 projects in total.
---------------------
Stats for wikimania2015wiki on date 20191101
Total size of page content dump files for articles, current content only:
15017153
Total size of page content dump files for all pages, current content only:
15907454
Total size of page content dump files for all pages, all revisions:
245100957
---------------------
Stats for enwiki on date 20191101
Total size of page content dump files for articles, current content only:
74373625381
Total size of page content dump files for all pages, current content only:
165746251934
Total size of page content dump files for all pages, all revisions:
19823027933877
---------------------
Sincerely,
Your friendly Wikimedia Dump Info Collector