----- Original Message -----
From: Brian J Mingus <brian.mingus@Colorado.EDU>
Date: Tuesday, March 29, 2011 7:15 pm
Subject: Re: [Wikitech-l] [Xmldatadumps-l] March 17 en wikipedia history bz2 files ready
To: Wikimedia developers <wikitech-l@lists.wikimedia.org>
Cc: Jamie Morken <jmorken@shaw.ca>, "Ariel T. Glenn" <ariel@wikimedia.org>, xmldatadumps-l@lists.wikimedia.org
> >
> According to this data the 7z dump for enwp will reach 1
> terabyte on Jan 2,
> 2145.
>
> =)
Hi,
I made a graph for the uncompressed XML file size for the enwiki pages-meta-history files over time, I thought that these files would be growing exponentially but they appear to grow linear. For comparison in 2145 the raw XML should be about 178 TB I think, so the 7z files are growing linearly about 180x faster than the raw XML.
"http://nekrom.com/wikipedia/enwiki%20history%20uncompressed%20XML%20dump%20file%20size%20over%20time.png"
(data below)
cheers,
Jamie
enwiki-20060816-pages-meta-history.xml 782741875000 (728.99 GB)
enwiki-20070402-pages-meta-history.xml 1763048493749 (1641.97 GB) (229 days since previous dump)
enwiki-20080103-pages-meta-history.xml 2807444044080 (2614.64 GB) (276 days since previous dump)
enwiki-20100130-pages-meta-history.xml 5873134833455 (5469.78 GB) (758 days since previous dump)
enwiki-20110115-pages-meta-history[1-15].xml 7218617857754 (6722.86 GB) (350 days since previous dump)
enwiki-20110115-pages-meta-history1.xml 1 080 719 385 129
enwiki-20110115-pages-meta-history2.xml 677 956 948 289
enwiki-20110115-pages-meta-history3.xml 550 889 319 423
enwiki-20110115-pages-meta-history4.xml 447 001 611 247
enwiki-20110115-pages-meta-history5.xml 453 700 983 270
enwiki-20110115-pages-meta-history6.xml 540 208 590 115
enwiki-20110115-pages-meta-history7.xml 458 817 000 243
enwiki-20110115-pages-meta-history8.xml 649 710 293 818
enwiki-20110115-pages-meta-history9.xml 471 183 250 318
enwiki-20110115-pages-meta-history10.xml 406 115 459 739
enwiki-20110115-pages-meta-history11.xml 342 840 308 580
enwiki-20110115-pages-meta-history12.xml 310 507 626 798
enwiki-20110115-pages-meta-history13.xml 362 264 384 002
enwiki-20110115-pages-meta-history14.xml 269 988 897 698
enwiki-20110115-pages-meta-history15.xml 196 713 799 085
>
> --
> Brian Mingus
> Graduate student
> Computational Cognitive Neuroscience Lab
> University of Colorado at Boulder
>