Hi Frederic, thanks for your work. Have you tested 7z?
We can compress to xz while the new disks arrive. I read that it is about 24 TB, so, we can revert to gzip in the future.
2011/1/2 Frédéric Schütz schutz@mathgen.ch
On 12/28/2010 03:52 PM, emijrp wrote:
Well, there's 986 Gigs in /mnt/user-store/stats
Please, do not delete that files, they are important for stats.
Following a suggestion made by River, I am currently recompressing all files using the newly-installed "xz" program. Based on an arbitrary sample (all files from 1 January 2011), we can expect a size reduction of about 25% -- or about 250 Gb freed in total. The compression time, however, is quite long, and I haven't measured how the decompression time compares with gzip.
Let me know if the xz format causes any problem; I have started recompressing the older files, which are not used so often, but if this is ok with all users, I may start in the future recompressing the new files as soon as they are downloaded.
Hopefully, with the new storage space, we'll be able to store these files in a more convenient way (not only raw files as they are now, but also the same information in a better format).
Frédéric
Toolserver-l mailing list (Toolserver-l@lists.wikimedia.org) https://lists.wikimedia.org/mailman/listinfo/toolserver-l Posting guidelines for this list: https://wiki.toolserver.org/view/Mailing_list_etiquette