dumps - Wikitech-l - lists.wikimedia.org

List overview All Threads
Download

dumps

dumps

On strike...

Domas Mituzas

16 May 2005 16 May '05

5:15 p.m.

Hello, new dumps (as of 20050516) available at http://download.wikimedia.org/ Please note, we no longer provide 2GB split files. I do sincerely believe, that HTTP has enough capability to do partial downloads of files ;-) And those splits simply double disk space requirements... Cheers, Domas

Reply

Show replies by date

Timwi

17 May 17 May

10:48 a.m.

Domas Mituzas wrote:

new dumps (as of 20050516) available at http://download.wikimedia.org/ Please note, we no longer provide 2GB split files. I do sincerely believe, that HTTP has enough capability to do partial downloads of files ;-) And those splits simply double disk space requirements...

Yay. I've always been wondering what sort of people would need the split files. For Windows users, I recommend GetRight to download the file in chunks and with interruptions.

Reply

Brion Vibber

11:30 a.m.

Timwi wrote:

new dumps (as of 20050516) available at http://download.wikimedia.org/ Please note, we no longer provide 2GB split files. I do sincerely believe, that HTTP has enough capability to do partial downloads of files ;-) And those splits simply double disk space requirements...

Yay. I've always been wondering what sort of people would need the split files.

We started providing split files simply because Apache on the servers they were running on would refuse to *serve* files larger than 2 gigabytes. If your OS is unable to handle files of that size, you're probably going to have problems with the database too. :) -- brion vibber (brion @ pobox.com)

Reply

Magnus Manske

1:36 p.m.

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Timwi schrieb:

For Windows users, I recommend GetRight to download the file in chunks and with interruptions.

What, you recommend mucho $$$ software when Free Download Manager [1] is available? Honestly, I *did* buy GetRight some years ago, but scrapped it for FDM. Magnus [1] http://www.freedownloadmanager.org/

...PGP SIGNATURE...

-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (MingW32) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCifNxCZKBJbEFcz0RAi47AJ98nLu5UW7aYjY/dRBlhPWf1mk4owCfX55Y O4fNeCuTBvhfY7NjNz30ov8= =TWwA -----END PGP SIGNATURE-----

Reply

Timwi

3:33 p.m.

Magnus Manske wrote:

Timwi schrieb:

For Windows users, I recommend GetRight to download the file in chunks and with interruptions.

What, you recommend mucho $$$ software

Uhm... no?... I recommend something which is free at least in the sense of "free beer".

[1] http://www.freedownloadmanager.org/

Haven't heard of that yet -- will try that later. I'll send my complaints about it to you via private mail, mwahaha :-)

Reply

Brian

6:43 p.m.

use curl to download large files on windows. http://curl.haxx.se/download.html On 5/17/05, Timwi <timwi(a)gmx.net> wrote:

Magnus Manske wrote:

Timwi schrieb:

For Windows users, I recommend GetRight to download the file in chunks and with interruptions.

What, you recommend mucho $$$ software

Uhm... no?... I recommend something which is free at least in the sense of "free beer".

[1] http://www.freedownloadmanager.org/

Haven't heard of that yet -- will try that later. I'll send my complaints about it to you via private mail, mwahaha :-) _______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

Reply

elwp＠gmx.de

18 May 18 May

8:59 a.m.

Domas Mituzas:

new dumps (as of 20050516) available at http://download.wikimedia.org/ Please note, we no longer provide 2GB split files. I do sincerely believe, that HTTP has enough capability to do partial downloads of files ;-) And those splits simply double disk space requirements...

The server doesn't handle requests for partial content correctly. This is the output of wget -d -c http://download.wikimedia.org/wikipedia/de/old_table.sql.gz after I interrupted the previous connection at a file size of 3907378719 Bytes. ----------------------------------------------------------------------- DEBUG output created by Wget 1.9+cvs-stable (Red Hat modified) on linux-gnu. --09:53:24-- http://download.wikimedia.org/wikipedia/de/old_table.sql.gz => `old_table.sql.gz' Auflösen des Hostnamen »download.wikimedia.org«.... 207.142.131.212 Caching download.wikimedia.org => 207.142.131.212 Verbindungsaufbau zu download.wikimedia.org[207.142.131.212]:80... verbunden. Created socket 3. Releasing 0x94a34c0 (new refcount 1). ---request begin--- GET /wikipedia/de/old_table.sql.gz HTTP/1.0 User-Agent: Wget/1.9+cvs-stable (Red Hat modified) Host: download.wikimedia.org Accept: */* Connection: Keep-Alive Range: bytes=3907378719- ---request end--- HTTP Anforderung gesendet, warte auf Antwort... HTTP/1.0 206 Partial Content Connection: keep-alive Date: Wed, 18 May 2005 07:51:15 GMT Content-Length: 12856795601 ETag: 1881960765 Accept-Ranges: bytes Content-Type: application/x-gzip Content-Range: bytes 2147483647-15004279247/15004279248 Server: Wikimedia dump service 20050409 (lighttpd) Found download.wikimedia.org in host_name_addresses_map (0x94a34c0) Registered fd 3 for persistent reuse. Closing fd 3 Releasing 0x94a34c0 (new refcount 1). Invalidating fd 3 from further reuse. Erneuter Versuch. ------------------------------------------------------------------- Obviously "Range" in the request and "Content-Range" in the response don't match. This results in wget trying to request the same over and over again. (Download doesn't start at all. The last line above, "Erneuter Versuch", means "next try".) Maybe this confused the T-Online proxies as well yesterday. My version of wget is the one that comes with Fedora Core 3 and claims to be able to handle files larger than 2GB. -- 5 GB Mailbox, 50 FreeSMS http://www.gmx.net/de/go/promail +++ GMX - die erste Adresse für Mail, Message, More +++

Reply

Alfio Puglisi

9:41 p.m.

Is a new stats run in progress? Alfio On Mon, 16 May 2005, Domas Mituzas wrote:

Hello, new dumps (as of 20050516) available at http://download.wikimedia.org/ Please note, we no longer provide 2GB split files. I do sincerely believe, that HTTP has enough capability to do partial downloads of files ;-) And those splits simply double disk space requirements... Cheers, Domas

Reply

Brian

9:44 p.m.

Yes. On 5/18/05, Alfio Puglisi <puglisi(a)arcetri.astro.it> wrote:

Is a new stats run in progress? Alfio On Mon, 16 May 2005, Domas Mituzas wrote:

Hello, new dumps (as of 20050516) available at http://download.wikimedia.org/ Please note, we no longer provide 2GB split files. I do sincerely

believe,

that HTTP has enough capability to do partial downloads of files ;-) And those splits simply double disk space requirements... Cheers, Domas

_______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

Reply

6967

days inactive

6969

days old

wikitech-l@lists.wikimedia.org

Manage subscription

8 comments

7 participants

tags (0)

participants (7)

Alfio Puglisi
Brian
Brion Vibber
Domas Mituzas
elwp＠gmx.de
Magnus Manske
Timwi