wikipedia downloads only available with 2GB?!

List overview All Threads
Download

newer

older

Re: Wikispammer honeypot

MediaWiki won first prize at les...

Merlin

19 May 2005 19 May '05

3:39 p.m.

Hi there, I am trying to download the german "old" table on: http://download.wikimedia.org/#wikipedia The link says that the size of that file is 13.97 GiB. Howver, the file is just 2GB (definatelly) and there are no aditional files like described on the wikipedia download help page. Can somebody please give me some help on where to download that proper file? I tried this one, but it is not complete (which gzip is telling me). My guess is that the file is stored on a server which does not support files > 2GB and that this is just a mistake. Any suggestions? Thank you in advance, merlin

Show replies by date

Kate Turner

19 May 19 May

4:03 p.m.

Merlin wrote in gmane.science.linguistics.wikipedia.technical:

...

I am trying to download the german "old" table on: http://download.wikimedia.org/#wikipedia The link says that the size of that file is 13.97 GiB.

...

Howver, the file is just 2GB (definatelly) and there are no aditional files like described on the wikipedia download help page.

are you sure the program you are using to download it supports large files? ? 101> curl >/dev/null http://download.wikimedia.org/wikipedia/de/20050516_old_table.sql.gz % Total % Received % Xferd Average Speed Time Curr. Dload Upload Total Current Left Speed 22 14.0G 22 3.1G 0 0 27.5M 0 0:08:40 0:01:55 0:06:45 28.2M

...

merlin

kate.

Merlin

23 May 23 May

12:48 p.m.

Hi Kate, are you sure? The line gives me that output: curl >/dev/null http://download.wikimedia.org/wikipedia/de/20050516_old_table.sql.gz % Total % Received % Xferd Average Speed Time Curr. Dload Upload Total Current Left Speed 0 2047M 0 2600k 0 0 449k 0 1:17:48 0:00:05 1:17:42 511k04 I am trying this on suse 9.1 and files over 2GB are supported on the file system. Regards, Merlin Kate Turner wrote:

...

Merlin wrote in gmane.science.linguistics.wikipedia.technical:

I am trying to download the german "old" table on: http://download.wikimedia.org/#wikipedia The link says that the size of that file is 13.97 GiB.

Howver, the file is just 2GB (definatelly) and there are no aditional files like described on the wikipedia download help page.

are you sure the program you are using to download it supports large files? ? 101> curl >/dev/null

http://download.wikimedia.org/wikipedia/de/20050516_old_table.sql.gz

...

% Total % Received % Xferd Average Speed Time Curr. Dload Upload Total Current Left Speed 22 14.0G 22 3.1G 0 0 27.5M 0 0:08:40 0:01:55 0:06:45 28.2M

merlin

kate. _______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

Alfio Puglisi

1:12 p.m.

Old versions of curl cannot download files >2GB. Intermediate versions have a bug on the display. This is probably your problem. Newer version of curl can download files >2GB without problems. Mine is: $ curl -V curl 7.12.1 (i386-redhat-linux-gnu) libcurl/7.12.1 The previous version I had (7.10 something) failed. Alfio On Mon, 23 May 2005, Merlin wrote:

...

Merlin wrote in gmane.science.linguistics.wikipedia.technical:

I am trying to download the german "old" table on: http://download.wikimedia.org/#wikipedia The link says that the size of that file is 13.97 GiB.

Howver, the file is just 2GB (definatelly) and there are no aditional files like described on the wikipedia download help page.

are you sure the program you are using to download it supports large files? ? 101> curl >/dev/null

http://download.wikimedia.org/wikipedia/de/20050516_old_table.sql.gz

% Total % Received % Xferd Average Speed Time Curr. Dload Upload Total Current Left Speed 22 14.0G 22 3.1G 0 0 27.5M 0 0:08:40 0:01:55 0:06:45 28.2M

merlin

kate. _______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

_______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

Merlin

24 May 24 May

1:04 p.m.

Kate Turner wrote:

...

Merlin wrote in gmane.science.linguistics.wikipedia.technical:

I am trying to download the german "old" table on: http://download.wikimedia.org/#wikipedia The link says that the size of that file is 13.97 GiB.

Howver, the file is just 2GB (definatelly) and there are no aditional files like described on the wikipedia download help page.

merlin

kate. _______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

Hi, I am having serious trouble in updating the curl sw for my running system. That is not as trivial as it seams. That said, I have read that many people do have the same problem in downloading the larger files. Is there are reason not to place the files on an ftp server? That seams to be the apropriate protocol for file transfers. Why http? I can not see a reason, but I might lack technical knowledge for this. Can somebody shed some light on this? Thank you in advance, merlin

Timwi

5:45 p.m.

Merlin wrote:

...

I am having serious trouble in updating the curl sw for my running system. That is not as trivial as it seams. That said, I have read that many people do have the same problem in downloading the larger files. Is there are reason not to place the files on an ftp server? That seams to be the apropriate protocol for file transfers. Why http? I can not see a reason, but I might lack technical knowledge for this. Can somebody shed some light on this?

Just because it's called "File Transfer Protocol", doesn't mean you shouldn't transfer files with anything else. In fact, I prefer HTTP because it uses ONE connection only. FTP uses two. Of course, this fact shouldn't really make any difference to you. Whether you use FTP or HTTP doesn't make the transfers any faster. Nor does it make curl (or any other download tool) work any better or worse. Greetings, Timwi

Neil Harris

5:59 p.m.

Timwi wrote:

...

Merlin wrote:

Yes. Since the advent of https and range transfers and MD5-digest authentication in HTTP 1.1, HTTP is now my preferred protocol for large file transfers. Being firewall-friendly and stateless is an added bonus. -- Neil

Ævar Arnfjörð Bjarmason

10:35 p.m.

...

> In fact, I prefer HTTP because it uses ONE connection only. FTP uses two.

Not just two connections, but two connections using two different ports and two different protocols.

Brian

11:59 p.m.

Is rsync still setup for those files? On 5/24/05, Ævar Arnfjörð Bjarmason <avarab(a)gmail.com> wrote:

...

> In fact, I prefer HTTP because it uses ONE connection only. FTP uses

two. Not just two connections, but two connections using two different ports and two different protocols. _______________________________________________ Wikitech-l mailing list Wikitech-l(a)wikimedia.org http://mail.wikipedia.org/mailman/listinfo/wikitech-l

-- "What we cannot speak of we must pass over in silence." - Ludwig Wittgenstein "I'd hate to die twice. It's so boring." - Richard Feynman's last words -- http://www.br1an.net (website)

Merlin

25 May 25 May

1:23 p.m.

Timwi wrote:

...

Merlin wrote:

Hi Timwi, that sounds plausible. However it creates a huge problem since (speaking for myself) I am not able to get CURL updated. An update of openssl is needed which does affect the currrent apache installation and so on. Dependencies over and over. I cant get a client on the suse 9.1 system running which does support that download size. Has anybody a suggestion? Thank you for any hint. Merlin

Timwi

26 May 26 May

9:45 p.m.

Merlin wrote:

...

I am not able to get CURL updated. An update of openssl is needed which does affect the currrent apache installation and so on. Dependencies over and over. I cant get a client on the suse 9.1 system running which does support that download size.

Hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahaha hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahaha hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahaha hahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahahaha All hail free software!

...

Has anybody a suggestion? Thank you for any hint.

Use Windows.

Yaroslav Fedevych

10:15 p.m.

On Thu, May 26, 2005 at 10:45:00PM +0100, Timwi wrote:

...

Merlin wrote:

Has anybody a suggestion? Thank you for any hint.

Use Windows.

s/Windows/Debian/g; I think that we've got a troll here. Is this list moderated? If not, I'm in a mood to start a flamewar ;-) -- X Windows: Armageddon never looked so good.

6915

days inactive

6922

days old

wikitech-l@lists.wikimedia.org

Manage subscription

11 comments

9 participants

tags (0)

participants (9)

Alfio Puglisi
Brian
Kate Turner
Merlin
Merlin
Neil Harris
Timwi
Yaroslav Fedevych
Ævar Arnfjörð Bjarmason