Considering that dumps.wikimedia.org serves files in excess of 100 GB, I don't think HTTP the protocol is the issue. However when you try to request the file (or even just do a HEAD request) the web server closes the connection before sending any data, suggesting its a web server issue, not an HTTP the protocol issue.
-bawolff
Message: 6 Date: Tue, 09 Nov 2010 09:51:48 +0100 From: Peter K?rner osm-lists@mazdermind.de Subject: Re: [Toolserver-l] Fwd: [Tagging] Download huge file via HTTP To: toolserver-l@lists.wikimedia.org Message-ID: 4CD90BA4.3070207@mazdermind.de Content-Type: text/plain; charset=ISO-8859-1; format=flowed
No comments on that?
Peter
Am 06.11.2010 20:18, schrieb Peter K?rner:
Hi
I tried to provide a download link to a complete OSM Render Stack in a Virtual Machine on the toolserver. The files are located at /mnt/user-store and symlinked into my public_html.
This works for the small files (md5sums) but the 3.1G VMDK File is not accessible: http://toolserver.org/~mazder/osm_vm/
Is it possible to serve such a large file via http?
Peter
On 09.11.2010 15:23, bawolff wrote:
Considering that dumps.wikimedia.org serves files in excess of 100 GB, I don't think HTTP the protocol is the issue. However when you try to request the file (or even just do a HEAD request) the web server closes the connection before sending any data, suggesting its a web server issue, not an HTTP the protocol issue.
-bawolff
Some servers, clients and proxies break on the 31 (or 32) bit boundary (2 or 4 GB, respectively) for the content-length field. wget is a prominent example for this (or was, when i last tried this a couple of years ago).
-- daniel
Am 09.11.2010 21:37, schrieb Daniel Kinzler:
Some servers, clients and proxies break on the 31 (or 32) bit boundary (2 or 4 GB, respectively) for the content-length field. wget is a prominent example for this (or was, when i last tried this a couple of years ago).
The server itsself reports 0 bytes and neither curl nor firefox nor chrome is able to download the file.
I tried telnet and it is definitively a server side issue:
[root@vm2007 ~]# telnet toolserver.org 80 Trying 91.198.174.203... Connected to toolserver.org. Escape character is '^]'. GET /~mazder/osm_vm/OSM%20Render%20Stack.vmdk HTTP/1.1 Host: toolserver.org
Connection closed by foreign host. [root@vm2007 ~]#
Peter
Peter Körner wrote:
The server itsself reports 0 bytes and neither curl nor firefox nor chrome is able to download the file.
I tried telnet and it is definitively a server side issue:
[root@vm2007 ~]# telnet toolserver.org 80 Trying 91.198.174.203... Connected to toolserver.org. Escape character is '^]'. GET /~mazder/osm_vm/OSM%20Render%20Stack.vmdk HTTP/1.1 Host: toolserver.org
Connection closed by foreign host. [root@vm2007 ~]#
mzmcbride@nightshade:~$ stat /home/mazder/public_html/osm_vm File: `/home/mazder/public_html/osm_vm' -> `/mnt/user-store/mazder/osm_vm/'
From basic testing I did just now, you can't symlink from user-store to a
publicly accessible directory like that. It will return the exact error you're describing. That's the cause of your problem.
I don't know whether this is intentional or fixable.
MZMcBride
Am 09.11.2010 22:38, schrieb MZMcBride:
mzmcbride@nightshade:~$ stat /home/mazder/public_html/osm_vm File: `/home/mazder/public_html/osm_vm' -> `/mnt/user-store/mazder/osm_vm/'
From basic testing I did just now, you can't symlink from user-store to a publicly accessible directory like that. It will return the exact error you're describing. That's the cause of your problem.
I don't know whether this is intentional or fixable.
But i can download other files from the same directory: http://toolserver.org/~mazder/osm_vm/OSM%20Render%20Stack.ovf
Mysterious.
Peter
On Tue, Nov 9, 2010 at 10:38 PM, MZMcBride z@mzmcbride.com wrote:
mzmcbride@nightshade:~$ stat /home/mazder/public_html/osm_vm File: `/home/mazder/public_html/osm_vm' -> `/mnt/user-store/mazder/osm_vm/'
From basic testing I did just now, you can't symlink from user-store to a publicly accessible directory like that. It will return the exact error you're describing. That's the cause of your problem.
The point is the rights set on osm_vm. global r-x needs to be set on the user-store directory so that webserver can read directory.
Marco
Daniel Kinzler wrote:
On 09.11.2010 15:23, bawolff wrote:
Considering that dumps.wikimedia.org serves files in excess of 100 GB, I don't think HTTP the protocol is the issue. However when you try to request the file (or even just do a HEAD request) the web server closes the connection before sending any data, suggesting its a web server issue, not an HTTP the protocol issue.
-bawolff
Some servers, clients and proxies break on the 31 (or 32) bit boundary (2 or 4 GB, respectively) for the content-length field. wget is a prominent example for this (or was, when i last tried this a couple of years ago).
-- daniel
wget works fine with big files. I remember that some old wget version had issues showing the downloading bar in that boundary (it went crazy) but still, I think it got the job done.
toolserver-l@lists.wikimedia.org