http://commons.wikimedia.org/wiki/Special:Preferences#mw-prefsection-uploads
There's a new user preference "chunked uploads for files over 1 MB" for Upload Wizard.
The intent of this feature is to reliably transfer large files, mitigating connection reliability issues. This will enable us to raise the file size limit for all users.
Please help test this feature and report issues (performance, file corruption, errors, etc.). The carrot is that with this preference enabled, you can upload files <500 MB today (as opposed to <100 MB) :-)
The feature currently only works with Upload Wizard.
More information: http://commons.wikimedia.org/wiki/Commons:Chunked_uploads
Please report bugs on the talk page or against UploadWizard in Bugzilla: https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&...
Am 04.05.2012 05:16, schrieb Erik Moeller:
http://commons.wikimedia.org/wiki/Special:Preferences#mw-prefsection-uploads
There's a new user preference "chunked uploads for files over 1 MB" for Upload Wizard.
The intent of this feature is to reliably transfer large files, mitigating connection reliability issues. This will enable us to raise the file size limit for all users.
Please help test this feature and report issues (performance, file corruption, errors, etc.). The carrot is that with this preference enabled, you can upload files <500 MB today (as opposed to <100 MB) :-)
The feature currently only works with Upload Wizard.
More information: http://commons.wikimedia.org/wiki/Commons:Chunked_uploads
Please report bugs on the talk page or against UploadWizard in Bugzilla: https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&...
Thanks for the info, Erik.
Does anyone know, if the pywikipediabot framework supports chunked uploading now or in the near future?
Raimond.
Hi Raimond,
Op 5-5-2012 10:46, Raimond Spekking schreef:
Am 04.05.2012 05:16, schrieb Erik Moeller:
http://commons.wikimedia.org/wiki/Special:Preferences#mw-prefsection-uploads
There's a new user preference "chunked uploads for files over 1 MB" for Upload Wizard.
The intent of this feature is to reliably transfer large files, mitigating connection reliability issues. This will enable us to raise the file size limit for all users.
Please help test this feature and report issues (performance, file corruption, errors, etc.). The carrot is that with this preference enabled, you can upload files<500 MB today (as opposed to<100 MB) :-)
The feature currently only works with Upload Wizard.
More information: http://commons.wikimedia.org/wiki/Commons:Chunked_uploads
Please report bugs on the talk page or against UploadWizard in Bugzilla: https://bugzilla.wikimedia.org/enter_bug.cgi?product=MediaWiki%20extensions&...
Thanks for the info, Erik.
Does anyone know, if the pywikipediabot framework supports chunked uploading now or in the near future?
I was about to forward this email to the pywikipedia-l list to suggest someone to implement this. Based on http://www.mediawiki.org/wiki/API:Upload#Chunked_uploading this shouldn't be too hard.
Maarten
Very nice news! I hope we'll soon be able to reach the sweetness of the archive.org uploader, which allowed me to upload files up to 7-8 GB smoothly and without errors (on Chromium).
Erik Moeller, 04/05/2012 05:16:
Please help test this feature and report issues (performance, file corruption, errors, etc.). The carrot is that with this preference enabled, you can upload files<500 MB today (as opposed to<100 MB) :-)
This is indeed a sweet carrot; I've tried to upload the new WMIT WikiGuide video (190 MB) but it failed with internal_api_error_UploadStashFileNotFoundException (took too long?). Now I'll try again, but I had also the following (non blocking) problems which make it unusable (or very user-unfriendly) at least on my platform: * upload is terribly slow, averaged at 25-50 KB/s and never went above 100 (but I didn't stare at it for hours); * especially when you add the file and start upload, it's very resource-intensive and made my browser stall for a short while: 400 MB of RAM used for several minutes, CPU usage hopping, a whole core waiting for disk (?). I suppose you're already working on these problems, but if they're platform-specific (I used Firefox on Ubuntu) and you need more details I can file some bugs.
Nemo
Hi,
On Mon, May 7, 2012 at 11:54 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
This is indeed a sweet carrot; I've tried to upload the new WMIT WikiGuide video (190 MB) but it failed with internal_api_error_UploadStashFileNotFoundException (took too long?).
I got the same error, but: The file was uploaded correctly to Commons.
- especially when you add the file and start upload, it's very
resource-intensive and made my browser stall for a short while: 400 MB of RAM used for several minutes, CPU usage hopping, a whole core waiting for disk (?).
Chromium on Ubuntu is just crashing, Firefox complains about a slow script, both on Windows and Ubuntu, but it still works.
Regards, Holger
Prolineserver, 07/05/2012 12:24:
- especially when you add the file and start upload, it's very
resource-intensive and made my browser stall for a short while: 400 MB of RAM used for several minutes, CPU usage hopping, a whole core waiting for disk (?).
Chromium on Ubuntu is just crashing, Firefox complains about a slow script, both on Windows and Ubuntu, but it still works.
On the contrary, with Chromium 18.0.1025.142 (Build 129054 Linux) I had a much better experience. It still failed though, filed as https://bugzilla.wikimedia.org/show_bug.cgi?id=36587
Nemo
On Mon, May 7, 2012 at 2:54 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
- especially when you add the file and start upload, it's very
resource-intensive and made my browser stall for a short while: 400 MB of RAM used for several minutes, CPU usage hopping, a whole core waiting for disk (?).
Yeah, UW was trying to extract JPEG metadata from video files. :-P This should be fixed once https://gerrit.wikimedia.org/r/#/c/6722/ is reviewed and deployed.
The internal API error bug will probably be a bit harder to track down; hazarding a guess it might be a timing issue with the copy operation on very large files.
Can you file a bug for the upload performance issue as well? That one is new to me.
Thanks, Erik
Erik Moeller, 07/05/2012 19:13:
On Mon, May 7, 2012 at 2:54 AM, Federico Leva (Nemo) wrote:
- especially when you add the file and start upload, it's very
resource-intensive and made my browser stall for a short while: 400 MB of RAM used for several minutes, CPU usage hopping, a whole core waiting for disk (?).
Yeah, UW was trying to extract JPEG metadata from video files. :-P This should be fixed once https://gerrit.wikimedia.org/r/#/c/6722/ is reviewed and deployed.
The internal API error bug will probably be a bit harder to track down; hazarding a guess it might be a timing issue with the copy operation on very large files.
What about the automatic clearing of the stash? Wasn't that made more aggressive at some point?
Can you file a bug for the upload performance issue as well? That one is new to me.
Done: https://bugzilla.wikimedia.org/show_bug.cgi?id=36599
Nemo
Limit still is 100MB. I am willing to test but I can't with the 100MB restriction.
-- とある白い猫 (To Aru Shiroi Neko)
On Mon, May 7, 2012 at 9:06 PM, Federico Leva (Nemo) nemowiki@gmail.comwrote:
Erik Moeller, 07/05/2012 19:13:
On Mon, May 7, 2012 at 2:54 AM, Federico Leva (Nemo) wrote:
- especially when you add the file and start upload, it's very
resource-intensive and made my browser stall for a short while: 400 MB of RAM used for several minutes, CPU usage hopping, a whole core waiting for disk (?).
Yeah, UW was trying to extract JPEG metadata from video files. :-P This should be fixed once https://gerrit.wikimedia.org/**r/#/c/6722/https://gerrit.wikimedia.org/r/#/c/6722/is reviewed and deployed.
The internal API error bug will probably be a bit harder to track down; hazarding a guess it might be a timing issue with the copy operation on very large files.
What about the automatic clearing of the stash? Wasn't that made more aggressive at some point?
Can you file a bug for the upload performance issue as well? That one
is new to me.
Done: https://bugzilla.wikimedia.**org/show_bug.cgi?id=36599https://bugzilla.wikimedia.org/show_bug.cgi?id=36599
Nemo
______________________________**_________________ Commons-l mailing list Commons-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/commons-lhttps://lists.wikimedia.org/mailman/listinfo/commons-l
On Tue, May 8, 2012 at 1:08 PM, とある白い猫 to.aru.shiroi.neko@gmail.com wrote:
Limit still is 100MB. I am willing to test but I can't with the 100MB restriction.
It's not -- if you get that message, it's likely that your browser doesn't support chunked uploading (or you don't have the user preference enabled). Which browser were you testing with?
Firefox, IE and Chorme on Windows 7
- IE upload just goes on forever. It attempted to upload for over a day and nothing happened. - Chrome just dies. - Firefox does nothing when I pick to upload any file
I am using Special:UploadWizard to upload a 1.4gb file.
-- とある白い猫 (To Aru Shiroi Neko)
On Tue, May 8, 2012 at 11:51 PM, Erik Moeller erik@wikimedia.org wrote:
On Tue, May 8, 2012 at 1:08 PM, とある白い猫 to.aru.shiroi.neko@gmail.com wrote:
Limit still is 100MB. I am willing to test but I can't with the 100MB restriction.
It's not -- if you get that message, it's likely that your browser doesn't support chunked uploading (or you don't have the user preference enabled). Which browser were you testing with?
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Commons-l mailing list Commons-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l
I get the same problem with a 173 mb file.
-- とある白い猫 (To Aru Shiroi Neko)
On Wed, May 9, 2012 at 11:56 PM, Federico Leva (Nemo) nemowiki@gmail.comwrote:
とある白い猫, 09/05/2012 23:18:
I am using Special:UploadWizard to upload a 1.4gb file.
Erik Moeller, 04/05/2012 05:16:
The carrot is that with this preference enabled, you can upload files<500 MB today (as opposed to<100 MB)
Nemo
______________________________**_________________ Commons-l mailing list Commons-l@lists.wikimedia.org https://lists.wikimedia.org/**mailman/listinfo/commons-lhttps://lists.wikimedia.org/mailman/listinfo/commons-l
On Fri, May 11, 2012 at 11:22 AM, とある白い猫 to.aru.shiroi.neko@gmail.com wrote:
I get the same problem with a 173 mb file.
Until this change is merged and deployed: https://gerrit.wikimedia.org/r/#/c/6722/
You'll experience browser crashes/slowness with very large files.
However, _if_ (and only if) you're able to manage to start a chunked upload of a large file in a recent version of Firefox and Chrome, please tell me if you're still seeing errors on the first step. We just deployed a timeout fix that may have resolved the error message sometimes returned on the first step.
I have a no go on IE (attempts to upload forever) and Chrome (crashes the first second). I managed to upload a file on Firefox: http://commons.wikimedia.org/wiki/File:Iridescent_Glory_of_Nearby_Helix_Nebu...
Could someone review if the upload is corrupted or not?
-- とある白い猫 (To Aru Shiroi Neko)
On Fri, May 11, 2012 at 10:34 PM, Erik Moeller erik@wikimedia.org wrote:
On Fri, May 11, 2012 at 11:22 AM, とある白い猫 to.aru.shiroi.neko@gmail.com wrote:
I get the same problem with a 173 mb file.
Until this change is merged and deployed: https://gerrit.wikimedia.org/r/#/c/6722/
You'll experience browser crashes/slowness with very large files.
However, _if_ (and only if) you're able to manage to start a chunked upload of a large file in a recent version of Firefox and Chrome, please tell me if you're still seeing errors on the first step. We just deployed a timeout fix that may have resolved the error message sometimes returned on the first step.
-- Erik Möller VP of Engineering and Product Development, Wikimedia Foundation
Support Free Knowledge: https://wikimediafoundation.org/wiki/Donate
Commons-l mailing list Commons-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/commons-l
On Fri, May 11, 2012 at 1:34 PM, Erik Moeller erik@wikimedia.org wrote:
On Fri, May 11, 2012 at 11:22 AM, とある白い猫 to.aru.shiroi.neko@gmail.com wrote:
I get the same problem with a 173 mb file.
Until this change is merged and deployed: https://gerrit.wikimedia.org/r/#/c/6722/
It is now. Please try chunked uploading again -- it should work now, at least in recent versions of Firefox and Chrome, for files up to 500MB (some steps may take a long time to complete, but you shouldn't get errors).