Hi,
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
Thanks for any pointers,
Daniel
[1] http://commons.wikimedia.org/wiki/User:Open_Access_Media_Importer_Bot [2] https://github.com/erlehmann/open-access-media-importer/issues/22#issuecomme...
Right now the wiki is setup to only allow up to 100 MB files. The only way I see to upload a file greater than that is to file a bugzilla request and a system administrator with shell access can manually upload it using a script.
Techmn224
On 2013-06-25, at 4:43 PM, Daniel Mietchen daniel.mietchen@googlemail.com wrote:
Hi,
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
Thanks for any pointers,
Daniel
[1] http://commons.wikimedia.org/wiki/User:Open_Access_Media_Importer_Bot [2] https://github.com/erlehmann/open-access-media-importer/issues/22#issuecomme...
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
<quote name="Techman224" date="2013-06-25" time="16:47:39 -0500">
Right now the wiki is setup to only allow up to 100 MB files. The only way I see to upload a file greater than that is to file a bugzilla request and a system administrator with shell access can manually upload it using a script.
I thought it was set to 500mb? https://git.wikimedia.org/blob/operations%2Fmediawiki-config/40f5cf38a00edce...
Greg
https://commons.wikimedia.org/w/index.php?title=Special:Upload shows 100 MB.
I turned on chunked uploads and it still shows 100 MB.
Techman224
On 2013-06-25, at 5:13 PM, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="Techman224" date="2013-06-25" time="16:47:39 -0500"> > Right now the wiki is setup to only allow up to 100 MB files. The only way I see to upload a file greater than that is to file a bugzilla request and a system administrator with shell access can manually upload it using a script.
I thought it was set to 500mb? https://git.wikimedia.org/blob/operations%2Fmediawiki-config/40f5cf38a00edce...
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
There's a comment for that variable, "Only affects URL uploads; web uploads are enforced by PHP."
Techman224
On 2013-06-25, at 5:18 PM, Techman224 techman224@techman224.ca wrote:
https://commons.wikimedia.org/w/index.php?title=Special:Upload shows 100 MB.
I turned on chunked uploads and it still shows 100 MB.
Techman224
On 2013-06-25, at 5:13 PM, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="Techman224" date="2013-06-25" time="16:47:39 -0500"> > Right now the wiki is setup to only allow up to 100 MB files. The only way I see to upload a file greater than that is to file a bugzilla request and a system administrator with shell access can manually upload it using a script.
I thought it was set to 500mb? https://git.wikimedia.org/blob/operations%2Fmediawiki-config/40f5cf38a00edce...
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Oh yea, I forgot it was for the Upload Wizard. It doesn't say the maximum file size allowed.
Techman224
On 2013-06-25, at 5:22 PM, Techman224 techman224@techman224.ca wrote:
There's a comment for that variable, "Only affects URL uploads; web uploads are enforced by PHP."
Techman224
On 2013-06-25, at 5:18 PM, Techman224 techman224@techman224.ca wrote:
https://commons.wikimedia.org/w/index.php?title=Special:Upload shows 100 MB.
I turned on chunked uploads and it still shows 100 MB.
Techman224
On 2013-06-25, at 5:13 PM, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="Techman224" date="2013-06-25" time="16:47:39 -0500"> > Right now the wiki is setup to only allow up to 100 MB files. The only way I see to upload a file greater than that is to file a bugzilla request and a system administrator with shell access can manually upload it using a script.
I thought it was set to 500mb? https://git.wikimedia.org/blob/operations%2Fmediawiki-config/40f5cf38a00edce...
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On 6/25/13, Greg Grossmeier greg@wikimedia.org wrote:
<quote name="Techman224" date="2013-06-25" time="16:47:39 -0500"> > Right now the wiki is setup to only allow up to 100 MB files. The only way > I see to upload a file greater than that is to file a bugzilla request and > a system administrator with shell access can manually upload it using a > script.
I thought it was set to 500mb? https://git.wikimedia.org/blob/operations%2Fmediawiki-config/40f5cf38a00edce...
Greg
-- | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D |
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Well as the comment says: "// Only affects URL uploads; web uploads are enforced by PHP."
That particular variable limits chunked uploads, and upload by url (which is only enabled for flickr). Direct normal (non-chunked) uploads get limited by the smallest of: MW config variable, upload_max_filesize, and post_max_size and hence are limited to 100mb.
Note: Special:Upload is always non-chunked. Upload Wizard uses chunked based on a preference, and you can use either method if you do things yourself using the API.
--bawolff
On Jun 25, 2013 5:44 PM, "Daniel Mietchen" daniel.mietchen@googlemail.com wrote:
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
this looks like the relevant section:
https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading
I don't know the current settings; you might need to enable chunked uploads in the MediaWiki prefs for the user you're uploading as.
-Jeremy
I would assume given the previous reply that regardless of upload method, chunked or otherwise, there is still the hard limit of how big the resultant file can be.
On Tue, Jun 25, 2013 at 5:49 PM, Jeremy Baron jeremy@tuxmachine.com wrote:
On Jun 25, 2013 5:44 PM, "Daniel Mietchen" <daniel.mietchen@googlemail.com
wrote:
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
this looks like the relevant section:
https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading
I don't know the current settings; you might need to enable chunked uploads in the MediaWiki prefs for the user you're uploading as.
-Jeremy _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
With the https://commons.wikimedia.org/wiki/Commons:Chunked_uploads enabled, users can currently upload files up to 500MB, when using a supported browser (e.g. a current Firefox or Chrome).
I think Daniel's question was about how to make this method work for a bot too, i.e. via the API rather than in a browser.
On Tue, Jun 25, 2013 at 2:52 PM, OQ overlordq@gmail.com wrote:
I would assume given the previous reply that regardless of upload method, chunked or otherwise, there is still the hard limit of how big the resultant file can be.
On Tue, Jun 25, 2013 at 5:49 PM, Jeremy Baron jeremy@tuxmachine.com wrote:
On Jun 25, 2013 5:44 PM, "Daniel Mietchen" <daniel.mietchen@googlemail.com
wrote:
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
this looks like the relevant section:
https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading
I don't know the current settings; you might need to enable chunked uploads in the MediaWiki prefs for the user you're uploading as.
-Jeremy _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
When browsing through commons, I happened to stumble upon https://commons.wikimedia.org/wiki/User:Smallman12q/PyCJWiki which appears to be a python bot that uses chunked uploading, and thus work with files up to 500 mb big. Perhaps it would be helpful to you.
--bawolff
On 6/25/13, Tilman Bayer tbayer@wikimedia.org wrote:
With the https://commons.wikimedia.org/wiki/Commons:Chunked_uploads enabled, users can currently upload files up to 500MB, when using a supported browser (e.g. a current Firefox or Chrome).
I think Daniel's question was about how to make this method work for a bot too, i.e. via the API rather than in a browser.
On Tue, Jun 25, 2013 at 2:52 PM, OQ overlordq@gmail.com wrote:
I would assume given the previous reply that regardless of upload method, chunked or otherwise, there is still the hard limit of how big the resultant file can be.
On Tue, Jun 25, 2013 at 5:49 PM, Jeremy Baron jeremy@tuxmachine.com wrote:
On Jun 25, 2013 5:44 PM, "Daniel Mietchen" <daniel.mietchen@googlemail.com
wrote:
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
this looks like the relevant section:
https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading
I don't know the current settings; you might need to enable chunked uploads in the MediaWiki prefs for the user you're uploading as.
-Jeremy _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Tilman Bayer Senior Operations Analyst (Movement Communications) Wikimedia Foundation IRC (Freenode): HaeB
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Thanks to all the replies so far. Looks like chunked upload is the way to go.
Cheers,
Daniel
On Wed, Jun 26, 2013 at 12:06 AM, Brian Wolff bawolff@gmail.com wrote:
When browsing through commons, I happened to stumble upon https://commons.wikimedia.org/wiki/User:Smallman12q/PyCJWiki which appears to be a python bot that uses chunked uploading, and thus work with files up to 500 mb big. Perhaps it would be helpful to you.
--bawolff
On 6/25/13, Tilman Bayer tbayer@wikimedia.org wrote:
With the https://commons.wikimedia.org/wiki/Commons:Chunked_uploads enabled, users can currently upload files up to 500MB, when using a supported browser (e.g. a current Firefox or Chrome).
I think Daniel's question was about how to make this method work for a bot too, i.e. via the API rather than in a browser.
On Tue, Jun 25, 2013 at 2:52 PM, OQ overlordq@gmail.com wrote:
I would assume given the previous reply that regardless of upload method, chunked or otherwise, there is still the hard limit of how big the resultant file can be.
On Tue, Jun 25, 2013 at 5:49 PM, Jeremy Baron jeremy@tuxmachine.com wrote:
On Jun 25, 2013 5:44 PM, "Daniel Mietchen" <daniel.mietchen@googlemail.com
wrote:
my bot[1] occasionally stumbles upon files that are above 100MB and thus does not upload them[2]. What do I have to do to get it set up for handling these files too?
this looks like the relevant section:
https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading
I don't know the current settings; you might need to enable chunked uploads in the MediaWiki prefs for the user you're uploading as.
-Jeremy _______________________________________________ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
-- Tilman Bayer Senior Operations Analyst (Movement Communications) Wikimedia Foundation IRC (Freenode): HaeB
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
We have several tools and bots using chunked upload via the API for files over 100 MB. Daniel, please report back if you encounter https://bugzilla.wikimedia.org/show_bug.cgi?id=36587 ; it seemed fixed at some point but Fastily (developer of one of those tools) reopened it.
Nemo
wikitech-l@lists.wikimedia.org