IA Upload queue - Wikisource-l - lists.wikimedia.org

List overview All Threads
Download

IA Upload queue

Edit of Page: and Index: pages...

Proofread status colour buttons

Andrea Zanni

12 Sep 2017 12 Sep '17

11:36 a.m.

Dear all, someone could help understand if we have an issue here? https://tools.wmflabs.org/ia-upload/commons/init Some librarians uploaded books months ago, but they were never processed. Is the tool working, or it simply never signals when it fails?

Attachments:

attachment.htm (text/html — 406 bytes)

Reply

Show replies by date

Ilario Valdelli

12 Sep 12 Sep

11:47 a.m.

[2017-08-02 08:00:49] LOG.CRITICAL: Client error: `POST https://commons.wikimedia.org/w/api.php` resulted in a `413 Request Entity Too Large` response: <html> <head><title>413 Request Entity Too Large</title></head> <body bgcolor="white"> <center><h1>413 Request Entity (truncated...) [] [] Or the program is not able to process huge files or, simply, the disk space is finished. Kind regards Sent from Mail for Windows 10 From: Andrea Zanni Sent: 12 September 2017 10:37 To: discussion list for Wikisource, the free library Subject: [Wikisource-l] IA Upload queue Dear all, someone could help understand if we have an issue here? https://tools.wmflabs.org/ia-upload/commons/init Some librarians uploaded books months ago, but they were never processed. Is the tool working, or it simply never signals when it fails? --- Questa e-mail è stata controllata per individuare virus con Avast antivirus. https://www.avast.com/antivirus

Reply

Sam Wilson

3:03 p.m.

Yes, this happens when the resultant DjVu file is larger than Commons will allow. I think 100 MB is the limit? I'm not sure how to get around this. Perhaps we resize the images smaller? But we don't want to do that every time, so perhaps we have to generate the DjVu, see how big it is, and if it's too big resize and build it again? Would that work? We could make the over-size DjVu available for download, and then the user could use a different method to upload to Commons (is there such a method?). Suggestions welcome! A related issue is https://phabricator.wikimedia.org/T161396 I can't find a ticket yet for the request-too-large problem, but I remember seeing one; anyway, I'll create it again, and perhaps Community Tech can look into it. There's also the slight possibility that IA can start creating DjVus again! Which would be brilliant, but I haven't heard anything about that since Wikimania. —Sam. On Tue, 12 Sep 2017, at 04:47 PM, Ilario Valdelli wrote:

[2017-08-02 08:00:49] LOG.CRITICAL: Client error: `POST https://commons.wikimedia.org/w/api.php` resulted in a `413 Request Entity Too Large` response: <html> <head><title>413 Request Entity Too Large</title></head> <body bgcolor="white"> <center><h1>413 Request Entity (truncated...) [] []>

Or the program is not able to process huge files or, simply, the disk space is finished.>

Kind regards

Sent from Mail[1] for Windows 10

> *From: *Andrea Zanni[2] *Sent: *12 September 2017 10:37 *To: > *discussion list for Wikisource, the free library[3] *Subject: *[Wikisource- > l] IA Upload queue

Dear all,

someone could help understand if we have an issue here? https://tools.wmflabs.org/ia-upload/commons/init> Some librarians uploaded books months ago,

but they were never processed.

Is the tool working, or it simply never signals when it fails?

>

> Mail priva di virus. www.avast.com[4] > > _________________________________________________ > Wikisource-l mailing list > Wikisource-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikisource-l Links: 1. https://go.microsoft.com/fwlink/?LinkId=550986 2. mailto:zanni.andrea84@gmail.com 3. mailto:wikisource-l@lists.wikimedia.org 4. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&ut…

Reply

Yann Forget

3:07 p.m.

Hi, The upload limit on Commons is 4 GB, so there should be any issue. If the script can't upload files bigger than 100 MB, it should be fixed. See how other tools work, as an example (i.e. video2commons). Regards, Yann 2017-09-12 14:03 GMT+02:00 Sam Wilson <sam(a)samwilson.id.au>au>:

Yes, this happens when the resultant DjVu file is larger than Commons will allow. I think 100 MB is the limit? I'm not sure how to get around this. Perhaps we resize the images smaller? But we don't want to do that every time, so perhaps we have to generate the DjVu, see how big it is, and if it's too big resize and build it again? Would that work? We could make the over-size DjVu available for download, and then the user could use a different method to upload to Commons (is there such a method?). Suggestions welcome! A related issue is https://phabricator.wikimedia.org/T161396 I can't find a ticket yet for the request-too-large problem, but I remember seeing one; anyway, I'll create it again, and perhaps Community Tech can look into it. There's also the slight possibility that IA can start creating DjVus again! Which would be brilliant, but I haven't heard anything about that since Wikimania. —Sam. On Tue, 12 Sep 2017, at 04:47 PM, Ilario Valdelli wrote: [2017-08-02 08:00:49] LOG.CRITICAL: Client error: `POST https://commons.wikimedia.org/w/api.php` <https://commons.wikimedia.org/w/api.php> resulted in a `413 Request Entity Too Large` response: <html> <head><title>413 Request Entity Too Large</title></head> <body bgcolor="white"> <center><h1>413 Request Entity (truncated...) [] [] Or the program is not able to process huge files or, simply, the disk space is finished. Kind regards Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10 *From: *Andrea Zanni <zanni.andrea84(a)gmail.com> *Sent: *12 September 2017 10:37 *To: *discussion list for Wikisource, the free library <wikisource-l(a)lists.wikimedia.org> *Subject: *[Wikisource-l] IA Upload queue Dear all, someone could help understand if we have an issue here? https://tools.wmflabs.org/ia-upload/commons/init Some librarians uploaded books months ago, but they were never processed. Is the tool working, or it simply never signals when it fails? <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> Mail priva di virus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> <#m_2376373167041964652_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> *_______________________________________________* Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l

Reply

Nicolas VIGNERON

3:17 p.m.

2017-09-12 14:07 GMT+02:00 Yann Forget <yannfo(a)gmail.com>om>:

Hi, The upload limit on Commons is 4 GB, so there should be any issue. If the script can't upload files bigger than 100 MB, it should be fixed. See how other tools work, as an example (i.e. video2commons).

100 MB is the « regular » limit. 4 GB is the limit for chunked upload : https://commons.wikimedia.org/wiki/Commons:Chunked_uploads (it's end on the same place but it's not the same system) Cdlt, ~nicolas

Reply

Sam Wilson

3:24 p.m.

Ah yes! So I think we just need to make ia-upload be better. :-) https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading On Tue, 12 Sep 2017, at 08:17 PM, Nicolas VIGNERON wrote:

2017-09-12 14:07 GMT+02:00 Yann Forget <yannfo(a)gmail.com>om>:

Hi, The upload limit on Commons is 4 GB, so there should be any issue. If the script can't upload files bigger than 100 MB, it should be fixed.>> See how other tools work, as an example (i.e. video2commons).

100 MB is the « regular » limit. 4 GB is the limit for chunked upload : https://commons.wikimedia.org/wiki/Commons:Chunked_uploads (it's end on the same place but it's not the same system)> Cdlt, ~nicolas _________________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l

Reply

Sam Wilson

3:23 p.m.

Oh right! I'm behind the times for sure. :-) I guess it's because we don't support multi-part uploads then? Each chunk must be smaller than some amount? New bug now at: https://phabricator.wikimedia.org/T175680 On Tue, 12 Sep 2017, at 08:07 PM, Yann Forget wrote:

Hi, The upload limit on Commons is 4 GB, so there should be any issue. If the script can't upload files bigger than 100 MB, it should be fixed.> See how other tools work, as an example (i.e. video2commons). Regards, Yann 2017-09-12 14:03 GMT+02:00 Sam Wilson <sam(a)samwilson.id.au>au>: > __ > Yes, this happens when the resultant DjVu file is larger than Commons > will allow. I think 100 MB is the limit?>> > I'm not sure how to get around this. Perhaps we resize the images > smaller? But we don't want to do that every time, so perhaps we have > to generate the DjVu, see how big it is, and if it's too big resize > and build it again? Would that work?>> > We could make the over-size DjVu available for download, and then the > user could use a different method to upload to Commons (is there such > a method?).>> > Suggestions welcome! > > A related issue is https://phabricator.wikimedia.org/T161396 > I can't find a ticket yet for the request-too-large problem, but I > remember seeing one; anyway, I'll create it again, and perhaps > Community Tech can look into it.>> > There's also the slight possibility that IA can start creating DjVus > again! Which would be brilliant, but I haven't heard anything about > that since Wikimania.>> > —Sam. > > > On Tue, 12 Sep 2017, at 04:47 PM, Ilario Valdelli wrote: >> [2017-08-02 08:00:49] LOG.CRITICAL: Client error: `POST >> https://commons.wikimedia.org/w/api.php` resulted in a `413 Request >> Entity Too Large` response: <html> <head><title>413 Request Entity >> Too Large</title></head> <body bgcolor="white"> <center><h1>413 >> Request Entity (truncated...) [] []>>>

>> Or the program is not able to process huge files or, simply, the >> disk space is finished.>>>

>> Kind regards

>>

>> Sent from Mail[1] for Windows 10

>>

>>> *From: *Andrea Zanni[2] *Sent: *12 September 2017 10:37 *To: >>> *discussion list for Wikisource, the free library[3] *Subject: *[Wikisource- >>> l] IA Upload queue

>>

>> Dear all,

>> someone could help understand if we have an issue here? >> https://tools.wmflabs.org/ia-upload/commons/init>>> Some librarians uploaded books months ago,

>> but they were never processed.

>> Is the tool working, or it simply never signals when it fails?

>>

>>>

>>

>>> Mail priva di virus. www.avast.com[4] >>> >>> _________________________________________________ >>> Wikisource-l mailing list >>> Wikisource-l(a)lists.wikimedia.org >>> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> > _________________________________________________ > Wikisource-l mailing list > Wikisource-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikisource-l Links: 1. https://go.microsoft.com/fwlink/?LinkId=550986 2. mailto:zanni.andrea84@gmail.com 3. mailto:wikisource-l@lists.wikimedia.org 4. https://www.avast.com/sig-email?utm_medium=email&utm_source=link&ut…

Reply

Federico Leva (Nemo)

3:09 p.m.

Sam Wilson, 12/09/2017 15:03:

I'm not sure how to get around this.

Probably by delegating the upload to a library which is able to use chuncked uploading, https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading Nemo

Reply

2412

days inactive

2412

days old

wikisource-l@lists.wikimedia.org

Manage subscription

7 comments

6 participants

tags (0)

participants (6)

Andrea Zanni
Federico Leva (Nemo)
Ilario Valdelli
Nicolas VIGNERON
Sam Wilson
Yann Forget