Large djvu files from IA - Wikisource-l - lists.wikimedia.org

List overview All Threads
Download

Large djvu files from IA

CfD on Commons (German wikisource...

change to parsing of bare...

Bodhisattwa Mandal

27 Feb 2016 27 Feb '16

2:38 a.m.

Hi, Is it possible to transfer large djvu files (more than 100 mb) from Internet Archive to Commons? I have tried IA-upload and url2commons tool but not succeeded. Regards, -- Bodhisattwa

Attachments:

attachment.htm (text/html — 385 bytes)

Reply

Show replies by date

Federico Leva (Nemo)

27 Feb 27 Feb

4:47 a.m.

Bodhisattwa Mandal, 26/02/2016 20:38:

Is it possible to transfer large djvu files (more than 100 mb) from Internet Archive to Commons?

Sure, you can upload files up to 2 GB with the standard upload interface. https://commons.wikimedia.org/wiki/Commons:Maximum_file_size

I have tried IA-upload and url2commons tool but not succeeded.

IA-upload has https://github.com/Tpt/ia-upload/issues/5 For other methods make sure to enter the correct URL, which must be like https://archive.org/download/bub_gb_D5a57FHN8-wC/bub_gb_D5a57FHN8-wC.djvu and not a /stream/* URL. Nemo

Reply

Yann Forget

5:16 a.m.

Hi, Admins on Commons can upload files larger than 100 MB via upload-by-url. There is currently a timeout issue, so the limit is not a fixed figure. 200 MB files work fine most of the time. I even once managed once to upload a 300 MB file. If you need help, please give a list of files on my Commons talk page. Regards, Yann https://commons.wikimedia.org/wiki/User_talk:Yann 2016-02-26 20:38 GMT+01:00 Bodhisattwa Mandal <bodhisattwa.rgkmc(a)gmail.com>om>:

Hi, Is it possible to transfer large djvu files (more than 100 mb) from Internet Archive to Commons? I have tried IA-upload and url2commons tool but not succeeded. Regards, -- Bodhisattwa _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l

Reply

Bodhisattwa Mandal

1 Mar 1 Mar

11:03 a.m.

Hi, Sorry for the late reply and thanks for your response. I will make a list and let you know. By the way, recently I noticed IA is not making djvu files for BUB uploads. If we are uploading files without using BUB, then IA is creating djvu and there is no problem then. Can this be fixed? Thanks again, Bodhisattwa On Feb 27, 2016 3:46 AM, "Yann Forget" <yannfo(a)gmail.com> wrote:

Hi, Admins on Commons can upload files larger than 100 MB via upload-by-url. There is currently a timeout issue, so the limit is not a fixed figure. 200 MB files work fine most of the time. I even once managed once to upload a 300 MB file. If you need help, please give a list of files on my Commons talk page. Regards, Yann https://commons.wikimedia.org/wiki/User_talk:Yann 2016-02-26 20:38 GMT+01:00 Bodhisattwa Mandal <bodhisattwa.rgkmc(a)gmail.com

: Hi, Is it possible to transfer large djvu files (more than 100 mb) from

Internet

Archive to Commons? I have tried IA-upload and url2commons tool but not succeeded. Regards, -- Bodhisattwa _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l

_______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l

Reply

Federico Leva (Nemo)

8:14 p.m.

Bodhisattwa Mandal, 01/03/2016 05:03:

By the way, recently I noticed IA is not making djvu files for BUB uploads.

Link?

If we are uploading files without using BUB, then IA is creating djvu and there is no problem then. Can this be fixed?

Might be https://github.com/rohit-dua/BUB/issues/49 Nemo

Reply

Bodhisattwa Mandal

10:59 p.m.

On 1 March 2016 at 18:44, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote: Link?

https://archive.org/details/bub_gal_ark_12148_bpt6k1040628d https://archive.org/details/bub_gb_lmhDAAAAcAAJ https://archive.org/details/bub_man_adee2c3832c5798abc76e7b65f62ef4e and many others. I have been noticing it for last 2-3 days. Before that, it was ok. > Might be https://github.com/rohit-dua/BUB/issues/49

Replied there about why we mark language English in stead of Bengali. Thanks -- Bodhisattwa

Reply

Federico Leva (Nemo)

11:29 p.m.

Bodhisattwa Mandal, 01/03/2016 16:59:

and many others. I have been noticing it for last 2-3 days. Before that, it was ok.

There used to be in logs: => Target bub_gb_d7FGAumLBVAC.djvu : "DjVu" https://catalogd.archive.org/log/466680564 (3.9 days ago) Now I only see: Source bub_man_adee2c3832c5798abc76e7b65f62ef4e_djvu.xml : "Djvu XML" => Target bub_man_adee2c3832c5798abc76e7b65f62ef4e_text.pdf : "Additional Text PDF" => Target bub_man_adee2c3832c5798abc76e7b65f62ef4e_pdf.zip : "Single Page Zipped PDF" => Target bub_man_adee2c3832c5798abc76e7b65f62ef4e_djvu.txt : "DjVuTXT" => Target bub_man_adee2c3832c5798abc76e7b65f62ef4e_scandata.xml : "Scandata" => Target bub_man_adee2c3832c5798abc76e7b65f62ef4e_toc.xml : "Contents" (https://catalogd.archive.org/log/467557801) Looks like they scrapped the DjVu and "Text PDF" format, I don't know whether intentionally. You should ask in their forum: https://archive.org/iathreads/post-new.php?forum=texts (be kind!). Nemo

Reply

Bodhisattwa Mandal

6 Mar 6 Mar

1:15 a.m.

Hi Nemo,

Looks like they scrapped the DjVu and "Text PDF" format, I don't know whether intentionally. You should ask in their forum: https://archive.org/iathreads/post-new.php?forum=texts (be kind!).

Just saw this forum post in IA. It seems IA is going to stop creating djvu files soon. https://archive.org/post/1053214/djvu-files-for-new-uploads Regards, -- Bodhisattwa

Reply

Yann Forget

7 Mar 7 Mar

1:50 a.m.

Hi, 2016-03-05 19:15 GMT+01:00 Bodhisattwa Mandal <bodhisattwa.rgkmc(a)gmail.com>om>:

Hi Nemo,

Looks like they scrapped the DjVu and "Text PDF" format, I don't know whether intentionally. You should ask in their forum: https://archive.org/iathreads/post-new.php?forum=texts (be kind!).

Just saw this forum post in IA. It seems IA is going to stop creating djvu files soon. https://archive.org/post/1053214/djvu-files-for-new-uploads

As IA is probably the biggest source of scans for Wikisource, this is worrying. I would expect an important change like this to be announced well in advance, but it wasn't. So we need to rewrite the BUB tool completely from scratch. I would also suggest uploading as many DJVU files as possible while they are still there.

Regards, -- Bodhisattwa

Thanks for this important info. Yann

Reply

Federico Leva (Nemo)

16 Mar 16 Mar

2:03 a.m.

Very topic indeed... I'm continuing on the IA thread. As you can see, there are already some users with excess baggage from previous conversations who started being less than constructive; let's all be respectful there, after all we've (ab)used IA a lot for Wikisource purposes. Yann Forget, 06/03/2016 19:50:

So we need to rewrite the BUB tool completely from scratch.

Well... running pdf2djvu and uploading the result is hardly a total rewrite, though it would be annoying.

I would also suggest uploading as many DJVU files as possible while they are still there.

Brewster explicitly said they aren't going to delete anything for now. I never discourage additional copies of files but personally I'm still trying to understand the situation better and I won't spread alarmism. Nemo

Reply

2976

days inactive

2994

days old

wikisource-l@lists.wikimedia.org

Manage subscription

9 comments

3 participants

tags (0)

participants (3)

Bodhisattwa Mandal
Federico Leva (Nemo)
Yann Forget