Congratulations, this is an amazing event for all the Wikisource community. Hopefully we'll use it to show other libraries that such things are possible, and we should all take this as a model. it would be really important to document everything is a common and visible location, where we can read and learn from this experience. Where would this be? Aubrey (In Gdansk too) 2010/7/10 Michael Jörgens <joergens.mic(a)googlemail.com>

Tue, 13 Jul 2010 17:17:12 +0200, Federico Leva (Nemo) <nemowiki(a)gmail.com> wrote: I don't know exactly how they have been selected, it seems the BnF wanted to give us a panel of books with different characteristics, I don't know the Wikisource community has given any wishlist. In the agreement nothing is expressed about that. This agreement is more an experiment to see what sort of partnership it would be possible to do (or not) in the future. The BnF gived us about 10% of its annual digitalization (I think but not sure) ; and they have nobody/no money to review the OCR they use. The agreement text is on our private wiki (for WMFR members) in French, I ask about a public version or about an internal-wiki version. ~ Seb35 [^_^]

Thank you, Seb35! Andrea Zanni, 15/07/2010 18:47: May I mention http://strategy.wikimedia.org/wiki/Proposal:Make_Wikisource_scale ? There are some references, numbers and so on. Nemo

On 07/15/2010 06:47 PM, Andrea Zanni wrote: Wikisource is a project that could have a great importance in the future, if it succeeds to grow. Scanned books should be useful references in Wikipedia. But it's far too early to say if it will be successful, because it is still too small. In proofreading from scanned images, it's still very small and growing fast, because it really only started very recently. Only 3 languages have proofread more than 50,000 pages and 8 more languages have between 1000 and 10,000 pages. http://wikisource.org/wiki/Wikisource:ProofreadPage_Statistics While 50,000 pages is big for Wikipedia, it only means 250 books of 200 pages each. Such a book is 1 cm thick (0.4 inches), so the entire bookshelf of 50,000 pages fits in 2.50 metres of shelving, less than a normal bookcase. Swedish Wikisource now has 5,500 proofread pages, but these belong to 237 different volumes, many being small pamphlets or individual issues of a newspaper. Only 15 volumes are fully proofread books with more than 100 pages. Many of the volumes are books that someone uploaded and started to proofread, but didn't quite finish. Many were uploaded because they were available (scanned by a library), not because they would be really useful as source documents. Of our 15 fully proofread books, I can name 5 that should be really useful as sources. The Wikisource communities are very small, counting 12 contributors (making at least 5 edits per month) in Italian, 25 in Polish, 60 in German, 100 in French and 100 in English. For most languages, every single new contributor can make a huge difference. This might attract some users who have plenty of time and want to make a big difference, such as those who write hundreds of Wikipedia articles. But as time goes by and the projects scale up, you will find very few such contributors. Even if we don't pay for their work, volunteers are a finite resource and we shouldn't waste their time. It's easy to waste time with manually proofreading text that has very poor OCR quality. Running a new OCR might save hours of work. It's even easier to waste time by proofreading a book that nobody finds useful. That's where I think we should start: Which books are really useful? How do we determine that? Unfortunately, link search in Wikipedia does not count links to Wikisource, so it's hard to get good statistics. The first two books that I put on Wikisource in the fall of 2005 were small encyclopedias in German and English. This was an experiment and it worked well as an experiment. But these small encyclopedias were next to useless, because they contained so much less information than was already in Wikipedia. These 1000 bytes on Elbe, http://en.wikisource.org/wiki/The_New_Student's_Reference_Work/Elbe were useless because Wikipedia in October 2005 already had 5 times as much (today, that is 20 times as much). For Arabic and other languages where Wikipedia is still rather small, finding and scanning such a small 5 volume encyclopedia could be a great help, but for the languages where Wikipedia is bigger (and this is true for all languages where Wikisource is now active, except maybe Armenian), we need to look for more specialized reference works to be really useful as sources. I'm making an experiment now with proofreading a newspaper. Not just articles, but entire issues. I have done three full weeks from January 1836. Fortunately, each daily issue is only 4 pages in 3 columns, or 75 kilobytes in total, e.g. http://sv.wikisource.org/wiki/Post-_och_Inrikes_Tidningar_1836-01-05 But it still takes a lot of work. On my own, I'm not able to proofread one issue per day, so I'm already lagging behind. Newspapers are useful as sources. Wikipedia often references articles in current newspapers, and it would be great to have complete year runs going back in time. Still, I doubt that we can find enough proofreading volunteers to cover any substantial timespan. -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

2010/7/21 Hélène Pedrosa-Masson <edhral(a)free.fr> I too think that wikisource should be much more a "container for arts and history" than a "container for update knowledge", since most works are ancient (a century or more); this is why - IMHO - my small field of interest, which brought me into source world: equitation ("the *art* and practice of riding"), that's much more an art than a science, is a rather interesting topic, just as poems and literature, old chronicles and old newspapers. This means too that users will be very few, just as are very few the users of old or ancient books. Therefore source can be a useful "source" for some particular topics only, from wikipedia point of view. No matter: the whole server space devoided to wikisource is so small... the whole unzipped dump of it.source "good content" is more or less 250 Mby, t.i. less than three full-size (100 Mby) Commons files. ;-) Alex

Hélène Pedrosa-Masson wrote : this is planned ; the <pages/> bug has been fixed.

On 07/21/2010 01:56 PM, Federico Leva (Nemo) wrote: Do you have any examples of dictionaries that have been proofread in Wikisource and were found to be useful for Wiktionary? What were the parameters? Was the OCR of good quality? Was the proofreading done by one person, or a small or large team? Did proofreading take more or less time (per page) than for other books? Did proofreaders have Wiktionary in mind? -- Lars Aronsson (lars(a)aronsson.se) Aronsson Datateknik - http://aronsson.se

2010/7/21 Federico Leva (Nemo) &lt;nemowiki(a)gmail.com&gt; I tested a simple script to parse text of a proofread page and to output the list of different words into the page. My aim was mainly to find my mistakes when working about ancient, or unknown, languages: see http://pt.wikisource.org/wiki/P%C3%A1gina:Bem_cavalgar.djvu/40 (a 15° century portuguese book) and its talk page, where there's an example of such bot-created list of words. This rough script could be expanded, to produce the whole list of different works into a book, anyone linked with the proofread page where the word has been used. Could this trick be useful as a link between wikisource work and wiktionaries? -- Alex

2010/7/22 Federico Leva (Nemo) &lt;nemowiki(a)gmail.com&gt; Well, I understand that a "rough" list of words isn't so useful; but a list of words, a*nyone linked with a full bibliographic source pointing to the image and text of the original page*, perhaps could be of some interest. No matter if it isn't.... simply, I'll not think about any more. Alex Alex

2010/7/26 John Vandenberg &lt;jayvdb(a)gmail.com&gt; Thanks John, yes, it's pretty simple to do such type of statistics. The trick is really simple, and - in my opinion - anyone could implement it with a python script much better than my one. It consists simply of a routine that converts a string into a python list where "words characters " and "other text characters" are separated, giving simply the "word character" list as a parameter (or, what's the same, the list of "not word characters" . I.e, "This could be a piece of raw wikitext splitted by [[python]] routine" is converted into list ["This"," ","could be"," ","a;" ","piece"," ","of"," ","raw"," ","wikitext"," ","splitted"," ", "by"," [[","python","]] ", "routine"] where "words and "not-words" regularly alternate and a simple "".join() method of the list gives back *exactly* the source string. Simply selecting "words" from such a list, you can do anything you like with them. Alex

2010/7/28 Lars Aronsson &lt;lars(a)aronsson.se&gt; This is very interesting. Can you tell us more details about? has been the job documented (in English, Swedish is "a little difficult" for me...) somewhere? I can produce lists by my rought script, but it works on raw wiki code and the result is "dirty" - it contains markup words, and obviously all wrong words too (seaching for wrong words was my fisrt aim...). Did you work on html dump perhaps? -- Alex

2010/7/29 Lars Aronsson &lt;lars(a)aronsson.se&gt; Thanks Lars for details! From xml dump: this is what I 'd like to know (the same I do). HTML is too interesting as a source, since "absolutely not well formed wiki syntax" is replaced by a "well formed html syntax", but so far I didn't explore it. Thanks too for your link. Alex