@Alex: but what do you think of storing the source information in "Index:" pages for all works stored in Wikisource, even if they don't have a supporting scan? That was the original question :) About your proposed library, it would be more useful if it could modify data in Wikidata, not only import it. Besides, if the Wikidata client is installed in Wikisource, the inclusion syntax already takes care of displaying data... Micru On Mon, Jun 10, 2013 at 5:38 PM, Alex Brollo <alex.brollo(a)gmail.com> wrote: -- Etiamsi omnes, ego non

No, it won't be stored in Wikisource, but still there is the need to present the information in a consistent manner. If you want to display the information on ns0, you will end up needing the same fields that the "Index:" page is using now. So why not to have the same solution for both? It could also be a template with a reduced set of fields that expands to show "Template:Book" with linked data from Wikidata, no matter if they have supporting scans or not. Micru On Mon, Jun 10, 2013 at 6:00 PM, Alex Brollo <alex.brollo(a)gmail.com> wrote: -- Etiamsi omnes, ego non

Sorry if my answer is off-topic but if metadata are stored in WIkidata, is it really needed to create index pages to store the same data as Wikidata? As I see the things, we'll have bibliographical metadata on Wikidata (title, author, date of publication...) and data related to proofreading (proofreading level, table of content...) on the Index: pages. More, as the Proofread Page extension considers that an Index page is about a scan (ie one or more files) I'm not sure that Index pages about books without scan will be managed well by the extension. {{header|index name}} is already done, for books with scan, by the Proofread Page extension with the header=1 feature. In fr Wikisource, we already use a Lua module to manage the Mediawiki:Proofreadpage_header_template template used by the header=1 feature. https://fr.wikisource.org/wiki/Module:Header_template This template outputs automatically metadata and navigation from the index page TOC (but it allows also to override data). Tpt Date: Tue, 11 Jun 2013 01:33:39 +0200 From: alex.brollo(a)gmail.com To: wikisource-l(a)lists.wikimedia.org Subject: Re: [Wikisource-l] About texts without supporting files and "Index:" pages I'm going to test what you are telling in a real Lua script; as you know, Lua can read the code of any page with one "expensive" server function only, so that a simple {{header|index name}} ns0 template call could read all the wiki code from index page, parse it, extract all its data content, and use it to build any html you like. No other field is needed. In it.wikisource we are testing something more complex, since we are exporting Index data into a local Lua data module, to be loaded with a mw.loadData function that is not listed as "server-expensive"; but I presume that wiki servers would not be overloaded by one server expensive call.... If Im not going wrong, such a script could be written tomorrow by a good Lua programmer.... I'll need some more time as a beginner. I'll test a "MediaWiki:Proofreadpage_index_template" Lua loader & parser working into ns0, just to see if all runs as I guess, then I'll tell you in this thread. In which wikisource project do you work usually? Alex 2013/6/11 David Cuenca <dacuetu(a)gmail.com> No, it won't be stored in Wikisource, but still there is the need to present the information in a consistent manner. If you want to display the information on ns0, you will end up needing the same fields that the "Index:" page is using now. So why not to have the same solution for both? It could also be a template with a reduced set of fields that expands to show "Template:Book" with linked data from Wikidata, no matter if they have supporting scans or not. Micru On Mon, Jun 10, 2013 at 6:00 PM, Alex Brollo <alex.brollo(a)gmail.com> wrote: Simply there is no need to store data twice or more, if they are dinamically imported from wikidata. Such data would be simply generated by a normal template. Something similar to Commons media sharing: most wikipedians but beginners know that when you want to edit a shared media file, you must do you edit in Commons; there's no need to host a media file locally. So, IMHO a good Lua wikidata-reading library could avoid at all to store data in wikisource, or wikipedia, or Commons. Alex 2013/6/10 David Cuenca <dacuetu(a)gmail.com> @Alex: but what do you think of storing the source information in "Index:" pages for all works stored in Wikisource, even if they don't have a supporting scan? That was the original question :) About your proposed library, it would be more useful if it could modify data in Wikidata, not only import it. Besides, if the Wikidata client is installed in Wikisource, the inclusion syntax already takes care of displaying data... Micru On Mon, Jun 10, 2013 at 5:38 PM, Alex Brollo <alex.brollo(a)gmail.com> wrote: I don't see the need to change deeply Index/ns0 relationship, while I appreciate the idea "promote coherence reducing redundance" (many years ago I painfully used dBase III - dBase IV and I learned that principle by "try and learn"). Here: http://www.mediawiki.org/wiki/Extension_talk:Scribunto/Brainstorming a brief message about relationship among wikidata, commons, wikisource and any other project. Don't follow the link, it's so short that I copy it here (but if you like it, comment it there): Scribunto-Lua and WikidataI'd like a library to get Wikidata content; it would be a good idea IMHO to access to Wikidata data in plain form, just as such data would be Lua tables/variables. --Alex brollo (talk) 13:06, 10 June 2013 (UTC) If such a Lua library could be built, to import data from wikidata would be as simple, as writing a template, and data will be self-aligned. Alex 2013/6/10 Aarti K. Dwivedi <ellydwivedi2093(a)gmail.com> Hi, There was a thread some time ago where there were talks of having books which were born digital. These pages wouldn't have scans. What the 'Index' page would have in these cases is something I am not very sure about. Cheers,Rtdwivedi On Mon, Jun 10, 2013 at 10:47 PM, David Cuenca <dacuetu(a)gmail.com> wrote: With the deployment of Wikidata it is a good moment to re-examine what "Index" pages are and what should be their function. The most direct transition to a Wikidata-supported Wikisource could be something like this: https://sites.google.com/site/dacuetu/BookData.pdf That would allow: - to share data book data between Commons, Wikisource and Wikipedia - to update it, when any of the sites has been updated - to facilitate better search functions (like searches by author, or topic, limiting the date range or the language) That would only apply to those texts which use a "Index:" page, so now the question is, what do we do with books that do not have supporting scans (and therefore no index page)? Some possible options: a) ignore pages without sources and focus only on works with supporting scans b) use ns0 pages also as data containers (instead of, or in addition to "Index" pages) c) create "Index:" pages for all works, with or without scans. Use that instead of "Template:Textinfo" Personally I prefer "option c", even if it would require to rename "Index:" to "Source:" to make more clear what are those pages, however I would like to hear the opinion of other wikisourcerors about this. Cheers, Micru _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l -- Aarti K. Dwivedi _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l -- Etiamsi omnes, ego non _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l -- Etiamsi omnes, ego non _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l _______________________________________________ Wikisource-l mailing list Wikisource-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikisource-l

@aarti: sometimes some books/text/documents are born-digital. Think about all the scientific literature, or Phd thesis. These files (if cc-by/sa licensed) could be stored in Wikisource, and be useful for the wikicommunity. We already have some means to link those text to their source (with a URL). It's a long time "controversy" if we must or must not allow documents without scans on Wikisource. Every community should decide by itself. My personal POV (also as a "librarian"), is that if we leave out born digital documents we are forgetting the bulk of the stuff. I think that one of the most important added values of Wikisource is integrating texts with other Wikimedia projects, and (wiki)linking and connecting each other. No other digital library do that on the Internet, and we can do it because we have a community. So, these texts will have a source. I do think that proofreading a born digital PDF is a waste of time. Aubrey On Tue, Jun 11, 2013 at 8:46 AM, Aarti K. Dwivedi <ellydwivedi2093(a)gmail.com > A slighly off-topic question: Even if we modify the extension to proofread > books which do not have scans( I am assuming books that were born digital > ), against what > will these books be proofread? > > > On Tue, Jun 11, 2013 at 12:11 PM, Thomas PT <thomaspt(a)hotmail.fr> >> Sorry if my answer is off-topic but if metadata are stored in WIkidata, >> is it really needed to create index pages to store the same data as >> Wikidata? >> As I see the things, we'll have bibliographical metadata on Wikidata >> (title, author, date of publication...) and data related to proofreading >> (proofreading level, table of content...) on the Index: pages. More, as the >> Proofread Page extension considers that an Index page is about a scan (ie >> one or more files) I'm not sure that Index pages about books without scan >> will be managed well by the extension. >> >> {{header|index name}} is already done, for books with scan, by the >> Proofread Page extension with the header=1 feature. In fr Wikisource, we >> already use a Lua module to manage the >> Mediawiki:Proofreadpage_header_template template used by the header=1 >> feature. https://fr.wikisource.org/wiki/Module:Header_template This >> template outputs automatically metadata and navigation from the index page >> TOC (but it allows also to override data). >> >> Tpt >> >> ------------------------------ >> Date: Tue, 11 Jun 2013 01:33:39 +0200 >> From: alex.brollo(a)gmail.com >> To: wikisource-l(a)lists.wikimedia.org >> Subject: Re: [Wikisource-l] About texts without supporting files and >> "Index:" pages >> >> >> I'm going to test what you are telling in a real Lua script; as you know, >> Lua can read the code of any page with one "expensive" server function >> only, so that a simple {{header|index name}} ns0 template call could read >> all the wiki code from index page, parse it, extract all its data content, >> and use it to build any html you like. No other field is needed. In >> it.wikisource we are testing something more complex, since we are exporting >> Index data into a local Lua data module, to be loaded with a mw.loadData >> function that is not listed as "server-expensive"; but I presume that wiki >> servers would not be overloaded by *one* server expensive call.... >> >> If Im not going wrong, such a script could be written tomorrow by a good >> Lua programmer.... I'll need some more time as a beginner. I'll test >> a "MediaWiki:Proofreadpage_index_template" Lua loader & parser working into >> ns0, just to see if all runs as I guess, then I'll tell you in this thread. >> In which wikisource project do you work usually? >> >> Alex >> >> >> >> 2013/6/11 David Cuenca <dacuetu(a)gmail.com> >> >> No, it won't be stored in Wikisource, but still there is the need to >> present the information in a consistent manner. >> If you want to display the information on ns0, you will end up needing >> the same fields that the "Index:" page is using now. >> So why not to have the same solution for both? >> >> It could also be a template with a reduced set of fields that expands to >> show "Template:Book" with linked data from Wikidata, no matter if they have >> supporting scans or not. >> >> Micru >> >> >> On Mon, Jun 10, 2013 at 6:00 PM, Alex Brollo <alex.brollo(a)gmail.com>wrote;wrote: >> >> Simply there is no need to store data twice or more, if they are >> dinamically imported from wikidata. Such data would be simply generated by >> a normal template. Something similar to Commons media sharing: most >> wikipedians but beginners know that when you want to edit a shared media >> file, you must do you edit in Commons; there's no need to host a media file >> locally. >> >> So, IMHO a good Lua wikidata-reading library could avoid at all to store >> data in wikisource, or wikipedia, or Commons. >> >> Alex >> >> >> 2013/6/10 David Cuenca <dacuetu(a)gmail.com> >> >> @Alex: but what do you think of storing the source information in >> "Index:" pages for all works stored in Wikisource, even if they don't have >> a supporting scan? >> >> That was the original question :) >> >> About your proposed library, it would be more useful if it could modify >> data in Wikidata, not only import it. Besides, if the Wikidata client is >> installed in Wikisource, the inclusion syntax already takes care of >> displaying data... >> >> Micru >> >> >> On Mon, Jun 10, 2013 at 5:38 PM, Alex Brollo <alex.brollo(a)gmail.com>wrote;wrote: >> >> I don't see the need to change deeply Index/ns0 relationship, while I >> appreciate the idea "promote coherence reducing redundance" (many years ago >> I painfully used dBase III - dBase IV and I learned that principle by "try >> and learn"). >> >> Here: >> http://www.mediawiki.org/wiki/Extension_talk:Scribunto/Brainstorming a >> brief message about relationship among wikidata, commons, wikisource and >> any other project. Don't follow the link, it's so short that I copy it here >> (but if you like it, comment it there): >> >> Scribunto-Lua and Wikidata >> I'd like a library to get Wikidata content; it would be a good idea IMHO >> to access to Wikidata data in plain form, just as such data would be Lua >> tables/variables. --Alex brollo (talk) 13:06, 10 June 2013 (UTC) >> >> >> If such a Lua library could be built, to import data from wikidata would >> be as simple, as writing a template, and data will be self-aligned. >> >> Alex >> >> >> 2013/6/10 Aarti K. Dwivedi <ellydwivedi2093(a)gmail.com> >> >> Hi, >> >> There was a thread some time ago where there were talks of having >> books which were born digital. These pages wouldn't have scans. >> What the 'Index' page would have in these cases is something I am not >> very sure about. >> >> Cheers, >> Rtdwivedi >> >> >> On Mon, Jun 10, 2013 at 10:47 PM, David Cuenca <dacuetu(a)gmail.com>> >> With the deployment of Wikidata it is a good moment to re-examine what >> "Index" pages are and what should be their function. >> The most direct transition to a Wikidata-supported Wikisource could be >> something like this: >> https://sites.google.com/site/dacuetu/BookData.pdf >> >> That would allow: >> - to share data book data between Commons, Wikisource and Wikipedia >> - to update it, when any of the sites has been updated >> - to facilitate better search functions (like searches by author, or >> topic, limiting the date range or the language) >> >> That would only apply to those texts which use a "Index:" page, so now >> the question is, what do we do with books that do not have supporting scans >> (and therefore no index page)? >> >> Some possible options: >> a) ignore pages without sources and focus only on works with supporting >> scans >> b) use ns0 pages also as data containers (instead of, or in addition to >> "Index" pages) >> c) create "Index:" pages for all works, with or without scans. Use that >> instead of "Template:Textinfo" >> >> Personally I prefer "option c", even if it would require to rename >> "Index:" to "Source:" to make more clear what are those pages, however I >> would like to hear the opinion of other wikisourcerors about this. >> >> Cheers, >> Micru >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> >> >> -- >> Aarti K. Dwivedi >> >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> >> >> -- >> Etiamsi omnes, ego non >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> >> >> -- >> Etiamsi omnes, ego non >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> >> >> _______________________________________________ Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> _______________________________________________ >> Wikisource-l mailing list >> Wikisource-l(a)lists.wikimedia.org >> https://lists.wikimedia.org/mailman/listinfo/wikisource-l >> >> > > > -- > Aarti K. Dwivedi > > > _______________________________________________ > Wikisource-l mailing list > Wikisource-l(a)lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikisource-l > >

@Billinghurst, I think Aubrey was referring mainly to pdf files, which sometimes have text and format but they are not that easy to represent in Wikisource. The main problem is that our current workflow always assume that we are going to proofread a text and have it stored as a web page. @others: for me it doesn't matter much if the representation of the metadata is done by a template, an index page, or something different (maybe related to the new Extension:BookManager?) However I think that from the user point of view it is better to have a consistent system that can handle: 1) representation of book/source metadata 2) give access to export/visualization options I'm preparing a document with some ideas that we can discuss here. Micru On Tue, Jun 11, 2013 at 7:48 AM, billinghurst <billinghurst(a)gmail.com>wrote;wrote: -- Etiamsi omnes, ego non

On Wed, Jun 12, 2013 at 1:32 PM, billinghurst <billinghurst(a)gmail.com>wrote;wrote: Thanks for clarifying this. I agree with you, and would welcome both solutions. But a lot of wikisourcerors don't think this way, so better discuss :-) Aubrey

On Wed, Jun 12, 2013 at 2:32 PM, Thibaut Horel <thibaut.horel(a)gmail.com>wrote;wrote: I couldn't agree more. I think this could be an opportunity also to make task *smaller* and *clearer* (in the direction of "microtask", which are contributions in crowdsourcing projects which are small, definite and simple. eg GalaxyZoo, reCAPTCHA). We could define some tasks as * corrected the page * proofread the text * formatted the page * validated the formatting * OPTIONAL added optional templates/links/annotations *... We could even have qualifiers (all/part of the page, ...) Is this idea crazy, or somewhat doable? Aubrey

On 06/12/2013 02:48 PM, Andrea Zanni wrote: Geotagged all the photos, ... The list doesn't end. You need a generic mechanism for any new feature you can invent. But aren't our existing templates and categories the best way to do this? You could just add to each page: {{done|proofread=user1|validated=user2|geotagged=user4|...}} -- Lars Aronsson (lars(a)aronsson.se) Project Runeberg - free Nordic literature - http://runeberg.org/

Nobody is saying anything about using copyrighted works, there are many books that have an open license that would allow to include them in Wikisource. For instance in ca-ws we have this translation from 2009: http://ca.wikisource.org/wiki/Llibre:El_secret_de_l%E2%80%99or_que_creix_%2… The original is in the PD, and the translator gave away his rights. It would have been much easier to work directly with the pdf, instead of converting to djvu. Micru On Wed, Jun 12, 2013 at 10:47 AM, Aarti K. Dwivedi < ellydwivedi2093(a)gmail.com&gt; wrote: -- Etiamsi omnes, ego non

On Wed, Jun 12, 2013 at 4:47 PM, Aarti K. Dwivedi &lt;ellydwivedi2093(a)gmail.com You are thinking of *books*, which are not the only documents Wikisource can host. For example, I am thinking about Open Access literature, which counts in hundred thousands CC-BY licensed articles, for example. Just look in DOAJ: http://www.doaj.org/ One of the wikimedians most involved in Open Access - Wiki collaboration is Daniel Mietchen (cc'ed). He's working on a bot who could grab the XML/HTML of an online article, format it in wikicode, and post it wherever he wants (maybe, Wikisources). The bot is aming to download automatically all images within the articles, and post them on Commons. I personally think that this project is beyond awesomeness, IF we manage to solve particular and specific issues (as converting hyperlinks to other articles in wikilinks to those articles posted on WIkisource...) As I said before, I see Wikisource as a broad, international, connected, hypertextual digital library, which has a thing no other digital library in the world has: a dedicated community[*]. It is my personal opinion, I know some people don't see it that way (like Alex :-D) Aubrey [*] there is Project Gutenberg, but I would argue they are not a digital library...

You're right Aubrey nevertheless while promoving a user friendly interface the result is that data and wiki code is extremely difficult to use as a clean "data base". Think only to wiki markup and the "simple" trick to mark bold and italic text with apostophes.... very user friendly, but something like a nightmare for a poor programmer which needs to find the algorithm to understand which apostophes are text and which are code. The server too can't solve solve apostrophes concatenation. Was it less user friendly to use something like <b>...</b>? Yes; but.... how much cleaner raw wiki text would be! Distributed Proofreaders uses a completely different approach: there's a rigid set of increasing abilitations for users, and unexperienced users can do simple task only. This is far from "wiki mentality", but we can't expect to keep things too much easy. Alex 2013/6/11 Andrea Zanni &lt;zanni.andrea84(a)gmail.com&gt;