As you know, wikisource needs robust, well-defined data, and there's a strict, deep relationship between wikisource and Commons since Commons hosts images of books, in .djvu or .pdf files. Commons shares both images and contents fo information page of images, so that any wiki project can visualize a view-only "pseudo-page" accessing to a local page named as the file name into Commons.
Working into self-made data semantization into it.wikisouce using a lot of creative tricks, we discovered that it's hard/almost impossible to read by AJAX calls the contents of pages of other projects since well-known same origin policy, but that File: local pages are considered as coming from "same origin" so that they can be read as any other local page, and this AJAX call asking for the content of i.e. File:Die_Judenfrage_in_Deutchland_1936.djvu:
html=$.ajax({url:" http://wikisource.org/wiki/File:Die_Judenfrage_in_Deutchland_1936.djvu ",async:false}).responseText;
gives back the html text of local File: view-only page, and this means that any data stored into information page into Commons is freely accessible by a javascript script and can easily used locally. In particular, data stored into information and/or (much better) Book and Creator templates can be retrieved and parsed
Has this been described/used before? It seems a plain, simple way to share and disseminate good, consistent metadata into any project; and this runs from today, without any change on current wiki software.
If you like, I'm sharing a practical test use of this trick into wikisource.org too, you can import User:Alex brollo/Library.js and a lot of smallo, original scripts will be loaded; click on "metadata" botton from any page connected to a File: page ( namespaces Index, Page) and you'll see a result coming from such an AJAX call.
Alex brollo, from it.wikisource
On Wed, Aug 29, 2012 at 12:39 AM, Alex Brollo alex.brollo@gmail.com wrote:
As you know, wikisource needs robust, well-defined data, and there's a strict, deep relationship between wikisource and Commons since Commons hosts images of books, in .djvu or .pdf files. Commons shares both images and contents fo information page of images, so that any wiki project can visualize a view-only "pseudo-page" accessing to a local page named as the file name into Commons.
Working into self-made data semantization into it.wikisouce using a lot of creative tricks, we discovered that it's hard/almost impossible to read by AJAX calls the contents of pages of other projects since well-known same origin policy, but that File: local pages are considered as coming from "same origin" so that they can be read as any other local page, and this AJAX call asking for the content of i.e. File:Die_Judenfrage_in_Deutchland_1936.djvu:
html=$.ajax({url:" http://wikisource.org/wiki/File:Die_Judenfrage_in_Deutchland_1936.djvu ",async:false}).responseText;
gives back the html text of local File: view-only page, and this means that any data stored into information page into Commons is freely accessible by a javascript script and can easily used locally. In particular, data stored into information and/or (much better) Book and Creator templates can be retrieved and parsed
Has this been described/used before? It seems a plain, simple way to share and disseminate good, consistent metadata into any project; and this runs from today, without any change on current wiki software. [..]
You can also do this more directly using JSON with callback - Define a function foo, and put http://commons.wikimedia.org/w/api.php?action=parse&page=Main_Page&f... as the src of the script tag. This works for certain "safe" api methods. In future we may support CORS which will allow full cross-origin js requests between Wikimedia sites
However, I think the main issue is that people want to do more things with metadata than just retrieving the information with js. I suppose it all boils down to use-cases. I'll admit I'm not overly familiar with Wikisource's metadata use case.
-- -bawolff
On Thu, Aug 30, 2012 at 12:25 AM, bawolff bawolff+wn@gmail.com wrote:
You can also do this more directly using JSON with callback - Define a function foo, and put http://commons.wikimedia.org/w/api.php?action=parse&page=Main_Page&f... as the src of the script tag. This works for certain "safe" api methods. In future we may support CORS which will allow full cross-origin js requests between Wikimedia sites
Has there been any discussion of CORS support in Mediawiki / WMF sites anywhere?
Has there been any discussion of CORS support in Mediawiki / WMF sites anywhere?
There was some talk of it in bug 32890 [0] in UploadWizard, and I tried to throw together code for it in a patchset [1], but I didn't spend much time on it (the effort was mostly to put the code into a workable patchset rather than just a snippet in a bug).
bawolff also commented on the patchset and linked to bug 20814 [2], which is further discussion of the topic in broader strokes.
[0] https://bugzilla.wikimedia.org/show_bug.cgi?id=32890 [1] https://gerrit.wikimedia.org/r/#/c/9718/ [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=20814
As mentioned in those bugs, mediawiki support a basic CORS implementation already. It looks like we haven't authorized any domain for wmf projects though.
On Wed, Aug 29, 2012 at 12:03 PM, Mark Holmquist mtraceur@member.fsf.org wrote:
Has there been any discussion of CORS support in Mediawiki / WMF sites anywhere?
There was some talk of it in bug 32890 [0] in UploadWizard, and I tried to throw together code for it in a patchset [1], but I didn't spend much time on it (the effort was mostly to put the code into a workable patchset rather than just a snippet in a bug).
bawolff also commented on the patchset and linked to bug 20814 [2], which is further discussion of the topic in broader strokes.
[0] https://bugzilla.wikimedia.org/show_bug.cgi?id=32890 [1] https://gerrit.wikimedia.org/r/#/c/9718/ [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=20814
-- Mark Holmquist Contractor, Wikimedia Foundation mtraceur@member.fsf.org http://marktraceur.info
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
On Wed, Aug 29, 2012 at 2:55 PM, Chris Steipp csteipp@wikimedia.org wrote:
As mentioned in those bugs, mediawiki support a basic CORS implementation already. It looks like we haven't authorized any domain for wmf projects though.
Looks like Roan is taking charge on it on bug 20814, yay. :)
There are also open requests about setting up CORS headers to allow read access to upload & bits: https://bugzilla.wikimedia.org/show_bug.cgi?id=28700 https://bugzilla.wikimedia.org/show_bug.cgi?id=25886
Access for upload is important for supporting web-based tools for SVG or raster image editing (eg deploying SVGEditor extension).
[Note that we want *other sites* to be able to load files from upload.wikimedia.org, not files on upload.wikimedia.org to be able to open anything else. :)]
-- brion
Thanks for comments.
Relationship between wikisource and Commons is very strict, and there's a large 1:1 match between structured wikisource data stored into well-formed templates (used into nsIndex and ns0) and Book template; there's too a 1:1 relationship between nsCreator into Commons and nsAuthor into wikisource.
Djvu and (less used) pdf files are already shared among different wikisource projects, but data stored into information page of files are not shared, so that any project rewrites them and stores them with a variety of formats and contents, introducing redundance and mining deeply coherence.
So, we are going to use Commons metadata - already stored into Book and Creator templates - to share them widely anong all projects that need them. When metadata are uploaded and parsed they can be used to feed local wikisource templates and/or to align data with automated procedures.
Thanks for API suggestion, but the question is: does it violates "same origin" AJAX policy? I can read anything by a bot from any project, but AJAX is great to enhance interactivity and to help user just when user needs data, i.e. in edit mode.
Yes, the solution will be CORS, but I can't wait for future enhancements when data can be accessed and used today, with present software.
Alex brollo
On Wed, Aug 29, 2012 at 2:24 PM, Alex Brollo alex.brollo@gmail.com wrote:
Thanks for comments.
[..]
Thanks for API suggestion, but the question is: does it violates "same origin" AJAX policy? I can read anything by a bot from any project, but AJAX is great to enhance interactivity and to help user just when user needs data, i.e. in edit mode.
No it doesn't violate the same origin policy. Same origin policy only prevents reading information from other websites, it does not stop you from executing content from other websites (Which always seemed an odd distinction to me...). Thus you can use the api with a callback parameter to get around the same origin policy.
Obviously CORS is a much nicer solution.
-bawolff
2012/8/29 bawolff bawolff+wn@gmail.com
On Wed, Aug 29, 2012 at 2:24 PM, Alex Brollo alex.brollo@gmail.com wrote:
Thanks for comments.
[..]
Thanks for API suggestion, but the question is: does it violates "same origin" AJAX policy? I can read anything by a bot from any project, but AJAX is great to enhance interactivity and to help user just when user needs data, i.e. in edit mode.
No it doesn't violate the same origin policy. Same origin policy only prevents reading information from other websites, it does not stop you from executing content from other websites (Which always seemed an odd distinction to me...). Thus you can use the api with a callback parameter to get around the same origin policy.
Obviously CORS is a much nicer solution.
-bawolff
Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
No it doesn't violate the same origin policy. Same origin policy only
prevents reading information from other websites, it does not stop you from executing content from other websites (Which always seemed an odd distinction to me...). Thus you can use the api with a callback parameter to get around the same origin policy.
Ouch.... this is a little bit above my skill & understanding (really I discovered AJAX not far ago). . Where can I find some examples of API inter-project data exchage wth callback parameter?
I.e: I'd like to get the raw content and the html rendering of File:I_promessi_sposi_(1840).djvu from wikisource, with an AJAX callback call. Which is the needed code?
:-(
Alex
On Wed, Aug 29, 2012 at 6:03 PM, Alex Brollo alex.brollo@gmail.com wrote:
Ouch.... this is a little bit above my skill & understanding (really I discovered AJAX not far ago). . Where can I find some examples of API inter-project data exchage wth callback parameter?
I.e: I'd like to get the raw content and the html rendering of File:I_promessi_sposi_(1840).djvu from wikisource, with an AJAX callback call. Which is the needed code?
Luckily, if you're using jQuery much of the low-level stuff can be taken care of for you. Something like this should work for API calls, automatically using the callback behind the scenes:
$.ajax({ url: 'https://commons.wikimedia.org/w/api.php', data: { format: 'json', // ... various settings ... }, dataType: 'jsonp' // this is the important one! }).done(function(data) { // do some work here alert(JSON.stringify(data)); });
You can look up the various parameters for different queries in the online help at https://commons.wikimedia.org/w/api.php and/or playing with the API sandbox at https://commons.wikimedia.org/wiki/Special:ApiSandbox
-- brion
2012/8/30 Brion Vibber brion@pobox.com
Luckily, if you're using jQuery much of the low-level stuff can be taken care of for you. Something like this should work for API calls, automatically using the callback behind the scenes:
Thanks! really I tested some interproject AJAX API call with no luck; but I never tested a similar syntax. Can I ask for something more help using your email, if needed (and I guess, it will...)? Obviuosly I'll share into this thread any good result.
Unluckily, I found that field of Information and Book templates are well, clearly tagged with informative ids, while Creator fields are not. I posted a proposal there, but I know that editing complex templates is a hard job.
Alex
Thanks again Brion, it runs perfectly and - strange to say - I got no hard difficulty, just a little bit of review of API calls and of structure of resulting formidable objects. It's really much simpler to parse original template contents than resulting html from their expansion.... ;-) and AJAX API calls are SO fast!
Alex
Just to give a final feedback to this talk, that has been very useful for my tries: woks are going on fastly, and are presently focused on alignement of some structures templates whose data are shared between Commons and Wikisource: Creator vs. Author; Book vs MediaWiki:Proofreadpage_index_template.
Thanks again to list members. :-)
Alex
On Wed, 29 Aug 2012 12:03:05 -0700, Mark Holmquist mtraceur@member.fsf.org wrote:
Has there been any discussion of CORS support in Mediawiki / WMF sites anywhere?
There was some talk of it in bug 32890 [0] in UploadWizard, and I tried to throw together code for it in a patchset [1], but I didn't spend much time on it (the effort was mostly to put the code into a workable patchset rather than just a snippet in a bug).
bawolff also commented on the patchset and linked to bug 20814 [2], which is further discussion of the topic in broader strokes.
[0] https://bugzilla.wikimedia.org/show_bug.cgi?id=32890 [1] https://gerrit.wikimedia.org/r/#/c/9718/ [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=20814
CORS will definitely be something we want to implement when OAuth/SomethingLikeIt comes around. JSONP is a horrid concept and we shouldn't encourage it's use. And it's impossible to do write actions or proper OAuth over JSONP. So CORS will be necessary for any pure-JS OAuth apps. eg: 3rd party anti-spam web apps that let you open them up in your browser, type in a wiki's url, connect with OAuth, and then make batch cleanups, patrolling, etc...
On Wed, 29 Aug 2012 12:03:05 -0700, Mark Holmquist mtraceur@member.fsf.org wrote:
Has there been any discussion of CORS support in Mediawiki / WMF sites anywhere?
There was some talk of it in bug 32890 [0] in UploadWizard, and I tried to throw together code for it in a patchset [1], but I didn't spend much time on it (the effort was mostly to put the code into a workable patchset rather than just a snippet in a bug).
bawolff also commented on the patchset and linked to bug 20814 [2], which is further discussion of the topic in broader strokes.
[0] https://bugzilla.wikimedia.org/show_bug.cgi?id=32890 [1] https://gerrit.wikimedia.org/r/#/c/9718/ [2] https://bugzilla.wikimedia.org/show_bug.cgi?id=20814
CORS will definitely be something we want to implement when OAuth/SomethingLikeIt comes around. JSONP is a horrid concept and we shouldn't encourage it's use. And it's impossible to do write actions or proper OAuth over JSONP. So CORS will be necessary for any pure-JS OAuth apps. eg: 3rd party anti-spam web apps that let you open them up in your browser, type in a wiki's url, connect with OAuth, and then make batch cleanups, patrolling, etc...
wikitech-l@lists.wikimedia.org