Hi everybody,
With the Structured Data for Commons project about to move into high gear, it seems to me that there's something the Wikidata community needs to have a serious discussion about, before APIs start getting designed and set in stone.
Specifically: when should an object have an item with its own Q-number created for it on Wikidata? What are the limits? (Are there any limits?)
The position so far seems to be essentially that a Wikidata item has only been created when an object either already has a fully-fledged Wikipedia article written for it, or reasonably could have.
So objects that aren't particularly notable typically have not had Wikidata items made for them.
Indeed, practically the first message Lydia sent to me when I started trying to work on Commons and Wikidata was to underline to me that Wikidata objects should generally not be created for individual Commons files.
But, if I'm reading the initial plans and API thoughts of the Multimedia team correctly, eg https://commons.wikimedia.org/w/index.php?title=File%3AStructured_Data_-_Sli... and https://docs.google.com/document/d/1tzwGtXRyK3o2ZEfc85RJ978znRdrf9EkqdJ0zVjm...
there seems to be the key assumption that, for any image that contains information relating to something beyond the immediate photograph or scan, there will be some kind of 'original work' item on main Wikidata that the file page will be able to reference, such that the 'original work' Wikidata item will be able to act as a place to locate any information specifically relating to the original work.
Now in many ways this is a very clean division to be able to make. It removes any question of having to judge "notability"; and it removes any ambiguity or diversity of where information might be located -- if the information relates to the original work, then it will be stored on Wikidata.
But it would appear to imply a potentially *huge* increase in the inclusion criteria for Wikidata, and the number of Wikidata items potentially creatable.
So it seems appropriate that the Wikidata community should discuss and sign off just what should and should not be considered appropriate, before things get much further.
For example, a year ago the British Library released 1 million illustrations from out-of-copyright books, which increasingly have been uploaded to Commons. Recently the Internet Archive has announced plans to release a further 12 million, with more images either already uploading or to follow from other major repositories including eg the NYPL, the Smithsonian, the Wellcome Foundation, etc, etc.
How many of these images, all scanned from old originals, are going to need new Q-numbers for those originals? Is this okay? Or are some of them too much?
For example, for maps, cf this data schema https://docs.google.com/spreadsheets/d/1Hn8VQ1rBgXj3avkUktjychEhluLQQJl5v6WR... , each map sheet will have a separate Northernmost, Southernmost, Easternmost, Westernmost bounding co-ordinates. Does that mean each map sheet should have its own Wikidata item?
For book illustrations, perhaps it is would be enough just to reference the edition of the book. But if individual illustrations have their own artist and engraver details, does that mean the illustration needs to have its own Wikidata item? Similarly, if the same engraving has appeared in many books, is that also a sign that it should have its own Wikidata item?
What about old photographs, or old postcards, similarly. When should these have their own Wikidata item? If they have their own known creator, and creation date, then is it most simple just to give them a Wikidata item, so that such information about an original underlying work is always looked for on Wikidata? What if multiple copies of the same postcard or photograph are known, published or re-published at different times? But the potential number of old postcards and photographs, like the potential number of old engravings, is *huge*.
What if an engraving was re-issued in different "states" (eg a re-issued engraving of a place might have been modified if a tower had been built). When should these get different items?
At https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Visual_arts#Wikidata... where I raised some of these issues a couple of weeks ago, there has even been the suggestion that particular individual impressions of an engraving might deserve their own separate items; or even everything with a separate accession number, so if a museum had three copies of an engraving, we would make three separate items, each carrying their own accession number, identifying the accession number that belonged to a particular File.
(See also other sections at https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Visual_arts for further relevant discussions on how to represent often quite complicated relations with Wikidata properties).
With enough items, we could re-create and represent essentially the entire FRBR tree.
We could do this. We may even need to do this, if MM team's outline for Commons is to be implemented in its apparent current form.
But it seems to me that we shouldn't just sleepwalk into it.
It does seem to me that this does represent (at least potentially) a *very* large expansion in the number of items, and widening of the inclusion criteria, for what Wikidata is going to encompass.
I'm not saying it isn't the right thing to do, but given the potential scale of the implications, I do think it is something we do need to have properly worked through as a community, and confirmed that it is indeed what we *want* to do.
All best,
James.
(Note that this is a slightly different discussion, though related, to the one I raised a few weeks ago as to whether Commons categories -- eg for particular sets of scans -- should necessarily have their own Q-number on Wikidata. Or whether some -- eg some intersection categories -- should just have an item on Commons data. But it's clearly related: is the simplest thing just to put items for everything on Wikidata? Or does one try to keep Wikidata lean, and no larger than it absolutely needs to be; albeit then having to cope with the complexity that some categories would have a Q-number, and some would not.)
Hi James,
thanks for starting this conversation! It is indeed important and has been overlooked by us.
there seems to be the key assumption that, for any image that contains
information relating to something beyond the immediate photograph or scan, there will be some kind of 'original work' item on main Wikidata that the file page will be able to reference, such that the 'original work' Wikidata item will be able to act as a place to locate any information specifically relating to the original work.
While we would like to keep track of some sort of "original work" entity when the file is a derived work, that entity doesn't necessarily has to be on Wikidata. When a Commons image is the derivative of another image, it makes more sense to refer to the Commonsdata item of that image. One possibility would be to generalize that and allow data items on Commons which are not attached to any file but instead refer to some external work such as a Flickr image.
Gergo,
Thanks for this -- and hoping you have a very productive set of sessions, to all of you in Berlin this week.
Yes, where one has a derivative work of another Commons work (a restoration, or a cropping, say), I can see it makes sense to point to the CommonsData entry for that other work.
But I guess you need to ask yourselves how much of a chain you're prepared to walk, if that file in turn points to data on an underlying physical work. Do you extract the "original creator" through the chain, or link directly? And what if there are multiple original creators?
I don't know whether it makes sense to have "original work" items on CommonsData rather than WikiData, more generally. (And here I'm talking about an "original work" in the sense of an original physical old photograph, or map sheet, or manuscript folio). As someone has pointed out, there are issues about having to deal with things in two places, and questions whether it would still be findable, if for example one were trying to search WikiData for all objects created by a particular creator.
I think this is something we simply have to defer to you, the technical designers of the system, for your considered view on what is the best way forward.
But I would commend some of the discussions at https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Visual_arts to you, not least for some examples file-cases that you might want to consider, for questions like:
* How to treat engravings from books? * What "original works" should (or should not) get their own WikiData items? * Could an edition item on Wikidata be enough to contain all the relevant information not held on CommonsData about an illustration? * What if the same engraving appears in different books?
* If there are a number of pictures from a particular scan-set, should the scan-set have a Wikidata item? (Because there is information we may want to store about the scan-set, eg its source; and we may want to filter by it; and sort members of it according to a qualifier, numerical position in sequence)
Also I would very much commend to you the existing Wikidata schemas for artworks, for book editions, and for book works, as well as the work done by the old maps project: https://www.wikidata.org/wiki/Wikidata:WikiProject_Visual_arts/Item_structur... https://www.wikidata.org/wiki/Wikidata:WikiProject_Books#Edition_item_proper... https://www.wikidata.org/wiki/Wikidata:WikiProject_Books#Work_item_propertie... https://docs.google.com/spreadsheets/d/1Hn8VQ1rBgXj3avkUktjychEhluLQQJl5v6WR...
which already go a long way towards creating a structure for storing information about many sorts of objects.
Finally, on a more general point, I would beg all of you in Berlin this week: don't despise wikitext.
It's easy to think that a shiny new system will displace everything. But there is lots of information, and tools, based on old-fashioned wikitext that it will need to integrate with for the foreseeable future. Wikitext is a straightforward API that a lot of content, and a lot of tools have been built on. So please do think how you can work with that, rather than simply deprecate it.
Vandal-fighting tools in particular are based on changes to Wikitext, so please do consider that it would be good to be able to represent changes in the WikiData or CommonsData in ways that tools based on existing wikitext file pages can pick up and if necessary revert.
Thanks, and all best for this week,
-- James.
On 05/10/2014 11:14, Gergo Tisza wrote:
Hi James,
thanks for starting this conversation! It is indeed important and has been overlooked by us.
there seems to be the key assumption that, for any image that contains
information relating to something beyond the immediate photograph or scan, there will be some kind of 'original work' item on main Wikidata that the file page will be able to reference, such that the 'original work' Wikidata item will be able to act as a place to locate any information specifically relating to the original work.
While we would like to keep track of some sort of "original work" entity when the file is a derived work, that entity doesn't necessarily has to be on Wikidata. When a Commons image is the derivative of another image, it makes more sense to refer to the Commonsdata item of that image. One possibility would be to generalize that and allow data items on Commons which are not attached to any file but instead refer to some external work such as a Flickr image.
Multimedia mailing list Multimedia@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/multimedia
Thanks for the pointers, James! I'll try to digest them.
Our thoughts on the issue of representing relationships between works are not fully formed yet, but the current idea is loosely that * if the original work has a Wikidata item (according to whatever notability guidelines the community prefers), link to that * otherwise if it is a Commons image, link to the local data item of that image * otherwise representing the relationships in full detail is probably not that important, so it's fine to just add the authors of the originals as contributors to the CommonsData entry with some generic role such as "author of a source work", without trying to represent the accurate relationship between them.
So, if there is a chain of "derivative of" relationships between works which have Wikidata or CommonsData items, we can walk the chain upon extraction and collect the authors. Where the theoretical chain extends outside Wikidata+CommonsData, the actual (as stored in Wikibase) chain would have author information from the outlying nodes "squashed" into the edge nodes.
On Mon, Oct 6, 2014 at 11:08 AM, James Heald j.heald@ucl.ac.uk wrote:
Gergo,
Thanks for this -- and hoping you have a very productive set of sessions, to all of you in Berlin this week.
Yes, where one has a derivative work of another Commons work (a restoration, or a cropping, say), I can see it makes sense to point to the CommonsData entry for that other work.
But I guess you need to ask yourselves how much of a chain you're prepared to walk, if that file in turn points to data on an underlying physical work. Do you extract the "original creator" through the chain, or link directly? And what if there are multiple original creators?
I don't know whether it makes sense to have "original work" items on CommonsData rather than WikiData, more generally. (And here I'm talking about an "original work" in the sense of an original physical old photograph, or map sheet, or manuscript folio). As someone has pointed out, there are issues about having to deal with things in two places, and questions whether it would still be findable, if for example one were trying to search WikiData for all objects created by a particular creator.
I think this is something we simply have to defer to you, the technical designers of the system, for your considered view on what is the best way forward.
But I would commend some of the discussions at https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Visual_arts to you, not least for some examples file-cases that you might want to consider, for questions like:
- How to treat engravings from books?
- What "original works" should (or should not) get their own WikiData
items?
- Could an edition item on Wikidata be enough to contain all the relevant
information not held on CommonsData about an illustration?
What if the same engraving appears in different books?
If there are a number of pictures from a particular scan-set, should the
scan-set have a Wikidata item? (Because there is information we may want to store about the scan-set, eg its source; and we may want to filter by it; and sort members of it according to a qualifier, numerical position in sequence)
Also I would very much commend to you the existing Wikidata schemas for artworks, for book editions, and for book works, as well as the work done by the old maps project: https://www.wikidata.org/wiki/Wikidata:WikiProject_Visual_ arts/Item_structure#Describing_individual_objects https://www.wikidata.org/wiki/Wikidata:WikiProject_Books# Edition_item_properties https://www.wikidata.org/wiki/Wikidata:WikiProject_Books# Work_item_properties https://docs.google.com/spreadsheets/d/1Hn8VQ1rBgXj3avkUktjychEhluLQQ Jl5v6WRlI0LJho/edit#gid=0
which already go a long way towards creating a structure for storing information about many sorts of objects.
Finally, on a more general point, I would beg all of you in Berlin this week: don't despise wikitext.
It's easy to think that a shiny new system will displace everything. But there is lots of information, and tools, based on old-fashioned wikitext that it will need to integrate with for the foreseeable future. Wikitext is a straightforward API that a lot of content, and a lot of tools have been built on. So please do think how you can work with that, rather than simply deprecate it.
Vandal-fighting tools in particular are based on changes to Wikitext, so please do consider that it would be good to be able to represent changes in the WikiData or CommonsData in ways that tools based on existing wikitext file pages can pick up and if necessary revert.
Gergo
One of the big advantages of commonsdata over wikitext is that commonsdata is Internationalised and ready for localisation.
For this reason alone I believe it is worth looking closely at all wikitext to see if it can be expressed as a Commonsdata statement.
Joe
On 10 Oct 2014 17:09, "Gergo Tisza" gtisza@wikimedia.org wrote:
Thanks for the pointers, James! I'll try to digest them.
Our thoughts on the issue of representing relationships between works are
not fully formed yet, but the current idea is loosely that
- if the original work has a Wikidata item (according to whatever
notability guidelines the community prefers), link to that
- otherwise if it is a Commons image, link to the local data item of that
image
- otherwise representing the relationships in full detail is probably not
that important, so it's fine to just add the authors of the originals as contributors to the CommonsData entry with some generic role such as "author of a source work", without trying to represent the accurate relationship between them.
So, if there is a chain of "derivative of" relationships between works
which have Wikidata or CommonsData items, we can walk the chain upon extraction and collect the authors. Where the theoretical chain extends outside Wikidata+CommonsData, the actual (as stored in Wikibase) chain would have author information from the outlying nodes "squashed" into the edge nodes.
On Mon, Oct 6, 2014 at 11:08 AM, James Heald j.heald@ucl.ac.uk wrote:
Gergo,
Thanks for this -- and hoping you have a very productive set of
sessions, to all of you in Berlin this week.
Yes, where one has a derivative work of another Commons work (a
restoration, or a cropping, say), I can see it makes sense to point to the CommonsData entry for that other work.
But I guess you need to ask yourselves how much of a chain you're
prepared to walk, if that file in turn points to data on an underlying physical work. Do you extract the "original creator" through the chain, or link directly? And what if there are multiple original creators?
I don't know whether it makes sense to have "original work" items on
CommonsData rather than WikiData, more generally. (And here I'm talking about an "original work" in the sense of an original physical old photograph, or map sheet, or manuscript folio). As someone has pointed out, there are issues about having to deal with things in two places, and questions whether it would still be findable, if for example one were trying to search WikiData for all objects created by a particular creator.
I think this is something we simply have to defer to you, the technical
designers of the system, for your considered view on what is the best way forward.
But I would commend some of the discussions at https://www.wikidata.org/wiki/Wikidata_talk:WikiProject_Visual_arts to you, not least for some examples file-cases that you might want to
consider, for questions like:
- How to treat engravings from books?
- What "original works" should (or should not) get their own WikiData
items?
- Could an edition item on Wikidata be enough to contain all the
relevant information not held on CommonsData about an illustration?
What if the same engraving appears in different books?
If there are a number of pictures from a particular scan-set, should
the scan-set have a Wikidata item? (Because there is information we may want to store about the scan-set, eg its source; and we may want to filter by it; and sort members of it according to a qualifier, numerical position in sequence)
Also I would very much commend to you the existing Wikidata schemas for
artworks, for book editions, and for book works, as well as the work done by the old maps project:
https://www.wikidata.org/wiki/Wikidata:WikiProject_Visual_arts/Item_structur...
https://www.wikidata.org/wiki/Wikidata:WikiProject_Books#Edition_item_proper...
https://www.wikidata.org/wiki/Wikidata:WikiProject_Books#Work_item_propertie...
https://docs.google.com/spreadsheets/d/1Hn8VQ1rBgXj3avkUktjychEhluLQQJl5v6WR...
which already go a long way towards creating a structure for storing
information about many sorts of objects.
Finally, on a more general point, I would beg all of you in Berlin this
week: don't despise wikitext.
It's easy to think that a shiny new system will displace everything. But
there is lots of information, and tools, based on old-fashioned wikitext that it will need to integrate with for the foreseeable future. Wikitext is a straightforward API that a lot of content, and a lot of tools have been built on. So please do think how you can work with that, rather than simply deprecate it.
Vandal-fighting tools in particular are based on changes to Wikitext, so
please do consider that it would be good to be able to represent changes in the WikiData or CommonsData in ways that tools based on existing wikitext file pages can pick up and if necessary revert.
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
One case that particularly comes to mind is where we have multiple different scans of the same work -- eg we have multiple (incomplete) sets of the early 1800s colour engravings from Ackermann's Microcosm of London, or Pyne's Royal Palaces, or Audubon's Birds of America etc.
It seems a shame not to be able to abstract the duplicated information between different scans -- eg the creatorship, the publication history, the topic list of items depicted -- given that they are versions of the same work.
However, if the different scans have been made independently, there is no chain of derivation between them. And - probably - the individual engravings would not pass WD notability, so would not have separate items, though the book they were collected in probably would.
So it doesn't seem that there would be an item on which to store the data that would be common between the different versions of the image.
(Similarly, multiple reproductions of the same vintage photograph, etc).
Perhaps there might be a case for CommonsData items for works that belong to a sequence, where the sequence has an item on Wikidata?
Or perhaps they should just have items on Wikidata?
-- James.
On 10/10/2014 17:08, Gergo Tisza wrote:
Thanks for the pointers, James! I'll try to digest them.
Our thoughts on the issue of representing relationships between works are not fully formed yet, but the current idea is loosely that
- if the original work has a Wikidata item (according to whatever
notability guidelines the community prefers), link to that
- otherwise if it is a Commons image, link to the local data item of that
image
- otherwise representing the relationships in full detail is probably not
that important, so it's fine to just add the authors of the originals as contributors to the CommonsData entry with some generic role such as "author of a source work", without trying to represent the accurate relationship between them.
So, if there is a chain of "derivative of" relationships between works which have Wikidata or CommonsData items, we can walk the chain upon extraction and collect the authors. Where the theoretical chain extends outside Wikidata+CommonsData, the actual (as stored in Wikibase) chain would have author information from the outlying nodes "squashed" into the edge nodes.
On Mon, Oct 6, 2014 at 11:08 AM, James Heald j.heald@ucl.ac.uk wrote:
Gergo,
I think the place for all data about an image should be Wikidata. It will be trivial to update a Wikidata item with an image when that image becomes available on Commons. Until that time, the item can point to a catalog's online or offline entry where the image can be viewed. I am thinking for example of a Salvador Dali work that cannot be included on Wikipedia due to copyright constraints. In this case the catalog entry at least points the user in a useful direction
On Fri, Oct 10, 2014 at 7:33 PM, James Heald j.heald@ucl.ac.uk wrote:
One case that particularly comes to mind is where we have multiple different scans of the same work -- eg we have multiple (incomplete) sets of the early 1800s colour engravings from Ackermann's Microcosm of London, or Pyne's Royal Palaces, or Audubon's Birds of America etc.
It seems a shame not to be able to abstract the duplicated information between different scans -- eg the creatorship, the publication history, the topic list of items depicted -- given that they are versions of the same work.
However, if the different scans have been made independently, there is no chain of derivation between them. And - probably - the individual engravings would not pass WD notability, so would not have separate items, though the book they were collected in probably would.
So it doesn't seem that there would be an item on which to store the data that would be common between the different versions of the image.
(Similarly, multiple reproductions of the same vintage photograph, etc).
Perhaps there might be a case for CommonsData items for works that belong to a sequence, where the sequence has an item on Wikidata?
Or perhaps they should just have items on Wikidata?
-- James.
On 10/10/2014 17:08, Gergo Tisza wrote:
Thanks for the pointers, James! I'll try to digest them.
Our thoughts on the issue of representing relationships between works are not fully formed yet, but the current idea is loosely that
- if the original work has a Wikidata item (according to whatever
notability guidelines the community prefers), link to that
- otherwise if it is a Commons image, link to the local data item of that
image
- otherwise representing the relationships in full detail is probably not
that important, so it's fine to just add the authors of the originals as contributors to the CommonsData entry with some generic role such as "author of a source work", without trying to represent the accurate relationship between them.
So, if there is a chain of "derivative of" relationships between works which have Wikidata or CommonsData items, we can walk the chain upon extraction and collect the authors. Where the theoretical chain extends outside Wikidata+CommonsData, the actual (as stored in Wikibase) chain would have author information from the outlying nodes "squashed" into the edge nodes.
On Mon, Oct 6, 2014 at 11:08 AM, James Heald j.heald@ucl.ac.uk wrote:
Gergo,
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
In my opinion "how many items will X add" is a false problem. If p.a. we moved file categories to subpages as we do on templates, we'd have 20 millions new Commons pages: but the question would be, are they as accessible as they were before? Similarly, the only danger is when items' statements are not transcluded outside Wikidata. When stuff is in use on projects, as for authority codes; and when it can be edited in-place, as we all do for sitelinks and ru.wiki does for much more: then there is nothing I worry about.
Nemo
@Daniel - the further back you go, the more notable the engravings in books become (see for example the whole family of engravings and copies thereof for the 17th-century "Counts of Holland" series) and sometimes engravings from books are the source for paintings. @Nemo - I don't follow your thinking on this one - when you say "new Commons pages, do you mean new Wikidata items based on Commons categories? I don't see a problem with that. Things that have needed a category on Wikimedia Commons are probably notable enough for Wikidata (though I can think of some non-notable categories like "1610 engravings" that would be unnecessary on Wikidata)
On Tue, Oct 14, 2014 at 8:39 AM, Federico Leva (Nemo) nemowiki@gmail.com wrote:
In my opinion "how many items will X add" is a false problem. If p.a. we moved file categories to subpages as we do on templates, we'd have 20 millions new Commons pages: but the question would be, are they as accessible as they were before? Similarly, the only danger is when items' statements are not transcluded outside Wikidata. When stuff is in use on projects, as for authority codes; and when it can be edited in-place, as we all do for sitelinks and ru.wiki does for much more: then there is nothing I worry about.
Nemo
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
multimedia@lists.wikimedia.org