Hey all,
I've created a very rough REST API for Wikidata and am looking for your feedback.
* About this API: http://queryr.wmflabs.org * Documentation: http://queryr.wmflabs.org/about/docs * API root: http://queryr.wmflabs.org/api
At present this is purely a demo. The data it serves is stale and potentially incomplete, the endpoints and formats they use are very much liable to change, the server setup is not reliable and I'm not 100% sure I'll continue with this little project.
The main thing I'm going for with this API compared to the existing one is greater ease of use for common use cases. Several factors make this a lot easier to do in a new API than in the existing one: no need the serve all use cases, no need to retain compatibility with existing users and no framework imposed restrictions. You can read more about the difference on the website.
You are invited to comment on the concept and on the open questions mentioned on the website.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Hey Jeroen, awesome. I think it's very important to make sure everyone can access Wikidata's data as easy as possible. For many, i suppose SPARQL is just a bit too complex, and the API is lacking in simplicity as well.
Considering common use cases, i've built the Sum of All Knowledge where i needed to tackle the same problem (read-only access), it might be interesting to compare how our approaches differ.
This is an example of a Sum page: http://sum.bykr.org/328523
Which maps to this Wikidata item: https://www.wikidata.org/wiki/Q328523
The Sum application has a strict separation of concerns model, where the backend is a separate Python application called Chantek: https://github.com/hay/chantek
Here's the JSON format that is served over the wire to the Sum frontend: http://api.haykranen.nl/wikidata/entity?q=Q328523
And here's the one from QueryR: http://queryr.wmflabs.org/api/items/Q328523
For comparison, here's the one from the official API: https://www.wikidata.org/w/api.php?action=wbgetentities&ids=Q328523&...
I suppose the main difference is that your API is even simpler than mine. One thing that might be difficult is having the property labels as keys in the JSON. Those things tend to change, and having the Property ID there is probably safer.
One other thing that i've added is a direct link to the image, otherwise that's another query.
-- Hay
On Mon, Nov 30, 2015 at 2:55 PM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey all,
I've created a very rough REST API for Wikidata and am looking for your feedback.
- About this API: http://queryr.wmflabs.org
- Documentation: http://queryr.wmflabs.org/about/docs
- API root: http://queryr.wmflabs.org/api
At present this is purely a demo. The data it serves is stale and potentially incomplete, the endpoints and formats they use are very much liable to change, the server setup is not reliable and I'm not 100% sure I'll continue with this little project.
The main thing I'm going for with this API compared to the existing one is greater ease of use for common use cases. Several factors make this a lot easier to do in a new API than in the existing one: no need the serve all use cases, no need to retain compatibility with existing users and no framework imposed restrictions. You can read more about the difference on the website.
You are invited to comment on the concept and on the open questions mentioned on the website.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
If you are using the P/Q/whatever markers in the id, then you should not differentiate on items and properties in the root.
The path /items/{item_id}/data/{property_label} should use the property id and not the property label. The later is not stable.
On Mon, Nov 30, 2015 at 2:55 PM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey all,
I've created a very rough REST API for Wikidata and am looking for your feedback.
- About this API: http://queryr.wmflabs.org
- Documentation: http://queryr.wmflabs.org/about/docs
- API root: http://queryr.wmflabs.org/api
At present this is purely a demo. The data it serves is stale and potentially incomplete, the endpoints and formats they use are very much liable to change, the server setup is not reliable and I'm not 100% sure I'll continue with this little project.
The main thing I'm going for with this API compared to the existing one is greater ease of use for common use cases. Several factors make this a lot easier to do in a new API than in the existing one: no need the serve all use cases, no need to retain compatibility with existing users and no framework imposed restrictions. You can read more about the difference on the website.
You are invited to comment on the concept and on the open questions mentioned on the website.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Seems like you filter siteids on language, I don't think this is a correct behaviour.
On Mon, Nov 30, 2015 at 5:38 PM, John Erling Blad jeblad@gmail.com wrote:
If you are using the P/Q/whatever markers in the id, then you should not differentiate on items and properties in the root.
The path /items/{item_id}/data/{property_label} should use the property id and not the property label. The later is not stable.
On Mon, Nov 30, 2015 at 2:55 PM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey all,
I've created a very rough REST API for Wikidata and am looking for your feedback.
- About this API: http://queryr.wmflabs.org
- Documentation: http://queryr.wmflabs.org/about/docs
- API root: http://queryr.wmflabs.org/api
At present this is purely a demo. The data it serves is stale and potentially incomplete, the endpoints and formats they use are very much liable to change, the server setup is not reliable and I'm not 100% sure I'll continue with this little project.
The main thing I'm going for with this API compared to the existing one is greater ease of use for common use cases. Several factors make this a lot easier to do in a new API than in the existing one: no need the serve all use cases, no need to retain compatibility with existing users and no framework imposed restrictions. You can read more about the difference on the website.
You are invited to comment on the concept and on the open questions mentioned on the website.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hoi, It may not be stable but it is what PEOPLE understand. What you can do is get the labels again each time when you show or edit the tool. In this the numbers are leading and PEOPLE can actually use it.
<grin> I salute the effort and I appreciate the critique </grin> however many approaches do not have ordinary people in mind but are from ones own perspective. When that is of a developer of a data scientist it is often correct but hardly usable. Thanks, GerardM
On 30 November 2015 at 17:38, John Erling Blad jeblad@gmail.com wrote:
If you are using the P/Q/whatever markers in the id, then you should not differentiate on items and properties in the root.
The path /items/{item_id}/data/{property_label} should use the property id and not the property label. The later is not stable.
On Mon, Nov 30, 2015 at 2:55 PM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey all,
I've created a very rough REST API for Wikidata and am looking for your feedback.
- About this API: http://queryr.wmflabs.org
- Documentation: http://queryr.wmflabs.org/about/docs
- API root: http://queryr.wmflabs.org/api
At present this is purely a demo. The data it serves is stale and potentially incomplete, the endpoints and formats they use are very much liable to change, the server setup is not reliable and I'm not 100% sure I'll continue with this little project.
The main thing I'm going for with this API compared to the existing one is greater ease of use for common use cases. Several factors make this a lot easier to do in a new API than in the existing one: no need the serve all use cases, no need to retain compatibility with existing users and no framework imposed restrictions. You can read more about the difference on the website.
You are invited to comment on the concept and on the open questions mentioned on the website.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hi!
It may not be stable but it is what PEOPLE understand. What you can do
This is not as simple as it seems. First, people usually understand only one language version - thus, we'd have 200 URIs referring to the same object, but that's not the main issue I see with it. The main issue is that the name is not always trivial to guess - so you'd have to go to wikidata and look it up anyway (especially if not all languages are supported). And, also, if you use English name and somebody uses Russian interface, they may not even know that's the same property without looking up on Wikidata. So yes, when displaying, label is what people want. But when using the API - not so sure.
<grin> I salute the effort and I appreciate the critique </grin> however many approaches do not have ordinary people in mind but are from ones own perspective. When that is of a developer of a data scientist it is often correct but hardly usable.
What you mean by "ordinary people" here? If you mean random person selected out of 7 billions living on a planet, chances are they won't know the first thing about what REST API is, what JSON is and what that thing is all about. So we are talking about very specific narrow category of people who do know what REST API is and need it and know how to use it. So you can make some assumptions here which are not true in general population, but may be true amongst REST-API-using population.
Hoi, You are right. However, Hay was critiqued for his approach. Arguably he is absolutely using the right approach for his use case.
When you state that people have to go back to Wikidata, it is easier to search for a label than it is to search for an ID. When you are developing software and you use whatever technology, please appreciate that in the final analysis what you create is to be used. JSON, the REST API are for developers but it is a technique not a tool. What Hay demonstrates is a usable tool. Thanks, GerardM
On 1 December 2015 at 09:14, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
It may not be stable but it is what PEOPLE understand. What you can do
This is not as simple as it seems. First, people usually understand only one language version - thus, we'd have 200 URIs referring to the same object, but that's not the main issue I see with it. The main issue is that the name is not always trivial to guess - so you'd have to go to wikidata and look it up anyway (especially if not all languages are supported). And, also, if you use English name and somebody uses Russian interface, they may not even know that's the same property without looking up on Wikidata. So yes, when displaying, label is what people want. But when using the API - not so sure.
<grin> I salute the effort and I appreciate the critique </grin> however many approaches do not have ordinary people in mind but are from ones own perspective. When that is of a developer of a data scientist it is often correct but hardly usable.
What you mean by "ordinary people" here? If you mean random person selected out of 7 billions living on a planet, chances are they won't know the first thing about what REST API is, what JSON is and what that thing is all about. So we are talking about very specific narrow category of people who do know what REST API is and need it and know how to use it. So you can make some assumptions here which are not true in general population, but may be true amongst REST-API-using population. -- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I have proposed several properties on Wikidata and discovered others by browsing items. Using shortcuts I don't need to type in the full names of things. Frankly there is no way I would be able to guess the property labels in English, let alone any other language. I still need to go to an item to look up both the property name and the property number I am looking for. Many properties have an item that links to an article somewhere that will tell you more, but most do not. I think it is important to keep to the Q- and P- numbers in anything one does on Wikidata, since that is one of the things it was designed to do, namely to create permanent identifiers for concepts that flip around a lot in terms of wiki titles.
On Tue, Dec 1, 2015 at 9:35 AM, Gerard Meijssen gerard.meijssen@gmail.com wrote:
Hoi, You are right. However, Hay was critiqued for his approach. Arguably he is absolutely using the right approach for his use case.
When you state that people have to go back to Wikidata, it is easier to search for a label than it is to search for an ID. When you are developing software and you use whatever technology, please appreciate that in the final analysis what you create is to be used. JSON, the REST API are for developers but it is a technique not a tool. What Hay demonstrates is a usable tool. Thanks, GerardM
On 1 December 2015 at 09:14, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
It may not be stable but it is what PEOPLE understand. What you can do
This is not as simple as it seems. First, people usually understand only one language version - thus, we'd have 200 URIs referring to the same object, but that's not the main issue I see with it. The main issue is that the name is not always trivial to guess - so you'd have to go to wikidata and look it up anyway (especially if not all languages are supported). And, also, if you use English name and somebody uses Russian interface, they may not even know that's the same property without looking up on Wikidata. So yes, when displaying, label is what people want. But when using the API - not so sure.
<grin> I salute the effort and I appreciate the critique </grin> however many approaches do not have ordinary people in mind but are from ones own perspective. When that is of a developer of a data scientist it is often correct but hardly usable.
What you mean by "ordinary people" here? If you mean random person selected out of 7 billions living on a planet, chances are they won't know the first thing about what REST API is, what JSON is and what that thing is all about. So we are talking about very specific narrow category of people who do know what REST API is and need it and know how to use it. So you can make some assumptions here which are not true in general population, but may be true amongst REST-API-using population. -- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hoi, In the browsers that I use, when you hover over a property it shows both in Reasonator and in Wikidata .. Hope it helps. Thanks, GerardM
On 1 December 2015 at 10:24, Jane Darnell jane023@gmail.com wrote:
I have proposed several properties on Wikidata and discovered others by browsing items. Using shortcuts I don't need to type in the full names of things. Frankly there is no way I would be able to guess the property labels in English, let alone any other language. I still need to go to an item to look up both the property name and the property number I am looking for. Many properties have an item that links to an article somewhere that will tell you more, but most do not. I think it is important to keep to the Q- and P- numbers in anything one does on Wikidata, since that is one of the things it was designed to do, namely to create permanent identifiers for concepts that flip around a lot in terms of wiki titles.
On Tue, Dec 1, 2015 at 9:35 AM, Gerard Meijssen <gerard.meijssen@gmail.com
wrote:
Hoi, You are right. However, Hay was critiqued for his approach. Arguably he is absolutely using the right approach for his use case.
When you state that people have to go back to Wikidata, it is easier to search for a label than it is to search for an ID. When you are developing software and you use whatever technology, please appreciate that in the final analysis what you create is to be used. JSON, the REST API are for developers but it is a technique not a tool. What Hay demonstrates is a usable tool. Thanks, GerardM
On 1 December 2015 at 09:14, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
It may not be stable but it is what PEOPLE understand. What you can do
This is not as simple as it seems. First, people usually understand only one language version - thus, we'd have 200 URIs referring to the same object, but that's not the main issue I see with it. The main issue is that the name is not always trivial to guess - so you'd have to go to wikidata and look it up anyway (especially if not all languages are supported). And, also, if you use English name and somebody uses Russian interface, they may not even know that's the same property without looking up on Wikidata. So yes, when displaying, label is what people want. But when using the API - not so sure.
<grin> I salute the effort and I appreciate the critique </grin>
however
many approaches do not have ordinary people in mind but are from ones own perspective. When that is of a developer of a data scientist it is often correct but hardly usable.
What you mean by "ordinary people" here? If you mean random person selected out of 7 billions living on a planet, chances are they won't know the first thing about what REST API is, what JSON is and what that thing is all about. So we are talking about very specific narrow category of people who do know what REST API is and need it and know how to use it. So you can make some assumptions here which are not true in general population, but may be true amongst REST-API-using population. -- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Gerard, I was referring to the user interface for the API to Wikidata, not the Wikidata user interface. So when I type in "Leo Gestel" to the [1] Api demo interface I get the info back, but only if I click the option "View on QueryR" do I see the access syntax (2]. I think the user interface should accept Q and P numbers, not labels, though it could provide a lookup gadget.
[1] http://queryr.wmflabs.org/about/demo [2] http://queryr.wmflabs.org/api/items/Q597999
On Tue, Dec 1, 2015 at 11:15 AM, Gerard Meijssen gerard.meijssen@gmail.com wrote:
Hoi, In the browsers that I use, when you hover over a property it shows both in Reasonator and in Wikidata .. Hope it helps. Thanks, GerardM
On 1 December 2015 at 10:24, Jane Darnell jane023@gmail.com wrote:
I have proposed several properties on Wikidata and discovered others by browsing items. Using shortcuts I don't need to type in the full names of things. Frankly there is no way I would be able to guess the property labels in English, let alone any other language. I still need to go to an item to look up both the property name and the property number I am looking for. Many properties have an item that links to an article somewhere that will tell you more, but most do not. I think it is important to keep to the Q- and P- numbers in anything one does on Wikidata, since that is one of the things it was designed to do, namely to create permanent identifiers for concepts that flip around a lot in terms of wiki titles.
On Tue, Dec 1, 2015 at 9:35 AM, Gerard Meijssen < gerard.meijssen@gmail.com> wrote:
Hoi, You are right. However, Hay was critiqued for his approach. Arguably he is absolutely using the right approach for his use case.
When you state that people have to go back to Wikidata, it is easier to search for a label than it is to search for an ID. When you are developing software and you use whatever technology, please appreciate that in the final analysis what you create is to be used. JSON, the REST API are for developers but it is a technique not a tool. What Hay demonstrates is a usable tool. Thanks, GerardM
On 1 December 2015 at 09:14, Stas Malyshev smalyshev@wikimedia.org wrote:
Hi!
It may not be stable but it is what PEOPLE understand. What you can do
This is not as simple as it seems. First, people usually understand only one language version - thus, we'd have 200 URIs referring to the same object, but that's not the main issue I see with it. The main issue is that the name is not always trivial to guess - so you'd have to go to wikidata and look it up anyway (especially if not all languages are supported). And, also, if you use English name and somebody uses Russian interface, they may not even know that's the same property without looking up on Wikidata. So yes, when displaying, label is what people want. But when using the API - not so sure.
<grin> I salute the effort and I appreciate the critique </grin>
however
many approaches do not have ordinary people in mind but are from ones own perspective. When that is of a developer of a data scientist it is often correct but hardly usable.
What you mean by "ordinary people" here? If you mean random person selected out of 7 billions living on a planet, chances are they won't know the first thing about what REST API is, what JSON is and what that thing is all about. So we are talking about very specific narrow category of people who do know what REST API is and need it and know how to use it. So you can make some assumptions here which are not true in general population, but may be true amongst REST-API-using population. -- Stas Malyshev smalyshev@wikimedia.org
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hey all,
Thanks a lot for your valuable input.
## Labels vs ids
This has been raised by several people and for good reason. It's very important to use the stability of ids, which the current item response format does not allow. This is because I've not figured out yet how to best **serve both** the users that want to use the ids, and those that do not want to be forced into dealing with them. I've written down some thoughts on what can be done in this issue [0], and welcome your input (be it here on the mailing list or on the issue).
[0] https://github.com/JeroenDeDauw/QueryrAPI/issues/36
If accessing data by relying on labels is ever a good idea has been put into question. The answer to this is not clear to me.
Most developers out there do not know about Wikidata, so the most common use case seems to be "get some information out of Wikipedia". In case such developers want to do something where for whichever reason high stability is not required, then it is presumably not nice to force them to figure out what these P123 things are and which one they need. I certainly understand the arguments of "not entirely correct" and "might cause them to make the wrong choice", though forcing casual users to do the more advanced thing rather than just recommending might end up being quite frustrating for them.
Note that the item response format is now actually documented :) http://queryr.wmflabs.org/about/docs/item
## /items and /properties vs /entities
Can the people that suggested this motivate why they think having a single endpoint would be better?
I know that is what the Wikidata API does, though think that is a design flaw which we've not been able to move away from with reasonable cost due to API compatibility concerns. Reasoning from my side is that the type of resource is different (both conceptually and in response format), thus that different endpoints make sense. An added advantage is that you have collection endpoints for just items and just properties. If you only have one for entities, it becomes a lot harder to just walk through properties.
## Entity lookup by non-id
So when I type in "Leo Gestel" to the [1] Api demo interface I get the
info back, but only if I click the option "View on QueryR" do I see the access syntax (2]. I think the user interface should accept Q and P numbers, not labels, though it could provide a lookup gadget.
[1] http://queryr.wmflabs.org/about/demo [2] http://queryr.wmflabs.org/api/items/Q597999
I agree it makes sense to use ids here (as is currently done). At the same time I think it's very important to provide convenient access by Wikipedia page name. That is what the demo search thing does (those are not labels). Such access is crucial for people that want to "get data for this/these Wikipedia page(s)" or "get this specific data for this/these Wikipedia page(s)".
The little demo thing currently resolves the page name via the Wikidata API as such lookup is not yet possible via QueryR. I'm thinking of having redirects to the canonical resource:
* /wikipedia/$enWikiPageName => /items/$itemId * /wikipedia/$subdomain/$pageName => /items/$itemId
$subdomain being the part before ".wikipedia.org" of the relevant Wikipedia ("en" for the English Wikipedia).
Does that sound reasonable? Is there a better way to do this?
(This is now also tracked by https://github.com/JeroenDeDauw/QueryrAPI/issues/37)
## Sum of All Knowledge
Interesting project, and nice separation of concerns! Too bad I was not aware of this when I started writing QueryR. Though it is now indeed certainly nice to learn by looking at the differences between them.
The point on having direct links to images is well taken - the reason this is not done yet is because I've yet to get to it.
One key difference is that Chantek forwards directly to the Wikidata API, while QueryR has its own persistence. (If you can get away with not having the persistence, then that is definitely the way to go, as it's significantly simpler.) Would you have done anything different in the format you created if you had such persistence?
## Not all values shown?
one value, shouldn't it return multiple ones?
Only the highest ranked values are returned. In case of this set of values, one is preferred and the rest are normal, so only the preferred one is shown. Do you think a different behaviour makes makes more sense?
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Hi Jeroen
I was interested to take a look at your API but I was surprised to see it described as REST. It doesn't look like hypertext to me. In a REST API, related resources should be represented using hyperlinks.
Conal
On 2 December 2015 at 09:23, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey all,
Thanks a lot for your valuable input.
## Labels vs ids
This has been raised by several people and for good reason. It's very important to use the stability of ids, which the current item response format does not allow. This is because I've not figured out yet how to best **serve both** the users that want to use the ids, and those that do not want to be forced into dealing with them. I've written down some thoughts on what can be done in this issue [0], and welcome your input (be it here on the mailing list or on the issue).
[0] https://github.com/JeroenDeDauw/QueryrAPI/issues/36
If accessing data by relying on labels is ever a good idea has been put into question. The answer to this is not clear to me.
Most developers out there do not know about Wikidata, so the most common use case seems to be "get some information out of Wikipedia". In case such developers want to do something where for whichever reason high stability is not required, then it is presumably not nice to force them to figure out what these P123 things are and which one they need. I certainly understand the arguments of "not entirely correct" and "might cause them to make the wrong choice", though forcing casual users to do the more advanced thing rather than just recommending might end up being quite frustrating for them.
Note that the item response format is now actually documented :) http://queryr.wmflabs.org/about/docs/item
## /items and /properties vs /entities
Can the people that suggested this motivate why they think having a single endpoint would be better?
I know that is what the Wikidata API does, though think that is a design flaw which we've not been able to move away from with reasonable cost due to API compatibility concerns. Reasoning from my side is that the type of resource is different (both conceptually and in response format), thus that different endpoints make sense. An added advantage is that you have collection endpoints for just items and just properties. If you only have one for entities, it becomes a lot harder to just walk through properties.
## Entity lookup by non-id
So when I type in "Leo Gestel" to the [1] Api demo interface I get the
info back, but only if I click the option "View on QueryR" do I see the access syntax (2]. I think the user interface should accept Q and P numbers, not labels, though it could provide a lookup gadget.
[1] http://queryr.wmflabs.org/about/demo [2] http://queryr.wmflabs.org/api/items/Q597999
I agree it makes sense to use ids here (as is currently done). At the same time I think it's very important to provide convenient access by Wikipedia page name. That is what the demo search thing does (those are not labels). Such access is crucial for people that want to "get data for this/these Wikipedia page(s)" or "get this specific data for this/these Wikipedia page(s)".
The little demo thing currently resolves the page name via the Wikidata API as such lookup is not yet possible via QueryR. I'm thinking of having redirects to the canonical resource:
- /wikipedia/$enWikiPageName => /items/$itemId
- /wikipedia/$subdomain/$pageName => /items/$itemId
$subdomain being the part before ".wikipedia.org" of the relevant Wikipedia ("en" for the English Wikipedia).
Does that sound reasonable? Is there a better way to do this?
(This is now also tracked by https://github.com/JeroenDeDauw/QueryrAPI/issues/37)
## Sum of All Knowledge
Interesting project, and nice separation of concerns! Too bad I was not aware of this when I started writing QueryR. Though it is now indeed certainly nice to learn by looking at the differences between them.
The point on having direct links to images is well taken - the reason this is not done yet is because I've yet to get to it.
One key difference is that Chantek forwards directly to the Wikidata API, while QueryR has its own persistence. (If you can get away with not having the persistence, then that is definitely the way to go, as it's significantly simpler.) Would you have done anything different in the format you created if you had such persistence?
## Not all values shown?
one value, shouldn't it return multiple ones?
Only the highest ranked values are returned. In case of this set of values, one is preferred and the rest are normal, so only the preferred one is shown. Do you think a different behaviour makes makes more sense?
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hey Conal,
I was interested to take a look at your API but I was surprised to see it
described as REST. It doesn't look like hypertext to me. In a REST API, related resources should be represented using hyperlinks.
There are hyperlinks in various places. For instance at [0], the items are linked, and pagination can be done via link headers in the response. Are these the type of hyperlinks you are looking for? In addition, are there links that you'd expect at other locations and that are not there yet? Note that a lot of stuff might not be there yet simply due to the very preliminary state of the API, for instance [1].
[0] http://queryr.wmflabs.org/api/items [1] https://github.com/JeroenDeDauw/QueryrAPI/issues/13
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Hi Jeroen
Yes I had expected to see links *almost everywhere*, given the nature of the underlying dataset. For instance I looked at the representation of New Zealand, and found a screed of data including the name of the Prime Minister, the name of the Capital City, the name of the Highest Peak, and so on, but with no links to any of those things.
At least the names of those things are meaningful data in themselves, but many of the other names are not at all useful, e.g.
"coat of arms": { "value": "Coat of arms of New Zealand", "type": "string" }
"history of topic": { "value": "history of New Zealand", "type": "string" }
On 2 December 2015 at 10:18, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey Conal,
I was interested to take a look at your API but I was surprised to see
it described as REST. It doesn't look like hypertext to me. In a REST API, related resources should be represented using hyperlinks.
There are hyperlinks in various places. For instance at [0], the items are linked, and pagination can be done via link headers in the response. Are these the type of hyperlinks you are looking for? In addition, are there links that you'd expect at other locations and that are not there yet? Note that a lot of stuff might not be there yet simply due to the very preliminary state of the API, for instance [1].
[0] http://queryr.wmflabs.org/api/items [1] https://github.com/JeroenDeDauw/QueryrAPI/issues/13
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
On Wed, Dec 2, 2015 at 8:27 AM, Conal Tuohy conal.tuohy@gmail.com wrote:
Hi Jeroen
Yes I had expected to see links almost everywhere, given the nature of the underlying dataset. For instance I looked at the representation of New Zealand, and found a screed of data including the name of the Prime Minister, the name of the Capital City, the name of the Highest Peak, and so on, but with no links to any of those things.
At least the names of those things are meaningful data in themselves, but many of the other names are not at all useful, e.g.
"coat of arms": { "value": "Coat of arms of New Zealand", "type": "string" } "history of topic": { "value": "history of New Zealand", "type": "string" }On 2 December 2015 at 10:18, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey Conal,
I was interested to take a look at your API but I was surprised to see it described as REST. It doesn't look like hypertext to me. In a REST API, related resources should be represented using hyperlinks.
There are hyperlinks in various places. For instance at [0], the items are linked, and pagination can be done via link headers in the response. Are these the type of hyperlinks you are looking for? In addition, are there links that you'd expect at other locations and that are not there yet? Note that a lot of stuff might not be there yet simply due to the very preliminary state of the API, for instance [1].
[0] http://queryr.wmflabs.org/api/items [1] https://github.com/JeroenDeDauw/QueryrAPI/issues/13
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-- Conal Tuohy http://conaltuohy.com/ @conal_tuohy +61-466-324297
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I agree it would be cool to use some standard format as an api output!
Am 02.12.2015 um 11:10 schrieb Martynas Jusevičius:
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
On Wed, Dec 2, 2015 at 8:27 AM, Conal Tuohy conal.tuohy@gmail.com wrote:
Hi Jeroen
Yes I had expected to see links almost everywhere, given the nature of the underlying dataset. For instance I looked at the representation of New Zealand, and found a screed of data including the name of the Prime Minister, the name of the Capital City, the name of the Highest Peak, and so on, but with no links to any of those things.
At least the names of those things are meaningful data in themselves, but many of the other names are not at all useful, e.g.
"coat of arms": { "value": "Coat of arms of New Zealand", "type": "string" } "history of topic": { "value": "history of New Zealand", "type": "string" }On 2 December 2015 at 10:18, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey Conal,
I was interested to take a look at your API but I was surprised to see it described as REST. It doesn't look like hypertext to me. In a REST API, related resources should be represented using hyperlinks.
There are hyperlinks in various places. For instance at [0], the items are linked, and pagination can be done via link headers in the response. Are these the type of hyperlinks you are looking for? In addition, are there links that you'd expect at other locations and that are not there yet? Note that a lot of stuff might not be there yet simply due to the very preliminary state of the API, for instance [1].
[0] http://queryr.wmflabs.org/api/items [1] https://github.com/JeroenDeDauw/QueryrAPI/issues/13
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-- Conal Tuohy http://conaltuohy.com/ @conal_tuohy +61-466-324297
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
On 02.12.2015 12:07, Bene* wrote:
I agree it would be cool to use some standard format as an api output!
I am usually all for Web standards and so on, but I have also seen with the Wikidata-to-RDF mapping work that it really is a lot of effort to do such a mapping properly. We already have two output formats (Wikibase custom JSON and RDF [in various formats]), and both of these are in active use and will not go away. I would add support for further standards only when there are clear use cases.
Markus
Am 02.12.2015 um 11:10 schrieb Martynas Jusevičius:
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
On Wed, Dec 2, 2015 at 8:27 AM, Conal Tuohy conal.tuohy@gmail.com
Hey,
Conal, thanks for explaining. The items are indeed not linked. Indeed, the current format hides the fact that they are items altogether. Perhaps this is going a step too far, and it is better to take an approach similar to that of Hay: still have a dedicated wikibase-item data type for which the value includes the id, but also the label. What are your thoughts on using this approach [0]? (Everyone is welcome to comment on this.) As you can see, it makes the format significantly more verbose and is not quite as trivial to use when you don't care about the links or ids. It's still a lot simpler than dealing with the canonical Wikibase item format, and perhaps strikes a better balance than what I created initially.
[0] https://gist.github.com/JeroenDeDauw/fc17f9fdd2e4567a17ff
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
Thanks for the suggestion. I'm not really familiar with JSON-LD and quickly read through the examples there. While this is certainly interesting, I wonder what the actual benefits are, going on the assumption that most developers are unfamiliar with this format. It does add complexity, so I'm quite hesitant to use this in the main format. That said, this might be a good candidate for an additional response format. (And adding additional formats to the API in a clean way ought to be quite easy, see https://github.com/JeroenDeDauw/QueryrAPI/issues/39)
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
JSON-LD does add complexity over plain JSON -- because it also can be interpreted as RDF. And that makes all the difference.
The importance of this distinction cannot be overstated. If one views some custom JSON and JSON-LD (and by extension, RDF) as two alternative formats for doing the same thing, than clearly one fails to grasp what the semantics and web of data are about.
On Wed, Dec 2, 2015 at 8:46 PM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
Conal, thanks for explaining. The items are indeed not linked. Indeed, the current format hides the fact that they are items altogether. Perhaps this is going a step too far, and it is better to take an approach similar to that of Hay: still have a dedicated wikibase-item data type for which the value includes the id, but also the label. What are your thoughts on using this approach [0]? (Everyone is welcome to comment on this.) As you can see, it makes the format significantly more verbose and is not quite as trivial to use when you don't care about the links or ids. It's still a lot simpler than dealing with the canonical Wikibase item format, and perhaps strikes a better balance than what I created initially.
[0] https://gist.github.com/JeroenDeDauw/fc17f9fdd2e4567a17ff
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
Thanks for the suggestion. I'm not really familiar with JSON-LD and quickly read through the examples there. While this is certainly interesting, I wonder what the actual benefits are, going on the assumption that most developers are unfamiliar with this format. It does add complexity, so I'm quite hesitant to use this in the main format. That said, this might be a good candidate for an additional response format. (And adding additional formats to the API in a clean way ought to be quite easy, see https://github.com/JeroenDeDauw/QueryrAPI/issues/39)
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hey Martynas,
The importance of this distinction cannot be overstated. If one views
some custom JSON and JSON-LD (and by extension, RDF) as two alternative formats for doing the same thing, than clearly one fails to grasp what the semantics and web of data are about.
I understand those are not the same thing, and that you can do things with JSON-LD that you cannot do with such a custom format. Keep in mind that the main goal of this API is not "to be part of the web of data". You can already get RDF via the Wikidata API. The goal here is simplicity for the target audience of your average developer, who happens to not really know what the web of data, JSON-LD or RDF are. As I mentioned before, this does not mean this API cannot also cater to a more specific audience by means of providing an additional format. However, the main format will not try to be interpretable as RDF using some reinvention of JSON-LD, as it will not try to be interpretable as RDF at all.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate Developer at Wikimedia Germany ~=[,,_,,]:3
On 02.12.2015 23:17, Martynas Jusevičius wrote:
JSON-LD does add complexity over plain JSON -- because it also can be interpreted as RDF. And that makes all the difference.
The importance of this distinction cannot be overstated. If one views some custom JSON and JSON-LD (and by extension, RDF) as two alternative formats for doing the same thing, than clearly one fails to grasp what the semantics and web of data are about.
Well, we have only one database. So, in our case, all exports are, indeed, "alternative formats for the same thing". :-)
But, more seriously: we already support RDF exports, LOD-style and in dumps. The question for JSON-LD is not "do we appreciate the semantic clarity of RDF" but "should we add another RDF syntax to our exports"? I guess most RDF tools will be happy with NTriples, so there might not be any benefit for them to have JSON-LD in addition. The other question is if the API that Jeroen announced here is actually something that RDF crawlers would like to use (in which case it would help to support at least the RDF formats we already have elsewhere). That's what I meant when referring to use cases.
Markus
On Wed, Dec 2, 2015 at 8:46 PM, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
Conal, thanks for explaining. The items are indeed not linked. Indeed, the current format hides the fact that they are items altogether. Perhaps this is going a step too far, and it is better to take an approach similar to that of Hay: still have a dedicated wikibase-item data type for which the value includes the id, but also the label. What are your thoughts on using this approach [0]? (Everyone is welcome to comment on this.) As you can see, it makes the format significantly more verbose and is not quite as trivial to use when you don't care about the links or ids. It's still a lot simpler than dealing with the canonical Wikibase item format, and perhaps strikes a better balance than what I created initially.
[0] https://gist.github.com/JeroenDeDauw/fc17f9fdd2e4567a17ff
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
Thanks for the suggestion. I'm not really familiar with JSON-LD and quickly read through the examples there. While this is certainly interesting, I wonder what the actual benefits are, going on the assumption that most developers are unfamiliar with this format. It does add complexity, so I'm quite hesitant to use this in the main format. That said, this might be a good candidate for an additional response format. (And adding additional formats to the API in a clean way ought to be quite easy, see https://github.com/JeroenDeDauw/QueryrAPI/issues/39)
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Jeroen I encourage you to look closely at JSON-LD, because this is precisely the use case it was designed for.
Depending on how you choose to encode your data in JSON-LD, it need not offer any additional complexities to a consumer of the data. For instance, you can supply a link to a JSON-LD Context in an HTTP Link Header.
The big advantages of JSON-LD over other JSON-based representations of linked data come from the use of the JSON-LD "Context". By associating the JSON object with a Context, you can map from the strings used as JSON names (such as "sister city") to full URIs such as (I'm guessing) < https://www.wikidata.org/wiki/Property:P123%3E. Data consumers who wish to treat the JSON object simply as JSON can use the strings as keys, but JSON-LD aware consumers can parse the data as RDF if they wish. Even developers who don't use the JSON-LD features at runtime can at least use them to identify the meanings of those strings when they are writing their Javascript code.
Conal
On 3 December 2015 at 05:46, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey,
Conal, thanks for explaining. The items are indeed not linked. Indeed, the current format hides the fact that they are items altogether. Perhaps this is going a step too far, and it is better to take an approach similar to that of Hay: still have a dedicated wikibase-item data type for which the value includes the id, but also the label. What are your thoughts on using this approach [0]? (Everyone is welcome to comment on this.) As you can see, it makes the format significantly more verbose and is not quite as trivial to use when you don't care about the links or ids. It's still a lot simpler than dealing with the canonical Wikibase item format, and perhaps strikes a better balance than what I created initially.
[0] https://gist.github.com/JeroenDeDauw/fc17f9fdd2e4567a17ff
And if the data serialization is JSON, why not use JSON-LD which is all the rage these days? http://www.w3.org/TR/json-ld/
Thanks for the suggestion. I'm not really familiar with JSON-LD and quickly read through the examples there. While this is certainly interesting, I wonder what the actual benefits are, going on the assumption that most developers are unfamiliar with this format. It does add complexity, so I'm quite hesitant to use this in the main format. That said, this might be a good candidate for an additional response format. (And adding additional formats to the API in a clean way ought to be quite easy, see https://github.com/JeroenDeDauw/QueryrAPI/issues/39)
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hey all,
I've made a number of changes based on your feedback that are now live:
* TL;DR: http://queryr.wmflabs.org/api/items/Q64 * The data is now indexed by property id rather than property label * The data sections now contain a property map with properties "label", "id" and "url" * Values of type entity id no longer get turned into strings. Instead they get turned into maps with properties "label", "id" and "url" * Item and property responses now contain links to get parts of their data, ie "data_url": "api/items/Q64/data" * Item responses now contain a link to Wikipedia * Property responses now include the data section * Bonus TL;DR: http://img.ctrlv.in/img/15/12/06/5663c0afad19a.jpg
Conal, Martynas, Bene and other JSON-LD proponents: I'm willing to spend some time on this. However, I'd appreciate if someone who is already familiar with JSON-LD can write the spec for the format. This does not need to be elaborate or formal. An example of a simple item response (including headers) with one or two values of different types will do plenty. Once I have this I'll review it and implement it in the API software.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Hi!
- About this API: http://queryr.wmflabs.org
- Documentation: http://queryr.wmflabs.org/about/docs
- API root: http://queryr.wmflabs.org/api
I like the idea of truly REST-form API for wikidata. Some comments from the first look:
* I'm not sure items and properties should really be separate, and not have common /entity/ endpoint.
* You can do /items/{item_id}/data/{property_label} but not by property ID. In fact, property label seems to be working only in English, and some properties have pretty unwieldy names even in English - e.g. P357 (I know it's obsolete, but it may still have data and people may have to use it)
* http://queryr.wmflabs.org/api/items/Q42/data/occupation returns only one value, shouldn't it return multiple ones?
Hey Jeroen,
An often-cited complaint to me about the Wikidata API is that it's obscure and hard to use. Simple things, like needing the QID of a page rather than being able to give it a URL of a page from a client wiki, can make a big difference to third parties that aren't super familiar with Wikidata. So, this looks really cool! Thanks for working on it. :-)
Dan
On 30 November 2015 at 05:55, Jeroen De Dauw jeroendedauw@gmail.com wrote:
Hey all,
I've created a very rough REST API for Wikidata and am looking for your feedback.
- About this API: http://queryr.wmflabs.org
- Documentation: http://queryr.wmflabs.org/about/docs
- API root: http://queryr.wmflabs.org/api
At present this is purely a demo. The data it serves is stale and potentially incomplete, the endpoints and formats they use are very much liable to change, the server setup is not reliable and I'm not 100% sure I'll continue with this little project.
The main thing I'm going for with this API compared to the existing one is greater ease of use for common use cases. Several factors make this a lot easier to do in a new API than in the existing one: no need the serve all use cases, no need to retain compatibility with existing users and no framework imposed restrictions. You can read more about the difference on the website.
You are invited to comment on the concept and on the open questions mentioned on the website.
Cheers
-- Jeroen De Dauw - http://www.bn2vs.com Software craftsmanship advocate ~=[,,_,,]:3
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
On Tue, Dec 1, 2015 at 5:13 PM, Dan Garry dgarry@wikimedia.org wrote:
An often-cited complaint to me about the Wikidata API is that it's obscure and hard to use. Simple things, like needing the QID of a page rather than being able to give it a URL of a page from a client wiki, can make a big difference to third parties that aren't super familiar with Wikidata.
You can use the URL of a page on a client wiki with action=wbgetentities. Just specify sites and titles parameters, e.g.
https://www.wikidata.org/w/api.php?action=wbgetentities&format=jsonfm&am... or from Special:ApiSandbox, https://www.wikidata.org/wiki/Special:ApiSandbox#action=wbgetentities&fo...
I didn't mention this in https://www.mediawiki.org/wiki/API:Presenting_Wikidata_knowledge because it's my understanding that entity IDs are more universal.
Cheers,
On 2 December 2015 at 13:55, S Page spage@wikimedia.org wrote:
You can use the URL of a page on a client wiki with action=wbgetentities.
That's true, and obvious to us, but often represents a barrier to third party users. From the technical side, there's also the performance implications of having chains of separate, dependent API requests on clients. Little things like this can make a difference. :-)
Thanks, Dan