I find the order of claims on web site useful, but when requesting entity data through API (action=wbgetentities) it got lost. How the claims are ordered on an entity web page and how to restore the order in API response?
-- Vlad
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Без вирусов. www.avast.ru https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
The order of properties as presented in the web interface is specified in https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties . You can use the same list to order the data you get from any API response accordingly.
Note that, technically, the order of elements in a JSON object (curly brackets) is unspecified. This is what the JSON specification dictates. This becomes relevant when, for example, you are using a Java Set. You can iterate a Set, but the order will be pretty much randomized.
The order you can see in the wikidata.org API responses is basically the order in which the statements are stored in the database. This order mostly reflects the order in which the statements have been created. But this is not guaranteed and can change any time.
Best Thiemo
Thanks for the link with sorted properties. Is this page updated automatically or maintained manually by someone? In latter case this looks weird to me, because the order may become not actual at some moment.
It is curious that when properties are used as qualifiers we have a separate field specifying the order (called ‘qualifiers-order’). Why not to add the same at the top-level of entity definition?
Vlad
29 нояб. 2017 г., в 12:06, Thiemo Kreuz thiemo.kreuz@wikimedia.de написал(а):
The order of properties as presented in the web interface is specified in https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties . You can use the same list to order the data you get from any API response accordingly.
Note that, technically, the order of elements in a JSON object (curly brackets) is unspecified. This is what the JSON specification dictates. This becomes relevant when, for example, you are using a Java Set. You can iterate a Set, but the order will be pretty much randomized.
The order you can see in the wikidata.org API responses is basically the order in which the statements are stored in the database. This order mostly reflects the order in which the statements have been created. But this is not guaranteed and can change any time.
Best Thiemo _______________________________________________ Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
On Wed, Nov 29, 2017 at 11:14 AM, Владимир Рябцев greatvovan@gmail.com wrote:
Thanks for the link with sorted properties. Is this page updated automatically or maintained manually by someone? In latter case this looks weird to me, because the order may become not actual at some moment.
Yes it is maintained by hand by the editors.
It is curious that when properties are used as qualifiers we have a separate field specifying the order (called ‘qualifiers-order’). Why not to add the same at the top-level of entity definition?
It is just a heading to make the page more manageable - it doesn't have a meaning beyond that.
Cheers Lydia
OK Lydia, what is the purpose of giving order of qualifiers then?
Along with helping to give a user a better representation of data, the order can be useful in automated processing of properties. To my mind, it starts with the most important entity data. Moreover, in case of contradiction, I would assume that first properties are “ranked” higher. After all we are humans and pay more attention to the top of page. Our mind may get bit tired by the end of page. In an ideal world you are right that order does not matter, but in the reality it may help algorithms.
Vlad
29 нояб. 2017 г., в 14:19, Lydia Pintscher lydia.pintscher@wikimedia.de написал(а):
On Wed, Nov 29, 2017 at 11:14 AM, Владимир Рябцев greatvovan@gmail.com wrote: Thanks for the link with sorted properties. Is this page updated automatically or maintained manually by someone? In latter case this looks weird to me, because the order may become not actual at some moment.
Yes it is maintained by hand by the editors.
It is curious that when properties are used as qualifiers we have a separate field specifying the order (called ‘qualifiers-order’). Why not to add the same at the top-level of entity definition?
It is just a heading to make the page more manageable - it doesn't have a meaning beyond that.
Cheers Lydia
-- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
what is the purpose of giving order of qualifiers then?
The JSON still contains fields called "qualifiers-order" on each qualifier, as well as "snaks-order" on each reference. These are fragments from a feature the Wikibase software once supported. In 2013 the interface had "move up" and "move down" buttons that allowed to manually change the order of certain elements. This feature was not intuitive, bogus, actually broken, and removed just a few months later.
Much later it was replaced by an order that is consistent across all entities: https://phabricator.wikimedia.org/T99243 .
The "qualifiers-order" and "snaks-order" fields are unused and entirely meaningless now. They should not be used for anything.
The specific order you need for your application mostly depends on your application. If you want "preferred" statements to show up first, then go ahead and order them accordingly.
A concept of "importance" does not exist in the Wikibase software. Applying ranking is the the job of a (secondary) search engine, like Google does, or our very own CirrusSearch index. The only thing that can be considered a "ranking" is what you see on https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties , which you can reuse if you want.
Best Thiemo
Dear Vlad,
Ordering claims on a page as you suggest would not work well, since several other orders must take precedence over the order you suggest. First of all, statements are grouped by property and you don't want to change this. Hence, you cannot use the order across statements of different properties, since this would force you in some cases to ungroup (which would have other disadvantages).
Second, it makes sense to order statements of one property by other aspects, e.g., by time, to make it possible for humans to find something. Hence, again, we are not free to use the order to encode further information.
So what remains is to order quantifiers inside statements, but there it is rarely relevant (usually there are only a few qualifiers and all of them can be seen at once, without getting tired).
In summary, order does not lend itself as a way to encode much additional information, since there are usability concerns that make you want to change order in different contexts (or maybe for different users), since order cannot be preserved when remixing data, and since it is overall too implicit for people to build up a shared understanding of what it is supposed to mean (you don't want fights about whether some item has to be in fourth or fifth position of some list based on some vague understanding of "quality" or "trustworthiness" -- it would be very hard to find objective arguments for or against a particular order).
Cheers,
Markus
On 29.11.2017 12:45, Владимир Рябцев wrote:
OK Lydia, what is the purpose of giving order of qualifiers then?
Along with helping to give a user a better representation of data, the order can be useful in automated processing of properties. To my mind, it starts with the most important entity data. Moreover, in case of contradiction, I would assume that first properties are “ranked” higher. After all we are humans and pay more attention to the top of page. Our mind may get bit tired by the end of page. In an ideal world you are right that order does not matter, but in the reality it may help algorithms.
Vlad
29 нояб. 2017 г., в 14:19, Lydia Pintscher lydia.pintscher@wikimedia.de написал(а):
On Wed, Nov 29, 2017 at 11:14 AM, Владимир Рябцев greatvovan@gmail.com wrote: Thanks for the link with sorted properties. Is this page updated automatically or maintained manually by someone? In latter case this looks weird to me, because the order may become not actual at some moment.
Yes it is maintained by hand by the editors.
It is curious that when properties are used as qualifiers we have a separate field specifying the order (called ‘qualifiers-order’). Why not to add the same at the top-level of entity definition?
It is just a heading to make the page more manageable - it doesn't have a meaning beyond that.
Cheers Lydia
-- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
Thiemo, thanks, I understand that I can use the order from that page as well as invent my own order. My point is that it would be nice to have a way to represent data same way as Wikidata site does. Since I see the same claims layout every time I refresh a page, I assume this order is fixed and stored somewhere in the system (not on a separate page).
Markus, yes I want to keep properties grouped, values within a group ordered and so on and so on... everything like we have on a web page. Whoever wants, can sort the data as he wish, be any existing property, but the ability to sort "as in the UI" is also desired.
My suggestions: • Implement API action (e.g. wbgetproperties) returning all Wikidata properties with the current UI order. • Add an integer value per claim and per value (and per qualified value?) indicating the current order of them in UI. No fights for the order. Who wants, will use it or will sort by other features.
Is it the right place to post such suggestions? If not please tell me the proper place.
Thanks, Vlad
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Без вирусов. www.avast.ru https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
2017-11-29 15:35 GMT+03:00 Markus Krötzsch markus@semantic-mediawiki.org:
Dear Vlad,
Ordering claims on a page as you suggest would not work well, since several other orders must take precedence over the order you suggest. First of all, statements are grouped by property and you don't want to change this. Hence, you cannot use the order across statements of different properties, since this would force you in some cases to ungroup (which would have other disadvantages).
Second, it makes sense to order statements of one property by other aspects, e.g., by time, to make it possible for humans to find something. Hence, again, we are not free to use the order to encode further information.
So what remains is to order quantifiers inside statements, but there it is rarely relevant (usually there are only a few qualifiers and all of them can be seen at once, without getting tired).
In summary, order does not lend itself as a way to encode much additional information, since there are usability concerns that make you want to change order in different contexts (or maybe for different users), since order cannot be preserved when remixing data, and since it is overall too implicit for people to build up a shared understanding of what it is supposed to mean (you don't want fights about whether some item has to be in fourth or fifth position of some list based on some vague understanding of "quality" or "trustworthiness" -- it would be very hard to find objective arguments for or against a particular order).
Cheers,
Markus
On 29.11.2017 12:45, Владимир Рябцев wrote:
OK Lydia, what is the purpose of giving order of qualifiers then?
Along with helping to give a user a better representation of data, the order can be useful in automated processing of properties. To my mind, it starts with the most important entity data. Moreover, in case of contradiction, I would assume that first properties are “ranked” higher. After all we are humans and pay more attention to the top of page. Our mind may get bit tired by the end of page. In an ideal world you are right that order does not matter, but in the reality it may help algorithms.
Vlad
29 нояб. 2017 г., в 14:19, Lydia Pintscher lydia.pintscher@wikimedia.de
написал(а):
On Wed, Nov 29, 2017 at 11:14 AM, Владимир Рябцев greatvovan@gmail.com
wrote: Thanks for the link with sorted properties. Is this page updated automatically or maintained manually by someone? In latter case this looks weird to me, because the order may become not actual at some moment.
Yes it is maintained by hand by the editors.
It is curious that when properties are used as qualifiers we have a
separate field specifying the order (called ‘qualifiers-order’). Why not to add the same at the top-level of entity definition?
It is just a heading to make the page more manageable - it doesn't have a meaning beyond that.
Cheers Lydia
-- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
As a follow-up, I analyzed the properties mentioned in https://www.wikidata .org/wiki/MediaWiki:Wikibase-SortedProperties and it turns out it lacks PLENTY of properties we usually work with. From 1400+ properties we normally use, there are only 480 on this page :(
Vlad
2017-11-29 16:13 GMT+03:00 Vladimir Ryabtsev greatvovan@gmail.com:
Thiemo, thanks, I understand that I can use the order from that page as well as invent my own order. My point is that it would be nice to have a way to represent data same way as Wikidata site does. Since I see the same claims layout every time I refresh a page, I assume this order is fixed and stored somewhere in the system (not on a separate page).
Markus, yes I want to keep properties grouped, values within a group ordered and so on and so on... everything like we have on a web page. Whoever wants, can sort the data as he wish, be any existing property, but the ability to sort "as in the UI" is also desired.
My suggestions: • Implement API action (e.g. wbgetproperties) returning all Wikidata properties with the current UI order. • Add an integer value per claim and per value (and per qualified value?) indicating the current order of them in UI. No fights for the order. Who wants, will use it or will sort by other features.
Is it the right place to post such suggestions? If not please tell me the proper place.
Thanks, Vlad
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Без вирусов. www.avast.ru https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#m_4939092080553920555_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
2017-11-29 15:35 GMT+03:00 Markus Krötzsch markus@semantic-mediawiki.org :
Dear Vlad,
Ordering claims on a page as you suggest would not work well, since several other orders must take precedence over the order you suggest. First of all, statements are grouped by property and you don't want to change this. Hence, you cannot use the order across statements of different properties, since this would force you in some cases to ungroup (which would have other disadvantages).
Second, it makes sense to order statements of one property by other aspects, e.g., by time, to make it possible for humans to find something. Hence, again, we are not free to use the order to encode further information.
So what remains is to order quantifiers inside statements, but there it is rarely relevant (usually there are only a few qualifiers and all of them can be seen at once, without getting tired).
In summary, order does not lend itself as a way to encode much additional information, since there are usability concerns that make you want to change order in different contexts (or maybe for different users), since order cannot be preserved when remixing data, and since it is overall too implicit for people to build up a shared understanding of what it is supposed to mean (you don't want fights about whether some item has to be in fourth or fifth position of some list based on some vague understanding of "quality" or "trustworthiness" -- it would be very hard to find objective arguments for or against a particular order).
Cheers,
Markus
On 29.11.2017 12:45, Владимир Рябцев wrote:
OK Lydia, what is the purpose of giving order of qualifiers then?
Along with helping to give a user a better representation of data, the order can be useful in automated processing of properties. To my mind, it starts with the most important entity data. Moreover, in case of contradiction, I would assume that first properties are “ranked” higher. After all we are humans and pay more attention to the top of page. Our mind may get bit tired by the end of page. In an ideal world you are right that order does not matter, but in the reality it may help algorithms.
Vlad
29 нояб. 2017 г., в 14:19, Lydia Pintscher lydia.pintscher@wikimedia.de
написал(а):
On Wed, Nov 29, 2017 at 11:14 AM, Владимир Рябцев greatvovan@gmail.com
wrote: Thanks for the link with sorted properties. Is this page updated automatically or maintained manually by someone? In latter case this looks weird to me, because the order may become not actual at some moment.
Yes it is maintained by hand by the editors.
It is curious that when properties are used as qualifiers we have a
separate field specifying the order (called ‘qualifiers-order’). Why not to add the same at the top-level of entity definition?
It is just a heading to make the page more manageable - it doesn't have a meaning beyond that.
Cheers Lydia
-- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
[…] it turns out it lacks PLENTY of properties we usually work with. From
1400+ properties we normally use, there are only 480 on this page
This is intended. The list was originally created with the most common properties, and can and should be expanded any time when the need to do so arises. Unlisted properties will be moved to the end, in their original order (as stored in the database). If you find specific properties that should move up to one of the groups currently specified in https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties , go ahead and suggest changes on the talk page.
As for the suggestions in your other mail: Even if I can understand that a more "nice" JSON would be – well – more "nice", I don't see what the specific benefit of that would be. As long as no specific use case arises I don't see a reason to invest resources in changing the current behavior.
Best Thiemo
The talk is not about JSON, I never mentioned it. JSON is a serialization format and it does not have to be "nice". The talk about the ability to do something.
As for use case, I have already described it: representing data in the same (familiar to user) layout as on Wikidata page. If you consider this use case unimportant or uncommon — well, OK then. At least I tried.
Best regard, Vlad
2017-11-29 21:28 GMT+03:00 Thiemo Kreuz thiemo.kreuz@wikimedia.de:
[…] it turns out it lacks PLENTY of properties we usually work with.
From 1400+ properties we normally use, there are only 480 on this page
This is intended. The list was originally created with the most common properties, and can and should be expanded any time when the need to do so arises. Unlisted properties will be moved to the end, in their original order (as stored in the database). If you find specific properties that should move up to one of the groups currently specified in https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties , go ahead and suggest changes on the talk page.
As for the suggestions in your other mail: Even if I can understand that a more "nice" JSON would be – well – more "nice", I don't see what the specific benefit of that would be. As long as no specific use case arises I don't see a reason to invest resources in changing the current behavior.
Best Thiemo
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
Hey Vladimir,
On Wed, Nov 29, 2017 at 10:54 PM, Vladimir Ryabtsev greatvovan@gmail.com wrote:
The talk is not about JSON, I never mentioned it. JSON is a serialization format and it does not have to be "nice". The talk about the ability to do something.
As for use case, I have already described it: representing data in the same (familiar to user) layout as on Wikidata page. If you consider this use case unimportant or uncommon — well, OK then. At least I tried.
I'd like to understand a bit more about what you are doing/trying to do. Would you mind sharing a bit more about what you are working on either here or with me off-list? Then I can better tell you if there are good ways to get what you want, if we can add some or not.
Cheers Lydia
Hey Lydia,
We have NLP system that processes news articles, and a part of the system is named entity linking module which works with both our own entities and imported from Wikidata. Our internal users work with entities data in our system for purposes of editing, modifying and merging of entities data. At some moments entity data is displayed on user's screen and it would be convenient for him/her to have the same data layout as on Wikidata page for cross-checking or editing procedures that could not be automated. To display it like this, we need information about how Wikidata site works to sort and group properties. This thing would improve productivity of users, especially when working on large entities with plenty of properties.
Best regards, Vlad
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Без вирусов. www.avast.ru https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
2017-11-30 17:36 GMT+03:00 Lydia Pintscher lydia.pintscher@wikimedia.de:
Hey Vladimir,
On Wed, Nov 29, 2017 at 10:54 PM, Vladimir Ryabtsev greatvovan@gmail.com wrote:
The talk is not about JSON, I never mentioned it. JSON is a serialization format and it does not have to be "nice". The talk about the ability to
do
something.
As for use case, I have already described it: representing data in the
same
(familiar to user) layout as on Wikidata page. If you consider this use
case
unimportant or uncommon — well, OK then. At least I tried.
I'd like to understand a bit more about what you are doing/trying to do. Would you mind sharing a bit more about what you are working on either here or with me off-list? Then I can better tell you if there are good ways to get what you want, if we can add some or not.
Cheers Lydia
-- Lydia Pintscher - http://about.me/lydia.pintscher Product Manager for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
As far as I can tell, this is not possible via the API directly. However, you can get the order of properties from MediaWiki:Wikibase-SortedProperties https://www.wikidata.org/wiki/MediaWiki:Wikibase-SortedProperties, and sort the response you get accordingly. If you’re using Lua, there’s also the mw.wikibase.getPropertyOrder() https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua#mw.wikibase.getPropertyOrder and mw.wikibase.orderProperties( propertyIds https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua#mw.wikibase.orderProperties ) https://www.mediawiki.org/wiki/Extension:Wikibase_Client/Lua#mw.wikibase.orderProperties functions.
2017-11-28 19:22 GMT+02:00 Vladimir Ryabtsev greatvovan@gmail.com:
I find the order of claims on web site useful, but when requesting entity data through API (action=wbgetentities) it got lost. How the claims are ordered on an entity web page and how to restore the order in API response?
-- Vlad
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Без вирусов. www.avast.ru https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#m_-3502740646293738920_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Wikidata-tech mailing list Wikidata-tech@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech
wikidata-tech@lists.wikimedia.org