How are value node IRIs generated?

List overview All Threads
Download

newer

older

Re: [Wikitech-l] Save the date!...

2020-12-09 Scrum of Scrums meeting...

Baskauf, Steven James

9 Dec 2020 9 Dec '20

9:16 a.m.

Datavalues with complex structure (time, quantity, globecoordinate) have properties (e.g. latitude, longitude, precision, amount, unit, etc.) that link from a node that is the value of a statement, reference, or qualifier. In my question, I'm going to refer to those nodes as "value nodes". When performing a SPARQL query at the WD Query Service (example: https://w.wiki/ptp), these value nodes are identified by an IRI such as wdv: 742521f02b14bf1a6cbf7d4bc599eb77 (http://www.wikidata.org/value/742521f02b14bf1a6cbf7d4bc599eb77). The local name part of this IRI seems to be a hash of something. However, when I compare the hash values from the snak JSON returned from the API for the same value node (see https://gist.github.com/baskaufs/8c86bc5ceaae19e31fde88a2880cf0e9 for the example), the hash associated with the value node (35976d7cb070b06a2dec1482aaca2982df3fedd4 in this case) does not have any relationship to the local name part if the IRI for that value node. This situation differs from that of identifiers for references, whose IRIs (wdref:8eb6208639efa82b5e7e4c709b7d18cbfca67411 = http://www.wikidata.org/reference/8eb6208639efa82b5e7e4c709b7d18cbfca67411 in this example) are clearly formed from the hash associated with the reference hash in the snak JSON returned from the API (8eb6208639efa82b5e7e4c709b7d18cbfca67411). I am using the JSON returned by the API after writing new data to track those data at a later time via the Query Service. So I would like to know if there is a way that the value node IRIs can be determined from information in the JSON returned from the API. This is easy to do for reference and statement IRIs, but is not obvious for value nodes. Thanks! Steve Baskauf -- Steven J. Baskauf, Ph.D. Data Science and Data Curation Specialist Jean & Alexander Heard Libraries, Vanderbilt University Nashville, TN 37235, USA Office: Eskind Biomedical Library, EMB 111 Phone: (615) 343-4582 https://my.vanderbilt.edu/baskauf/

Attachments:

attachment.htm (text/html — 5.4 KB)

Show replies by date

petr.kadlec＠gmail.com

9 Dec 9 Dec

9:22 a.m.

On Wed, Dec 9, 2020 at 6:17 PM Baskauf, Steven James < steve.baskauf(a)vanderbilt.edu> wrote:

...

When performing a SPARQL query at the WD Query Service (example: https://w.wiki/ptp), these value nodes are identified by an IRI such as wdv: 742521f02b14bf1a6cbf7d4bc599eb77 ( http://www.wikidata.org/value/742521f02b14bf1a6cbf7d4bc599eb77). The local name part of this IRI seems to be a hash of something. However, when I compare the hash values from the snak JSON returned from the API for the same value node (see https://gist.github.com/baskaufs/8c86bc5ceaae19e31fde88a2880cf0e9 for the example), the hash associated with the value node (35976d7cb070b06a2dec1482aaca2982df3fedd4 in this case) does not have any relationship to the local name part if the IRI for that value node.

https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format#Value_repr… states:

...

Full values are represented as nodes having prefix wdv: and the local

David Causse

10 Dec 10 Dec

1:39 a.m.

Hi, On Wed, Dec 9, 2020 at 6:22 PM <petr.kadlec(a)gmail.com> wrote:

...

On Wed, Dec 9, 2020 at 6:17 PM Baskauf, Steven James < steve.baskauf(a)vanderbilt.edu> wrote:

https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format#Value_repr… states:

Full values are represented as nodes having prefix wdv: and the local

David Causse

2:14 a.m.

On Thu, Dec 10, 2020 at 10:39 AM David Causse <dcausse(a)wikimedia.org> wrote:

...

About 35976d7cb070b06a2dec1482aaca2982df3fedd4 which I think you obtained from the wbgetclaims api[0]? I think this hash identifies the Snak while the one you see from the query service identifies the Value, the former will uniquely identifies the Snak so that for another entity using the same value[1] the Snak hash is different (8eb6208639efa82b5e7e4c709b7d18cbfca67411) but the value is identical (+2019-12-14T00:00:00Z).

Please read a249e4ebdd5022be6cc7fe25feb93e0503eac6ee instead of 8eb6208639efa82b5e7e4c709b7d18cbfca67411 (wrong copy/paste).

Baskauf, Steven James

5:57 a.m.

On Wed, Dec 9, 2020 at 6:22 PM <petr.kadlec@gmail.com<mailto:petr.kadlec@gmail.com>> wrote: On Wed, Dec 9, 2020 at 6:17 PM Baskauf, Steven James <steve.baskauf@vanderbilt.edu<mailto:steve.baskauf@vanderbilt.edu>> wrote: When performing a SPARQL query at the WD Query Service (example: https://w.wiki/ptp<https://nam04.safelinks.protection.outlook.com/?url=h…)mp;reserved=0>), these value nodes are identified by an IRI such as wdv: 742521f02b14bf1a6cbf7d4bc599eb77 (http://www.wikidata.org/value/742521f02b14bf1a6cbf7d4bc599eb77<https://n…)mp;reserved=0>). The local name part of this IRI seems to be a hash of something. However, when I compare the hash values from the snak JSON returned from the API for the same value node (see https://gist.github.com/baskaufs/8c86bc5ceaae19e31fde88a2880cf0e9<https:… for the example), the hash associated with the value node (35976d7cb070b06a2dec1482aaca2982df3fedd4 in this case) does not have any relationship to the local name part if the IRI for that value node. https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format#Value_repr… states:

...

Full values are represented as nodes having prefix wdv: and the local name being the hash of the value contents (e.g. wdv:382603eaa501e15688076291fc47ae54). There is no guarantee of the value of the hash except for the fact that different values will be represented by different hashes, and same value mentioned in different places will have the same hash.

indeed no assumptions should be made on this hash value, the initial goal was (I think) for two unrelated claims that have the same complex value elements to share it instead of reifying one for each claim/reference. I would strongly advise against storing this hash for later use. About 35976d7cb070b06a2dec1482aaca2982df3fedd4 which I think you obtained from the wbgetclaims api[0]? I think this hash identifies the Snak while the one you see from the query service identifies the Value, the former will uniquely identifies the Snak so that for another entity using the same value[1] the Snak hash is different (8eb6208639efa82b5e7e4c709b7d18cbfca67411) but the value is identical (+2019-12-14T00:00:00Z). I don't think you can extract the hash of the value using wbgetclaims but it is visible using the RDF output[2]. 0: https://www.wikidata.org/w/api.php?action=wbgetclaims&entity=Q42352198&… 1: https://www.wikidata.org/w/api.php?action=wbgetclaims&entity=Q232113&am… 2: https://www.wikidata.org/wiki/Special:EntityData/Q42352198.ttl?flavor=dump&… ----- The difference between the snak and the value makes sense to me now. I was obtaining the hash from the JSON that is returned from the API as a response to a write operation. I wanted to use the hash as a way to record that the value had been written to the API. The problem with using the Query Service to get the value node IRI is the delay in the Query Service updater -- the value node IRI would not be immediately available from the Query Service but the response JSON from the API is available immediately after writing. Is the RDF output [2] also dependent on the Query Service Updater, or is it immediately available without the lag? Thanks for taking the time to answer my question. Steve

Lucas Werkmeister

6:12 a.m.

The RDF output is available immediately (it’s where the query service updater gets its data from). Am Do., 10. Dez. 2020 um 14:58 Uhr schrieb Baskauf, Steven James < steve.baskauf(a)vanderbilt.edu>gt;:

...

On Wed, Dec 9, 2020 at 6:22 PM <petr.kadlec(a)gmail.com> wrote: On Wed, Dec 9, 2020 at 6:17 PM Baskauf, Steven James < steve.baskauf(a)vanderbilt.edu> wrote: When performing a SPARQL query at the WD Query Service (example: https://w.wiki/ptp <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw.wiki%2Fptp&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766366522%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=yHJ2oaww4%2BpyEQN9KcjIJR0uR%2B1N1kjH2zPUwu9Ie9k%3D&reserved=0>), these value nodes are identified by an IRI such as wdv: 742521f02b14bf1a6cbf7d4bc599eb77 ( http://www.wikidata.org/value/742521f02b14bf1a6cbf7d4bc599eb77 <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.wikidata.org%2Fvalue%2F742521f02b14bf1a6cbf7d4bc599eb77&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766376521%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=tn2JjhMNCrK8BSMyfnnXjsKEkJgBsFihTlZkFUmiUOM%3D&reserved=0>). The local name part of this IRI seems to be a hash of something. However, when I compare the hash values from the snak JSON returned from the API for the same value node (see https://gist.github.com/baskaufs/8c86bc5ceaae19e31fde88a2880cf0e9 <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Fbaskaufs%2F8c86bc5ceaae19e31fde88a2880cf0e9&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766386508%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=O1rElEdCDv8%2FS6NWpfxKU%2Bqnv8Caj%2FcEhDPgw06K4nk%3D&reserved=0> for the example), the hash associated with the value node (35976d7cb070b06a2dec1482aaca2982df3fedd4 in this case) does not have any relationship to the local name part if the IRI for that value node. https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format#Value_repr… <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.mediawiki.org%2Fwiki%2FWikibase%2FIndexing%2FRDF_Dump_Format%23Value_representation&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766386508%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=CNe570q%2FLtRzI02LA%2FPHj59VgLto7LAcLlvCb7LHsB8%3D&reserved=0> states:

Full values are represented as nodes having prefix wdv: and the local

name being the hash of the value contents (e.g. wdv:382603eaa501e15688076291fc47ae54). There is no guarantee of the value of the hash except for the fact that different values will be represented by different hashes, and same value mentioned in different places will have the same hash. indeed no assumptions should be made on this hash value, the initial goal was (I think) for two unrelated claims that have the same complex value elements to share it instead of reifying one for each claim/reference. I would strongly advise against storing this hash for later use. About 35976d7cb070b06a2dec1482aaca2982df3fedd4 which I think you obtained from the wbgetclaims api[0]? I think this hash identifies the Snak while the one you see from the query service identifies the Value, the former will uniquely identifies the Snak so that for another entity using the same value[1] the Snak hash is different (8eb6208639efa82b5e7e4c709b7d18cbfca67411) but the value is identical (+2019-12-14T00:00:00Z). I don't think you can extract the hash of the value using wbgetclaims but it is visible using the RDF output[2]. 0: https://www.wikidata.org/w/api.php?action=wbgetclaims&entity=Q42352198&… <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.wikidata.org%2Fw%2Fapi.php%3Faction%3Dwbgetclaims%26entity%3DQ42352198%26property%3DP496%26formatversion%3D2&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766396506%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=O%2BQ%2BETPVG6hkxy%2FiLegpp5BHYtaDDXnevmPK3ZTm200%3D&reserved=0> 1: https://www.wikidata.org/w/api.php?action=wbgetclaims&entity=Q232113&am… <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.wikidata.org%2Fw%2Fapi.php%3Faction%3Dwbgetclaims%26entity%3DQ232113%26property%3DP570%26formatversion%3D2&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766406507%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=m1N0TUImMZKjSYtuN7K%2B2kRtw6iVmR4tMPcyDE6SCHE%3D&reserved=0> 2: https://www.wikidata.org/wiki/Special:EntityData/Q42352198.ttl?flavor=dump <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.wikidata.org%2Fwiki%2FSpecial%3AEntityData%2FQ42352198.ttl%3Fflavor%3Ddump&data=04%7C01%7Csteve.baskauf%40vanderbilt.edu%7C797bb01c807541e0427e08d89cef8134%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C637431899766406507%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=m83NmCpSBZWff6dIyccNzoihvNf2eVW2yqYAPYHMPUQ%3D&reserved=0> ----- The difference between the snak and the value makes sense to me now. I was obtaining the hash from the JSON that is returned from the API as a response to a write operation. I wanted to use the hash as a way to record that the value had been written to the API. The problem with using the Query Service to get the value node IRI is the delay in the Query Service updater -- the value node IRI would not be immediately available from the Query Service but the response JSON from the API is available immediately after writing. Is the RDF output [2] also dependent on the Query Service Updater, or is it immediately available without the lag? Thanks for taking the time to answer my question. Steve _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

-- Lucas Werkmeister (he/er) Full Stack Developer Wikimedia Deutschland e. V. | Tempelhofer Ufer 23-24 | 10963 Berlin Phone: +49 (0)30 219 158 26-0 https://wikimedia.de Imagine a world in which every single human being can freely share in the sum of all knowledge. Help us to achieve our vision! https://spenden.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 B. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

1230

days inactive

1231

days old

wikitech-l@lists.wikimedia.org

Manage subscription

5 comments

4 participants

tags (0)

participants (4)

Baskauf, Steven James
David Causse
Lucas Werkmeister
petr.kadlec＠gmail.com