Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019 - introduction to ShEx http://shex.io/shex-primer/ - more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it. - A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. - The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2. - The external tool shex-simple https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page - If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag shape-expressions - Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model? - You can also get more information about how to create a Schema https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board https://phabricator.wikimedia.org/tag/shape_expressions/ - Technical documentation of the extension https://meta.wikimedia.org/wiki/Extension:EntitySchema - To enhance the interface, you can use this user script https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
I definitely think this is a big step forward that will reduce maintenance effort, will boost consensus processes on how to represent knowledge on Wikidata and will ensure that the data is consistently represented so that it can be processed unambiguously and automagically to an extent previously unthinkable. And I can't wait to start creating Schemas. :-)
Congratulations to all who are making this possible!
On 5/19/19 15:32, Léa Lacroix wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata.
*First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx.
*What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org http://test.wikidata.org on May 21st and on wikidata.org http://wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features.
*How can you help?*
- Before the release, you can try to edit or create Shape Expressions on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag |shape-expressions|
- Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
In other words: the benefits of standardized schemata but with less gatekeeping and exclusivity. Cheers!
On Sun, May 19, 2019 at 3:32 PM Léa Lacroix lea.lacroix@wikimedia.de wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx
https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema
https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape
Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases
(quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple
https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions
on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to
create a new task on Phabricator with the tag shape-expressions
- Once Schemas are enabled, you can discuss about it on your favorite
wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board
https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension
https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 https://maps.google.com/?q=Tempelhofer+Ufer+23-24+10963+Berlin&entry=gmail&source=g 10963 Berlin https://maps.google.com/?q=Tempelhofer+Ufer+23-24+10963+Berlin&entry=gmail&source=g www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
This is fantastic news, as always. :) I've got a question: can/will those entities also be recalled from Lua templates to help build infoboxes? (Sorry for being cryptic, I'm busy and cannot further explain at the moment, but I cannot help my imagination run wild!) :)))
L.
Il dom 19 mag 2019, 15:32 Léa Lacroix lea.lacroix@wikimedia.de ha scritto:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx
https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema
https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape
Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases
(quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple
https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions
on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to
create a new task on Phabricator with the tag shape-expressions
- Once Schemas are enabled, you can discuss about it on your favorite
wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board
https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension
https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hey Luca, For now, we didn't build any specific Lua support for shape expressions, but that's something we could consider if there is the need for it. i'd be glad to discuss about your ideas and needs when you have a bit more time :)
On Mon, 20 May 2019 at 10:53, Luca Martinelli martinelliluca@gmail.com wrote:
This is fantastic news, as always. :) I've got a question: can/will those entities also be recalled from Lua templates to help build infoboxes? (Sorry for being cryptic, I'm busy and cannot further explain at the moment, but I cannot help my imagination run wild!) :)))
L.
Il dom 19 mag 2019, 15:32 Léa Lacroix lea.lacroix@wikimedia.de ha scritto:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx
https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema
https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape
Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases
(quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple
https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions
on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to
create a new task on Phabricator with the tag shape-expressions
- Once Schemas are enabled, you can discuss about it on your favorite
wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board
https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension
https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Il giorno mar 21 mag 2019 alle ore 15:07 Léa Lacroix lea.lacroix@wikimedia.de ha scritto:
For now, we didn't build any specific Lua support for shape expressions, but that's something we could consider if there is the need for it. i'd be glad to discuss about your ideas and needs when you have a bit more time :)
Ok, just give me the time to wrap my head around these things, and to discuss it with somebody else. No rush. ;)
L.
Wonderful! Awesome! Thank you very much, Léa, to announce us this arrival. And, of course, thank you to everyone who is working in this project.
I think ShEx is going to help to make the data of Wikidata more consistent and trustworthy. I am wishing to learn how to create and use this ShEx schemas!
Regards, Iván
On 19/5/19 14:32, Léa Lacroix wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata.
*First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx.
*What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org http://test.wikidata.org on May 21st and on wikidata.org http://wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features.
*How can you help?*
- Before the release, you can try to edit or create Shape Expressions on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag |shape-expressions|
- Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hello all,
As previously announced, we just released shape expressions on Wikidata. You can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new EntitySchema https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx - introduction to ShEx http://shex.io/shex-primer/ - more details about the language http://shex.io/shex-semantics/ - More information about how to create a Schema https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F - Phabricator tag: shape-expressions https://phabricator.wikimedia.org/tag/shape_expressions/ - User script https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any question or encounter issues, feel free to ping me. Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix lea.lacroix@wikimedia.de wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx
https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema
https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape
Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases
(quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple
https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions
on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to
create a new task on Phabricator with the tag shape-expressions
- Once Schemas are enabled, you can discuss about it on your favorite
wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board
https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension
https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Hi Léa,
Thanks to all the team for this.
I've proposed a property,
https://www.wikidata.org/wiki/Wikidata:Property_proposal/Shape_Expression_fo...
To make this work, is it possible to have a Shape Expression as the value of a statement on Wikidata (and the RDF dump, and WDQS) ?
Is there a timescale in which this should become possible ?
Thanks,
James.
On 28/05/2019 17:04, Léa Lacroix wrote:
Hello all,
As previously announced, we just released shape expressions on Wikidata. You can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new EntitySchema https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx> - introduction to ShEx <http://shex.io/shex-primer/> - more details about the language <http://shex.io/shex-semantics/> - More information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F> - Phabricator tag: shape-expressions <https://phabricator.wikimedia.org/tag/shape_expressions/> - User script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to highlight items and properties in the schema code and turn the IDs into linksIf you have any question or encounter issues, feel free to ping me. Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix lea.lacroix@wikimedia.de wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg> made by community members during the Wikimedia hackathon 2019 - introduction to ShEx <http://shex.io/shex-primer/> - more details about the language <http://shex.io/shex-semantics/>*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema <https://www.mediawiki.org/wiki/Extension:EntitySchema>, defining the Schema namespace and its behavior as well as special pages related to it. - A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. - The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) <http://shex.io/shex-semantics/#shexc>. You can see an example here <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. - The external tool shex-simple <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2> is directly linked from the Schema pages in order to check entities of your choice against the schema.*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page> - If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag shape-expressions - Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model? - You can also get more information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F>*See also: *
- Main Phabricator board <https://phabricator.wikimedia.org/tag/shape_expressions/> - Technical documentation of the extension <https://meta.wikimedia.org/wiki/Extension:EntitySchema> - To enhance the interface, you can use this user script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to highlight items and properties in the schema code and turn the IDs into linksIf you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Thanks for your feedback! There is already a ticket about adding a new data type allowing to link EntitySchemas from statements: https://phabricator.wikimedia.org/T214884 If we don't encounter any major technical issue, this could be done in the incoming weeks.
On Tue, 28 May 2019 at 19:18, James Heald jpm.heald@gmail.com wrote:
Hi Léa,
Thanks to all the team for this.
I've proposed a property,
https://www.wikidata.org/wiki/Wikidata:Property_proposal/Shape_Expression_fo...
To make this work, is it possible to have a Shape Expression as the value of a statement on Wikidata (and the RDF dump, and WDQS) ?
Is there a timescale in which this should become possible ?
Thanks,
James.On 28/05/2019 17:04, Léa Lacroix wrote:
Hello all,
As previously announced, we just released shape expressions on Wikidata. You can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new EntitySchema https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx> - introduction to ShEx <http://shex.io/shex-primer/> - more details about the language <http://shex.io/shex-semantics/> - More information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
- Phabricator tag: shape-expressions <https://phabricator.wikimedia.org/tag/shape_expressions/> - User script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E
to highlight items and properties in the schema code and turn theIDs into
linksIf you have any question or encounter issues, feel free to ping me.
Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix lea.lacroix@wikimedia.de
wrote:
Hello all,
After several months of development and testing together with the
WikiProject
ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a
concise,
formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the
case
of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg> made by community members during the Wikimedia hackathon 2019 - introduction to ShEx <http://shex.io/shex-primer/> - more details about the language <http://shex.io/shex-semantics/>*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe
what
the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what
is
expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible
errors
or lack of information. Subsets of the Wikidata graph can be tested to
see
whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially
useful
for wiki projects to more easily discuss and ensure the modeling of
items
in their domain. In the spirit of Wikidata not restricting the world,
Shape
Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future,
for
example building a tool that would suggest, when creating a new item,
what
would be the basic structure for this item, and helping adding
statements
or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema <https://www.mediawiki.org/wiki/Extension:EntitySchema>, definingthe
Schema namespace and its behavior as well as special pages relatedto it.
- A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. - The Schemas will have multilingual labels, descriptions andaliases
(quite similar to the termbox on Items), and the schema text onecan fill
with a syntax called ShEx Compact Syntax (ShExC) <http://shex.io/shex-semantics/#shexc>. You can see an example here <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. - The external tool shex-simple <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex...
is directly linked from the Schema pages in order to check entitiesof your
choice against the schema.*When is this happening?*
Schemas will be enabled on on test.wikidata.org on May 21st and on wikidata.org on May 28th. After this release, they will be integrated
to
the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create ShapeExpressions
on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page%3E
- If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag shape-expressions - Once Schemas are enabled, you can discuss about it on yourfavorite
wikiprojects: for example, what types of items would you like tomodel?
- You can also get more information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
*See also: *
- Main Phabricator board <https://phabricator.wikimedia.org/tag/shape_expressions/> - Technical documentation of the extension <https://meta.wikimedia.org/wiki/Extension:EntitySchema> - To enhance the interface, you can use this user script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E
to highlight items and properties in the schema code and turn theIDs into
linksIf you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das
Finanzamt
für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Very interesting as Wikidata starts beeing part of more external data “flows” I would also like to see that we easy can tell * this is the schema we use inside Wikidata * this is external related schemas other organisations has created for this type of data * how this WD schema relate to an external schema. What parts we map etc.
In Sweden the goverment speaks about basic data (swe. Grunddata) that they will define that I hope the data in Wikidata can “plug-in” and add value to.
Regards Magnus Sälgö 0046-705937579 Salgo60@msn.commailto:Salgo60@msn.com
29 maj 2019 kl. 09:44 skrev Léa Lacroix <lea.lacroix@wikimedia.demailto:lea.lacroix@wikimedia.de>:
Thanks for your feedback! There is already a ticket about adding a new data type allowing to link EntitySchemas from statements: https://phabricator.wikimedia.org/T214884 If we don't encounter any major technical issue, this could be done in the incoming weeks.
On Tue, 28 May 2019 at 19:18, James Heald <jpm.heald@gmail.commailto:jpm.heald@gmail.com> wrote: Hi Léa,
Thanks to all the team for this.
I've proposed a property,
https://www.wikidata.org/wiki/Wikidata:Property_proposal/Shape_Expression_fo...
To make this work, is it possible to have a Shape Expression as the value of a statement on Wikidata (and the RDF dump, and WDQS) ?
Is there a timescale in which this should become possible ?
Thanks,
James.
On 28/05/2019 17:04, Léa Lacroix wrote:
Hello all,
As previously announced, we just released shape expressions on Wikidata. You can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new EntitySchema https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx> - introduction to ShEx <http://shex.io/shex-primer/> - more details about the language <http://shex.io/shex-semantics/> - More information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F> - Phabricator tag: shape-expressions <https://phabricator.wikimedia.org/tag/shape_expressions/> - User script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to highlight items and properties in the schema code and turn the IDs into linksIf you have any question or encounter issues, feel free to ping me. Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix <lea.lacroix@wikimedia.demailto:lea.lacroix@wikimedia.de> wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg> made by community members during the Wikimedia hackathon 2019 - introduction to ShEx <http://shex.io/shex-primer/> - more details about the language <http://shex.io/shex-semantics/>*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx. *What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema <https://www.mediawiki.org/wiki/Extension:EntitySchema>, defining the Schema namespace and its behavior as well as special pages related to it. - A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. - The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) <http://shex.io/shex-semantics/#shexc>. You can see an example here <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. - The external tool shex-simple <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2> is directly linked from the Schema pages in order to check entities of your choice against the schema.*When is this happening?*
Schemas will be enabled on on test.wikidata.orghttp://test.wikidata.org on May 21st and on wikidata.orghttp://wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?*
- Before the release, you can try to edit or create Shape Expressions on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page> - If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag shape-expressions - Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model? - You can also get more information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F>*See also: *
- Main Phabricator board <https://phabricator.wikimedia.org/tag/shape_expressions/> - Technical documentation of the extension <https://meta.wikimedia.org/wiki/Extension:EntitySchema> - To enhance the interface, you can use this user script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to highlight items and properties in the schema code and turn the IDs into linksIf you have any questions, feel free to reach me. Cheers,
Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.dehttp://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.orgmailto:Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.dehttp://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.orgmailto:Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I sure hope that E10 is *not* the shape for human. It certainly isn't a correct shape for humans that belong to subclasses of human (such as Old Croghan Man (Q166790) or Delina Filkins (Q1408186)). E10 is also currently silent on what information should be present for humans, which I take it to be the point of having ShEx in Wikidata.
It is also unclear what is means to be the shape for human. The shape E10- does not have any information on which items are to be considered against the shape. Are all items in Wikidata to be considered (as in the definition of ShEx)? That doesn't seem right. Are all direct instances of human? That seems to limiting. Are all indirect instances of human? This seems the most natural, but where is this behaviour given?
Peter F. Patel-Schneider Samsung Research America
On 5/28/19 12:04 PM, Léa Lacroix wrote:
Hello all,
As previously announced, we just released shape expressions on Wikidata. You can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new EntitySchema https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
- More information about how to create a Schema https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
- Phabricator tag: shape-expressions https://phabricator.wikimedia.org/tag/shape_expressions/
- User script https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any question or encounter issues, feel free to ping me. Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix <lea.lacroix@wikimedia.de mailto:lea.lacroix@wikimedia.de> wrote:
Hello all, After several months of development and testing together with the WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx>, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?* ShEx (Q29377880) <https://www.wikidata.org/wiki/Q29377880> is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled. See also: * a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg> made by community members during the Wikimedia hackathon 2019 * introduction to ShEx <http://shex.io/shex-primer/> * more details about the language <http://shex.io/shex-semantics/> *What can it be used for?* On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements. Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors. On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle <https://tools.wmflabs.org/wikidata-todo/cradle/#/>, that is currently not based on ShEx. *What is going to change on Wikidata?* * A new extension will be added to Wikidata: EntitySchema <https://www.mediawiki.org/wiki/Extension:EntitySchema>, defining the Schema namespace and its behavior as well as special pages related to it. * A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. * The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) <http://shex.io/shex-semantics/#shexc>. You can see an example here <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. * The external tool shex-simple <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2> is directly linked from the Schema pages in order to check entities of your choice against the schema. *When is this happening?* Schemas will be enabled on on test.wikidata.org <http://test.wikidata.org> on May 21st and on wikidata.org <http://wikidata.org> on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features. *How can you help?* * Before the release, you can try to edit or create Shape Expressions on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page> * If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag |shape-expressions| * Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model? * You can also get more information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F> *See also: * * Main Phabricator board <https://phabricator.wikimedia.org/tag/shape_expressions/> * Technical documentation of the extension <https://meta.wikimedia.org/wiki/Extension:EntitySchema> * To enhance the interface, you can use this user script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to highlight items and properties in the schema code and turn the IDs into links If you have any questions, feel free to reach me. Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de <http://www.wikimedia.de> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
The schemas can strike a practical balance between capturing current practice and describing a todo list of things to fix on current practice. It's possible we will want to separate those roles. In the meantime, can you survey existing instances and propose a shape which is not too far from the deployed instances?
On Tue, May 28, 2019 at 10:13 PM Peter F. Patel-Schneider < pfpschneider@gmail.com> wrote:
I sure hope that E10 is *not* the shape for human. It certainly isn't a correct shape for humans that belong to subclasses of human (such as Old Croghan Man (Q166790) or Delina Filkins (Q1408186)). E10 is also currently silent on what information should be present for humans, which I take it to be the point of having ShEx in Wikidata.
It is also unclear what is means to be the shape for human. The shape E10- does not have any information on which items are to be considered against the shape. Are all items in Wikidata to be considered (as in the definition of ShEx)? That doesn't seem right. Are all direct instances of human? That seems to limiting. Are all indirect instances of human? This seems the most natural, but where is this behaviour given?
Peter F. Patel-Schneider Samsung Research America
On 5/28/19 12:04 PM, Léa Lacroix wrote:
Hello all,
As previously announced, we just released shape expressions on Wikidata.
You
can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new
EntitySchema
https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx <
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx%3E
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
- More information about how to create a Schema <
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
- Phabricator tag: shape-expressions https://phabricator.wikimedia.org/tag/shape_expressions/
- User script <
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E to
highlight items and properties in the schema code and turn the IDsinto links
If you have any question or encounter issues, feel free to ping me.
Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix <lea.lacroix@wikimedia.de mailto:lea.lacroix@wikimedia.de> wrote:
Hello all, After several months of development and testing together with the WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx>, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?* ShEx (Q29377880) <https://www.wikidata.org/wiki/Q29377880> is aconcise,
formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. Inthe case
of Wikidata, this would be sets of properties, qualifiers andreferences
that describe the domain being modeled. See also: * a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg%3E
made by community members during the Wikimedia hackathon 2019 * introduction to ShEx <http://shex.io/shex-primer/> * more details about the language <http://shex.io/shex-semantics/> *What can it be used for?* On Wikidata, the main goal of Shape Expressions would be to describewhat
the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and manyother
important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detailwhat is
expected in the structure of items, statements and values of these statements. Once Schemas are created for various types of items, it is possibleto
test some existing items against the Schema, and highlight possibleerrors
or lack of information. Subsets of the Wikidata graph can be testedto see
whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure themodeling of
items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent,errors.
On top of this, one could imagine other uses of Schemas in thefuture, for
example building a tool that would suggest, when creating a newitem, what
would be the basic structure for this item, and helping addingstatements
or values. A bit like this existing tool, Cradle <https://tools.wmflabs.org/wikidata-todo/cradle/#/>, that iscurrently not
based on ShEx. *What is going to change on Wikidata?* * A new extension will be added to Wikidata: EntitySchema <https://www.mediawiki.org/wiki/Extension:EntitySchema>,defining the
Schema namespace and its behavior as well as special pagesrelated to it.
* A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. * The Schemas will have multilingual labels, descriptions andaliases
(quite similar to the termbox on Items), and the schema text onecan
fill with a syntax called ShEx Compact Syntax (ShExC) <http://shex.io/shex-semantics/#shexc>. You can see an examplehere
<https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. * The external tool shex-simple <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex...
is directly linked from the Schema pages in order to checkentities of
your choice against the schema. *When is this happening?* Schemas will be enabled on on test.wikidata.org <on May 21st and on wikidata.org <http://wikidata.org> on May 28th.After
this release, they will be integrated to the regular maintenancejust like
the rest of Wikidata’s features. *How can you help?* * Before the release, you can try to edit or create ShapeExpressions on
our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page%3E
* If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag |shape-expressions| * Once Schemas are enabled, you can discuss about it on yourfavorite
wikiprojects: for example, what types of items would you like tomodel?
* You can also get more information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
*See also: * * Main Phabricator board <https://phabricator.wikimedia.org/tag/shape_expressions/> * Technical documentation of the extension <https://meta.wikimedia.org/wiki/Extension:EntitySchema> * To enhance the interface, you can use this user script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E to
highlight items and properties in the schema code and turn theIDs
into links If you have any questions, feel free to reach me. Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de <http://www.wikimedia.de> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e.V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch dasFinanzamt
für Körperschaften I Berlin, Steuernummer 27/029/42207.-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter
der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Anyone know of a Language Server & Client for ShEx in Javascript ? It would be great to have one so all IDE's can take advantage of it. https://code.visualstudio.com/api/language-extensions/overview
I've opened an issue on github:shexSpec/shex.js to inquire as well... https://github.com/shexSpec/shex.js/issues/58
Thad https://www.linkedin.com/in/thadguidry/
On Tue, May 28, 2019 at 3:48 PM Andra Waagmeester andra@micel.io wrote:
The schemas can strike a practical balance between capturing current practice and describing a todo list of things to fix on current practice. It's possible we will want to separate those roles. In the meantime, can you survey existing instances and propose a shape which is not too far from the deployed instances?
On Tue, May 28, 2019 at 10:13 PM Peter F. Patel-Schneider < pfpschneider@gmail.com> wrote:
I sure hope that E10 is *not* the shape for human. It certainly isn't a correct shape for humans that belong to subclasses of human (such as Old Croghan Man (Q166790) or Delina Filkins (Q1408186)). E10 is also currently silent on what information should be present for humans, which I take it to be the point of having ShEx in Wikidata.
It is also unclear what is means to be the shape for human. The shape E10- does not have any information on which items are to be considered against the shape. Are all items in Wikidata to be considered (as in the definition of ShEx)? That doesn't seem right. Are all direct instances of human? That seems to limiting. Are all indirect instances of human? This seems the most natural, but where is this behaviour given?
Peter F. Patel-Schneider Samsung Research America
On 5/28/19 12:04 PM, Léa Lacroix wrote:
Hello all,
As previously announced, we just released shape expressions on
Wikidata. You
can for example have a look at E10, the shape for human https://www.wikidata.org/wiki/EntitySchema:E10, or create a new
EntitySchema
https://www.wikidata.org/wiki/Special:NewEntitySchema.
A few useful links:
- WikiProject ShEx <
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx%3E
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
- More information about how to create a Schema <
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
- Phabricator tag: shape-expressions https://phabricator.wikimedia.org/tag/shape_expressions/
- User script <
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E to
highlight items and properties in the schema code and turn the IDsinto links
If you have any question or encounter issues, feel free to ping me.
Cheers,
Léa
On Sun, 19 May 2019 at 15:32, Léa Lacroix <lea.lacroix@wikimedia.de mailto:lea.lacroix@wikimedia.de> wrote:
Hello all, After several months of development and testing together with the WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx>, Shape Expressions are about to be enabled on Wikidata. *First of all, what are Shape Expressions?* ShEx (Q29377880) <https://www.wikidata.org/wiki/Q29377880> is aconcise,
formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. Inthe case
of Wikidata, this would be sets of properties, qualifiers andreferences
that describe the domain being modeled. See also: * a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg%3E
made by community members during the Wikimedia hackathon 2019 * introduction to ShEx <http://shex.io/shex-primer/> * more details about the language <http://shex.io/shex-semantics/ *What can it be used for?* On Wikidata, the main goal of Shape Expressions would be todescribe what
the basic structure of an item would be. For example, for a human,we
probably want to have a date of birth, a place of birth, and manyother
important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detailwhat is
expected in the structure of items, statements and values of these statements. Once Schemas are created for various types of items, it is possibleto
test some existing items against the Schema, and highlight possibleerrors
or lack of information. Subsets of the Wikidata graph can be testedto see
whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure themodeling of
items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent,errors.
On top of this, one could imagine other uses of Schemas in thefuture, for
example building a tool that would suggest, when creating a newitem, what
would be the basic structure for this item, and helping addingstatements
or values. A bit like this existing tool, Cradle <https://tools.wmflabs.org/wikidata-todo/cradle/#/>, that iscurrently not
based on ShEx. *What is going to change on Wikidata?* * A new extension will be added to Wikidata: EntitySchema <https://www.mediawiki.org/wiki/Extension:EntitySchema>,defining the
Schema namespace and its behavior as well as special pagesrelated to it.
* A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E. * The Schemas will have multilingual labels, descriptions andaliases
(quite similar to the termbox on Items), and the schema textone can
fill with a syntax called ShEx Compact Syntax (ShExC) <http://shex.io/shex-semantics/#shexc>. You can see an examplehere
<https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. * The external tool shex-simple <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex...
is directly linked from the Schema pages in order to checkentities of
your choice against the schema. *When is this happening?* Schemas will be enabled on on test.wikidata.org <on May 21st and on wikidata.org <http://wikidata.org> on May 28th.After
this release, they will be integrated to the regular maintenancejust like
the rest of Wikidata’s features. *How can you help?* * Before the release, you can try to edit or create ShapeExpressions on
our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page%3E
* If you find any issue or feature you’d like to have, feel freeto
create a new task on Phabricator with the tag|shape-expressions|
* Once Schemas are enabled, you can discuss about it on yourfavorite
wikiprojects: for example, what types of items would you liketo model?
* You can also get more information about how to create a Schema <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
*See also: * * Main Phabricator board <https://phabricator.wikimedia.org/tag/shape_expressions/> * Technical documentation of the extension <https://meta.wikimedia.org/wiki/Extension:EntitySchema> * To enhance the interface, you can use this user script <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E to
highlight items and properties in the schema code and turn theIDs
into links If you have any questions, feel free to reach me. Cheers, -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de <http://www.wikimedia.de> Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissense. V.
Eingetragen im Vereinsregister des AmtsgerichtsBerlin-Charlottenburg
unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch dasFinanzamt
für Körperschaften I Berlin, Steuernummer 27/029/42207.-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg
unter
der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
It is not really possible to determine what a reasonable shape is before determining which Wikidata items are considered to be instances of human. For example, bog body (Q199414) is a subclass of human (Q5) but its instances are quite different from other instances human.
In any case, shouldn't some proponent of this addition to Wikidata be producing examples of reasonable shapes? I could propose reasonable constraints for instances of human, but I would do so in a formalism that I much prefer. Someone could, of course, translate these into ShEx, assuming that ShEx could represent the constraints (which I'm not sure of at all).
To see what the differences (and difficulties) are consider a very reasonable constraint - all the relatives of humans are humans (in my preferred syntax human <= all relative human). This *should* put a requirement on fathers, mothers, children, etc. of humans as these are all sub-properties of relative. Is this going to work in ShEx? I think that the answer is that it depends on what RDF graph ShEX is going to run over.
peter
On 5/28/19 4:47 PM, Andra Waagmeester wrote:
The schemas can strike a practical balance between capturing current practice and describing a todo list of things to fix on current practice. It's possible we will want to separate those roles. In the meantime, can you survey existing instances and propose a shape which is not too far from the deployed instances?
On Tue, May 28, 2019 at 10:13 PM Peter F. Patel-Schneider <pfpschneider@gmail.com mailto:pfpschneider@gmail.com> wrote:
I sure hope that E10 is *not* the shape for human. It certainly isn't a correct shape for humans that belong to subclasses of human (such as Old Croghan Man (Q166790) or Delina Filkins (Q1408186)). E10 is also currently silent on what information should be present for humans, which I take it to be the point of having ShEx in Wikidata. It is also unclear what is means to be the shape for human. The shape E10- does not have any information on which items are to be considered against the shape. Are all items in Wikidata to be considered (as in the definition of ShEx)? That doesn't seem right. Are all direct instances of human? That seems to limiting. Are all indirect instances of human? This seems the most natural, but where is this behaviour given? Peter F. Patel-Schneider Samsung Research America On 5/28/19 12:04 PM, Léa Lacroix wrote: > Hello all, > > As previously announced, we just released shape expressions on Wikidata. You > can for example have a look at E10, the shape for human > <https://www.wikidata.org/wiki/EntitySchema:E10>, or create a new EntitySchema > <https://www.wikidata.org/wiki/Special:NewEntitySchema>. > > A few useful links: > > * WikiProject ShEx <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx> > * introduction to ShEx <http://shex.io/shex-primer/> > * more details about the language <http://shex.io/shex-semantics/> > * More information about how to create a Schema > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F> > * Phabricator tag: shape-expressions > <https://phabricator.wikimedia.org/tag/shape_expressions/> > * User script > <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to > highlight items and properties in the schema code and turn the IDs into links > > If you have any question or encounter issues, feel free to ping me. Cheers, > > Léa > > > On Sun, 19 May 2019 at 15:32, Léa Lacroix <lea.lacroix@wikimedia.de <mailto:lea.lacroix@wikimedia.de> > <mailto:lea.lacroix@wikimedia.de <mailto:lea.lacroix@wikimedia.de>>> wrote: > > Hello all, > > After several months of development and testing together with the > WikiProject ShEx > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx>, Shape > Expressions are about to be enabled on Wikidata. > > *First of all, what are Shape Expressions?* > > ShEx (Q29377880) <https://www.wikidata.org/wiki/Q29377880> is a concise, > formal modeling and validation language for RDF structures. Shape > Expressions can be used to define shapes within the RDF graph. In the case > of Wikidata, this would be sets of properties, qualifiers and references > that describe the domain being modeled. > > See also: > > * a short video about ShEx <https://www.youtube.com/watch?v=AR75KhEoRKg> > made by community members during the Wikimedia hackathon 2019 > * introduction to ShEx <http://shex.io/shex-primer/> > * more details about the language <http://shex.io/shex-semantics/> > > *What can it be used for?* > > On Wikidata, the main goal of Shape Expressions would be to describe what > the basic structure of an item would be. For example, for a human, we > probably want to have a date of birth, a place of birth, and many other > important statements. But we would also like to make sure that if a > statement with the property “children” exists, the value(s) of this > property should be humans as well. Schemas will describe in detail what is > expected in the structure of items, statements and values of these > statements. > > Once Schemas are created for various types of items, it is possible to > test some existing items against the Schema, and highlight possible errors > or lack of information. Subsets of the Wikidata graph can be tested to see > whether or not they conform to a specific shape through the use of > validation tools. Therefore, Schemas will be very useful to help the > editors improving the data quality. We imagine this to be especially > useful for wiki projects to more easily discuss and ensure the modeling of > items in their domain. In the spirit of Wikidata not restricting the > world, Shape Expressions are a tool to highlight, not prevent, errors. > > On top of this, one could imagine other uses of Schemas in the future, for > example building a tool that would suggest, when creating a new item, what > would be the basic structure for this item, and helping adding statements > or values. A bit like this existing tool, Cradle > <https://tools.wmflabs.org/wikidata-todo/cradle/#/>, that is currently not > based on ShEx. > > *What is going to change on Wikidata?* > > * A new extension will be added to Wikidata: EntitySchema > <https://www.mediawiki.org/wiki/Extension:EntitySchema>, defining the > Schema namespace and its behavior as well as special pages related to it. > * A new entity type, EntitySchema, will be enabled to store Shape > Expressions. Schemas will be identified with the letter E. > * The Schemas will have multilingual labels, descriptions and aliases > (quite similar to the termbox on Items), and the schema text one can > fill with a syntax called ShEx Compact Syntax (ShExC) > <http://shex.io/shex-semantics/#shexc>. You can see an example here > <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. > * The external tool shex-simple > <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2> > is directly linked from the Schema pages in order to check entities of > your choice against the schema. > > *When is this happening?* > > Schemas will be enabled on on test.wikidata.org <http://test.wikidata.org> <http://test.wikidata.org> > on May 21st and on wikidata.org <http://wikidata.org> <http://wikidata.org> on May 28th. After > this release, they will be integrated to the regular maintenance just like > the rest of Wikidata’s features. > > *How can you help?* > > * Before the release, you can try to edit or create Shape Expressions on > our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page> > * If you find any issue or feature you’d like to have, feel free to > create a new task on Phabricator with the tag |shape-expressions| > * Once Schemas are enabled, you can discuss about it on your favorite > wikiprojects: for example, what types of items would you like to model? > * You can also get more information about how to create a Schema > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F> > > *See also: * > > * Main Phabricator board > <https://phabricator.wikimedia.org/tag/shape_expressions/> > * Technical documentation of the extension > <https://meta.wikimedia.org/wiki/Extension:EntitySchema> > * To enhance the interface, you can use this user script > <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> to > highlight items and properties in the schema code and turn the IDs > into links > > If you have any questions, feel free to reach me. Cheers, > > -- > Léa Lacroix > Project Manager Community Communication for Wikidata > > Wikimedia Deutschland e.V. > Tempelhofer Ufer 23-24 > 10963 Berlin > www.wikimedia.de <http://www.wikimedia.de> <http://www.wikimedia.de> > > Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. > > Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg > unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt > für Körperschaften I Berlin, Steuernummer 27/029/42207. > > > > -- > Léa Lacroix > Project Manager Community Communication for Wikidata > > Wikimedia Deutschland e.V. > Tempelhofer Ufer 23-24 > 10963 Berlin > www.wikimedia.de <http://www.wikimedia.de> <http://www.wikimedia.de> > > Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. > > Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter > der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für > Körperschaften I Berlin, Steuernummer 27/029/42207. > > _______________________________________________ > Wikidata mailing list > Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> > https://lists.wikimedia.org/mailman/listinfo/wikidata > _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Hello,
Could you explain why the non-standard ShEx has been chosen rather than the W3C Recommendation SHACL?
I would assume that if one has several options for bringing a functionality to something that largely promotes interoperability (like Wikidata), the default choice should be a standard, and /only if/ one has a carefully crafted argumentation to reject it, one would opt for something else.
For those who may not know, the W3C RDF Data Shapes Working Group worked between 2014 and 2017 on defining a standard for describing data shapes in RDF. ShEx existed already and was a candidate for standardisation. Eventually, another standard emerged, Shapes Constraint Language (SHACL, see https://www.w3.org/TR/shacl/).
Disclaimer: I did not contribute to either SHACL or ShEx, and I do not know them enough to judge which one is better.
Best, --AZ
On 19/05/2019 15:32, Léa Lacroix wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata.
*First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a concise, formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx.
*What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2 is directly linked from the Schema pages in order to check entities of your choice against the schema.
*When is this happening?*
Schemas will be enabled on on test.wikidata.org http://test.wikidata.org on May 21st and on wikidata.org http://wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features.
*How can you help?*
- Before the release, you can try to edit or create Shape Expressions on our test system https://wikidata-shex.wmflabs.org/wiki/Main_Page
- If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag |shape-expressions|
- Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to model?
- You can also get more information about how to create a Schema https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F
*See also: *
- Main Phabricator board https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js to highlight items and properties in the schema code and turn the IDs into links
If you have any questions, feel free to reach me. Cheers,
-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I really don't see the issue here. SHACL, like ShEx is a language to express data shapes. I adopted using ShEx in a wikidata context 2016 when ShEx was demonstrated at a tutorial at the SWAT4HCLS conference [1] in Amsterdam, where it was discussed in both a tutorial and a hackathon topic. At that conferene, I was convinced that ShEx is helpful in maintaining quality in Wikidata. ShEx offers not only the means to validate data shapes in Wikidata, but it also provides a way to document how primary data is expressed in Wikidata. In 2016 I joined the ShEx community group [2]. Since I have been actively using ShEx in defining shapes in various projects on Wikidata (e.g. Gene Wiki and Wikicite). It is not that this happened in secrecy. On the contrary, it was discussed at both Wikimedia [3,4] and non-Wikimedia events [5,6,7].
It is also not the case that SHACL has not been discussed in this context, on the contrary, I have very good memories of a workshop where both were debated (see page 24 ;) ) [8]
IMHO the statement that we all should adhere to one standard, simply because it is a standard, is not a valid argument. Imagine having to dictate that we all should speak English because it is the standard language. In every single talk that I have given since 2016, proponents of SHACL have been very vocal in asking the same question over and over again "why not SHACL?", where the discussion never went beyond, "You should because it is a standard". It is also a bit disingenuous to suggest we all should adhere to SHACL because it is the standard, while in the same sentence calling it a "Recommendation".
Although initially, I was open to SHACL as well (I use both Mac and Linux, so why not open up to different alternatives in data shapes), (Some) Arguments for me to prefer ShEx over SHACL are: 1. Already in 2017 there were different (open) implementations. At the time SHACL didn't have much tooling to choose from, other than one javascript implementation and a proprietary software package. 2. ShEx has a more intuitive way of describing Shapes, which is the compact syntax (ShExC). SHACL seems to have adopted the compact syntax as well, but only yesterday [9]. 3. The culture in the Shape Expression community group aligns well with the culture in Wikidata. 4. I don't want to be shackled to one standard (pun intended). I assume the name was chosen with a shackle in mind, which puts constraints at the core of the language. Wikidata already has different methods in place to deal with constraints and constraint violations. In the context of Wikidata, ShEx should specifically not be intended to impose constraints, on the contrary, it allows expressing of disagreement or variants of different shapes, whether conflict or not. Which fits well with the NPOV concept. Symbols do matter.
For a less personal comparison, I refer to the "Validating RDF data" book which describes both ShEx and SHACL, and has a specific chapter on how they compare and differ [10]
Up until now, I have been using ShEx in repositories outside the Wikidata ecosystem (e.g. Github), but I am really excited about the release of this extension. I am curious about how the wiki extension will influence the maintenance of schemas. Schemas are currently often expressed as static images, while in practice the schemas are as fluid as the underlying data itself. Being able to document these changes dynamically (the wiki way), can be very interesting. One specific expectation I have is that it might make it easier to write federated SPARQL queries. Currently, when writing these federated queries we often have to rely on either a set of example queries or a one-time schema description, which makes it hard to write those queries, because of schemas changing constantly. Federated SPARQL queries now really is a process of "slot machine" querying, where one has to explore the underlying schema, query by query. With a wiki in place and a community maintaining these ever-changing schema's, I expect better documentation.
The data shape community, instead of adhering to one language, should really be proud to have produced two very helpful languages. ShEx and SHACL are similar but do have differences so both have merit to exist and I wish we could steer away from this ShEx vs SHACL feud. It really isn't helping the cause, i.e. being able to express schemas in a formal language. Honestly, this fued really reminds me of the famous monty python sketch, "The machine that says Bing". Let us focus on the patient and not on the "Bing".
Just my 2ct.
[1] http://www.swat4ls.org/workshops/amsterdam2016/ [2] https://www.w3.org/community/shex/ [3] https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Submissions/Using_Sh... [4] https://meta.wikimedia.org/wiki/WikiCite_2017/Program [5] https://figshare.com/articles/Using_Shape_Expressions_ShEx_to_model_validate... [6] https://2017.semantics.cc/satellite-events/linked-data-quality-assessment-an... [7] http://swib.org/swib18/programme.html [8] https://upload.wikimedia.org/wikipedia/commons/d/d6/WikiCite_2017_report.pdf [9] https://lists.w3.org/Archives/Public/public-shacl/2019May/0012.html [10] http://book.validatingrdf.com/
On Wed, May 29, 2019 at 10:05 PM Antoine Zimmermann < antoine.zimmermann@emse.fr> wrote:
Hello,
Could you explain why the non-standard ShEx has been chosen rather than the W3C Recommendation SHACL?
I would assume that if one has several options for bringing a functionality to something that largely promotes interoperability (like Wikidata), the default choice should be a standard, and /only if/ one has a carefully crafted argumentation to reject it, one would opt for something else.
For those who may not know, the W3C RDF Data Shapes Working Group worked between 2014 and 2017 on defining a standard for describing data shapes in RDF. ShEx existed already and was a candidate for standardisation. Eventually, another standard emerged, Shapes Constraint Language (SHACL, see https://www.w3.org/TR/shacl/).
Disclaimer: I did not contribute to either SHACL or ShEx, and I do not know them enough to judge which one is better.
Best, --AZ
On 19/05/2019 15:32, Léa Lacroix wrote:
Hello all,
After several months of development and testing together with the WikiProject ShEx https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx, Shape Expressions are about to be enabled on Wikidata.
*First of all, what are Shape Expressions?*
ShEx (Q29377880) https://www.wikidata.org/wiki/Q29377880 is a
concise,
formal modeling and validation language for RDF structures. Shape Expressions can be used to define shapes within the RDF graph. In the case of Wikidata, this would be sets of properties, qualifiers and references that describe the domain being modeled.
See also:
- a short video about ShEx https://www.youtube.com/watch?v=AR75KhEoRKg made by community members during the Wikimedia hackathon 2019
- introduction to ShEx http://shex.io/shex-primer/
- more details about the language http://shex.io/shex-semantics/
*What can it be used for?*
On Wikidata, the main goal of Shape Expressions would be to describe what the basic structure of an item would be. For example, for a human, we probably want to have a date of birth, a place of birth, and many other important statements. But we would also like to make sure that if a statement with the property “children” exists, the value(s) of this property should be humans as well. Schemas will describe in detail what is expected in the structure of items, statements and values of these statements.
Once Schemas are created for various types of items, it is possible to test some existing items against the Schema, and highlight possible errors or lack of information. Subsets of the Wikidata graph can be tested to see whether or not they conform to a specific shape through the use of validation tools. Therefore, Schemas will be very useful to help the editors improving the data quality. We imagine this to be especially useful for wiki projects to more easily discuss and ensure the modeling of items in their domain. In the spirit of Wikidata not restricting the world, Shape Expressions are a tool to highlight, not prevent, errors.
On top of this, one could imagine other uses of Schemas in the future, for example building a tool that would suggest, when creating a new item, what would be the basic structure for this item, and helping adding statements or values. A bit like this existing tool, Cradle https://tools.wmflabs.org/wikidata-todo/cradle/#/, that is currently not based on ShEx.
*What is going to change on Wikidata?*
- A new extension will be added to Wikidata: EntitySchema https://www.mediawiki.org/wiki/Extension:EntitySchema, defining the Schema namespace and its behavior as well as special pages related to it.
- A new entity type, EntitySchema, will be enabled to store Shape Expressions. Schemas will be identified with the letter E.
- The Schemas will have multilingual labels, descriptions and aliases (quite similar to the termbox on Items), and the schema text one can fill with a syntax called ShEx Compact Syntax (ShExC) http://shex.io/shex-semantics/#shexc. You can see an example here https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2.
- The external tool shex-simple <
https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex...
is directly linked from the Schema pages in order to check entities of your choice against the schema.*When is this happening?*
Schemas will be enabled on on test.wikidata.org http://test.wikidata.org on May 21st and on wikidata.org http://wikidata.org on May 28th. After this release, they will be integrated to the regular maintenance just like the rest of Wikidata’s features.
*How can you help?*
Before the release, you can try to edit or create Shape Expressions on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page
If you find any issue or feature you’d like to have, feel free to create a new task on Phabricator with the tag |shape-expressions|
Once Schemas are enabled, you can discuss about it on your favorite wikiprojects: for example, what types of items would you like to
model?
- You can also get more information about how to create a Schema <
https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
*See also: *
- Main Phabricator board https://phabricator.wikimedia.org/tag/shape_expressions/
- Technical documentation of the extension https://meta.wikimedia.org/wiki/Extension:EntitySchema
- To enhance the interface, you can use this user script <
https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E
to highlight items and properties in the schema code and turn the IDs into linksIf you have any questions, feel free to reach me. Cheers,
-- Léa Lacroix Project Manager Community Communication for Wikidata
Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de http://www.wikimedia.de
Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V.
Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
The history of ShEx is quite complex.
I don't think that one can say that there were complete and conforming implementations of ShEx in 2017 because the main ShEX specification, http://shex.io/shex-semantics-20170713/ was ill-founded. I pointed this out in https://lists.w3.org/Archives/Public/public-shex/2018Mar/0008.html
There were several quite different semantics proposed for ShEx somewhat earlier, all with significant problems.
peter
On 5/30/19 12:34 AM, Andra Waagmeester wrote:
I really don't see the issue here. SHACL, like ShEx is a language to express data shapes. I adopted using ShEx in a wikidata context 2016 when ShEx was demonstrated at a tutorial at the SWAT4HCLS conference [1] in Amsterdam, where it was discussed in both a tutorial and a hackathon topic. At that conferene, I was convinced that ShEx is helpful in maintaining quality in Wikidata. ShEx offers not only the means to validate data shapes in Wikidata, but it also provides a way to document how primary data is expressed in Wikidata. In 2016 I joined the ShEx community group [2]. Since I have been actively using ShEx in defining shapes in various projects on Wikidata (e.g. Gene Wiki and Wikicite). It is not that this happened in secrecy. On the contrary, it was discussed at both Wikimedia [3,4] and non-Wikimedia events [5,6,7].
It is also not the case that SHACL has not been discussed in this context, on the contrary, I have very good memories of a workshop where both were debated (see page 24 ;) ) [8]
IMHO the statement that we all should adhere to one standard, simply because it is a standard, is not a valid argument. Imagine having to dictate that we all should speak English because it is the standard language. In every single talk that I have given since 2016, proponents of SHACL have been very vocal in asking the same question over and over again "why not SHACL?", where the discussion never went beyond, "You should because it is a standard". It is also a bit disingenuous to suggest we all should adhere to SHACL because it is the standard, while in the same sentence calling it a "Recommendation".
Although initially, I was open to SHACL as well (I use both Mac and Linux, so why not open up to different alternatives in data shapes), (Some) Arguments for me to prefer ShEx over SHACL are:
- Already in 2017 there were different (open) implementations. At the time
SHACL didn't have much tooling to choose from, other than one javascript implementation and a proprietary software package. 2. ShEx has a more intuitive way of describing Shapes, which is the compact syntax (ShExC). SHACL seems to have adopted the compact syntax as well, but only yesterday [9]. 3. The culture in the Shape Expression community group aligns well with the culture in Wikidata. 4. I don't want to be shackled to one standard (pun intended). I assume the name was chosen with a shackle in mind, which puts constraints at the core of the language. Wikidata already has different methods in place to deal with constraints and constraint violations. In the context of Wikidata, ShEx should specifically not be intended to impose constraints, on the contrary, it allows expressing of disagreement or variants of different shapes, whether conflict or not. Which fits well with the NPOV concept. Symbols do matter.
For a less personal comparison, I refer to the "Validating RDF data" book which describes both ShEx and SHACL, and has a specific chapter on how they compare and differ [10]
Up until now, I have been using ShEx in repositories outside the Wikidata ecosystem (e.g. Github), but I am really excited about the release of this extension. I am curious about how the wiki extension will influence the maintenance of schemas. Schemas are currently often expressed as static images, while in practice the schemas are as fluid as the underlying data itself. Being able to document these changes dynamically (the wiki way), can be very interesting. One specific expectation I have is that it might make it easier to write federated SPARQL queries. Currently, when writing these federated queries we often have to rely on either a set of example queries or a one-time schema description, which makes it hard to write those queries, because of schemas changing constantly. Federated SPARQL queries now really is a process of "slot machine" querying, where one has to explore the underlying schema, query by query. With a wiki in place and a community maintaining these ever-changing schema's, I expect better documentation.
The data shape community, instead of adhering to one language, should really be proud to have produced two very helpful languages. ShEx and SHACL are similar but do have differences so both have merit to exist and I wish we could steer away from this ShEx vs SHACL feud. It really isn't helping the cause, i.e. being able to express schemas in a formal language. Honestly, this fued really reminds me of the famous monty python sketch, "The machine that says Bing". Let us focus on the patient and not on the "Bing".
Just my 2ct.
[1] http://www.swat4ls.org/workshops/amsterdam2016/ [2] https://www.w3.org/community/shex/ [3] https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Submissions/Using_Sh... [4] https://meta.wikimedia.org/wiki/WikiCite_2017/Program [5] https://figshare.com/articles/Using_Shape_Expressions_ShEx_to_model_validate... [6] https://2017.semantics.cc/satellite-events/linked-data-quality-assessment-an... [7] http://swib.org/swib18/programme.html [8] https://upload.wikimedia.org/wikipedia/commons/d/d6/WikiCite_2017_report.pdf [9] https://lists.w3.org/Archives/Public/public-shacl/2019May/0012.html [10] http://book.validatingrdf.com/
On Wed, May 29, 2019 at 10:05 PM Antoine Zimmermann <antoine.zimmermann@emse.fr mailto:antoine.zimmermann@emse.fr> wrote:
Hello, Could you explain why the non-standard ShEx has been chosen rather than the W3C Recommendation SHACL? I would assume that if one has several options for bringing a functionality to something that largely promotes interoperability (like Wikidata), the default choice should be a standard, and /only if/ one has a carefully crafted argumentation to reject it, one would opt for something else. For those who may not know, the W3C RDF Data Shapes Working Group worked between 2014 and 2017 on defining a standard for describing data shapes in RDF. ShEx existed already and was a candidate for standardisation. Eventually, another standard emerged, Shapes Constraint Language (SHACL, see https://www.w3.org/TR/shacl/). Disclaimer: I did not contribute to either SHACL or ShEx, and I do not know them enough to judge which one is better. Best, --AZ On 19/05/2019 15:32, Léa Lacroix wrote: > Hello all, > > After several months of development and testing together with the > WikiProject ShEx > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx>, Shape > Expressions are about to be enabled on Wikidata. > > *First of all, what are Shape Expressions?* > > ShEx (Q29377880) <https://www.wikidata.org/wiki/Q29377880> is a concise, > formal modeling and validation language for RDF structures. Shape > Expressions can be used to define shapes within the RDF graph. In the > case of Wikidata, this would be sets of properties, qualifiers and > references that describe the domain being modeled. > > See also: > > * a short video about ShEx > <https://www.youtube.com/watch?v=AR75KhEoRKg> made by community > members during the Wikimedia hackathon 2019 > * introduction to ShEx <http://shex.io/shex-primer/> > * more details about the language <http://shex.io/shex-semantics/> > > *What can it be used for?* > > On Wikidata, the main goal of Shape Expressions would be to describe > what the basic structure of an item would be. For example, for a human, > we probably want to have a date of birth, a place of birth, and many > other important statements. But we would also like to make sure that if > a statement with the property “children” exists, the value(s) of this > property should be humans as well. Schemas will describe in detail what > is expected in the structure of items, statements and values of these > statements. > > Once Schemas are created for various types of items, it is possible to > test some existing items against the Schema, and highlight possible > errors or lack of information. Subsets of the Wikidata graph can be > tested to see whether or not they conform to a specific shape through > the use of validation tools. Therefore, Schemas will be very useful to > help the editors improving the data quality. We imagine this to be > especially useful for wiki projects to more easily discuss and ensure > the modeling of items in their domain. In the spirit of Wikidata not > restricting the world, Shape Expressions are a tool to highlight, not > prevent, errors. > > On top of this, one could imagine other uses of Schemas in the future, > for example building a tool that would suggest, when creating a new > item, what would be the basic structure for this item, and helping > adding statements or values. A bit like this existing tool, Cradle > <https://tools.wmflabs.org/wikidata-todo/cradle/#/>, that is currently > not based on ShEx. > > *What is going to change on Wikidata?* > > * A new extension will be added to Wikidata: EntitySchema > <https://www.mediawiki.org/wiki/Extension:EntitySchema>, defining > the Schema namespace and its behavior as well as special pages > related to it. > * A new entity type, EntitySchema, will be enabled to store Shape > Expressions. Schemas will be identified with the letter E. > * The Schemas will have multilingual labels, descriptions and aliases > (quite similar to the termbox on Items), and the schema text one can > fill with a syntax called ShEx Compact Syntax (ShExC) > <http://shex.io/shex-semantics/#shexc>. You can see an example here > <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. > * The external tool shex-simple > <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex-simple.html?schemaURL=https%3A%2F%2Fwikidata-shex.wmflabs.org%2Fwiki%2FSpecial%3AEntitySchemaText%2FE2> > is directly linked from the Schema pages in order to check entities > of your choice against the schema. > > *When is this happening?* > > Schemas will be enabled on on test.wikidata.org <http://test.wikidata.org> > <http://test.wikidata.org> on May 21st and on wikidata.org <http://wikidata.org> > <http://wikidata.org> on May 28th. After this release, they will be > integrated to the regular maintenance just like the rest of Wikidata’s > features. > > *How can you help?* > > * Before the release, you can try to edit or create Shape Expressions > on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page> > * If you find any issue or feature you’d like to have, feel free to > create a new task on Phabricator with the tag |shape-expressions| > * Once Schemas are enabled, you can discuss about it on your favorite > wikiprojects: for example, what types of items would you like to model? > * You can also get more information about how to create a Schema > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3F> > > *See also: * > > * Main Phabricator board > <https://phabricator.wikimedia.org/tag/shape_expressions/> > * Technical documentation of the extension > <https://meta.wikimedia.org/wiki/Extension:EntitySchema> > * To enhance the interface, you can use this user script > <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js> > to highlight items and properties in the schema code and turn the > IDs into links > > If you have any questions, feel free to reach me. Cheers, > > -- > Léa Lacroix > Project Manager Community Communication for Wikidata > > Wikimedia Deutschland e.V. > Tempelhofer Ufer 23-24 > 10963 Berlin > www.wikimedia.de <http://www.wikimedia.de> <http://www.wikimedia.de> > > Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. > > Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg > unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das > Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. > > > _______________________________________________ > Wikidata mailing list > Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> > https://lists.wikimedia.org/mailman/listinfo/wikidata > _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
I'd like to restate the initial question.
Why did wikidata choose shex instead of other approaches?
From this very detailed comparison
http://book.validatingrdf.com/bookHtml013.html (thank you Andra!) I could see arguments in both directions. I'm curious to know what swayed the wikidata software team as my group is currently grappling with the same decision.
On Thu, May 30, 2019 at 7:55 AM Peter F. Patel-Schneider < pfpschneider@gmail.com> wrote:
The history of ShEx is quite complex.
I don't think that one can say that there were complete and conforming implementations of ShEx in 2017 because the main ShEX specification, http://shex.io/shex-semantics-20170713/ was ill-founded. I pointed this out in https://lists.w3.org/Archives/Public/public-shex/2018Mar/0008.html
There were several quite different semantics proposed for ShEx somewhat earlier, all with significant problems.
peter
On 5/30/19 12:34 AM, Andra Waagmeester wrote:
I really don't see the issue here. SHACL, like ShEx is a language to
express
data shapes. I adopted using ShEx in a wikidata context 2016 when ShEx
was
demonstrated at a tutorial at the SWAT4HCLS conference [1] in Amsterdam,
where
it was discussed in both a tutorial and a hackathon topic. At that
conferene,
I was convinced that ShEx is helpful in maintaining quality in Wikidata.
ShEx
offers not only the means to validate data shapes in Wikidata, but it
also
provides a way to document how primary data is expressed in Wikidata.
In 2016
I joined the ShEx community group [2]. Since I have been actively using
ShEx
in defining shapes in various projects on Wikidata (e.g. Gene Wiki and Wikicite). It is not that this happened in secrecy. On the contrary, it
was
discussed at both Wikimedia [3,4] and non-Wikimedia events [5,6,7].
It is also not the case that SHACL has not been discussed in this
context, on
the contrary, I have very good memories of a workshop where both were
debated
(see page 24 ;) ) [8]
IMHO the statement that we all should adhere to one standard, simply
because
it is a standard, is not a valid argument. Imagine having to dictate
that we
all should speak English because it is the standard language. In every
single
talk that I have given since 2016, proponents of SHACL have been very
vocal in
asking the same question over and over again "why not SHACL?", where the discussion never went beyond, "You should because it is a standard". It
is
also a bit disingenuous to suggest we all should adhere to SHACL because
it is
the standard, while in the same sentence calling it a "Recommendation".
Although initially, I was open to SHACL as well (I use both Mac and
Linux, so
why not open up to different alternatives in data shapes), (Some)
Arguments
for me to prefer ShEx over SHACL are:
- Already in 2017 there were different (open) implementations. At the
time
SHACL didn't have much tooling to choose from, other than one javascript implementation and a proprietary software package. 2. ShEx has a more intuitive way of describing Shapes, which is the
compact
syntax (ShExC). SHACL seems to have adopted the compact syntax as well,
but
only yesterday [9]. 3. The culture in the Shape Expression community group aligns well with
the
culture in Wikidata. 4. I don't want to be shackled to one standard (pun intended). I assume
the
name was chosen with a shackle in mind, which puts constraints at the
core of
the language. Wikidata already has different methods in place to deal
with
constraints and constraint violations. In the context of Wikidata, ShEx
should
specifically not be intended to impose constraints, on the contrary, it
allows
expressing of disagreement or variants of different shapes, whether
conflict
or not. Which fits well with the NPOV concept. Symbols do matter.
For a less personal comparison, I refer to the "Validating RDF data" book which describes both ShEx and SHACL, and has a specific chapter on how
they
compare and differ [10]
Up until now, I have been using ShEx in repositories outside the Wikidata ecosystem (e.g. Github), but I am really excited about the release of
this
extension. I am curious about how the wiki extension will influence the maintenance of schemas. Schemas are currently often expressed as static images, while in practice the schemas are as fluid as the underlying data itself. Being able to document these changes dynamically (the wiki way),
can
be very interesting. One specific expectation I have is that it might
make it
easier to write federated SPARQL queries. Currently, when writing these federated queries we often have to rely on either a set of example
queries or
a one-time schema description, which makes it hard to write those
queries,
because of schemas changing constantly. Federated SPARQL queries now
really is
a process of "slot machine" querying, where one has to explore the
underlying
schema, query by query. With a wiki in place and a community maintaining these ever-changing schema's, I expect better documentation.
The data shape community, instead of adhering to one language, should
really
be proud to have produced two very helpful languages. ShEx and SHACL are similar but do have differences so both have merit to exist and I wish we could steer away from this ShEx vs SHACL feud. It really isn't helping
the
cause, i.e. being able to express schemas in a formal language.
Honestly, this
fued really reminds me of the famous monty python sketch, "The machine
that
says Bing". Let us focus on the patient and not on the "Bing".
Just my 2ct.
[1] http://www.swat4ls.org/workshops/amsterdam2016/ [2] https://www.w3.org/community/shex/ [3]
https://www.wikidata.org/wiki/Wikidata:WikidataCon_2017/Submissions/Using_Sh...
[4] https://meta.wikimedia.org/wiki/WikiCite_2017/Program [5]
https://figshare.com/articles/Using_Shape_Expressions_ShEx_to_model_validate...
[6]
https://2017.semantics.cc/satellite-events/linked-data-quality-assessment-an...
https://upload.wikimedia.org/wikipedia/commons/d/d6/WikiCite_2017_report.pdf
[9] https://lists.w3.org/Archives/Public/public-shacl/2019May/0012.html [10] http://book.validatingrdf.com/
On Wed, May 29, 2019 at 10:05 PM Antoine Zimmermann <antoine.zimmermann@emse.fr mailto:antoine.zimmermann@emse.fr> wrote:
Hello, Could you explain why the non-standard ShEx has been chosen ratherthan
the W3C Recommendation SHACL? I would assume that if one has several options for bringing a functionality to something that largely promotes interoperability(like
Wikidata), the default choice should be a standard, and /only if/ one has a carefully crafted argumentation to reject it, one would opt for something else. For those who may not know, the W3C RDF Data Shapes Working Groupworked
between 2014 and 2017 on defining a standard for describing datashapes
in RDF. ShEx existed already and was a candidate for standardisation. Eventually, another standard emerged, Shapes Constraint Language(SHACL,
see https://www.w3.org/TR/shacl/). Disclaimer: I did not contribute to either SHACL or ShEx, and I donot
know them enough to judge which one is better. Best, --AZ On 19/05/2019 15:32, Léa Lacroix wrote: > Hello all, > > After several months of development and testing together with the > WikiProject ShEx > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx>, Shape > Expressions are about to be enabled on Wikidata. > > *First of all, what are Shape Expressions?* > > ShEx (Q29377880) <https://www.wikidata.org/wiki/Q29377880> is aconcise,
> formal modeling and validation language for RDF structures. Shape > Expressions can be used to define shapes within the RDF graph. Inthe
> case of Wikidata, this would be sets of properties, qualifiers and > references that describe the domain being modeled. > > See also: > > * a short video about ShEx > <https://www.youtube.com/watch?v=AR75KhEoRKg> made bycommunity
> members during the Wikimedia hackathon 2019 > * introduction to ShEx <http://shex.io/shex-primer/> > * more details about the language <http://shex.io/shex-semantics/%3E
> > *What can it be used for?* > > On Wikidata, the main goal of Shape Expressions would be todescribe
> what the basic structure of an item would be. For example, for ahuman,
> we probably want to have a date of birth, a place of birth, andmany
> other important statements. But we would also like to make surethat if
> a statement with the property “children” exists, the value(s) ofthis
> property should be humans as well. Schemas will describe in detailwhat
> is expected in the structure of items, statements and values ofthese
> statements. > > Once Schemas are created for various types of items, it ispossible to
> test some existing items against the Schema, and highlight possible > errors or lack of information. Subsets of the Wikidata graph can be > tested to see whether or not they conform to a specific shapethrough
> the use of validation tools. Therefore, Schemas will be veryuseful to
> help the editors improving the data quality. We imagine this to be > especially useful for wiki projects to more easily discuss andensure
> the modeling of items in their domain. In the spirit of Wikidatanot
> restricting the world, Shape Expressions are a tool to highlight,not
> prevent, errors. > > On top of this, one could imagine other uses of Schemas in thefuture,
> for example building a tool that would suggest, when creating a new > item, what would be the basic structure for this item, and helping > adding statements or values. A bit like this existing tool, Cradle > <https://tools.wmflabs.org/wikidata-todo/cradle/#/>, that iscurrently
> not based on ShEx. > > *What is going to change on Wikidata?* > > * A new extension will be added to Wikidata: EntitySchema > <https://www.mediawiki.org/wiki/Extension:EntitySchema>,defining
> the Schema namespace and its behavior as well as special pages > related to it. > * A new entity type, EntitySchema, will be enabled to store Shape > Expressions. Schemas will be identified with the letter E. > * The Schemas will have multilingual labels, descriptions andaliases
> (quite similar to the termbox on Items), and the schema textone can
> fill with a syntax called ShEx Compact Syntax (ShExC) > <http://shex.io/shex-semantics/#shexc>. You can see anexample here
> <https://wikidata-shex.wmflabs.org/wiki/EntitySchema:E2>. > * The external tool shex-simple > <https://tools.wmflabs.org/shex-simple/wikidata/packages/shex-webapp/doc/shex...
> is directly linked from the Schema pages in order to checkentities
> of your choice against the schema. > > *When is this happening?* > > Schemas will be enabled on on test.wikidata.org <> <http://test.wikidata.org> on May 21st and on wikidata.org <http://wikidata.org> > <http://wikidata.org> on May 28th. After this release, they willbe
> integrated to the regular maintenance just like the rest ofWikidata’s
> features. > > *How can you help?* > > * Before the release, you can try to edit or create ShapeExpressions
> on our test system <https://wikidata-shex.wmflabs.org/wiki/Main_Page%3E
> * If you find any issue or feature you’d like to have, feel freeto
> create a new task on Phabricator with the tag|shape-expressions|
> * Once Schemas are enabled, you can discuss about it on yourfavorite
> wikiprojects: for example, what types of items would you liketo model?
> * You can also get more information about how to create a Schema > <https://www.wikidata.org/wiki/Wikidata:WikiProject_ShEx/How_to_get_started%3...
> > *See also: * > > * Main Phabricator board > <https://phabricator.wikimedia.org/tag/shape_expressions/> > * Technical documentation of the extension > <https://meta.wikimedia.org/wiki/Extension:EntitySchema> > * To enhance the interface, you can use this user script > <https://www.wikidata.org/wiki/User:Zvpunry/EntitySchemaHighlighter.js%3E
> to highlight items and properties in the schema code and turnthe
> IDs into links > > If you have any questions, feel free to reach me. Cheers, > > -- > Léa Lacroix > Project Manager Community Communication for Wikidata > > Wikimedia Deutschland e.V. > Tempelhofer Ufer 23-24 > 10963 Berlin > www.wikimedia.de <http://www.wikimedia.de> <> > Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissense. V.
> > Eingetragen im Vereinsregister des AmtsgerichtsBerlin-Charlottenburg
> unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das > Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. > > > _______________________________________________ > Wikidata mailing list > Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> > https://lists.wikimedia.org/mailman/listinfo/wikidata > _______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
On 30/05/2019 17:45, Benjamin Good wrote:
I'd like to restate the initial question.
Why did wikidata choose shex instead of other approaches?
From this very detailed comparison http://book.validatingrdf.com/bookHtml013.html (thank you Andra!) I could see arguments in both directions. I'm curious to know what swayed the wikidata software team as my group is currently grappling with the same decision.
One of the key differences would seem to be that SHACL has been deliberately constructed to directly representable in RDF -- so a SHACL expression could be put straight into WDQS and made queryable for what it pertains to.
I'm not clear whether there is an RDF representation of ShEx that could be added to WDQS -- this is a point that would be useful to clarify; and, if not, whether work is going on in this direction.
IMO, it would be a very important asset to be able to query the Shape specifications using SPARQL -- querying not for compliance, but for what the specifications actually contain.
On the other hand, SHACL seems very strongly based on shapes for members that are connected by an "is a" relationship.
It's not immediately clear to me whether SHACL adapts easily to memberships defined by eg P31 "instance on" or P279 "subclass of" statements, etc; also memberships possibly further defined or limited by other statements.
It would seem a basic requirement, but I didn't see it on a first quick skim-read.
-- James.
Hi James,
I'm not the best person to answer your questions (I've never actually used ShEx or contributed to it) but I hope to be able to answer your questions.
I'm not clear whether there is an RDF representation of ShEx that could
be added to WDQS -- this is a point that would be useful to clarify; and, if not, whether work is going on in this direction.
TL;DR: yes but it does not seem much used at the moment.
ShEx specifies [1] two syntaxes, ShExC that is the "plain text" syntax used on Wikidata and ShExJ that is based on JSON-LD. JSON-LD beeing an RDF seralization, there is indeed an RDF representation of ShEx. To get plain triples one could use the JSON-LD to RDF triples conversion algorithm that is implemented in most JSON-LD libraries. The old ShEx documentation pages refers to a ShExR serialization of ShEx to RDF but I believe it has been dropped in favor of ShExJ+JSON-LD to RDF conversion.
It's not immediately clear to me whether SHACL adapts easily to
memberships defined by eg P31 "instance on" or P279 "subclass of" statements, etc; also memberships possibly further defined or limited by other statements.
Indeed Shacl 1.0 does not seem to be able to express it. There is an extension [1] that allows to specify targets using a sparql query, just like what is done with the ShEx playground using the focus nodes sparql query.
Thomas
[1] http://shex.io/shex-semantics/ [2] https://www.w3.org/2018/jsonld-cg-reports/json-ld-api/#deserialize-json-ld-t... [3] https://www.w3.org/TR/shacl-af/#SPARQLTarget
Le ven. 31 mai 2019 à 11:06, James Heald jpm.heald@gmail.com a écrit :
On 30/05/2019 17:45, Benjamin Good wrote:
I'd like to restate the initial question.
Why did wikidata choose shex instead of other approaches?
From this very detailed comparison http://book.validatingrdf.com/bookHtml013.html (thank you Andra!) I
could
see arguments in both directions. I'm curious to know what swayed the wikidata software team as my group is currently grappling with the same decision.
One of the key differences would seem to be that SHACL has been deliberately constructed to directly representable in RDF -- so a SHACL expression could be put straight into WDQS and made queryable for what it pertains to.
I'm not clear whether there is an RDF representation of ShEx that could be added to WDQS -- this is a point that would be useful to clarify; and, if not, whether work is going on in this direction.
IMO, it would be a very important asset to be able to query the Shape specifications using SPARQL -- querying not for compliance, but for what the specifications actually contain.
On the other hand, SHACL seems very strongly based on shapes for members that are connected by an "is a" relationship.
It's not immediately clear to me whether SHACL adapts easily to memberships defined by eg P31 "instance on" or P279 "subclass of" statements, etc; also memberships possibly further defined or limited by other statements.
It would seem a basic requirement, but I didn't see it on a first quick skim-read.
-- James.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
On Fri, May 31, 2019 at 08:02:03PM +0200, Thomas Tanon wrote:
Hi James,
I'm not the best person to answer your questions (I've never actually used ShEx or contributed to it) but I hope to be able to answer your questions.
I'm not clear whether there is an RDF representation of ShEx that could
be added to WDQS -- this is a point that would be useful to clarify; and, if not, whether work is going on in this direction.
TL;DR: yes but it does not seem much used at the moment.
ShEx specifies [1] two syntaxes, ShExC that is the "plain text" syntax used on Wikidata and ShExJ that is based on JSON-LD. JSON-LD beeing an RDF seralization, there is indeed an RDF representation of ShEx. To get plain triples one could use the JSON-LD to RDF triples conversion algorithm that is implemented in most JSON-LD libraries. The old ShEx documentation pages refers to a ShExR serialization of ShEx to RDF but I believe it has been dropped in favor of ShExJ+JSON-LD to RDF conversion.
We actually do test the ShExR with a Turtle representation. We have ~500 round-trip tests between the three representations: https://github.com/shexSpec/shexTest/tree/extends/schemas
It would be pretty easy to add a translation to RDF in the backend of a schema update. That would give you all the clever meta queries you'd like for dependency-checking and auditing without making people edit triples.
It's not immediately clear to me whether SHACL adapts easily to
memberships defined by eg P31 "instance on" or P279 "subclass of" statements, etc; also memberships possibly further defined or limited by other statements.
Indeed Shacl 1.0 does not seem to be able to express it. There is an extension [1] that allows to specify targets using a sparql query, just like what is done with the ShEx playground using the focus nodes sparql query.
Thomas
[1] http://shex.io/shex-semantics/ [2] https://www.w3.org/2018/jsonld-cg-reports/json-ld-api/#deserialize-json-ld-t... [3] https://www.w3.org/TR/shacl-af/#SPARQLTarget
Le ven. 31 mai 2019 à 11:06, James Heald jpm.heald@gmail.com a écrit :
On 30/05/2019 17:45, Benjamin Good wrote:
I'd like to restate the initial question.
Why did wikidata choose shex instead of other approaches?
From this very detailed comparison http://book.validatingrdf.com/bookHtml013.html (thank you Andra!) I
could
see arguments in both directions. I'm curious to know what swayed the wikidata software team as my group is currently grappling with the same decision.
One of the key differences would seem to be that SHACL has been deliberately constructed to directly representable in RDF -- so a SHACL expression could be put straight into WDQS and made queryable for what it pertains to.
I'm not clear whether there is an RDF representation of ShEx that could be added to WDQS -- this is a point that would be useful to clarify; and, if not, whether work is going on in this direction.
IMO, it would be a very important asset to be able to query the Shape specifications using SPARQL -- querying not for compliance, but for what the specifications actually contain.
On the other hand, SHACL seems very strongly based on shapes for members that are connected by an "is a" relationship.
It's not immediately clear to me whether SHACL adapts easily to memberships defined by eg P31 "instance on" or P279 "subclass of" statements, etc; also memberships possibly further defined or limited by other statements.
It would seem a basic requirement, but I didn't see it on a first quick skim-read.
-- James.
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata