Significant change: new data type for tabular data files

List overview All Threads
Download

newer

older

Voting has begun in 2017 Wikimedia...

Weekly Summary #258

Léa Lacroix

2 May 2017 2 May '17

4:37 p.m.

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th* . The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org (example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207.

Attachments:

attachment.htm (text/html — 1.9 KB)

Show replies by date

Jane Darnell

2 May 2 May

6:33 p.m.

New subject: Significant change: new data type for tabular data files

Interesting, thanks! I have been waiting for more developments on this since it was shown by User:TheDJ at the developer's showcase in january (link here at 5 minutes in https://www.youtube.com/watch?v=j2pR21imm9A) I was wondering if this could be used in the case of painting items being linked to old art sale catalogs. So instead of bothering with wikisource, no matter what language the catalog is in I could link to a catalog entry on commons by line and column (theoretically two columns: one column for catalog identifier, and second columns for full catalog entry, generally less than 300 characters of text). On Tue, May 2, 2017 at 10:37 AM, Léa Lacroix <lea.lacroix(a)wikimedia.de> wrote:

...

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th*. The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org ( example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

John Erling Blad

9:47 p.m.

New subject: Significant change: new data type for tabular data files

You know that this has pretty huge implications for the data model, and that data stored in a tabular file might invalidate the statement where it is referenced? And both the statement and the data file might be valid in isolation? (It is two valid propositions but from different worlds.) On Tue, May 2, 2017 at 12:33 PM, Jane Darnell <jane023(a)gmail.com> wrote:

...

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th*. The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org ( example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Jane Darnell

9:56 p.m.

New subject: Significant change: new data type for tabular data files

Not sure what you mean - if the datafile stored on commons links in turn to the source (e.g. the sale catalog hosted somewhere) then the datafile only acts as a transformation engine enabling blobbish text to be accessed as citable material for machine reading bots. Seems nifty to me. On Tue, May 2, 2017 at 3:47 PM, John Erling Blad <jeblad(a)gmail.com> wrote:

...

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th*. The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org ( example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

John Erling Blad

10:10 p.m.

New subject: Significant change: new data type for tabular data files

A statement in a world might be valid and true in that world, but false in another world. Two statements, each in its own world might use a name (parameter or instantiated) that is similar by accident, and chained together they will form an invalid statement. If values from the data files are available in the same RDF without preventive measures (aka same world) it will create problems. This is well known in logic, so it should not be a surprise. There are several proposed solutions for RDF-graphs, but it gets slightly (a lot) more complex. On Tue, May 2, 2017 at 3:56 PM, Jane Darnell <jane023(a)gmail.com> wrote:

...

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th*. The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org ( example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

John Erling Blad

10 p.m.

New subject: Significant change: new data type for tabular data files

One of the most important use of this is probably JSONstat, but note that it is _not_ obvious which categories maps to which items, or that a category maps to the same item at all. For example "Oslo" might be used as the name of the city in Norway in some contexts, while it might be the name of the county in other contexts, or it might be the unincorporated community in Southern Florida. If a tag function is made to make and reapply queries to the API-console now in use at several census bureaus, then the JSONstat for specific data can be automatically updated. This will create some additional problems, as those stats can't be manually updated. Yes I have done some experiments on this, no it has not been possible to get this up and running for various reasons. (There must be a working cache with high availability, the json-lib in Scribunto is flaky, etc.) On Tue, May 2, 2017 at 3:47 PM, John Erling Blad <jeblad(a)gmail.com> wrote:

...

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th*. The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org ( example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

John Erling Blad

10:18 p.m.

New subject: Significant change: new data type for tabular data files

The API console for extraction of statistics is separate from use of JSONstat, but it can export JSONstat. Statistics Norway have a user manual for how to use the console.[2][1] This is a common collaboration between several Scandinavian census bureaus. [1] http://ssb.no/en/omssb/tjenester-og-verktoy/api/_attachment/248250?_ts=15b4… [2] http://ssb.no/en/omssb/tjenester-og-verktoy/api On Tue, May 2, 2017 at 4:00 PM, John Erling Blad <jeblad(a)gmail.com> wrote:

...

Hello all, We’ve been working on a new data type that allows you to link to the *tabular data files <https://www.mediawiki.org/wiki/Help:Tabular_Data>* that are now stored on Commons. This data type will be deployed on Wikidata on *May 15th*. The property creators will be able to create properties with this tabular data type by selecting “tabular data” in the data type list. When the property is created, you can use it in statements, and when filling the value, if you start typing a string, you can choose the name of a file in the list of what exists on Commons. Before the deployment, you can test it on http://test.wikidata.org ( example <https://test.wikidata.org/wiki/Q59992>). One thing to note: We currently do not export statements that use this datatype to RDF. They can therefore not be queried in the Wikidata Query Service. The reason is that we are still waiting for tabular data files to get stable URIs. This is handled in this ticket <https://phabricator.wikimedia.org/T161527>. If you have any question, feel free to ask! -- Léa Lacroix Project Manager Community Communication for Wikidata Wikimedia Deutschland e.V. Tempelhofer Ufer 23-24 10963 Berlin www.wikimedia.de Wikimedia Deutschland - Gesellschaft zur Förderung Freien Wissens e. V. Eingetragen im Vereinsregister des Amtsgerichts Berlin-Charlottenburg unter der Nummer 23855 Nz. Als gemeinnützig anerkannt durch das Finanzamt für Körperschaften I Berlin, Steuernummer 27/029/42207. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

_______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

2558

days inactive

2558

days old

wikidata@lists.wikimedia.org

Manage subscription

6 comments

3 participants

tags (0)

participants (3)

Jane Darnell
John Erling Blad
Léa Lacroix