I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata?
We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley < brylie@gnumedia.org> wrote:
I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
I'm intensely interested in links to shapefiles from databases such as Wikidata, DBpedia and Freebase. In particular I'd like to get Natural Earth hooked up
http://www.naturalearthdata.com/
It's definitely a weakness of current generic databases that they use the 'point GIS' model that is so popular in the social media world.
On Tue, Aug 5, 2014 at 10:14 AM, Magnus Manske magnusmanske@googlemail.com wrote:
We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley < brylie@gnumedia.org> wrote:
I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
There's a proposal I posted a while ago to store generic datasets that can be represented in a tabular or JSON format in a dedicated project namespace with dedicated handlers:
http://meta.wikimedia.org/wiki/DataNamespace
There's some good discussion on the talk page on the differences between this type of data and structure data hosted on Wikidata and where this thing could live (it could live on any Wikimedia wiki, including Commons or Meta). It looks like this could be a good fit for shapefiles and I'd love to hear your thoughts if you have a moment to read this.
Dario
On Aug 5, 2014, at 8:43, Paul Houle ontology2@gmail.com wrote:
I'm intensely interested in links to shapefiles from databases such as Wikidata, DBpedia and Freebase. In particular I'd like to get Natural Earth hooked up
http://www.naturalearthdata.com/
It's definitely a weakness of current generic databases that they use the 'point GIS' model that is so popular in the social media world.
On Tue, Aug 5, 2014 at 10:14 AM, Magnus Manske magnusmanske@googlemail.com wrote: We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley brylie@gnumedia.org wrote: I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com _______________________________________________ Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
This can already be done by changing JsonConfig configuration. I propose we add a "Data" namespace to the *commons https://commons.wikimedia.org/*. Moreover, with the recent work on Graph https://www.mediawiki.org/wiki/Extension:Graph extension, I was thinking of storing graphing related data there as well. JsonConfig currently supports php-code-based validation, but adding json-schema https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=json%20schema&safe=off validation should not be too difficult.
Data:* -- accepts any valid JSON, without additional validation Data:SubNamespace:* -- Custom validated domain-specific data
For graphs, we might have shape data as well as statistics data. So we could declare: Data:Graph:* -- accepts JSON that represents entire graph -- vega http://trifacta.github.io/vega/ grammar, and Data:SomethingElse:* for all snippets of graphs that will be pulled in by the vega dynamically.
Also, I could fairly easily add support for CSV and TSV if we decide that it is needed, so that Data:Tsv:* pages would be forced to have the same number of columns on each line.
On Wed, Aug 13, 2014 at 11:48 AM, Dario Taraborelli < dtaraborelli@wikimedia.org> wrote:
There's a proposal I posted a while ago to store generic datasets that can be represented in a tabular or JSON format in a dedicated project namespace with dedicated handlers:
http://meta.wikimedia.org/wiki/DataNamespace http://meta.m.wikimedia.org/wiki/DataNamespace
There's some good discussion on the talk page on the differences between this type of data and structure data hosted on Wikidata and where this thing could live (it could live on any Wikimedia wiki, including Commons or Meta). It looks like this could be a good fit for shapefiles and I'd love to hear your thoughts if you have a moment to read this.
Dario
On Aug 5, 2014, at 8:43, Paul Houle ontology2@gmail.com wrote:
I'm intensely interested in links to shapefiles from databases such as Wikidata, DBpedia and Freebase. In particular I'd like to get Natural Earth hooked up
http://www.naturalearthdata.com/
It's definitely a weakness of current generic databases that they use the 'point GIS' model that is so popular in the social media world.
On Tue, Aug 5, 2014 at 10:14 AM, Magnus Manske < magnusmanske@googlemail.com> wrote:
We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley < brylie@gnumedia.org> wrote:
I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Yuri and others, I asked not long ago in Commons and there is no opposition as long as it is for supporting visualizations. It would require modifying what is allowed in commons, though. There are interesting comments on the talk page https://meta.wikimedia.org/wiki/Requests_for_comment/How_to_deal_with_open_d...
Data.wikisource.org could be another option, but it would require more effort to set up, and some time for discussions.
Cheers, Micru
On Wed, Aug 13, 2014 at 6:15 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
This can already be done by changing JsonConfig configuration. I propose we add a "Data" namespace to the *commons https://commons.wikimedia.org/*. Moreover, with the recent work on Graph https://www.mediawiki.org/wiki/Extension:Graph extension, I was thinking of storing graphing related data there as well. JsonConfig currently supports php-code-based validation, but adding json-schema https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=json%20schema&safe=off validation should not be too difficult.
Data:* -- accepts any valid JSON, without additional validation Data:SubNamespace:* -- Custom validated domain-specific data
For graphs, we might have shape data as well as statistics data. So we could declare: Data:Graph:* -- accepts JSON that represents entire graph -- vega http://trifacta.github.io/vega/ grammar, and Data:SomethingElse:* for all snippets of graphs that will be pulled in by the vega dynamically.
Also, I could fairly easily add support for CSV and TSV if we decide that it is needed, so that Data:Tsv:* pages would be forced to have the same number of columns on each line.
On Wed, Aug 13, 2014 at 11:48 AM, Dario Taraborelli < dtaraborelli@wikimedia.org> wrote:
There's a proposal I posted a while ago to store generic datasets that can be represented in a tabular or JSON format in a dedicated project namespace with dedicated handlers:
http://meta.wikimedia.org/wiki/DataNamespace http://meta.m.wikimedia.org/wiki/DataNamespace
There's some good discussion on the talk page on the differences between this type of data and structure data hosted on Wikidata and where this thing could live (it could live on any Wikimedia wiki, including Commons or Meta). It looks like this could be a good fit for shapefiles and I'd love to hear your thoughts if you have a moment to read this.
Dario
On Aug 5, 2014, at 8:43, Paul Houle ontology2@gmail.com wrote:
I'm intensely interested in links to shapefiles from databases such as Wikidata, DBpedia and Freebase. In particular I'd like to get Natural Earth hooked up
http://www.naturalearthdata.com/
It's definitely a weakness of current generic databases that they use the 'point GIS' model that is so popular in the social media world.
On Tue, Aug 5, 2014 at 10:14 AM, Magnus Manske < magnusmanske@googlemail.com> wrote:
We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley < brylie@gnumedia.org> wrote:
I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Hi all,
Sorry for coming to this conversation late.
I've been preparing for a public beta release for the WikiBrain project I have been developing with Brent Hecht and others (http://wikibrainapi.org - docs not yet finalized). WikiBrain supports a variety of state-of-the-art algorithmic layers on top of Wikipedia, include spatial relationships, multi-lingual alignment, and semantic relatedness.
As part of this process, we've developed a mapping from NaturalEarth to Wikipedia at both the country and state level. I'd be happy to share it with you if you're interested.
-Shilad
On Wed, Aug 13, 2014 at 12:19 PM, David Cuenca dacuetu@gmail.com wrote:
Yuri and others, I asked not long ago in Commons and there is no opposition as long as it is for supporting visualizations. It would require modifying what is allowed in commons, though. There are interesting comments on the talk page
https://meta.wikimedia.org/wiki/Requests_for_comment/How_to_deal_with_open_d...
Data.wikisource.org could be another option, but it would require more effort to set up, and some time for discussions.
Cheers, Micru
On Wed, Aug 13, 2014 at 6:15 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
This can already be done by changing JsonConfig configuration. I propose we add a "Data" namespace to the *commons https://commons.wikimedia.org/*. Moreover, with the recent work on Graph https://www.mediawiki.org/wiki/Extension:Graph extension, I was thinking of storing graphing related data there as well. JsonConfig currently supports php-code-based validation, but adding json-schema https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=json%20schema&safe=off validation should not be too difficult.
Data:* -- accepts any valid JSON, without additional validation Data:SubNamespace:* -- Custom validated domain-specific data
For graphs, we might have shape data as well as statistics data. So we could declare: Data:Graph:* -- accepts JSON that represents entire graph -- vega http://trifacta.github.io/vega/ grammar, and Data:SomethingElse:* for all snippets of graphs that will be pulled in by the vega dynamically.
Also, I could fairly easily add support for CSV and TSV if we decide that it is needed, so that Data:Tsv:* pages would be forced to have the same number of columns on each line.
On Wed, Aug 13, 2014 at 11:48 AM, Dario Taraborelli < dtaraborelli@wikimedia.org> wrote:
There's a proposal I posted a while ago to store generic datasets that can be represented in a tabular or JSON format in a dedicated project namespace with dedicated handlers:
http://meta.wikimedia.org/wiki/DataNamespace http://meta.m.wikimedia.org/wiki/DataNamespace
There's some good discussion on the talk page on the differences between this type of data and structure data hosted on Wikidata and where this thing could live (it could live on any Wikimedia wiki, including Commons or Meta). It looks like this could be a good fit for shapefiles and I'd love to hear your thoughts if you have a moment to read this.
Dario
On Aug 5, 2014, at 8:43, Paul Houle ontology2@gmail.com wrote:
I'm intensely interested in links to shapefiles from databases such as Wikidata, DBpedia and Freebase. In particular I'd like to get Natural Earth hooked up
http://www.naturalearthdata.com/
It's definitely a weakness of current generic databases that they use the 'point GIS' model that is so popular in the social media world.
On Tue, Aug 5, 2014 at 10:14 AM, Magnus Manske < magnusmanske@googlemail.com> wrote:
We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley < brylie@gnumedia.org> wrote:
I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Etiamsi omnes, ego non
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Shapefiles are group of 3 to 10 individual files from the pov of non-gis software. I could see some difficulties uploading them to wikidata/commons since the model we are used to is self contained files
--Tom
On Wed, Aug 13, 2014 at 8:27 PM, Shilad Sen ssen@macalester.edu wrote:
Hi all,
Sorry for coming to this conversation late.
I've been preparing for a public beta release for the WikiBrain project I have been developing with Brent Hecht and others (http://wikibrainapi.org - docs not yet finalized). WikiBrain supports a variety of state-of-the-art algorithmic layers on top of Wikipedia, include spatial relationships, multi-lingual alignment, and semantic relatedness.
As part of this process, we've developed a mapping from NaturalEarth to Wikipedia at both the country and state level. I'd be happy to share it with you if you're interested.
-Shilad
On Wed, Aug 13, 2014 at 12:19 PM, David Cuenca dacuetu@gmail.com wrote:
Yuri and others, I asked not long ago in Commons and there is no opposition as long as it is for supporting visualizations. It would require modifying what is allowed in commons, though. There are interesting comments on the talk page
https://meta.wikimedia.org/wiki/Requests_for_comment/How_to_deal_with_open_d...
Data.wikisource.org could be another option, but it would require more effort to set up, and some time for discussions.
Cheers, Micru
On Wed, Aug 13, 2014 at 6:15 PM, Yuri Astrakhan yastrakhan@wikimedia.org wrote:
This can already be done by changing JsonConfig configuration. I propose we add a "Data" namespace to the commons. Moreover, with the recent work on Graph extension, I was thinking of storing graphing related data there as well. JsonConfig currently supports php-code-based validation, but adding json-schema validation should not be too difficult.
Data:* -- accepts any valid JSON, without additional validation Data:SubNamespace:* -- Custom validated domain-specific data
For graphs, we might have shape data as well as statistics data. So we could declare: Data:Graph:* -- accepts JSON that represents entire graph -- vega grammar, and Data:SomethingElse:* for all snippets of graphs that will be pulled in by the vega dynamically.
Also, I could fairly easily add support for CSV and TSV if we decide that it is needed, so that Data:Tsv:* pages would be forced to have the same number of columns on each line.
On Wed, Aug 13, 2014 at 11:48 AM, Dario Taraborelli dtaraborelli@wikimedia.org wrote:
There's a proposal I posted a while ago to store generic datasets that can be represented in a tabular or JSON format in a dedicated project namespace with dedicated handlers:
http://meta.wikimedia.org/wiki/DataNamespace
There's some good discussion on the talk page on the differences between this type of data and structure data hosted on Wikidata and where this thing could live (it could live on any Wikimedia wiki, including Commons or Meta). It looks like this could be a good fit for shapefiles and I'd love to hear your thoughts if you have a moment to read this.
Dario
On Aug 5, 2014, at 8:43, Paul Houle ontology2@gmail.com wrote:
I'm intensely interested in links to shapefiles from databases such as Wikidata, DBpedia and Freebase. In particular I'd like to get Natural Earth hooked up
http://www.naturalearthdata.com/
It's definitely a weakness of current generic databases that they use the 'point GIS' model that is so popular in the social media world.
On Tue, Aug 5, 2014 at 10:14 AM, Magnus Manske magnusmanske@googlemail.com wrote:
We don't have shapefiles yet, but a lot of property types such as geographic coordinates (as in, one per item, ideally...), external identifiers (e.g. VIAF), dates, etc.
A (reasonably) simple way to mass-add statements to Wikidata is this tool: http://tools.wmflabs.org/wikidata-todo/quick_statements.php
A combination of spreadsheet apps, shell commands, and/or a good text editor should allow you to convert many CSVs into the tool's input format.
Cheers, Magnus
On Tue, Aug 5, 2014 at 3:01 PM, Brylie Christopher Oxley brylie@gnumedia.org wrote:
I would like to contribute data to Wikidata that is in the form of CSV files, geospatial shapefiles, etc.
Is there currently, or planned, functionality to store general structured data on Wikidata? -- Brylie Christopher Oxley http://gnumedia.org
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Paul Houle Expert on Freebase, DBpedia, Hadoop and RDF (607) 539 6254 paul.houle on Skype ontology2@gmail.com
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Etiamsi omnes, ego non
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l
-- Shilad W. Sen Associate Professor Mathematics, Statistics, and Computer Science Dept. Macalester College ssen@macalester.edu http://www.shilad.com https://www.linkedin.com/in/shilad 651-696-6273
Wikidata-l mailing list Wikidata-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-l