Hello all,
Yesterday, an announcement (Now live: Shared structured data), incorrectly
stated that Structured Data had been launched on Commons.
The feature which was inaccurately named “Structured Data”, enables users
to add tabular data to the data namespace on Commons via the regular page
editor and to further display and/or visualize that data from other wikis.
This work is unrelated to an ongoing project called Structured Data on
Commons. For more on the newly launched feature, see the Tabular Data [1]
and Map Data [2] help pages on MediaWiki.org.
For information on the Structured Data on Commons project, designed to
associate structured data with media files on Commons to improve their
discoverability, please visit the project page on Commons. [3]
Thank you,
-Katie
[1] - https://www.mediawiki.org/wiki/Help:Tabular_Data
[2] - https://www.mediawiki.org/wiki/Help:Map_Data
[3] - https://commons.wikimedia.org/wiki/Commons:Structured_data
Gift season! We have launched structured data on Commons, available from
all wikis.
TLDR; One data store. Use everywhere. Upload table data to Commons, with
localization, and use it to create wiki tables, lists, or use directly in
graphs. Works for GeoJSON maps too. Must be licensed as CC0. Try this
per-state GDP map demo, and select multiple years. More demos at the bottom.
US Map state highlight
<https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight>
Data can now be stored as *.tab and *.map pages in the data namespace on
Commons. That data may contain localization, so a table cell could be in
multiple languages. And that data is accessible from any wikis, by Lua
scripts, Graphs, and Maps.
Lua lets you generate wiki tables from the data by filtering, converting,
mixing, and formatting the raw data. Lua also lets you generate lists. Or
any wiki markup.
Graphs can use both .tab and .map directly to visualize the data and let
users interact with it. The GDP demo above uses a map from Commons, and
colors each segment with the data based on a data table.
Kartographer (<maplink>/<mapframe>) can use the .map data as an extra layer
on top of the base map. This way we can show endangered species' habitat.
== Demo ==
* Raw data example
<https://commons.wikimedia.org/wiki/Data:Weather/New_York_City.tab>
* Interactive Weather data
<https://en.wikipedia.org/wiki/Template:Graph:Weather_monthly_history>
* Same data in Weather template
<https://en.wikipedia.org/wiki/User:Yurik/WeatherDemo>
* Interactive GDP map
<https://en.wikipedia.org/wiki/Template:Graph:US_Map_state_highlight>
* Endangered Jemez Mountains salamander - habitat
<https://en.wikipedia.org/wiki/Jemez_Mountains_salamander#/maplink/0>
* Population history
<https://en.wikipedia.org/wiki/Template:Graph:Population_history>
* Line chart <https://en.wikipedia.org/wiki/Template:Graph:Lines>
== Getting started ==
* Try creating a page at data:Sandbox/<user>.tab on Commons. Don't forget
the .tab extension, or it won't work.
* Try using some data with the Line chart graph template
A thorough guide is needed, help is welcome!
== Documentation links ==
* Tabular help <https://www.mediawiki.org/wiki/Help:Tabular_Data>
* Map help <https://www.mediawiki.org/wiki/Help:Map_Data>
If you find a bug, create Phabricator ticket with #tabular-data tag, or
comment on the documentation talk pages.
== FAQ ==
* Relation to Wikidata: Wikidata is about "facts" (small pieces of
information). Structured data is about "blobs" - large amounts of data like
the historical weather or the outline of the state of New York.
== TODOs ==
* Add a nice "table editor" - editing JSON by hand is cruel. T134618
* "What links here" should track data usage across wikis. Will allow
quicker auto-refresh of the pages too. T153966
* Support data redirects. T153598
* Mega epic: Support external data feeds.
Svetlana, thanks for suggestion. I think we should create a portal similar
to the Structured Data one, and put some examples there. Deciding on the
name is difficult :) "Commons Datasets" does sound good.
There has been a very prolonged discussion on where to host this feature -
https://meta.wikimedia.org/wiki/User:Yurik/Storing_data. Wikidata would
have been a good choice, but users expect all the data there to be in
public domain, and we may add more licensing choices later.
> An inline example with English commentary -- straight on the first page about
this new technology without making users click links -- could be nice. The
text you typed up does not seem to be on a wiki page, so I am unable to
edit it...
Which page are you referring to?
On Thu, Dec 22, 2016 at 4:03 PM Svetlana Tkachenko <svetlana(a)members.fsf.org>
wrote:
> Hello,
>
> Maybe 'commons store' or 'commons datasets' could work? I would suggest
> that the name reflects on the fact that the datasets are shared
> ('common') and are not on Wikidata.
>
> If I may ask, why is it in commons.wikimedia.org/wiki/Data:* and not at
> Meta (like Global user pages) or Wikidata (like structured data about
> lots of things)?
>
> An inline example with English commentary -- straight on the first page
> about this new technology without making users click links -- could be
> nice. The text you typed up does not seem to be on a wiki page, so I am
> unable to edit it...
>
> Svetlana.
>
Micru, thanks, I think Datasets sounds like a good name too!
On Thu, Dec 22, 2016 at 2:44 PM David Cuenca Tudela <dacuetu(a)gmail.com>
wrote:
> On Thu, Dec 22, 2016 at 8:38 PM, Brad Jorsch (Anomie) <
> bjorsch(a)wikimedia.org
> > wrote:
>
> > On Thu, Dec 22, 2016 at 2:30 PM, Yuri Astrakhan <
> yastrakhan(a)wikimedia.org>
> > wrote:
> >
> > > Gift season! We have launched structured data on Commons, available
> from
> > > all wikis.
> > >
> >
> > I was momentarily excited, then I read a little farther and discovered
> this
> > isn't about https://commons.wikimedia.org/wiki/Commons:Structured_data.
> >
>
> Same here, I think it needs a better name...
>
> What about calling it datasets or structured datasets?
>
> Cheers,
> Micru
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Yes, there seem to have been a bit of a naming collision. Tabular data and
map data have been jointly known as structured data, but there is also the
Structured Data project, which IMO should be called Structured Metadata
project :) Naming suggestions are welcome!
P.S. Brad, I'm sorry tabular and map data did not excite you :(
On Thu, Dec 22, 2016 at 2:38 PM Brad Jorsch (Anomie) <bjorsch(a)wikimedia.org>
wrote:
> On Thu, Dec 22, 2016 at 2:30 PM, Yuri Astrakhan <yastrakhan(a)wikimedia.org>
> wrote:
>
> > Gift season! We have launched structured data on Commons, available from
> > all wikis.
> >
>
> I was momentarily excited, then I read a little farther and discovered this
> isn't about https://commons.wikimedia.org/wiki/Commons:Structured_data.
>
>
> --
> Brad Jorsch (Anomie)
> Senior Software Engineer
> Wikimedia Foundation
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l(a)lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l