Hi everybody,
(With apologies for cross-posting...)
You may have seen the recent communication [1
<https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes>]
about the product and tech tune-up which went live the week of June 5th,
2017. In that communication, we promised an update on the future of
Discovery projects and we will talk about those in this email.
The Discovery team structure has now changed, but the new teams will still
work together to complete the goals as listed in the draft annual plan.[2]
A summary of their anticipated work, as we finalize these changes, is
below. We plan on doing a check-in at the end of the calendar year to see
how our goals are progressing with the new smaller and separated team
structure.
Here is a list of the various projects under the Discovery umbrella, along
with the goals that they will be working on:
Search Backend
Improve search capabilities:
-
Implement ‘learning to rank’ [3] and other advanced machine learning
methodologies
-
Improve support for languages using new analyzers
-
Maintain and expand power user search functionality
Search Frontend
Improve user interface of the search results page with new functionality:
-
Implement explore similar [4]
<https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testin…>
-
Update the completion suggester box [5]
<https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester>
-
Investigate the usage of a Wiktionary widget for English Wikipedia [6]
Wikidata Query Service
Expand and scale:
-
Improve ability to support power features on-wiki for readers
-
Improve full text search functionality
-
Implement SPARQL federation support
Portal
Create and implement automated language statistics and translation updates
for Wikipedia.org
Analysis
Provide in-depth analytics support:
-
Perform experimental design, data collection, and data analysis
-
Perform ad-hoc analyses of Discovery-domain data
-
Maintain and augment the Discovery Dashboards,[7] which allow the teams
to track their KPIs and other metrics
Maps
Map support:
-
Implement new map style
-
Increase frequency of OSM data replication
-
As needed, assist with individual language Wikipedia’s implementation of
mapframe [8] <https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps>
Note: There is a possibility that we can do more with maps in the coming
year; we are currently evaluating strategic, partnership, and resourcing
options.
Structured Data on Commons
Extend structured data search on Commons, as part of the structured data
grant [9] via:
-
Research and implement advanced search capabilities
-
Implement new elements, filters, relationships
Graphs and Tabular Data on Commons
We will be re-evaluating this functionality against other Commons
initiatives such as the structured data grant. As with maps, we will
provide updates when we know more.
We are still working out all the details with the new team structure and
there might be some turbulence; let us know if there are any concerns and
we will do our best to answer them.
Best regards,
Deborah Tankersley, Product Manager, Discovery
Erika Bjune, Engineering Manager, Search Platform
Jon Katz, Reading Product Lead
Toby Negrin, Interim Vice President of Product
Victoria Coleman, Chief Technology Officer
[1] https://www.mediawiki.org/wiki/Wikimedia_Engineering/June_2017_changes
[2]
https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2017-2018/…
[3] https://en.wikipedia.org/wiki/Learning_to_rank
[4]
https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testin…
[5]
https://www.mediawiki.org/wiki/Extension:CirrusSearch/CompletionSuggester
[6]
https://www.mediawiki.org/wiki/Cross-wiki_Search_Result_Improvements/Testin…
[7] https://discovery.wmflabs.org/
[8] https://www.mediawiki.org/wiki/Maps/how_to:_embedded_maps
[9] https://commons.wikimedia.org/wiki/Commons:Structured_data
Forwarding to the Wikimedia Maps mailing list.
Pine
---------- Forwarded message ----------
From: Edward Betts <edward(a)4angle.com>
Date: Wed, Jun 7, 2017 at 11:39 PM
Subject: [Wikidata] Tool for users to add wikidata tags to OpenStreetMap
To: wikidata(a)lists.wikimedia.org
I've built a tool for mappers to match things in OSM with Wikidata and add
the
appropriate wikidata tag to OSM.
https://osm.wikidata.link/
Users can search for a administrative area or city, then pick an area to
analyse. It works best if mappers pick areas they're familiar with.
The matcher can take a few minutes to run. It grabs items from Wikidata and
figures out a target set of tags and keys to search for. Then it downloads
OSM
data and looks for matches. The matching is based on tags and names,
currently
the wikipedia tags aren't considered.
Map data is from the OSM Overpass API. Large (more than 1,000 sq km) or
dense
areas might fail with a timeout error.
The results are cached in the system and are available here:
https://osm.wikidata.link/places
Once the matching process is complete the mapper is given a tabbed page with
the results. The five tabs are:
Match candidates - things on OSM that might be considered for tagging
Already tagged - matches that are already tagged in OSM
No match - items from Wikidata with no match found in OSM
Wikidata query - the query used to find Wikidata items
Overpass query - the Overpass query to find OSM objects in this area
To start tagging the mapper is able to login to OSM via OAuth. Tick boxes
will
appear next to the likely matches, the mapper can tick the box next to the
matches they want to upload, then add a change comment and upload them using
their own OSM account. Uploads within an area are combined into a single
changeset.
If the mapper sees an obviously incorrect match they can use the 'report bad
match' option to warn other mappers and provide feedback that I can use to
improve the algorithm.
This tools doesn't add any new objects to OSM. The only change it makes is
adding a wikidata tag to existing things.
My approach is to aim for a one-to-one mapping between Wikidata and OSM. If
there are two or more things in OSM that look like a Wikidata item then it
isn't a good match. This means for example that most road and rail bridges
won't be tagged because they are represented as two OSM ways. I might change
this at some point.
There are occasional duplicates in Wikidata, this tool should spot them and
refuse to add wikidata tags until the Wikidata duplicate is resolved.
The bug/todo list is here: https://github.com/EdwardBetts/osm-wikidata/
issues
Any ideas or suggestions are welcome.
--
Edward.
_______________________________________________
Wikidata mailing list
Wikidata(a)lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata
Joshua Minor, 08/06/2017 00:53:
> The Places feature currently uses the iOS provided Apple MapKit library
> and Apple maps. This is not a decision the team made easily, but after
> much feedback and research, we believe this is the only option for now.
> A couple important notes:
>
> 1. This is only used in the Places tab, as a means of search and
> discovery. These maps are NOT used in the encyclopedia itself, and
> do not replace, prevent or circumvent work by editors to add open
> maps to articles on wikis.
> 2. Our users' privacy is preserved. We don't share what articles you're
> reading with Apple, and Apple has put significant technical and
> legal oomph into preserving privacy for their built-in maps library.
>
> If you have concerns or questions, please check out the page we wrote
> explaining the situation in much more detail[1]. We also welcome your
> addition to the Discussion either on wiki, on the task request to
> replace Apple Maps[3] or via email.
>
> [1] https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/iOS/Maps_service
> <https://www.mediawiki.org/wiki/Wikimedia_Apps/Team/iOS/Maps_service>
> [2] https://phabricator.wikimedia.org/T157763
Thanks. I've replied there:
https://phabricator.wikimedia.org/T157763#3337113
Nemo
Forwarding.
Pine
---------- Forwarded message ----------
From: Yuri Astrakhan <yuriastrakhan(a)gmail.com>
Date: Tue, Jun 6, 2017 at 9:07 PM
Subject: [OSM-talk] OSM+Wikidata intro video
To: OpenStreetMap talk mailing list <talk(a)openstreetmap.org>, "
tagging(a)openstreetmap.org" <tagging(a)openstreetmap.org>
The RDF/SPARQL database that has both OpenStreetMap and Wikidata data in
the same table is alive and well, and getting considerable usage. To make
it better understood by even wider community, I made an intro video with
some examples. This database mostly benefits the object tag validation and
research at this moment, as its geometry support is still in the works.
The wiki page has also seen a lot of cleanup, explaining how quality
control can be done. I hope that other tools such as JOSM and especially
MapRoulette will be able to use it directly. Also, please contribute your
SPARQL queries to the wiki page.
https://www.youtube.com/watch?v=KDiKzbuIhtshttps://wiki.openstreetmap.org/wiki/Wikidata_RDF_database
_______________________________________________
talk mailing list
talk(a)openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk