Re: [Wikidata-tech] Question on Wikidata/Wikibase reconciliation service for OpenRefine

It looks like the openrefine-wikibase reconciliation service uses the wbsearchentities API to find items. As far as I’m aware, the default SQL-based Wikibase search also searches other languages, but still, I think the most likely reason you’re getting different results is that Wikidata uses WikibaseCirrusSearch <https://www.mediawiki.org/wiki/Extension:WikibaseCirrusSearch>, and I assume your wiki doesn’t. If you use entity search on your wiki directly (i. e. not via Special:Search, but in suggestion boxes), does cross-language search work as it should or does it have the same problem? Cheers, Lucas On 18.09.19 22:55, Régis Robineau wrote: Hi all, I'd need help on the Wikidata/Wikibase reconciliation service for OpenRefine. Context: I have my own Wikibase and WDQS instances in production, and I want to set up a reconciliation service on top of it, so that users can perform matchings from their local OpenRefine. I'm using the same tool as Wikidata, i.e. https://github.com/wetneb/openrefine-wikibase. The web service works fine, I can reconcile strings in OpenRefine against the data stored in my Wikibase instance... Issue: But there is a noteworthy difference compared to how the Wikidata reconciliation service works: - with Wikidata, i.e. by using the web service URL with the "en" language prefix (https://tools.wmflabs.org/openrefine-wikidata/en/api), I am able to find matches among labels in any other language of a Wikidata item. For instance, if I send a request for "Jacopo de Fazio", which is an alias in French and Italian for Q313460 <https://www.wikidata.org/wiki/Q313460>, OpenRefine will match Q313460 as expected, even if I'm using the "en" language code in the web service url. - Whereas in my own instance, i.e. by using my own "openrefine-wikibase" reconciliation service, it can only perform matching of labels/aliases in the same language: e.g. if I use " https://my-service.org/openrefine-wikidata/en/api"quot;, the web service only searches for labels in English in my Wikibase. This means that I am forced to launch the reconciliation process in OpenRefine for every single language, one by one. I do not know how the Wikidata reconciliation service is able to take into account all the labels/aliases in all the languages of a given Wikidata item. The data is modeled in the same way in Wikidata and in my Wikibase, and I do not see any difference between the two in the way the RDF data is structured into the respective triplestores... How can I enable the same behaviour as in the Wikidata reconciliation service? (i.e. to look for labels/aliases in every languages in one API call) This would heavily improve the reconciliation process in OpenRefine for my users. Thanks a lot for your help! Régis _______________________________________________ Wikidata-tech mailing listWikidata-tech@lists.wikimedia.orghttps://lists.wikimedia.org/mailman/listinfo/wikidata-tech _______________________________________________ Wikidata-tech mailing list Wikidata-tech(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-tech