On Oct 22, 2015, at 10:25 AM, Thomas Douillard &lt;thomas.douillard(a)gmail.com&gt; wrote: Great tool ! The error detection is precious ! 2015-10-22 17:31 GMT+02:00 Markus Kroetzsch &lt;markus.kroetzsch(a)tu-dresden.de <mailto:markus.kroetzsch@tu-dresden.de>>: Hi all, I am happy to announce a new tool [1], written by Serge Stratan, which allows you to browse the taxonomy (subclass of & instance of relations) between Wikidata's most important class items. For example, here is the Wikidata taxonomy for Pizza (discussed recently on this list): http://sergestratan.bitbucket.org?draw=true&optid=s0&item=177,2095,… <http://sergestratan.bitbucket.org/?draw=true&optid=s0&item=177,2095,7802,28877,35120,223557,386724,488383,666242,736427,746549,2424752,15222213,16686448> == What you see there == Solid green lines mean "subclass of" relations (subclasses are lower), while dashed purple lines are "instance of" relations (instances are lower). Drag and zoom the view as usual. Hover over items for more information. Click on arrows with numbers to display upper or lower neighbours. Right-click on classes to get more options. The sidebar on the left shows statistics and presumed problems in the data (redundancies and likely errors). You can select a report type to see the reports, and click on any line to show the error. If you search for a class in the search field, the errors will be narrowed down to issues related to the taxonomy of this class. The toolbar at the top has options to show and hide items based on the current selection (left click on any box). Edges in red are the wrong way around (top to bottom). This occurs only when there are cycles in the "taxonomy". == Micro tutorial == (1) Enter "Unicorn" in the search box, press return. (2) Zoom out a bit by scrolling your mouse/touchpad (3) Click on the "Unicorn" item box. It becomes blue (selected). (4) Click "Expand up" in the toolbar at the top (5) Zoom out to see the taxonomy of unicorn (6) Find the class "Fictional Horse" (directly above unicorn) and click its downwards arrow labelled "3" to see all three children items of "fictional horse". (7) Click the share button on the top right to get a link to this view. You can also create your own share link manually by just changing the Qids in the URL as you like. == Status and limitations == This is a prototype and it still has some limits: * It only shows "proper" classes that have at least one instance or subclass. This is to reduce the overall data size and load time. * The data is based on dumps (the date is shown on the right). It is not a live view. * The layout is sometimes too dense. You can find a "hidden" option to make it more spacy behind the sidebar (click "Sidebar" to see it). This helps to disentangle larger graphs. * There are some minor bugs in the UI. You sometimes need to click more than once until the right thing happens. * The help page at http://sergestratan.bitbucket.org/howtouse.html <http://sergestratan.bitbucket.org/howtouse.html> does not explain everything in detail yet. It is planned to work on some of these limitations in the future. The hope is that this tool will reveal many errors in Wikidata's taxonomy that are otherwise hard to detect. For example, you can see easily that every "Ship" is an "Event" in Wikidata, that every "Hobbit" is a "Fantasy Race", and that every "Monday" is both a "Mathematical object" and a "Unit of measurement". Feedback is welcome (on the tool; better start new threads for feedback on the Wikidata taxonomy ;-), Markus [1] http://sergestratan.bitbucket.org <http://sergestratan.bitbucket.org/> -- Markus Kroetzsch Faculty of Computer Science Technische Universität Dresden +49 351 463 38486 <tel:%2B49%20351%20463%2038486> http://korrekt.org/ <http://korrekt.org/> _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata <https://lists.wikimedia.org/mailman/listinfo/wikidata> _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

I’m constantly getting 500 errors.

I am having the same kinds of 500 problems. Bitbucket is generally suffering today: http://status.bitbucket.org

On Thu, Oct 22, 2015 at 12:27 PM, Markus Kroetzsch &lt;markus.kroetzsch(a)tu-dresden.de <mailto:markus.kroetzsch@tu-dresden.de>> wrote: On 22.10.2015 19:29, Dario Taraborelli wrote: I’m constantly getting 500 errors. I also observed short outages in the past, and I sometimes had to run a request twice to get an answer. It seems that the hosting on bitbucket is not very reliable. At the moment, this is still a first preview of the tool without everything set up as it should be. The tool should certainly move to Wikimedia labs in the future. Markus -- Markus Kroetzsch Faculty of Computer Science Technische Universität Dresden +49 351 463 38486 <tel:%2B49%20351%20463%2038486> http://korrekt.org/ _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

On 22.10.2015 19:29, Dario Taraborelli wrote: I also observed short outages in the past, and I sometimes had to run a request twice to get an answer. It seems that the hosting on bitbucket is not very reliable. At the moment, this is still a first preview of the tool without everything set up as it should be. The tool should certainly move to Wikimedia labs in the future. Markus -- Markus Kroetzsch Faculty of Computer Science Technische Universität Dresden +49 351 463 38486 http://korrekt.org/ _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Hoi, The problem with tools like this is that they get a moment attention. Particularly when they are stand alone, not integrated, they will lose interest.

Would it be an option to host this tool on Labs?

On 22 October 2015 at 21:27, Markus Kroetzsch &lt;markus.kroetzsch(a)tu-dresden.de <mailto:markus.kroetzsch@tu-dresden.de>> wrote: On 22.10.2015 19:29, Dario Taraborelli wrote: I’m constantly getting 500 errors. I also observed short outages in the past, and I sometimes had to run a request twice to get an answer. It seems that the hosting on bitbucket is not very reliable. At the moment, this is still a first preview of the tool without everything set up as it should be. The tool should certainly move to Wikimedia labs in the future. Markus -- Markus Kroetzsch Faculty of Computer Science Technische Universität Dresden +49 351 463 38486 <tel:%2B49%20351%20463%2038486> http://korrekt.org/ _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org <mailto:Wikidata@lists.wikimedia.org> https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

Hi! Very nice! One specially interesting to me is cycle detection - I just discovered there are many (as in, hundreds) cycles in various important properties such as P131 which break a lot of queries, and was wondering what could be done to expose/eliminate those better.

Hi! We can do it, but I'd rather we didn't. The reason is that it would require engine that queries such data (e.g. SPARQL engine) to be comfortable with cycles in property paths (especially ones with + and *), and not every one is (Blazegraph for example looks like does not handle them out of the box). It can be dealt with, I assume, but why create trouble for ourselves?

"Said to be the same as" is a good example of a case where cycles are unavoidable. A possible workaround in this case is to make sure that the transitive closure of "said to be the same as" is already in the data, such that the path "P460+" returns the same results as a mere "P460" would. It's not ideal, but maybe workable.

It's not the /graphs/ that are causing the problem here, because Blazegraph can handle either of them by themself and give the right answer.

On 24/10/2015 00:50, Stas Malyshev wrote: It should be a basic requirement of any SPARQL engine that it should be able to handle path queries that contain cycles. For example, consider equivalence relationships like P460 "said to be the same as", which is being used to link given names together. If we want to find all the names in a particular equivalence class, and eg rank them by their incidence count, as is done in the 'query' columns at https://www.wikidata.org/wiki/Wikidata:WikiProject_Names/given-name_variants then being able to handle cycles in path queries is a basic requirement for the job. -- James. _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata

On 24.10.2015 12:29, Martynas Jusevičius wrote: We were talking about *cyclic data* not cyclic queries (which you can also create easily using BGPs, but that's unrelated here). Apparently, BlazeGraph has performance issues when computing a path expression over a cyclic graph. Markus

On 25.10.2015 02:18, Kingsley Idehen wrote: You mean a query with BlazeGraph having performance issues? That problem was reported by Stas. He should have examples. In any case, it is always a combination of query and data.

On 25.10.2015 02:18, Kingsley Idehen wrote: You mean a query with BlazeGraph having performance issues? That problem was reported by Stas. He should have examples. In any case, it is always a combination of query and data. Markus

It should be a basic requirement of any SPARQL engine that it should be able to handle path queries that contain cycles.

Blazegraph for example looks like does not

Hi! We can do it, but I'd rather we didn't. The reason is that it would require engine that queries such data (e.g. SPARQL engine) to be comfortable with cycles in property paths (especially ones with + and *), and not every one is (Blazegraph for example looks like does not handle them out of the box). It can be dealt with, I assume, but why create trouble for ourselves? Right. So I think we need to mark properties that should not form cycles with https://www.wikidata.org/wiki/Q18647519 (asymmetric property) and have constraints checking scripts/bots find out such cases and alert about them. -- Stas Malyshev smalyshev(a)wikimedia.org _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata