On 08.09.2015 16:40, James Heald wrote:
It may not be time to retire WDQ quite just yet...
Further to what Markus wrote earlier, it does seem that there are still some queries that are a *lot* faster on WDQ than on this initial release of WQS.
For example, as described on Project Chat here,
https://www.wikidata.org/wiki/Wikidata:Project_chat#Wikidata_Query_Service I found that a query to count the number of classes in a particular class tree took 30 times longer to execute on the SPARQL service than with WDQ.
As a result, a lot of queries -- particularly ones which involve recursive extraction as performed by eg the TREE command in WDQ -- look as if they may very easily run in to the 30-second time limit for queries on the new SPARQL service. (For example, I wasn't able to count the items that were instances of classes in the above class tree using SPARQL).
Yes, path queries (called TREE queries in WDQ) are usually faster in WDQ. I think that WDQ is better optimised for this type of queries. This is also what I had in mind with what I wrote: if you narrow down your query language to specific use cases and (possibly) a subset of the data, then you may be able to achieve a better performance in return. There is always a trade-off there. SPARQL is rather complex (if you look at the query examples page, you get an idea of the possibilities), but there is a price to pay for this. I still hope that path queries in particular can be made faster in the future (it still is a rather recent SPARQL feature and I am sure BlazeGraph are continuously working on improving their code).
Markus
But this isn't to take away from a fabulous achievement by the team getting this fantastic new service up and running -- a service which will presumably only get quicker and quicker as it gets more and more optimised and more and more scaled up. So a very great thanks to all concerned!
All best,
James.On 08/09/2015 11:03, Magnus Manske wrote:
FWIW, I'd be happy to turn WDQ into a "redirect" that takes WDQ commands, uses the WDQ-to-SPARQL translator, runs SPARQL, and returns the results in WDQ format. THis would be for backwards compatibility only. I hope people will start using SPARQL instead of WDQ where possible, and I will add SPARQL options to my tools where appropriate over time.
As it is, some queries (e.g. large result sets like "all humans without image") do not work on SPARQL yet. I will try to keep WDQ running until all WDQ queries can run as SPARQL as well.
On Tue, Sep 8, 2015 at 9:27 AM Markus Krötzsch < markus@semantic-mediawiki.org> wrote:
Great, congratulations to getting this deployed! Now we can start developing downstream applications :-)
Markus
P.S. @Gerard: it is not meant to move Magnus's query tools to SPARQL. He could of course do this, if he thinks it is easier for him regarding future maintenance/feature extensions. But since his tools are working well, there is no reason to change the underlying technology. In the end, anything that works is fine. Having other solutions for specific query features can always be a good idea. SPARQL supports very powerful queries (like "find all statements with references based on 'Le Figaro'"). More specialised query services that do not support all of this can achieve higher performance. This is why more specialised solutions such as WDQ can also have advantages. Moreover, no matter which query service is used in the backend, it always makes sense to develop more user interfaces that simplify query construction.
On 08.09.2015 00:29, Dan Garry wrote:
The Discovery Department at the Wikimedia Foundation is pleased to announce the release of the Wikidata Query Service https://www.mediawiki.org/wiki/Wikidata_query_service! You can find the interface for the service at https://query.wikidata.org.
The Wikidata Query Service is designed to let users run queries on the data contained in Wikidata. The service uses SPARQL https://en.wikipedia.org/wiki/SPARQL as the query language. You can see some example queries in the user manual https://www.mediawiki.org/wiki/Wikidata_query_service/User_Manual.
Right now, the service is still in beta. This means that our goal <
https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q2_Goals#Wikida...
is
to monitor of the service usage and collect feedback about what people think should be next. To do that, we've created the Wikidata Query Service dashboard https://searchdata.wmflabs.org/wdqs/ to track usage of the service, and we're in the process https://phabricator.wikimedia.org/T111403 of setting up a feedback mechanism for users of the service. Once we've got monitored the usage of the service for a while and got user feedback, we'll decide on what's next for development of the service.
If you have any feedback, suggestions, or comments, please do send an email to the Discovery Department's public mailing list, wikimedia-search@lists.wikimedia.org mailto:wikimedia-search@lists.wikimedia.org.
Thanks, Dan
-- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation
Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata