Hi Ben, hi Max, hi Wikidata-istas (another one for the list), thanks for the support. Good question, and an important one too! I did not want to expand the proposal much more, so there is only one little paragraph I added about upcoming Wikidata query functionality ("Wikibase query services" under "Tools, technologies, and techniques"). Here I will expand a bit more on why these things are quite different. Question: What is the difference between the planned Wikidata query service and the proposed Wikidata Toolkit? Answer: First, let me assure you that a Wikidata query service is coming. This has been an important project for a while now and a lot of work in that direction is under way. My proposal neither replaces nor prepares or supersedes this; they are just two different things. The proposed toolkit is meant to support developers who want to work on the data. A basic requirement to do this will be to run some forms of queries, so this should somehow be supported. However, it is not clear yet what form these queries should take. I am definitely interested in features that resemble the "tree" feature in Magnus's Wikidata Query (a kind of transitive closure over properties). This is something that is not currently planned for Wikidata, and indeed it requires a type of recursion that is best implemented in memory (like Magnus does) but hard to get done efficiently if you delegate queries to MySQL (as Wikidata does). I am also interested in constraints and in rules (again, Magnus has some precedence for this in his Reasonator and bot proposals). This, too, will require some form of recursion which is very hard to realise on a relational DB (I tried it; others tried it; it just does not get anywhere near the performance of in-memory systems, even on large data, whether you use MySQL or Oracle). Running some operation on data recursively is quite natural when you are in a programming environment and your data is in a data structure. I would like the Wikidata Toolkit to support this way of handling data. This is very far from a (complex or easy) query language that captures all possible queries in one general format. Of course there are query languages that support regular expressions on binary relations (so-called path queries), and there are also query languages with recursion (e.g., Datalog), but my goal is not to create a service for one such language. One could do that on top of the Toolkit for more than one query language. Besides all this querying, there are also some other tasks that are hard to phrase as a query but are rather a kind of computational analysis. You might want to do this in a programming language, or you might just want to use the Toolkit to export a large matrix file that you can feed into R or Matlab. Again, this is not something you would get from the query service, even though you might need queries to get the data you need. Now compared to this, the Wikidata plans are much more focussed and (for this particular task) much more advanced than my proposal (which proposes to start the work by working out what to do, see Task T1 ;-). The plans for Wikidata are based on a language in the style of SMW's #ask, i.e., a language where you have neither JOINs nor variables explicitly -- instead, JOINS are implicit and "tree-shaped", i.e., there are no cyclic relationships. A simple example of a query that is not tree-shaped is "which people who were born in the same town that they died in?" another is "which people are the child of married parents?". Neither of these can be asked in SMW. There are many other features where query languages can differ. It is clear to me that the requirements should not all be satisfied by a single Wikidata query service -- that would probably lead to a rather bloated and inefficient service, too. Instead, Wikidata will focus on the most important Wikipedia-based use cases first. The Toolkit should be "compatible" with Wikidata's query support (maybe even have a representation for Wikibase query objects), but it should also allow to explore other query types. I hope this clarifies a bit the differences between Wikibase's upcoming query web service and the Toolkit. Both activities should still benefit one another: the Toolkit will be a good basis for exploring new query features and implementation approaches; the web service will be a convenient way to access live data (even our little Wikidata Analytics script already accesses the Wikidata Web API when it needs data that would take too long to seek inside a huge dump). So, summing up, all will be good. :-) Cheers, Markus On 30/09/13 21:43, Benjamin Good wrote: