[Wikidata] Make federated queries possible / was: SPARQL CONSTRUCT results truncated

From Stas' answer to https://phabricator.wikimedia.org/T127070 I learned the Wikidata Query Service does not "allow external federated queries ... for security reasons (it's basically open proxy)." Now, obviously endpoints referenced in a federated query via a service clause have to be open - so any attacker could send his queries directly instead of squeezing them through some other endpoint. The only scenario I can think of is that an attackers IP already is blocked by the attacked site. If (instead of much more common ways to fake an IP) the attacker would choose to do it by federated queries through WDQS, this _could_ result in WDQS being blocked by this endpoint. This is a quite unlikely scenario - in the last 7 years I'm on SPARQL mailing lists I cannot remember this kind of attack of ever having been reported - but of cause it is legitimate to secure production environments against any conceivable attack vector. However, I think it should be possible to query Wikidata with this kind of query. Federated SPARQL queries are a basic building block for Linked Open Data, and blocking it breaks many uses Wikidata could provide for the linked data cloud. This must not involve the highly-protected production environment, but could be solved by an additional unstable/experimental endpoint under another address. As an additional illustrating argument: There is an immense difference between referencing something in a service clause and getting a result in a few seconds, or having to use the Wikidata toolkit. To get the initial query for this thread answered by the example program Markus kindly provided at https://github.com/Wikidata/Wikidata-Toolkit-Examples/blob/master/src/examp… (and which worked perfectly - thanks again!), it took me - more than five hours to download the dataset (in my work environment wired to the DFN network) - 20 min to execute the query - considerable time to fiddle with the Java code for the query if I had to adapt it (+ another 20 min to execute it again) For many parts of the world, or even for users in Germany with a slow DSL connection, the first point alone would prohibit any use. And even with a good internet connection, a new or occasional user would quite probably turn away when offered this procedure instead of getting a "normal" LOD conformant query answered in a few seconds. Again, I very much value your work and your determination to set up a service with very high availability and performance. Please, make the great Wikidata LOD available in less demanding settings, too. It should be possible for users to do more advanced SPARQL queries for LOD uses in an environment where you can not guarantee a high level of reliability. Cheers, Joachim -----Ursprüngliche Nachricht----- Von: Wikidata [mailto:wikidata-bounces@lists.wikimedia.org] Im Auftrag von Neubert, Joachim Gesendet: Dienstag, 16. Februar 2016 15:48 An: 'Discussion list for the Wikidata project.' Betreff: Re: [Wikidata] SPARQL CONSTRUCT results truncated Thanks Markus, I've created https://phabricator.wikimedia.org/T127070 with the details. -----Ursprüngliche Nachricht----- Von: Wikidata [mailto:wikidata-bounces@lists.wikimedia.org] Im Auftrag von Markus Krötzsch Gesendet: Dienstag, 16. Februar 2016 14:57 An: Discussion list for the Wikidata project. Betreff: Re: [Wikidata] SPARQL CONSTRUCT results truncated Hi Joachim, I think SERVICE queries should be working, but maybe Stas knows more about this. Even if they are disabled, this should not result in some message rather than in a NullPointerException. Looks like a bug. Markus On 16.02.2016 13:56, Neubert, Joachim wrote: _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata _______________________________________________ Wikidata mailing list Wikidata(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata