On Sat, Dec 10, 2016 at 5:30 PM, MZMcBride <z(a)mzmcbride.com> wrote: Not directly, but there are bots which can emulate it, such as Listeria by Magnus: http://magnusmanske.de/wordpress/?p=301 _______________________________________________ Wikitech-l mailing list Wikitech-l(a)lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Hi! So this is kind of can of worms which we I guess eventually have to open, but very carefully. So I want to state my _current_ opinion on the matters - please note, it can change at any time due to changing circumstances, persuasion, experience, revelation, etc. 1. Technically, anything that can access a web-service and speak JSON, can talk to SPARQL server. So, in theory, making some way to do this, *in theory*, would not be very hard. But - please keep reading. 2. I am very apprehensive about having direct link between any wiki pages and SPARQL server without heavy caching and rate limiting in between. We don't have super-strong setup there and I'm afraid making such link would just knock our setup over, especially if people start putting queries into frequently-used templates. 3. We have number of bot setups (Listeria etc.) which can auto-update lists from SPARQL periodically. This works reasonably well (excepting occasional timeout on tricky queries, etc.) and does not require requesting the info too frequently. 4. If we want more direct page-to-SPARQL-to-page interface, we need to think about storing/caching data, and not for 5 minutes like it's cached now but for much longer time, probably in storage other than varnish. Ideally, that storage would be more of a persistent store than a cache - i.e. it would always (or nearly always) be available but periodically updated. Kind of like bots mentioned above but more generic. I don't have any more design for it beyond that but that's I think the direction we should be looking into. No, and there are tricky parts there. Consider https://www.wikidata.org/wiki/Q735712. Yes, Lex Luthor held the office of the President of the USA. In a fictional universe, of course. But the naive query - every Wikidata item where position held includes "President of the United States" - would return Lex Luthor as the president as legitimate as Abraham Lincoln. In fact, there are 79 US presidents judging by "position held" alone. So clearly, there need to be some limits. And those limits would be on case-by-case basis. Right now the best way is use one of the list-maintaining bots I think. Unless you're talking about pulling a small set of values, in which case Lua/templates are probably the best venue. We're working on it (mostly thinking right now, but correct design is 80% of the work, so...). Visualizations already have query capabilities (mainly because they have strong caching model embedded and because there are not too many of them and you need to create them so we can watch the load carefully). Other pages can gain them - probably via some kind of Lua functionality - as soon as we figure out what's the right way to do it, hopefully somewhere within the next year (no promise, but hopefully).

Sure, but I'm not really worried about potential false positives. I'm worried that we're building a giant write-only data store.

I'm not sure what small means here. We have about 46 U.S. Presidents, is that small enough? Which Lua functions and templates could I use?

Wikidata began in October 2012. I thought it might take till 2014 or even 2015 to get querying capability into a usable state, but we're now looking

to even contribute to it when it feels like putting data into a giant system that you can't really get back out. I love Magnus and I have a ton

Actually, specifically for list of presidents you don't need bot.

3. It is limited to simple lists (you can't have list of Republican presidents - because it requires additional filters and you don't want to create new property for it)

4. Internationalization - What if yi Wikipedia wants to create list of governors of some small country where there are no yi labels for the presidents? The list would be partially in yi partially in en - is this desired behavior? or they can show only presidents who have label in yi - but this would give partial data - is this the desired behavior? [Probably the correct solution is to do show the fallback labels in en, but add some tracking category for pages requires label translation or [translate me] links)