Re: [discovery] [Ops] Using Wikidata Query Service in a production context - Discovery

13 Oct 2016

Hi!

...
  3) Move the service to labs, not providing any firm
guarantee of service
 level ? 
I don't see any reason to move it to labs, and I don't think labs has
infrastructure to handle it. It can not run on virtual machines - in
fact, the only reason why it can't be as stable as we wish is because we
don't have hardware to run fully redundant configuration with capacity
to serve all cases. I don't see how using labs would help there - does
labs have more hardware capacity than production that is available for
wdqs use?

...
  I think there was one exception, which is services
that needed a lot of
 resources so they could not run on vms, but don't we have a prototype of
 "labs on real hardware"? 
I really wouldn't want to run this service on prototype that wasn't
tried before and introduce both additional point of failure and
unnecessary administration burden. I fail to see any existing issue that
this would solve, but it certainly has potential to introduce some.

...
  that you are describing- running easily out of
resources (DOS). Even
 quarry, which I have publicly complained about in the past, for what you
 say, has a better resource management than wqs (30-minute limit
 execution, concurrency control, etc.). 
We have 30-second limit now. People complain about it all the time
because it's too short, but see above about the hardware.

...
  icinga. But running an unstable service (wdqs) on top
of another
 unstable service (wikidata data handling) will never be stable.
 Everytime a bot starts writing to wikidata 600 times per second, s5 dbs
 shake (that is why we are creating s8) and wqs goes down. :-)
 I would suggest using wqs on labs (or anywhere, non-production) with
 regular imports rather than real-time updates. Less headaches. I am
 literally aiming for that for labsdbs, too. 
I don't think this is a good scenario. Delayed updates means severely
degraded user experience and won't save much performance as the data
needs to be moved over anyway. We could save something if we had proper
change tracking service that doesn't use recentchages API directly on
wikidata (so we get several uncached rc API calls per each edit) but has
some faster and more efficient intermediary, but AFAIK we don't have
that still. That would be a direction to look if updating is too
resource-consuming.
-- 
Stas Malyshev
smalyshev(a)wikimedia.org