Hi!
- Move the service to labs, not providing any firm guarantee of service
level ?
I don't see any reason to move it to labs, and I don't think labs has infrastructure to handle it. It can not run on virtual machines - in fact, the only reason why it can't be as stable as we wish is because we don't have hardware to run fully redundant configuration with capacity to serve all cases. I don't see how using labs would help there - does labs have more hardware capacity than production that is available for wdqs use?
I think there was one exception, which is services that needed a lot of resources so they could not run on vms, but don't we have a prototype of "labs on real hardware"?
I really wouldn't want to run this service on prototype that wasn't tried before and introduce both additional point of failure and unnecessary administration burden. I fail to see any existing issue that this would solve, but it certainly has potential to introduce some.
that you are describing- running easily out of resources (DOS). Even quarry, which I have publicly complained about in the past, for what you say, has a better resource management than wqs (30-minute limit execution, concurrency control, etc.).
We have 30-second limit now. People complain about it all the time because it's too short, but see above about the hardware.
icinga. But running an unstable service (wdqs) on top of another unstable service (wikidata data handling) will never be stable. Everytime a bot starts writing to wikidata 600 times per second, s5 dbs shake (that is why we are creating s8) and wqs goes down. :-) I would suggest using wqs on labs (or anywhere, non-production) with regular imports rather than real-time updates. Less headaches. I am literally aiming for that for labsdbs, too.
I don't think this is a good scenario. Delayed updates means severely degraded user experience and won't save much performance as the data needs to be moved over anyway. We could save something if we had proper change tracking service that doesn't use recentchages API directly on wikidata (so we get several uncached rc API calls per each edit) but has some faster and more efficient intermediary, but AFAIK we don't have that still. That would be a direction to look if updating is too resource-consuming.