I’d like to provide a bit more background and summarize a bit our work on the new WDQS updater, from the technical perspective.

It has been common knowledge that the old updater had its issues. Main ones among them:

We’ll be publishing a series of blog posts that will provide a more in-depth description of the architecture and the challenges during development - stay tuned!

In the meantime, I want to explain a few things that about the new updater:

The new Streaming Updater didn’t resolve all the issues magically and there are still two main ones, that we need to address:

I hope that answers at least some of the concerns already raised. Rest assured that we are working on way more things to improve the experience than the updater, all of which is, as always, available to see on our backlog board ([7]) and workboard ([8]).

Any and all feedback welcome!



[1] https://grafana.wikimedia.org/d/000000489/wikidata-query-service?viewPanel=8&orgId=1&from=1628010995348&to=1628065289312

[2] https://grafana-rw.wikimedia.org/d/yCBd7Tdnk/wdqs-lag-slo

[3] https://phabricator.wikimedia.org/T272120

[4] https://phabricator.wikimedia.org/T291609

[5] https://phabricator.wikimedia.org/T279541

[6] https://phabricator.wikimedia.org/T293862

[7] https://phabricator.wikimedia.org/tag/wikidata-query-service/

[8] https://phabricator.wikimedia.org/project/view/1227/


Zbyszko Papierski (He/Him)

Senior Software Engineer

Wikimedia Foundation