Hello all!
I hope you are all doing well in these interesting times! We are doing our
best to continue moving forward, but there is no doubt that the COVID-19
situation is affecting our work. Unexpected schedules, kids homeschooling,
loved ones and family affected, or just the overall stress level claiming
some of our brain capacity.
That being said, we are moving forward with our new updater [1]. We almost
have a first working implementation of the simplest use case: creating a
turtle stream from revision create events. More work is needed to include
more complex use cases (visibility changes, renames, ...), but we are
focusing on testing the non functional requirements and making sure this
solution is robust. We are working on metrics, monitoring, validating
latencies.
We've been mostly focused on this scaling work, but we know we also need to
address some of the shorter term issues as well. We are trying to dedicate
some time to this. In particular:
* fixing a bug about categories reloading [2]
<https://phabricator.wikimedia.org/T246568>
* update to wikipathways federation endpoint (thanks to RhinosF1 for doing
the actual work! we'll just do the deployment next Monday) [3]
We are hoping to be able to address more of those day to day operations
going forward, but we are stretched thin at the moment.
Keep safe!
Guillaume
[1]
https://phabricator.wikimedia.org/T244590
[2]
https://phabricator.wikimedia.org/T246568
[3]
https://phabricator.wikimedia.org/T249041
--
Guillaume Lederrey
Engineering Manager, Search Platform
Wikimedia Foundation
UTC+1 / CET