Hello all!

I hope you are all doing well in these interesting times! We are doing our best to continue moving forward, but there is no doubt that the COVID-19 situation is affecting our work. Unexpected schedules, kids homeschooling, loved ones and family affected, or just the overall stress level claiming some of our brain capacity.

That being said, we are moving forward with our new updater [1]. We almost have a first working implementation of the simplest use case: creating a turtle stream from revision create events. More work is needed to include more complex use cases (visibility changes, renames, ...), but we are focusing on testing the non functional requirements and making sure this solution is robust. We are working on metrics, monitoring, validating latencies.

We've been mostly focused on this scaling work, but we know we also need to address some of the shorter term issues as well. We are trying to dedicate some time to this. In particular:

* fixing a bug about categories reloading [2]
* update to wikipathways federation endpoint (thanks to RhinosF1 for doing the actual work! we'll just do the deployment next Monday) [3]

We are hoping to be able to address more of those day to day operations going forward, but we are stretched thin at the moment.

Keep safe!


[1] https://phabricator.wikimedia.org/T244590
[2] https://phabricator.wikimedia.org/T246568
[3] https://phabricator.wikimedia.org/T249041
Guillaume Lederrey
Engineering Manager, Search Platform
Wikimedia Foundation