Hello,
As of now, https://superset-next.wikimedia.org is running on Kubernetes. We also introduced changes and new features as we migrated it over, which should lead to reduce manual maintenance as well as faster browsing.
- Logging now goes through our OIDC server (CAS) - This allows us to map your LDAP groups to Superset roles. Without going into too much detail, everyone should get Alpha and sql_lab permissions, which are automatically managed by Superset. Gone should be the days of hand-managing permissions. - We have enabled data caching (caching of charts and dashboard data) while making sure that the cached data is scoped per user. This means that if a given user cannot access some data from, say, HDFS, they won't be able to access the cached version of it either. This should lead to a faster browsing, as data computed via expensive queries should be fetched much faster the next time.
As the Data Platform SRE team will be in San Fransisco next week, we plan to perform the https://superset.wikimedia.org migration the week after (March 18th-22nd). If you could take the time to log into superset-next and make sure everything looks up to snuff, that would be very helpful to us.
Kind regards Balthazar
On Mon, Feb 26, 2024 at 11:57 AM Ben Tullis btullis@wikimedia.org wrote:
Hello (especially to Superset users),
As you may know, the Data Platform SRE team is currently working on migrating the Analytics Superset instances to Kubernetes (under ticket T347710 https://phabricator.wikimedia.org/T347710) and, happily, I can report that we are making good progress.
This is just a courtesy email to let you know that we plan to switch our staging instance (superset-next.wikimedia.org) to over to Kubernetes over the next day or two. This is unlikely to affect anyone's work at the moment, given that both the staging and production instances of Superset have been on version 3.1.0 for a while.
However, given that this staging instance is available for you to use at any time, we thought it best to let you know that we are currently working on it and that it may be in a state of flux for a while.
Once it is stable on Kubernetes, we may well contact you again and ask you kindly to test superset-next for us and report your findings. At the moment though, we're just working on the transition itself so there won't be much for you to test.
As ever, if you have any queries or concerns, please don't hesitate to let us know.
Kind regards, Ben -- *Ben Tullis* (he/him) Senior Site Reliability Engineer Wikimedia Foundation https://wikimediafoundation.org/