Hi everybody,
as FYI I am going to upgrade Superset tomorrow (May 15th) to 0.32. This will involve moving to a new host based on Debian Buster and Python 3.7, so the move will require some time and it will be hopefully fully done early during the EU morning.
Tracking task: https://phabricator.wikimedia.org/T211706
Luca (on behalf of the Analytics team)
Hello,
Superset is now been upgraded, there are notable fixes on this version and now you can go crazy creating histograms cause they actually work.
An example: histogram of response sizes as reported by varnish last week: https://bit.ly/2vYB966
Also, there is a new dataset available called edit_hourly that is the edit equivalent of pageview_hourly.
Examples of graphs on top of this data:
Pages created per project last month (in content namespaces): https://bit.ly/2LUdBua
Edits per platform in Indonesia and Arabic Wikipedia last month: https://bit.ly/2JlKNIG
Thanks,
Nuria
On Tue, May 14, 2019 at 10:57 AM Luca Toscano ltoscano@wikimedia.org wrote:
Hi everybody,
as FYI I am going to upgrade Superset tomorrow (May 15th) to 0.32. This will involve moving to a new host based on Debian Buster and Python 3.7, so the move will require some time and it will be hopefully fully done early during the EU morning.
Tracking task: https://phabricator.wikimedia.org/T211706
Luca (on behalf of the Analytics team) _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Hi everybody,
I want to add some info about why it took so long to deploy this new version. The Superset upstream project is currently into the Apache incubator (https://incubator.apache.org/), that requires projects to be in line with the Apache foundation guidelines. The Superset dev struggled a lot to release the first Apache-licensed version of Superset, and we preferred not to deploy anything to our infrastructure that was in the process of changing license (we had a bad experience in the past). Moreover, we have filed several issues and pull requests to github, some of them still not merged (few upstream devs, a lot of bugs). If you want a complete list, please check https://phabricator.wikimedia.org/T211706, I'll keep the summary up to date.
We also created a staging environment for Superset, so that we will be able in the future to test new versions (and file bug reports in advance) without affecting the version currently deployed in production. We are currently using our own "patched" version of Superset ( https://github.com/wikimedia/incubator-superset/tree/wikimedia) but ideally in the future it would be great to use the official releases without any backport. We are not there yet :)
Last but not the least, we are aware of the annoying bug upon first user login (a horrible python stacktrace) that requires a manual fix: it should be fixed upstream now, but the change is big to backport at the moment (since it is located into a separate package, FlaskAppBuilder, that requires a major version bump) so we preferred not to delay the deployment of 0.32 further. We hope to fix it in the next release :)
If you guys encounter problems or bugs please report them to us, so we'll be able to follow up with upstream!
Thanks,
Luca
Il giorno mer 15 mag 2019 alle ore 21:17 Nuria Ruiz nuria@wikimedia.org ha scritto:
Hello,
Superset is now been upgraded, there are notable fixes on this version and now you can go crazy creating histograms cause they actually work.
An example: histogram of response sizes as reported by varnish last week: https://bit.ly/2vYB966
Also, there is a new dataset available called edit_hourly that is the edit equivalent of pageview_hourly.
Examples of graphs on top of this data:
Pages created per project last month (in content namespaces): https://bit.ly/2LUdBua
Edits per platform in Indonesia and Arabic Wikipedia last month: https://bit.ly/2JlKNIG
Thanks,
Nuria
On Tue, May 14, 2019 at 10:57 AM Luca Toscano ltoscano@wikimedia.org wrote:
Hi everybody,
as FYI I am going to upgrade Superset tomorrow (May 15th) to 0.32. This will involve moving to a new host based on Debian Buster and Python 3.7, so the move will require some time and it will be hopefully fully done early during the EU morning.
Tracking task: https://phabricator.wikimedia.org/T211706
Luca (on behalf of the Analytics team) _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Thank you so much, Luca and Nuria, for updating Superset and providing these details! Superset is noticeably easier to use!!
Warm regards, Kate
--
Kate Zimmerman (she/they) Head of Product Analytics Wikimedia Foundation
On Fri, May 17, 2019 at 7:47 AM Luca Toscano ltoscano@wikimedia.org wrote:
Hi everybody,
I want to add some info about why it took so long to deploy this new version. The Superset upstream project is currently into the Apache incubator (https://incubator.apache.org/), that requires projects to be in line with the Apache foundation guidelines. The Superset dev struggled a lot to release the first Apache-licensed version of Superset, and we preferred not to deploy anything to our infrastructure that was in the process of changing license (we had a bad experience in the past). Moreover, we have filed several issues and pull requests to github, some of them still not merged (few upstream devs, a lot of bugs). If you want a complete list, please check https://phabricator.wikimedia.org/T211706, I'll keep the summary up to date.
We also created a staging environment for Superset, so that we will be able in the future to test new versions (and file bug reports in advance) without affecting the version currently deployed in production. We are currently using our own "patched" version of Superset ( https://github.com/wikimedia/incubator-superset/tree/wikimedia) but ideally in the future it would be great to use the official releases without any backport. We are not there yet :)
Last but not the least, we are aware of the annoying bug upon first user login (a horrible python stacktrace) that requires a manual fix: it should be fixed upstream now, but the change is big to backport at the moment (since it is located into a separate package, FlaskAppBuilder, that requires a major version bump) so we preferred not to delay the deployment of 0.32 further. We hope to fix it in the next release :)
If you guys encounter problems or bugs please report them to us, so we'll be able to follow up with upstream!
Thanks,
Luca
Il giorno mer 15 mag 2019 alle ore 21:17 Nuria Ruiz nuria@wikimedia.org ha scritto:
Hello,
Superset is now been upgraded, there are notable fixes on this version and now you can go crazy creating histograms cause they actually work.
An example: histogram of response sizes as reported by varnish last week: https://bit.ly/2vYB966
Also, there is a new dataset available called edit_hourly that is the edit equivalent of pageview_hourly.
Examples of graphs on top of this data:
Pages created per project last month (in content namespaces): https://bit.ly/2LUdBua
Edits per platform in Indonesia and Arabic Wikipedia last month: https://bit.ly/2JlKNIG
Thanks,
Nuria
On Tue, May 14, 2019 at 10:57 AM Luca Toscano ltoscano@wikimedia.org wrote:
Hi everybody,
as FYI I am going to upgrade Superset tomorrow (May 15th) to 0.32. This will involve moving to a new host based on Debian Buster and Python 3.7, so the move will require some time and it will be hopefully fully done early during the EU morning.
Tracking task: https://phabricator.wikimedia.org/T211706
Luca (on behalf of the Analytics team) _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics
Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics