Hi everybody,
I recently learned from Superset Upstream that the suggested way to query Druid is using the SQLAlchemy connector, not the Druid one (that is "deprecated"). I digged a little bit more about this and I found the following:
- Druid Analytics supports SQL querying on the brokers, so it was a matter of adding a new "database" called "Druid Analytics SQL" to Superset. - "Druid Analytics SQL" can be used for data querying and exploration in the SQL Lab (feel free to test it!). - Using SQLAlchemy should be more quick/convenient since we wouldn't need to define mapping for Druid datasources manually (need to better investigate this but I am reasonably sure).
Moving all the charts already in Superset to the new database/datasource is probably a ton of work, and we are not really in a hurry (the "deprecated" above means that they still support it but a bug to that code will be considered not a blocker), but what I'd ask it to start using it and see if there are pros/cons in your daily workflows.
I created a task to report thoughts/suggestions/bugs/etc.. to avoid spamming too many people: https://phabricator.wikimedia.org/T249681
Thanks!
Luca
Hi Folks,
A disclaimer on the previous message: While an SQL querying endpoint has been added to Druid, the computation-engine is still very specific and by no mean a general-SQL engine. You'll probably very quickly find the limits of querying Druid via SQL.
For a full SQL engine use Presto :)
Joseph
On Wed, Apr 8, 2020 at 8:08 AM Luca Toscano ltoscano@wikimedia.org wrote:
Hi everybody,
I recently learned from Superset Upstream that the suggested way to query Druid is using the SQLAlchemy connector, not the Druid one (that is "deprecated"). I digged a little bit more about this and I found the following:
- Druid Analytics supports SQL querying on the brokers, so it was a matter
of adding a new "database" called "Druid Analytics SQL" to Superset.
- "Druid Analytics SQL" can be used for data querying and exploration in
the SQL Lab (feel free to test it!).
- Using SQLAlchemy should be more quick/convenient since we wouldn't need
to define mapping for Druid datasources manually (need to better investigate this but I am reasonably sure).
Moving all the charts already in Superset to the new database/datasource is probably a ton of work, and we are not really in a hurry (the "deprecated" above means that they still support it but a bug to that code will be considered not a blocker), but what I'd ask it to start using it and see if there are pros/cons in your daily workflows.
I created a task to report thoughts/suggestions/bugs/etc.. to avoid spamming too many people: https://phabricator.wikimedia.org/T249681
Thanks!
Luca _______________________________________________ Analytics mailing list Analytics@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/analytics