Felix Möller on 04 Dec 2020 12:54:53
Currently the databricks connector does not allow to see the folded query. This is a major drawback when trying to understand what queries are sent to databricks.
Recently the same support was added to Analysis Services Connector (see https://blog.crossjoin.co.uk/2020/10/04/view-native-query-now-works-for-analysis-services-data-sources/ )
So please add it to Databricks Connector as well.
- Comments (2)
RE: View Native Query for Databricks Connector
Here is a workaround to check for query folding in Databricks. Basically, you can view the query history in Databricks and see if the query generated by Power Query looks like it folded.https://medium.com/@harmonybi/power-query-query-folding-with-databricks-d51dc8c6029b
RE: View Native Query for Databricks Connector
This would be a good feature to have.
Native Query allows query folding and therefore incremental refresh.
Workaround is to create odbc connection to databricks sql endpoint and modifying the query in Power Query. However this then requires the query to route through the data gateway which is not ideal.
Best method at the moment is to have the databricks ingestion populate data into Azure SQL and therefore allow native query.