Skip to main content
Microsoft Idea

Data Factory

Needs Votes

Highlight in color the UI part where a M query has staging enabled but shouldn't--here's why.

Vote (2) Share
Alex Blake's profile image

Alex Blake on 06 Mar 2024 03:47:57

I just ran into an issue that reminded me that there are so many moving parts, so many details to keep in mind when using a dataflow gen2, let alone the full suite of Fabric artifacts, that to say the cognitive load on short term memory is great would be the understatement of the year.


Here is an example: In Power Query Dataflow Gen2, I just found out that if a query returns a single numerical value but at the same time has staging enabled, and is referenced in another query in a comparison op, it ends up blowing up the whole DFg2 refresh.


So, when a query has staging disabled, the only visual indication in the GUI is that its name is in italics. My beef is that if you have a huge screen, high res, the font change might not be immediately visible, or there is a great chance of not paying attention to it because it does not stand out much, which I miss all the time. Also, when you wear glasses, you field of vision is constrained by the lenses and you don't have a wide of an angle as when you had 20/20 vision, which compounds the issue.


As I was musing over this, and over why I had the issue above, I thought it might be a good thing to have a stronger visual indicator around query names that should NOT have staging enabled, like in my example above.


So I suggest that the PQ editor does an analysis of the queries and if it finds one or more queries that return only a single value and are not meant to be translated into tables, then to highlight the names of these queries, say with a strong background color behind the query name and not let the user publish the DF until staging is disabled for such queries. Of course, appropriate visual feedback should be given so that the user can understand why she can't suddenly publish.