Skip to main content
Microsoft Idea

Synapse

New

PIPELINE::COPY DATA::JSON OUTPUT LOSING KEY:VALUE PAIR

Vote (1) Share
Alex Blake's profile image

Alex Blake on 25 Apr 2024 19:51:39

If rowsCopied <> 0, then the key rowsCopied will be in the JSON output of the Copy data activity.

If rowsCopied == 0, then the key rowsCopied will NOT be in the JSON output of the Copy data activity.


Why is this an issue? Because if downstream you use a Teams activity to generate a logging message that prints how many rows have been copied, 0 or more, you end up with a pipeline run fail because when rowsCopied cannot be referenced anymore in the Expression builder when 0 rows were copied as this key is not included in the JSON output when the value is 0.


Yes, the user could create a boiler plate IF..THEN check with new branching to overcome this. But WHY should the user have to add additional complexity to a pipeline for something this trivial when it is already half baked in? (rowCopied is in the JSON output when it has a value > 0).


All that needs to be done is leave the rowsCopied key in the JSON output no matter what the value. That way you can just reference this key without the need for extra boiler plate overhead in your expression code with more IF..THEN checks. It's already clunky enough as it is, no need to make it more so is what I am saying.


Here's the link to the forum post where the 2 JSON outputs are shown.


https://community.fabric.microsoft.com/t5/General-Discussion/BUG-PIPELINE-EXPRESSION-BUILDER-EDITS-NOT-SAVED/m-p/3852282/highlight/true#M6643