Skip to main content
Microsoft Idea

Power BI

Needs Votes

Incremental Refresh using custom date format

Vote (14) Share
Ahmad Anuar's profile image

Ahmad Anuar on 15 Nov 2018 12:01:28

Please enable custom date format in incremental refresh of dataflow just like in Power BI desktop features.

Comments (1)
Ahmad Anuar's profile image Profile Picture

Pär Adeen on 11 Aug 2020 16:28:50

RE: Incremental Refresh using custom date format

To bad I can't give more than one vote for this!

The support for incremental refresh using PBI Dataflows are just horrible. If you have a datetime column in your data source it's ok, but if you just have a DW date key it's horrible. In my case I have a YYYYMM key in the data source, but it's like as bad if you have a YYYYMMDD key. The problem is to re-format the YYYYMMDD key to datetime while keeping query folding.
The workaround is to first create some fake datetime column, maybe using:
Table.AddColumn(source, "Fake", each DateTime.FixedLocalNow(), type datetime)
This will make it possible to configure incremental refresh.
Once this is done you can edit the entity and re-write the automatically populated filter, in my case I used the recommended method described here: https://docs.microsoft.com/en-us/power-bi/admin/service-premium-incremental-refresh#rangestart-and-rangeend-parameters.
So what I did was to re-write the automated filter from:
Table.SelectRows(#"Added Fake date column", each DateTime.From([Fake]) >= RangeStart and DateTime.From([Fake]) < RangeEnd)
To:
Table.SelectRows(#"Added Fake date column", each [DateKey] >= dateToYYYYMM(RangeStart) and [DateKey] < dateToYYYYMM(RangeEnd))
Where dateToYYYYMM = (x as datetime) => Date.Year(x)*10000 + Date.Year(x)*100

But doint this, everytime I update the incremental refresh parameters, I need to re-write the filter.

It looks to me that MS has done a pretty good implementation for datasets, but totally forgotten to do the same for Power BI Dataflows