Incremental Data Loads
Is it possible to add incremental load functionality to Power Query? Pull and load just new or changed rows from source system
We’re pleased to announce that in the May release of Power BI Desktop you can now define incremental refresh policies for your models.
At present these only apply once the model is published to a Premium workspace in the Power BI service; we’ll be bringing it to all Pro users in the future.
You can read more about it in our blog here: https://powerbi.microsoft.com/en-us/blog/power-bi-desktop-may-2018-feature-summary/#incrementalRefresh
We’re going to leave this idea open for now, so please give us your feedback while we’re in this preview phase!
Use Incremental Refresh without having premium: https://biztics.com/power-bi-real-incremental-refresh-maxim-zelensky/
Jeffrey Brady commented
The issue that i am having is that incremental refresh does not work well for sources that do not support query folding leaving me stuck.
Mike Seisbye commented
Very excited about the incremental load option. But if I feel the direction of the setup is wrong. Is it then a reply here, or a new idea?
The implemented solution are not very smart. you set up at date interval from where you want data. and also which date to "refresh". This seems like a very basic approach, leaving out all the "needed" solutions people really use when they setup up incremental loads in DWH.
Most people would not setup on date. Lets assume you look into an ERP system and the General ledger. If you use the posting date, you would need to refresh ALL dates, leaving the incremental almost useless. Since accounting departments sometimes need to backdate changes etc. why most erp systems have either timestamps or simular to flag changes. So one could simple note the timestamp at the lastest load, and then simply collect data with a higher timestamp. in Dynamics products(microsoft) it's in almost all tables. Making it possible to even to incremental on big dimension tables.
Pretty sad it's only for Premium, and super annoying that o365 Developer Subscriptions aren't Pro or Premium...that would be REALLY helpful for building an ISV solution
Hi there is NOTHING planned regarding incremental loads for PBI Pro: https://docs.microsoft.com/en-us/power-platform-release-plan/2019wave2/business-intelligence/planned-features
need that functions. please !!
Rafael Lucredio commented
Please make it to us Pro users!!
you need to merge this one with this and deliver it to the community..it does not make sense to offer it to premium only and ignore more than 8,000 users https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/34198180-incremental-refresh-for-power-bi-pro
pleaseeeeeee!!!!!! taking into account that this threat was opened in 2015, now it's supose to be 'future'
Power Bi Pro needs this feature soon!
Also please add this functionality for SAP BW On-Premise Connector.
Lazar Jelicic commented
First off thanks for keeping this open and acknowledging the need for this in Pro. Previously too many ideas have been closed by admins when only a part of the requested functionality has been implemented.
Any update on Pro timescales. We are already seeing fairly small SQL Server datasets taking 15 minutes for a gateway refresh?
This is so important it should be part of the Power BI roadmap, with a planned date.
Power Bi Pro needs this feature soon!
Kabir Laiwalla commented
Waiting to get this feature in PowerBI Pro. I am loading a table that has 10 million rows and everytime i make change on desktop and refresh, it loads all 10 million rows although I have added a filter to only load rows that are for fiscal year 2018 & 2019, but it goes through the entire data set of 10 million rows and takes 4-5 hours.
Diego Lopez commented
Needs to be at least Pro, too
Cristian O. commented
Really needing this in Pro...
This is July 2nd 2019, and we are still waiting for incremental data load on Pro.
Please add in Pro