How can we improve Power BI?

Incremental Data Loads

Is it possible to add incremental load functionality to Power Query? Pull and load just new or changed rows from source system

3,273 votes
Sign in
Check!
(thinking…)
Reset
or sign in with
  • facebook
  • google
    Password icon
    Signed in as (Sign out)

    We’ll send you updates on this idea

    Javier Recasens shared this idea  ·   ·  Flag idea as inappropriate…  ·  Admin →

    We’re pleased to announce that in the May release of Power BI Desktop you can now define incremental refresh policies for your models.

    At present these only apply once the model is published to a Premium workspace in the Power BI service; we’ll be bringing it to all Pro users in the future.

    You can read more about it in our blog here: https://powerbi.microsoft.com/en-us/blog/power-bi-desktop-may-2018-feature-summary/#incrementalRefresh

    We’re going to leave this idea open for now, so please give us your feedback while we’re in this preview phase!

    367 comments

    Sign in
    Check!
    (thinking…)
    Reset
    or sign in with
    • facebook
    • google
      Password icon
      Signed in as (Sign out)
      Submitting...
      • Kevin H commented  ·   ·  Flag as inappropriate

        (a) There seems to be an issue where an incremental refresh policy disables ALL parameters in the Power BI Service:
        https://community.powerbi.com/t5/Service/Incremental-Refresh-and-report-parameters/m-p/525773

        (b) Sometimes I want my data model to do a full refresh, and ignore incremental refresh policies. E.g., when scheduling refreshes in the Service, for each scheduled time, could there please be an option for either incremental or full? And for dataset refreshes triggered by the Power BI API, could there please be an option to trigger an incremental or full refresh?

      • Anonymous commented  ·   ·  Flag as inappropriate

        It will be great to have incremental data load feature added to Power BI desktop for pro accounts soon. I have a query with historical data and another query with new data. Both these queries are appended into a new query and it is painful to watch million of rows from historical data refresh every day. Hope you add it soon!!!

      • Todd Chittenden commented  ·   ·  Flag as inappropriate

        Found a link to this idea from a forum post on strange behavior of "Include in Report Refresh".

        I'm trying to have an Excel connection in the model that is static data, so does NOT need to be refreshed by the service. But the Gateway requires info on the source file and will NOT do any refreshes until that is resolved.

        WTF!!! Disabling in Report Refresh should NOT be dependent on if you are using the Desktop or the Service. It is a BUG in my opinion. Or at least, poorly thought-out requirements.

      • Aalok commented  ·   ·  Flag as inappropriate

        Power Bi Premium is not an option for us right now. It would be good if incremental refresh feature for Odata is enabled in Powerbi Pro.

        Thanks.

      • Leigh commented  ·   ·  Flag as inappropriate

        Still looking for pro support. Seems like the shared capacity scenario is the one which would most benefit from incremental.
        -thanks
        -e

      • Usman commented  ·   ·  Flag as inappropriate

        Announced feature is different than incremental load, in incremental load we wanted to append new data to the original data. For example I connected to a website which updates on daily basis, I want to make database of that data to analyze it over time. And of course it should be automated and work in service.

      • Ryan R commented  ·   ·  Flag as inappropriate

        I am thankful that you will be sharing this with all pro users. I agree with the others, it would be nice to know when you think this will be available for Pro users?

      • Josh commented  ·   ·  Flag as inappropriate

        Are there any plans to add incremental refresh to the Power BI Report Server implementation?

      • Michael Bristol commented  ·   ·  Flag as inappropriate

        i am looking forward to this coming to Pro. Our organization is not big enough to justify Premium as of yet.

      • Hanka commented  ·   ·  Flag as inappropriate

        It would be fantastic to have this also in Power BI Desktop. Currently we have to use some not ideal DAX workarounds how to load new data only :(

      • Anonymous commented  ·   ·  Flag as inappropriate

        Have to agree with what Withold mentioned earlier. Also was my understanding that this idea was to solve the problem of taking snapshots of a database, that involves having to continually download excel files and dump them into a folder.

      • Preeti commented  ·   ·  Flag as inappropriate

        Please introduce this for Power BI Desktop to connect to SharePoint Lists for ODataFeed and SharePoint List Connectors.

      • Joe Champagne commented  ·   ·  Flag as inappropriate

        This is a great feature. Why can't you download a pbix file from the service once incremental refresh is on?

      • Andrew commented  ·   ·  Flag as inappropriate

        This is a great feature, however, for certain reference data sources that don't need a daily refresh or a refreshed schedule at all, it would be great to be able to designate a lesser frequency for the entire data source. An example might be - a refresh frequency setting - daily, monthly, quarterly?

      • David commented  ·   ·  Flag as inappropriate

        Is there a scheduled date for PRO users?
        When will we have other users?

      ← Previous 1 3 4 5 18 19

      Feedback and Knowledge Base

      Ready to get started?

      Try new features of Power BI today by signing up and learn more about our powerful suite of apps.