Skip to main content
Microsoft Idea

Power BI

Needs Votes

More flexible schedule refresh

Vote (8) Share
Michael Hughes's profile image

Michael Hughes on 23 Mar 2021 17:44:36

Currently PowerBI does not provide the ability to define granular refresh schedules. For example, imagine a customer has a dataset which includes a dozen query / data sources. Often some of these data sources contain data that changes either infrequently (or potentially never) while other data sources contain transactional data that is frequently updated. In our case, we have a few large datasets that pull from a SQL DB which contain historic transaction data which never changes. Imagine an accounting system whereby previous accounting periods are “closed” and therefore data does not change. We may need this data to aggregate into our operational dataset and generate historical measures, etc. However, when we publish this operational data set and establish its refresh schedule, we are forced to refresh all queries within the dataset based on a single refresh schedule. This is significantly taxing the Azure compute and wasting time refreshing data that really need not be refreshed frequently.

Suggestion: Implement the ability within PowerBI desktop to create a “Refresh Pattern” template. For example, you create a refresh pattern that includes a few tiers as follows: Refresh Tier A, Refresh Tier B, Refresh Tier C, etc. For each refresh tier, you identify which of the queries in your model align to which refresh tier. There may even be situations where data from a query never changes at all so it need not be associated with any refresh tier.

Then in the PowerBI service under schedule refresh, provide an option of setting up different refresh intervals for each tier that was established in the model. E.g. one refresh schedule could refresh your frequently changing data entities hourly while another refresh tier may be scheduled to only trigger weekly or even monthly refreshes.

This capability will lead to far quicker refreshes and a significant reduction in Azure compute. Our implementation alone would significantly reduce processing. Imagine how many organizations are frequently refreshing data sources that contain data that is rarely changing, just because “some” of their data within their overall dataset change frequently.
Separating your data into separate datasets is not a good answer considering these data entities are often tightly related and involve measures and calculations that can only be accomplished within the same data set.