Gateway Usage monitoring tools for Power BI Service
As Power Bi and other applications in the Power Platform become more prolific within the BI Cloudscape and Landscape integration, gateways will become used more and more. Having visibility to the capacity of a gateway, the volume of traffic through the gateway, which datasources are used more frequently than others, which types, etc would be a very useful resource to have within the Gateway management system.
Alex Robé commented
In case of having multiple datasets in variable sizes up to multiple 100 of MB we sometimes face refresh issues, that relate to simultaneous / overlapping refreshes of datasets, causing "usually transient" issues.
If we have an easily accessible chance to find empty time slots we could balance the gateway better.
Dreaming allowed? Using a Gantt Chart visual showing runtimes of each dataset :-)
This is essential for administration, especially to avoid duplicate connections, inactive connections, tracking the users in that connections etc.
Joshua Cameon commented
Yes this would be very helpful to organizations that have a lot of import or mixed query-type reports. Either by Gateway or workspace or even overall as an organization high-level would be great use.
Need to be able to monitor health of refreshed dashboards and troubleshoot if necessary. From time to time a dataset will successfully refresh with zero rows due to operational issues. Can additional dataset refresh metadata be made available to give visibility?
Therese Devine commented
We need better/more enterprise tools for IT to SUPPOT all of BI; especially the disabled gateway refreshes after a connectivity/ server issue. We are supporting 30+ workspaces and that is growing every day. When the schedule report refreshes are automatically disabled we have to manually access each workspace, each report etc. to turn the refresh back on. We need enterprise views to support and fix issues like this across all of our workspaces, gateway tools etc. to review total space utilization etc.
Chris Amore commented
This type of monitoring, along with error reporting, is a much much needed component of this product.
It would be very useful to have gateway usage reports that are similar to the report usage reports. Specifically seeing a list of all reports being refreshed through a specific gateway cluster, which gateway within the cluster is being used, the refresh times, and other relevant resource metrics (e.g. RAM usage, overall RAM consumption throughout time, network throughput, amount of data written)
Really like this idea for any serious organization, this would bring PowerBI to the next level
It is a must to support organisms with an important Power BI infrastructure
Christian Harrington commented
We need ability to monitor gateway refreshes in real-time:
-have an overall view of what's running right now: datasets, workspace, gateways, memory, CPU. We have it in the PBI capacity metrics, but only, the day after that is not helping for live monitoring.
-ability to cancel a refresh when it has started
-use past stats to provide some ETA on refresh currently running, and provide a way to display that ETA on a report
-see the "true" current state (not just the wheel running) often it's broken but we see the error only 2 hours after
Chris Preston commented
In addition to this, allowing multiple administrators to manage and change an individual dataset's refresh schedule would be essential. Right now we have two people that share the responsibility of managing datasets and if my partner sets a schedule, not only can't I edit the schedule, but I can't even view the schedule. Unless I take over the schedule, in which case my partner then can't view it, so it becomes a constant tug of war over the schedule ownership, making keeping track of them difficult (or relegated to an Excel spreadsheet). At least I can still view the history, but I have to go to each individual dataset, which is another area where this overview would be handy.
Anand Mishra commented
Usage Report for Enterprise Gateway - The log data should be available so that we can create dashboards based on Gateway Usage data.
Metrics like the datasets name using Gateway, workspace name, owner, scheduled refresh frequency, adhoc refresh, run duration and other parameters for all dependent reports should be available. This will help in scaling the Enterprise Gateway correctly and avoid bottlenecks. Currently there is no visibility to the work spaces for which we don't have access however responsible for maintaining gateway.
Kawabata Yoshihiro commented
Yes, I want to analyze all On-Premises Data Gateway in our organization,
Which one need more performance about CPU, Memory?
Which one need more availability by adding a node?
Which one need more node for load balancing?
It would be very interesting to have the option to see the activity carried out through the gateway in the Power BI service, so that it can be seen in a simple way, the actions of updating the data sets that has been carried out from a gateway / connection, by way of the update history of the data sets but from the administration of the gateway.
Agreed, it would be nice to have a page dedicate to all the datasets and refresh schedules in order to better manage them. Once the number of datasets grows, it becomes more time consuming to keep clicking into each dataset in order to view their status.
When you have multiple data sets set to Schedule Refresh across multiple workspaces it can quickly become difficult to keep track of what is scheduled to run when. It can also become difficult to manage older versions of reports that no longer need to run, and ensure only what is required is scheduled.
Other BI tools have a screen where you can see and manage all scheduled refreshes in one place. Without this some companies many struggle to replace other tools with Power BI.
We use a scheduled powershell script that will alert us if the gateway service is not running. However, we have had instances where the gateway service is running but not responding. When trying to restart the service it gets hung in a "stopping" status. We have to use these commands to force stop the service then restart it again:
sc queryex PBIEgwService
taskkill /f /pid [PID]
Note: the PID comes from the sc queryex PBIEgwService command
We have to keep on monitoring at regular intervals which is difficult. It would be good if we get notification from cloud if gateway goes offline.