A Power BI cloud connector is a business analytics service that gives a single view of your most critical business data and supports report editing and collaboration for teams and organizations. Using OvalEdge, you can crawl the Reports, Report Columns, Dashboards, Tiles, Pages, Datasets and Dataflows existing in the Power BI Cloud and build the lineage for Report and Report Columns.
The following is the list of objects and data types supported by the Power BI-OnCloud connector
Support Data Objects
This section lists the prerequisites to establish a connection between the connector and OvalEdge Application.
- Configuration in Office 365
- Configuration in Azure
- Configuration in Power BI application
- Service Account Minimum Read Permissions
- Configure environment variables (Optional)
Note: Create a Service User or use any existing user in OvalEdge
Configuration in Azure
1. Creating an App
- Sign in to the Azure portal, and search for Azure AD in the Azure services text box. Under Manage, click on App registrations.
- Click on New Registration.
- Register an application - give a user-facing display name for the app, select the supported account types, and specify the redirect URI as https://app.powerbi.com.
- Click on Register.
- Give API Permissions
- From the above screenshot, we can see the Directory Tenant ID, and App Client ID.
2. Enabling API permission in Azure
- Click on the View API Permissions button to see an API permission window. Select API Permissions from the Manage list.
- Based on the below screenshot, enable the Request API Permissions—Microsoft APIs
- In the Request API Permissions > Application permissions > select the Tenant Read All permissions.
Note: Make sure that Tenant.Read.All must be available for Delegated and Application type with admin consent set to YES.
- Create Client Secrets - Click on Certificates & Secrets—New Client secret, Add Client Secret.
- Note the Secret ID Value.
3. Creating a new Security Group
Step 1: Create a new Security Group in Azure Active Directory. Read more about creating a basic group and adding members using Azure Active Directory. You can skip this step if you already have a security group you would like to use. Make sure to select Security as the Group type.
Step 2: Add the created service user and app which is created in step 1 to the current security group.
Configuration in Office 365
To enable Power BI Admin APIS for the created service user, you have two options:
- Either enable the OvalEdge service user as a Power BI Administrator role or you can use an existing user with the Power BI Administrator role
Next, you need to configure permissions in Power BI Admin Portal.
Configuration in Power BI Application
Login with the Power BI administrator user and enable the below settings:
1. Enable the Power BI service admin setting
- Log in to the Power BI admin portal with the service user credentials. You need to be a Power BI admin to see the tenant settings page.
- Under Admin API settings, you'll see Allow service principals to use read-only Power BI admin APIs. Set the toggle to Enabled, select the Specific security groups radio button and add the security group you created in Step 2 in the text field that appears.
- To enable these settings, go to Admin portal > Tenant settings > Developer settings.
- Select the entire organization option and enable the settings in the Admin Portal > Tenant Settings > Developer settings > Embed Content in apps.
- In Developer settings > Allow service principals to use Power BI APIs, select specific security groups, and enable the settings.
- In Developer settings > Allow service principals to create and use profiles, select specific security groups and enable the settings.
- In the Admin Portal > Tenant Settings > Admin API settings > Allow service principals to use read-only Power BI admin, select specific security groups, and enable the settings.
- In the Admin API settings > Enhance admin API responses with detailed metadata, select the entire organization option, and enable the settings.
- In the Admin API settings > Enhance admin APIs responses with DAX and mashup expressions, select the entire organization option and enable the settings.
2. Create a workspace in the Power BI instance
The following are prerequisites for connecting the Power BI database.
To connect to Power BI from OvalEdge, creating a workspace in the Power BI instance is required. If you already created it, you can skip Creating a Workspace section and move to Workspace Access section.Creating a workspace
- Navigate to app.powerbi.com.
- The First step is to create a workspace (Premium / Non-premier) in Power BI.
- Click on Create a workspace button.
- Enter the workspace name
- Click on the Save button in the advanced tab, select the option for Specific users and groups, and then enter the users and groups.
- Once the workspace is successfully created, search the workspace name.
Workspace AccessSearch results will display the recently created workspace name; next, click on three dots that will display options for the Workspace settings and workspace access.
- Click on the workspace access.
- Click on Settings
- An access pop-up window is displayed. Verify the permission details and member permission required on a particular workspace.
Service Account with Minimum Permissions
The following are the minimum privileges required for a service account user to crawl and profile a connector.
Configure environment variables (Optional)
This section describes the settings or instructions that you should be aware of prior to establishing a connection. If your environments have been configured, skip this step.
Configure Environment Names
The Environment Names allow you to select the environment configured for the specific connector from the dropdown list in the Add Connector pop-up window.
You might want to consider crawling the same schema in both stage and production environments for consistency. The typical environments for crawling are PROD, STG, or Temporary, and may also include QA or other environments. Additionally, crawling a temporary environment can be useful for schema comparisons, which can later be deleted, especially during application upgrade assistance.
Steps to Configure the Environment
- Navigate to Administration > System Settings.
- Select the Connector tab.
- Find the Key name “connector.environment”.
- Enter the desired environment values (PROD, STG) in the value column.
- Click ✔ to save.
Establish a Connection
To connect to Power BI using the OvalEdge application, complete the following steps:
- Log into the OvalEdge application.
- Navigate to Administration > Connectors.
- Click on the + icon, and the Add Connector with Search Connector pop-up window is displayed.
- Select the connection type as Power. The Add Connector with Power BI details pop-up window is displayed.
The selected connection type Power BI is displayed by default.
If required, the drop-down menu allows you to change the connector type and based on the selection of the connection type, the fields associated with the selected connection type are displayed.
Select the option from the drop-down menu, where you want to save your credentials:
OE Credential Manager: The Power BI connection is configured with the basic Username and Password of the service account in real-time when OvalEdge establishes a connection to the Power BI database. Users need to add the credentials manually if the database option is selected.
HashiCorp: The credentials are stored in the HashiCorp database server and fetched from HashiCorp to OvalEdge.
AWS Secrets Manager: The credentials are stored in the AWS Secrets Manager database server and fetched from the AWS Secrets Manager to OvalEdge.
Azure Key Vault: Click here
For more information on Credential Manager, refer to Credential Manager
All the connectors will have a Base Connector License by default that allows you to crawl and profile to obtain the metadata and statistical information from a data source.
OvalEdge supports various License Add-Ons based on the connector’s functionality requirements.
You choose either OneDrive or Local Drive as the source.
Note: However, it's important to note that if you opt for OneDrive, you need to provide a OneDrive connection and specify the name of the OneDrive folder.
Enter a Connection name for PowerBI-OnCloud.
Users can specify a connection name to identify the PowerBI-OnCloudsa connection in OvalEdge.
The environment drop-down menu allows you to select the environment configured for the connector from the dropdown list. For example, PROD, or STG.
The purpose of the environment field is to help you understand that the new connector is established in an environment available at the Production, STG, and QA.
Note: The steps to set up environment variables are explained in the prerequisite section.
Client ID is generated after the app is registered in Power BI.
A secret is known only to the application authorization server.
The default value (the organization that owns and manages a specific instance of Microsoft Cloud services)
Enter Tenant ID
A Username is required to connect to the Power BI-On Cloud server. Enter the Service Account Name established to access the Power BI-On Cloud environment.
Note: Sometimes, this field is autofill filled by the web browser with the current OvalEdge user login. Please enter the Power BI-On Cloud Service Account name.
Select the option for Premium report. When the option is Yes, the user can crawl the report's dataset and view the report when the premium option is No.
If Okta is enabled for the given service user, enter ‘Y’; otherwise enter ‘N’.
Read From NFS(Y/N)
To retrieve reports directly from the folder without connecting to the Power BI service, enter 'Y'; otherwise, enter 'N'.
Enter the server files path. A user needs to provide a temp path to store the exported PBIX files.
Crawl Hidden Pages(Y/N)
To crawl the hidden pages, enter ‘Y’; otherwise, enter 'N'.
Password required to connect to the Power BI-On Cloud server
Default Governance Roles*
Users can select a specific user or a team from the governance roles (Steward, Custodian, Owner) that get assigned for managing the data asset.
Note: The drop-down list displays all the configurable roles (single user or a team) as per the configurations made in the OvalEdge Security | Governance Roles section.
Select the required admin roles for this connector.
No of Archive Objects*
The number of archive objects indicates the number of recent metadata modifications made to a dataset at a remote/source location. By default, the archive objects feature is deactivated. However, users may enable it by clicking the Archive toggle button and specifying the number of objects they wish to archive.
With the OvalEdge Bridge component, any cloud-hosted server can connect with any on-premise or public cloud data sources without modifying firewall rules. A bridge provides real-time control that makes managing data movement between any source and destination easy.
When the bridge is configured and added, the Bridge ID will be displayed in the drop-down menu, or it will be displayed as "NO BRIDGE."
For more information, refer to Bridge Overview
6. Click on the Save button to save the connection. Alternatively, the user can also directly click on the Save & Configure button that displays the Connection Settings pop-up window to configure the settings for the selected Connector. The Save & Configure button is displayed only for the Connectors for which the settings configuration is required.
Note: * (asterisk) indicates the mandatory field required to establish a connection. Once the connection is validated and saved, it will be displayed on the Connectors home page.
Note: You can either save the connection details first, or you can validate the connection first and then save it.
Connection Validation Errors
The following are the possible error messages encountered during the validation.
Failed to establish a connection; please check the credentials.
Username and Password validation.
Note: If you have any issues creating a connection, please contact your assigned OvalEdge Customer Success Management (CSM) team.
Once the connection is validated successfully, various settings are provided to retrieve and display the information from the data source.
Crawler settings are configured to connect to a data source and collect and catalog all the data elements in the form of metadata. Check out the crawler options to set the crawler's behavior in the Crawler & Profiler Settings.
The lineage settings allow you to configure multiple dialects (by Selecting Source Server Type for lineage) and connection priority lists to pick the tables to build lineage.
Access Instruction allows the data owner to instruct others on using the objects in the application.
Business Glossary Settings
The Business Glossary setting provides flexibility and control over how they view and manage term association within the context of a business glossary at the connector level.
The Send Metadata Changes Notifications option is used to set the change notification about the metadata changes of the data objects.
You can use the toggle button to set the Default Governance Roles (Steward, Owner Custodian, etc.)
From the drop-down menu, you can select the role and team to receive the notification of metadata changes
Note: For more information, refer to the Connector Settings.
The Crawling of Report Group
A Crawl/Profile button allows you to select one or more Report Group Names for crawling.
- Navigate to the Connectors page, and click Crawl/Profile. It allows you to select the Report Groups that need to be crawled.
- The crawl option is selected by default.
- Click on the Run button that gathers all metadata from the connected source into the OvalEdge Data Catalog. After a successful crawl, all the information is displayed in the Data Catalog > Report / Report Column Tab.
Note: For more information on Scheduling, refer to Scheduling Connector
The below steps are followed to build the lineage.
- Building a lineage with the help of PowerBI PBXFiles and Power BI report metadata which is coming from Admin API.
- If the PBX files are not generated at the time of crawling due to the size of the report, in such cases you need to run an advanced job to export the PBX files.
- While generating the Power BI metadata from Admin API, there are some limitations.
- Datasets that have not been refreshed or republished will be returned in API responses but without their detailed low-level information and expressions. For example, you will see the dataset name and lineage in the response but not the dataset's table and column names.
- Datasets containing only DirectQuery tables will return low-level details only if they have been republished since enhanced metadata scanning has been enabled. DirectQuery datasets don't use the regular Power BI refresh flow that triggers caching. If a dataset also contains tables that use import mode, caching takes place upon dataset refresh as described above, and it is not necessary for the dataset to be republished for low-level details to be returned.
- Real-time datasets, datasets with object-level security, datasets with a live connection to AS-Azure and AS on-prem, and Excel full-fidelity datasets are not supported for detailed metadata. The response returns the reason for not getting detailed metadata about the dataset for unsupported datasets. It is found in a field named schemaRetrievalError, for example, schemaRetrievalError: Unsupported request for RealTime model.
- The API doesn't return sub-artifact metadata for datasets that are larger than 1GB in shared workspaces. For Premium workspaces, there is no size limitation.