Dremio

Connectivity Summary

An out of the box connector is available for Dremio databases. It provides support for crawling database objects, profiling of sample data and lineage building.

The drivers used by the connector are given below:

Driver / API: Dremio JDBC Driver

Version: 4.3.1 

Details: Driver can be downloaded from the official website of Dremio given below:
https://www.dremio.com/drivers/

Connector Capabilities

The connector capabilities are shown below:

Crawling

Supported objects and data types for Crawling are:

• Tables
• Table Columns
• Views

Note: Unsupported Data Types: Binary Varying
Please see this article Crawling Data for more details on crawling.  

Profiling

Please see Profiling Data for more details on profiling.

Feature	Support	Remarks
Table Profiling	Row count, Columns count, View sample data
View Profiling	Row count, Columns count, View sample data	View is treated as a table for profiling purposes
Column Profiling	Min, Max, Null count, distinct, top 50 values
Full Profiling	Supported
Sample Profiling	Supported

Lineage Building

Querying 

By default the service account provided for the connector will be used for any query operations. If the service account has write privileges, then Insert / Update / Delete queries can be executed.

Pre-requisites

To use the connector, the following need to be available:

• Connection details as specified in the following section should be available.
• A service account with read privileges.
• Add Dremio client drivers into the OvalEdge Jar path to communicate with the Dremio database.  

Check the Configuration section for further details on how to add the drivers to the jar path.

Connection Details

The following connection settings should be added for connecting to a Dremio database:

• Database Type: Dremio
• License Type: Standard, Auto Lineage
• Connection Name: Select a Connection name for the Dremio database. The name that you specify is a reference name to easily identify your Dremio database connection in OvalEdge. Example: Dremio Connection1
• Server: Database instance URL (on-premises/cloud-based)
  Example: 34.54.56.78
• Port Number: 31010
• Driver Name: JDBC driver name for Dremio. It will be auto-populated.
  Example: com.dremio.jdbc.Driver
• Username: User account login credential (only for Dremio Authentication
• Password: Password (only for Dremio Authentication)
• Connection String: Dremio  connection string. Set the Connection string toggle button to automatic, to get the details automatically from the credentials provided. Alternatively, you can manually enter the string.
  Format: jdbc:dremio:direct={server}:31010
  Example: jdbc:dremio:direct=34.54.56.78:31010

Once connectivity is established, additional configurations for Crawling and Profiling can be specified.

Property	Details
Crawler configurations
Tables, Views, Columns	By default it is selected true
Relationships	Need to select for crawl relationships
Procedures, Functions, View, Source codes	Need to select for crawl Procedures, functions view source codes
Crawler rules	Default include tables regex has  regex = (.*) , which can crawl all the tables
Profile Settings
Tables and columns	By default it is selected true
Views and columns	By default it is selected true
Profile Rules	Default include tables regex has  regex = (.*) , which can profile all the tables
Query Log Setting