OvalEdge uses a JDBC driver to connect to the data source, which allows the user to crawl and profile data objects (Tables, Table Columns, etc.) and build lineage.
Connector Capabilities
The connector capabilities are shown below:
Crawling
Features |
Supported Objects |
Remarks |
---|---|---|
Crawling |
Tables |
- |
Crawling |
Table Columns |
- |
Crawling |
Views |
- |
Crawling |
Stored Procedures |
- |
Crawling |
Functions |
- |
Profiling
Features |
Details |
Remarks |
Table Profiling |
Row count, Columns count, View sample data |
Not Supported Data Types:
|
View Profiling |
Row count, Columns count, View sample data |
- |
Column Profiling |
Min, Max, Null count, distinct, top 50 values |
- |
Full Profiling |
Supported |
- |
Sample Profiling |
Supported |
- |
Lineage Building
Lineage Entities |
Details |
Table lineage |
Supported |
Column lineage |
Supported |
Lineage Sources |
Stored Procedures, Functions, and Views |
Querying
Operation |
Details |
Select |
Supported |
Insert |
Supported, provided its service account has the permission |
Update |
Supported, provided its service account has the permission |
Delete |
Supported, provided its service account has the permission |
Joins within database |
Supported |
Joins outside database |
Not supported |
Aggregations |
Supported |
Group By |
Supported |
Order By |
Supported |
Note: By default, the service account provided for the connector will be used for any query operations. If the service account has write privileges, then Insert/Update/Delete queries can be executed.
Prerequisites
Driver Details
JDBC driver is provided by default over the OvalEdge installation. If it needs to be changed, add Teradata drivers into the OvalEdge Jar path (/home/ovaledge/jarpath) to communicate with the Teradata database.
Driver |
Version |
Details |
JDBC Driver |
19.3.0.0 |
Is JDBC 1.0 Note: The latest version is 2.0 |
Configuring Environment Variables
Configuring environment names enables users to select the appropriate environment from the drop-down list when adding a connector. This allows for consistent crawling of schemas across different environments, such as production (PROD), staging (STG), or temporary environments. It also facilitates schema comparisons and assists in application upgrades by providing a temporary environment that can later be deleted.
Before establishing a connection, it is important to configure the environment names for the specific connector. If your environments have been configured, skip this step.
Steps to Configure the Environment
- Log into the OvalEdge application.
- Navigate to Administration > System Settings.
- Select the Connector tab.
- Find the key name “connector.environment”.
- Enter the desired environment values (PROD, STG) in the Value column.
- Click ✔ to Save.
Service Account Permissions
Create a Teradata account to connect to OvalEdge with the following permissions to crawl metadata into OvalEdge. Required permissions and corresponding Teradata commands are given below for your reference:
Operation |
Minimum Access Permission |
---|---|
Connection Validation |
Read |
Crawl Schemas, Tables |
Read |
Profile Schemas, Tables |
Read, Select |
Establish a Connection
To connect to Teradata using the OvalEdge application, complete the following steps:
- Log in to the OvalEdge application.
- Navigate to Administration > Connectors.
- Click on the + (New Connector) icon.
- The Add Connector pop-up window is displayed, and users can search for the Teradata connector.
5. The Add Connector with Connector Type specific details pop-up window is displayed. Enter the relevant information to configure the Teradata connection.
Note: An asterisk (*) denotes a mandatory field for establishing a connection.
Field Name |
Description |
|
Connector Type |
This field allows users to select the connector from the drop-down list provided. By default, 'Teradata' is displayed as the selected connector type. |
|
Credential Manager* |
Select the option from the drop-down menu where users want to save their credentials: OE Credential Manager: The Teradata connection is configured with the basic Username and Password of the service account in real-time when OvalEdge establishes a connection to the Teradata database. Users must manually add the credentials if the OE Credential Manager option is selected. HashiCorp: The credentials are stored in the HashiCorp database server and fetched from HashiCorp to OvalEdge. AWS Secrets Manager: The credentials are stored in the AWS Secrets Manager database server and fetched from the AWS Secrets Manager to OvalEdge. For more information on Azure Key Vault, refer to Azure Key Vault. For more information on Credential Manager, refer to Credential Manager. |
|
License Add Ons |
All the connectors will have a Base Connector License by default, which allows users to crawl and profile to obtain metadata and statistical information from a data source. OvalEdge supports various License Add-Ons based on the connector’s functionality requirements.
|
|
Connector Name* |
The connection name refers to the Teradata database connection in the OvalEdge application. |
|
Connector Environment |
The Connector Environment drop-down list allows users to select the environment configured for the connector from the drop-down list. For example, select PROD or STG (based on the items configured in the OvalEdge configuration for the connector environment). The purpose of the environment field is to help users identify which connector is connecting to what type of system environment (Production, STG, or QA). Note: The Configuring Environment Variables section explains setting up environment variables. |
|
Server* |
Specify the name of the Teradata database instance server URL, which is accessible by the OvalEdge application. Format: <account>.teradatacomputing.com |
|
Port Number* |
Port number 1025 is the default port Teradata uses when connecting to a server over the internet. |
|
Database* |
Provide the name of the database that is associated with the Teradata connection, if necessary. It is a user-specified database name to crawl the metadata of selected connectors. |
|
Driver* |
A JDBC driver is a Java library file with the extension .jar that connects to a database. The driver details associated with the Teradata database will be auto-populated by default. Example: com.teradata.jdbc.TeraDriver |
|
Username* |
Enter the Service Account Username for the Teradata Server name. |
|
Password* |
Enter the password for the Teradata Server name. |
|
Connection String |
Set the Connection string toggle button to automatic, to get the details automatically from the credentials provided. Alternatively, you can manually enter the string. Format: jdbc:teradata://{server}/DBS_PORT={port},database={sid} |
|
Plugin Server |
Enter the server name if you are running this as a plugin. |
|
Plugin Port |
The port number on which the plugin is running. |
|
Default Governance Roles |
||
Steward* |
Select the Steward from the drop-down list options. |
|
Custodian* |
Select the Custodian from the drop-down list options. |
|
Owner* |
Select the Owner from the drop-down list options. |
|
Governance Roles 4, 5, 6* |
Select the respective user from the drop-down options. Note: The drop-down list displays all the configurable roles (single user or a team) as per the configurations made in the OvalEdge Security > Governance Roles section. |
|
Admin Roles |
||
Integration Admins* |
To add Integration Admin Roles, search for or select one or more roles from the Integration Admin options, then click the Apply button. |
|
Security and Governance Admins* |
To add Security and Governance Admin roles, search for or select one or more roles from the list and then click the Apply button.
|
|
No. of Archive Objects* |
The number of archive objects indicates the number of recent metadata modifications made to a dataset at a remote/source location. By default, the archive objects feature is deactivated. However, users may enable it by clicking the Archive toggle button and specifying the number of objects they wish to archive. |
|
Select Bridge* |
With the OvalEdge Bridge component, any cloud-hosted server can connect with any on-premise or public cloud data source(s) without modifying firewall rules. A bridge provides real-time control, making data movement between source and destination easy. For more information, refer to |
-
- Validate: Click the Validate button to verify the connection details. This ensures that the provided information is accurate and enables successful connection establishment.
- Save: Click on the Save button to store the connection details. Once saved, the connection will be added to the Connectors home page for easy access.
- Save & Configure: For specific Connectors requiring additional configuration settings, click the Save & Configure button. This will open the Connection Settings pop-up window, allowing users to configure the necessary settings before saving the connection.
7. Once the connection is validated and saved, it will be displayed on the Connectors home page.
Note: Users can either save the connection details first or validate the connection first and then save it.
Connection Validation Details
S.No |
Error Message(s) |
Description |
1 |
Failed to establish a connection. Please check the credentials. |
Invalid credentials are provided or the user or role does not have access. |
2 |
Connection Timeout |
Invalid credentials are provided or the server is not running. |
Note: If users encounter connection issues, please contact your assigned OvalEdge Customer Success Management (CSM) team.
Connector Settings
Once the connection is successfully established, various settings are provided to fetch and analyze the information from the data source.
The connection settings include Crawler, Profiler, Query Policies, Access Instruction, Business Glossary Settings, and Others.
To view the Connector Settings page,
- Go to the Connectors page.
- From the 9- dots, select the Settings option.
- This will display the Connector Settings page, where users can view all the connector settings.
- After making the desired changes, click on Save Changes. All setting changes will be applied to the metadata.
The following is a list of connection settings and their corresponding descriptions.
Connection Settings |
Description |
Crawler |
Crawler settings are configured to connect to a data source and collect and catalog all the data elements in metadata. |
Profiler |
Profiler settings govern gathering statistics and informative summaries about the connected data source(s). These statistics can help assess the quality of data sources before using them for analysis. Profiling is always optional; crawling can be run without profiling. |
Query Policies |
A query policy enforces security by preventing users with specific roles from performing certain query functions on the data source. |
Access Instruction |
Access Instruction allows the data owner to instruct others on using the objects in the application. |
Business Glossary Settings |
The Business Glossary Settings provide flexibility and control over how users view and manage term association within a business glossary at the connector level. |
Others |
The Enable/Disable Metadata Change Notifications option sets the change notification about metadata changes of the data objects.
|
Note: For more information, refer to the Connector Settings.
Crawling of Schema(s)
The Crawl/Profile option allows users to select the schema for the following operations:
Crawl, Crawl & Profile, Profile, or Profile Unprofiled. Under the Action section, the defined run date and time are displayed for any scheduled crawlers and profilers.
- Navigate to the Connectors page and click on the Crawl/Profile button.
Select Schemas to Crawl & Profile pop-up window is displayed. - Select the schema.
- The list of actions below is displayed in the Action section.
- Crawl: This allows the selected schema(s) metadata to be crawled.
- Crawl & Profile: This allows the metadata of the selected schema(s) and profiles of the sample data to be crawled.
- Profile: This allows the collection of table column statistics.
- Profile Unprofiled: This allows data that has not been profiled to be profiled.
- Schedule: Connectors can also be scheduled in advance to run crawling and/or profiling at prescribed times and selected intervals.
Note: For more information on Scheduling, refer to Scheduling Connector.
Click on the Run button. This gathers all metadata from the connected source and puts it into the OvalEdge Data Catalog.
Copyright © 2024, OvalEdge LLC, Peachtree Corners GA USA