ETLs

SAP BODS

OvalEdge connects to the SAP Business Object Data Services (BODS) Oracle database using the JDBC driver to crawl metadata, execute queries, and create lineage.

image1-1

Overview 

Connector Details

Connector Category

ETL System

Connector Version

[OvalEdge Connector Version Information]

Release6.3.1

Connectivity

[How OvalEdge connects to SAP BODS ]

JDBC 

SAP BODS Versions Supported 

[SAP BODS Version from which OvalEdge started providing support]

4.2 Onwards

OvalEdge Releases Supported (Available from)

[OvalEdge Release Version in which the SAP BODS was included in the OvalEdge Application]

Release5.2 Onwards

Connector Features

Crawling of Metadata Objects 

Supported

Query Sheet

Supported

Lineage 

Supported

Auto Lineage Sources

Parsing Source Code of Data Flows XML Files.

Authentication via Credential Manager

Supported

Data Quality

Not Supported

DAM (Data Access Management)

Not Supported

Bridge

Supported

Getting ready to establish a connection 

Prerequisites 

The following are the prerequisites required for establishing a connection:  

Whitelisting Ports

Ensure the inbound port “1521” is whitelisted for OvalEdge to connect to the Oracle database.

Service Account Permissions

Administrator privileges are required to run these commands.


Administrators can use existing users or create new users and then grant the necessary privileges.

SQL

-- Create a user 

CREATE USER oesauser IDENTIFIED BY oesauser; 

-- Grant permissions 

GRANT CONNECT TO oesauser; 

GRANT EXECUTE ON SCHEMA INFORMATION_SCHEMA TO oesauser; 

GRANT SELECT, REFERENCES ON SCHEMA INFORMATION_SCHEMA TO oesauser;


Replace "INFORMATION_SCHEMA" with your actual schema name before executing the command in production.

Validating Service Account Permissions 

  1. Using the service account credentials created, connect to any third-party tool that supports Oracle connections, such as DBeaver.
  2. Using the service account details, create a new connection in the tool.
  3. Test the connection. Once it is successful, click Ok to save the connection.
  4. The connection will be added and displayed in the DBeaver navigation pane, showing all the databases and schemas to which the user has access.

Setup a connection

Only a user with a Connector Creator role can set up a connection in OvalEdge.

  1. Log into OvalEdge, go to Administration > Connectors, click + (New Connector), search for SAP BODS, and complete the specific parameters.

    Note: Fields marked with an asterisk (*) are mandatory for establishing a connection.

    Field Name

    Description

    Connector Type

    By default, "SAP BODS" is displayed as the selected connector type.

    Connector Settings

    Credential Manager*

    Select the desired credentials manager from the dropdown list. Relevant parameters will be displayed based on your selection.

    Supported Credential Managers:

    • OE Credential Manager
    • AWS Secrets Manager
    • HashiCorp Vault
    • Azure Key Vault

    License Add Ons

    OvalEdge connectors have a default license add-on for data crawling and profiling.

    • Select the checkbox for Auto Lineage Add-On to build data lineage automatically.

    Connector Name*

    Enter a unique name for the SAP BODS connection.

    (Example: "SAPBODS_Prod").

    Connector Environment

    Select the environment (Example: PROD, STG) configured for the connector. 

    Refer to the "Connector Environment" document for more details.

    Server*

    Enter the Oracle database Server name or IP address (Example: oracle.example.com or 192.168.1.10).

    Port*

    By default, the port number for the Oracle database "1521" is auto-populated. If necessary, you can change this to a different port number.

    Database*

    Enter the database name to which the service account user has access within the Oracle server.

    Database Type

    Select the database type from the dropdown list. These database types are static values set by OvalEdge.

    • Use SID if you're working with a single Oracle instance.
    • Use SERVICE_NAME in distributed environments with high-availability setups (like Oracle RAC).

    Driver*

    By default, the Oracle driver details (Example: oracle.jdbc.driver.OracleDriver) are auto-populated. OvalEdge artifacts include the required drivers for supported databases.

    If needed, you can download the jar file from the following link and place it in the external jar folder:

    https://ovaledge-jars.s3.amazonaws.com/third_party_jars/third_party_jars_latest/ojdbc8.jar

    Schema*

    Enter the schema name from the provided database with which the OvalEdge should connect.

    Username*

    Enter the service account username set up to access the Oracle database (Example: "oesauser").

    Password*

    Enter the password associated with the service account user (Example: "password").

    Connection String

    Configure the connection string for the Oracle database:

    • Automatic Mode: The system generates a connection string based on the provided credentials.
      • Example (Oracle): 

    jdbc:oracle:thin:@{server}:{port}:{sid}

    • Manual Mode: Manually enter a valid connection string.
      • Example (Oracle): jdbc:oracle:thin:@{server}:{1521}:{sid}

    Replace placeholders with actual database details.

    {sid} refers to Database Name

    Default Governance Roles

    Default Governance Roles*

    Select the appropriate users or teams for each governance role from the dropdown list. All users and teams configured in OvalEdge Security are displayed for selection. 

    Admin Roles

    Admin Roles*

    Select one or more users from the dropdown list for Integration Admin and Security and Governance Admin. All users configured in OvalEdge Security are available for selection.

    No Of Archive Objects*

    It indicates the number of recent metadata changes to a dataset at the source. By default, it is off. You can enable it by toggling the Archive button and specifying the number of objects to archive.

    Example: Setting it to 4 retrieves the last 4 changes, shown in the 'version' column of the 'Metadata Changes' module.

    Bridge

    Select Bridge*

    The dropdown displays all the active and inactive bridges configured in the OvalEdge. Select the appropriate bridge that enables seamless connectivity between data sources without altering firewall rules.

  2. After entering all connection details, you can perform the following actions:
    1. Click Validate to verify the connection.
    2. Click Save to store the connection for future use.
    3. Click Save & Configure to apply additional settings before saving.
  3. The saved connection will appear on the Connectors home page.

Connectivity Troubleshooting

If incorrect parameters are provided, you may encounter error messages. To resolve these issues, ensure all input is correct. If problems persist, contact your assigned OvalEdge support team.

  • Authentication Errors

S.No.

Error Description

Resolution

1

"Invalid username or password."

Verify that the correct username and password are entered. Ensure the service account has the necessary permissions, and the credentials are up-to-date.

2

"Authentication failed for the user."

Confirm that the service account has the correct privileges to access the repository database. Check that the account is not locked or disabled.

  • Datasource Errors

S.No.

Error Description

Resolution

1.

"Could not establish a connection to the database."

Check that the server address, port number, and connection string are correctly specified. Ensure the database is running and accessible from the OvalEdge server. Verify network connectivity and firewall settings.

2.

"Database server not reachable."

Ensure that the database server is up and running. Verify that the correct server address and port are used and that no network issues or firewall restrictions prevent access.

3.

"Oracle connection failed."

Verify that the correct Oracle driver is used and that the server address, port, and database name are correctly specified. Check that the Oracle instance is up and accessible.

Manage Connector Operations

Configure Settings for Connector Operations

Lineage Settings for ETL Connectors: Lineage settings are enabled by default, allowing you to select multiple sources for lineage building. Choose server dialects for parsing source codes, ensuring at least one is selected. Configure table priority to specify the order in which tables with the same name or signature are considered for lineage.

Crawl/Profile

Integration Admin privileges are required for Crawl/Profile operations.

Crawl and Profile operations enable you to select one or more jobs from a list of all available jobs within a specific database. This allows you to customize the selection of crawling operations according to your requirements. Furthermore, it provides the option to schedule crawling.

Other Operations

The Connectors page in OvalEdge provides a centralized view of all configured connectors, including their health status. You can view, edit, validate, build lineage, and delete connectors using the Nine Dots menu.

Managing connectors includes:

  • Connectors Health: Displays performance with a green (active) or red (inactive) icon, helping monitor data flow and address issues early.
  • Viewing: Shows connector details (e.g., Databases, Tables, Table Columns, and Codes) via the View icon.

Nine Dots Menu Options:

  • Edit Connector: Update and revalidate the data source.
  • Validate Connector: Check the connection's integrity.
  • Settings: Modify connector settings.
  • Build Lineage: Automatically build data lineage by parsing source code of data flows XML files.
  • Delete Connector: Remove connectors or schemas with confirmation.

Limitations

S.No.

Description

None

FAQs

  • What is the current SSL/TLS version used for OvalEdge connectors? 
    • OvalEdge uses the TLS 1.2/1.3 version.
  • Is everything crawled using https TLS 1.2? 
    (Do all OvalEdge connectors use HTTPS/TLS 1.2?).
    • Support for SSL/TLS is not mandated in the JDBC specification. So, we cannot expect it from every driver. PostgreSQL and Redshift do have SSL support. Other connectors mentioned in your list use HTTPS.

Planned Upgrades

S.No.

Expected Date

Upgrade Version

Description

None