Amazon S3 Connector

OvalEdge connects to Amazon S3 using the AWS S3 SDK, allowing users to catalog metadata objects such as Buckets, Folders, and Files.

Overview

Connector Details

Connector Features

Getting Ready to Establish a Connection 

Prerequisites

The following are the prerequisites for establishing a connection between S3 and OvalEdge.

Service Account User Permissions

Setup a Connection

1. Log into OvalEdge, go to Administration > Connectors, click + (New Connector), search for Amazon S3, and complete the specific parameters.
   Note: Fields marked with an asterisk (*) are mandatory for establishing a connection.
   
   In the OvalEdge application, the S3 connector allows you to catalog the buckets and file data objects using IAM User Authentication and Role-Based Authentication.
   1. IAM User Authentication: IAM user authentication in Amazon S3 securely controls access to AWS services using AWS Identity and Access Management (IAM).
   2. Role-Based Authentication: Role-based authentication (RBAC) in Amazon S3 grants access to an S3 bucket using an AWS role instead of access/secret keys.
      

      Field Name	Description
      Connector Type	By default, "Amazon S3" is displayed as the selected connector type.
      Connector Settings
      Authentication	OvalEdge supports the following two types of authentication for Amazon S3: IAM User Authentication Role-Based Authentication
      Credential Manager*	Select the desired credentials manager from the dropdown list. Relevant parameters will be displayed based on your selection. Supported Credential Managers: OE Credential Manager AWS Secrets Manager HashiCorp Vault Azure Key Vault
      License Add Ons	Auto Lineage Not Supported Data Quality Supported Data Access Not Supported Select the checkbox for Data Quality Add-On to enable the data quality functionality.
      Connector Environment	Select the environment (e.g., PROD, STG) configured for the connector.
      Connector Name*	Enter a unique name for the Amazon S3 connection (Example: "Amazon S3”
      Access key*	Enter a unique identifier that is part of the credential pair, like a username. Note: This Field is available when the Authentication mechanism is selected as "IAM User Authentication."
      Secret key*	A secret, like a password, is used to sign requests to AWS. Note: This Field is available when the Authentication mechanism is selected as "IAM User Authentication."
      Filter by tags	Specify tags as filters to limit the scope of objects the connector will interact with. Example: ovaledge=9,region=ohio,region=oregon
      Region	The Region refers to the specific geographical location where your Amazon S3 bucket resides.  Example: us-east-1
      SSO Connection Id	The SSO connection ID is used to authenticate and authorize access to Amazon S3.
      SSO Application Id	This specific application ID within the SSO system connects to Amazon S3.
      SSO Role Prefix	Specifies the roles in the SSO system that have access to Amazon S3 and manage permissions.
      Cross Account Role ARN	Enter the Cross Account Role ARN Name to access the S3 buckets from other AWS accounts. Example: CrossAccountS3AccessRole Note: This Field is available when the Authentication mechanism is selected as "Role-Based Authentication."

      
      

      Default Governance Roles
      Default Governance Roles*	Select the appropriate users or teams for each governance role from the dropdown list. All users and teams configured in OvalEdge Security are displayed for selection.
      Admin Roles
      Admin Roles*	Select one or more users from the dropdown list for Integration Admin and Security and Governance Admin. All users configured in OvalEdge Security are available for selection.
      No Of Archive Objects*	It indicates the number of recent metadata changes to a dataset at the source. By default, it is off. You can enable it by toggling the Archive button and specifying the number of objects to archive. Example: Setting it to 4 retrieves the last 4 changes, shown in the 'version' column of the 'Metadata Changes' module.
      Bridge
      Select Bridge*	If applicable, select the bridge from the drop-down list. The drop-down list displays all active bridges configured in OvalEdge. These bridges enable communication between data sources and OvalEdge without altering firewall rules.

2. After entering all connection details, you can perform the following actions:
   1. Click Validate to verify the connection.
   2. Click Save to store the connection for future use.
   3. Click Save & Configure to apply additional settings before saving.
3. The saved connection will appear on the Connectors home page.

Connectivity Troubleshooting

If incorrect parameters are provided, you may encounter error messages. To resolve these issues, ensure all input is correct. If problems persist, contact your assigned OvalEdge support team.

Manage Connector Operations

Crawl/Profile

1. Once the user configures the desired connector settings, click the Crawl/Profile button to initiate the cataloging process of the S3 buckets. A message will appear confirming the successful submission to the catalog bucket job.
2. After completing the job, all the buckets will be cataloged and displayed in the File Manager. Select the specific folder(s) or file(s) from your buckets, and then select the "Catalog Files/Folders" option from the Nine Dots menu to add them to the Data Catalog.

Other Operations

The Connectors page in OvalEdge provides a centralized view of all configured connectors, including their health status. 

Managing connectors includes:

• Connectors Health: Displays performance with a green (active) or red (inactive) icon, helping monitor data flow and address issues early.
• Viewing: Shows connector details (e.g., File Folders, Files, File Columns) via the View icon. 

Nine Dots Menu Options:

You can view, edit, validate, and delete connectors using the Nine Dots menu.

• Edit Connector: Update and revalidate the data source.
• Validate Connector: Check the connection's integrity.
• Settings: Modify connector settings.
  • Crawler: Configure metadata that needs to be extracted.
  • Access Instructions: Specify how data can be accessed as a note.
  • Business Glossary Settings: Manage term associations at the connector level.
• Delete Connector: Remove connectors with confirmation.

Metadata Mapping

Limitations

Copyright © 2025, OvalEdge LLC, Peachtree Corners GA USA