AWS Glue

Connectivity Summary

An out of the box connector is available for the AWS Glue Catalog. It provides support for crawling cataloged objects.

AWS GlueThe connectivity to AWS Glue is performed via AWS SDK for AWS Glue Catalog, which is included in the platform. 

The connector supports any version of AWS Glue.

The drivers used by the connector are given below:

Driver/API: AWS SDK for AWS Glue
Details: SDK is provided by AWS to communicate with AWS Glue

Technical Specifications

The core components of the Java binding are files named AWS SDK for Glue, which contain the Java classes that provide the connection and caching mechanisms for communication with the Glue.

The connector capabilities are shown below:

Crawling

Feature Supported Objects Remarks
Crawling Tables  
Table Columns

Supports all the standard datatypes related to AWS Glue.

Profiling

Feature Supported Objects Remarks
Table Profiling Not Supported  
View Profiling Not Supported  
Column Profiling Not Supported  
Full Profiling  Not Supported  
Sample Profiling Not Supported  

Lineage Building

Feature Supported Objects Remarks
Table Lineage Not Supported  
Column Lineage Not Supported  
Lineage Sources Not Supported  

Querying

Feature Supported Objects Remarks
Select Not Supported  
Insert Not Supported  
Update Not Supported  
Delete Not Supported  
Join within database Not Supported  
Join outside database Not Supported  
Aggregations Not Supported  
Group By Not Supported  
Order By Not Supported  

Note: By default, the service account provided for the connector will be used for any query operations.

Pre-requisites

To use the connector, the following need to be available:

Connection details as specified in the following section should be available.

An admin/service account for crawling and profiling. If using ROLE ARN needs to have access over the entities like databases, tables.  etc.

Connection Details

The following are the connection settings that should be added for connecting to an AWS Glue Catalog:

General Authentication

AWS Connector

Property

Details

Database Type

AWS Glue

Connection Name

Select a Connection name for the AWS Glue catalog. The name that you specify is a reference name to easily identify your AWS Glue catalog connection in OvalEdge.

 Example: AWS Glue Connection

Access key

Access key

Secret key

Secret key

Database region

Region of database

Role Based Authentication

AWS Connector2

Property

Details

Database Type

AWS Glue

Connection Name

Select a Connection name for the AWS Glue catalog. The name that you specify is a reference name to easily identify your AWS Glue catalog connection in OvalEdge.

 Example: AWS Glue Connection

Cross account Role ARN

Provide role as ARN

Database region

Region of database

Once connectivity is established, additional configurations for crawling and profiling can be specified:

AWS G Settings

Property

Details

Setting > Crawler Options

Tables, Views columns

By default, it is selected true

Relationships

no need to select

Procedures, functions view source codes

no need to select

Settings > Profiler

N/A

 

FAQs

  1. How much does the driver cost?
    There is no need for any additional driver for communication between the OvalEdge and AWS Glue.

Copyright © 2019, OvalEdge LLC, Peachtree Corners GA USA