Knowledge Base

Connectors

No SQL

MongoDB

MongoDB is a document database used to build highly available and scalable internet applications. With its flexible schema approach, it is popular with development teams using agile methodologies.

OvalEdge uses MongoClient API to connect to the data source, which allows you to crawl the data objects (Tables, Columns, and Views) and execute Queries.

MongoDB_arch

Connector Capabilities

The following is the list of objects supported by the MongoDB connector.

Functionality	Supported Data Objects
Crawling	Table, Table Columns
Profiling	Table Profiling, Column Profiling, Sample Profiling
Query Execution	Select, Aggregations, Group By, Order By

Prerequisites

The following are prerequisites to establishing a connection to MongoDB.

API/Driver Details
Set up a Service Account with the required permission
Configure environment variables (Optional)

API/Driver Details

The APIs/ drivers used by the connector are given below:

S.No.	Drivers/API	Versions	Details
1	MongoDB driver	3.12.11	https://mvnrepository.com/artifact/org.mongodb/mongo-java-driver/3.12.11 *Note:* Latest version is 3.12.8
2	SQL-to-mongo-db-query-converter	1.11	https://mvnrepository.com/artifact/com.github.vincentrussell/sql-to-mongo-db-query-converter/1.1 *Note:* Latest version is 1.18.

Service Account with Minimum Permissions

The minimum privileges required are:

Operation	Minimum Access Permission
Connection validation	Read
Crawl schema/table	Read
Profile Schemas, tables	Read

Establish Environment Variables (Optional)

This section describes the settings or instructions that you should be aware of prior to establishing a connection. If your environments have been configured, skip this step.

Configure Environment Names

The Environment Names allow you to select the environment configured for the specific connector from the dropdown list in the Add Connector pop-up window.
You might want to consider crawling the same schema in both stage and production environments for consistency. The typical environments for crawling are PROD, STG, or Temporary, and may also include QA or other environments. Additionally, crawling a temporary environment can be useful for schema comparisons, which can later be deleted, especially during application upgrade assistance.

Steps to Configure the Environment

Navigate to Administration | Configuration.
Select the Connector tab.
Find the Key name “connector.environment”.
Enter the desired environment values (PROD, STG) in the value column.
Click ✔ to save.

Establish a Connection

To establish a Connection to MongoDB:

Log in to the OvalEdge application
In the left menu, click on the Administration module name, and click on the Connectors sub-module name. The Connectors Information page is displayed.
Click on + New Connector. The Add Connector pop-up window is displayed.

Fields	Details
Connection Type	The selected connection type ‘MongoDB’ is displayed by default. If required, the dropdown menu allows you to change the connector type
Authentication	OvalEdge supports two types of authentication to establish a connection with MongoDB. i. Standalone: In a standalone deployment, MongoDB is installed on a single machine, and all the data is stored on that machine's local storage. It requires the host address, port, and credential details to set up the connector. ii. Service: It is a cloud-based MongoDB service. For more information refer to Authentication.
License Add-Ons*	All the connectors will have a Base Connector License by default that allows you to crawl and profile to obtain the metadata and statistical information from a datasource. OvalEdge supports various License Add-Ons based on the connector’s functionality requirements. Select the Auto Lineage Add-On license that enables the automatic construction of the Lineage of data objects for a connector with the Lineage feature. Select the Data Quality Add-On license to identify, report, and resolve the data quality issues for a connector whose data supports data quality, using DQ Rules/functions, Anomaly detection, Reports, and more. Select the Data Access Add-On license that will enforce connector access via OvalEdge with Remote Data Access Management (RDAM) feature enabled
Connection Name*	Select a connection name for the MongoDB connection. You must specify a reference name for the easy identification of the MongoDB connection in OvalEdge easily. Example: MongoDB_Connection_DB1
Environment	The environment dropdown menu allows you to select the environment configured for the connector from the dropdown list. For example, PROD, or STG (based on the configured items in the OvalEdge configuration for the connector.environment). The purpose of the environment field is to help you identify which connector is connecting what type of system environment (Production, STG, or QA). Note: The steps to set up environment variables in explained in the prerequisite section.
Server*	Enter the Database instance URL (standalone/cloud-based) Example: 3.19.58.148
Port*	Port number 27017 is the default port used by MongoDB when connecting to a server over the internet.
Database*	Provide the name of the database that is associated with the MongoDB connection.
Driver*	It is the JDBC driver name for the MongoDB server that will be auto-populated.
Username*	Enter the Username to access the MongoDB Service account.
Password*	Enter the Password to access the MongoDB Service account.
Connection String	Set the Connection string toggle button to automatic to get the details automatically from the credentials provided. Alternatively, you can manually enter the string. Example: {server}:12345/{xyz}
Plugin Server	Provide the server name if you are running this as a plugin.
Plugin Port	Provide the port number on which the plugin is running.
Default Governance Roles*	You can select a specific user or a team from the governance roles (Steward, Custodian, Owner) that get assigned for managing the data asset. Note: The dropdown list displays all the configurable roles (single user or a team) as per the configurations made in the OvalEdge Security \| Governance Roles section.
Admin Roles	Select the required admin roles for this connector. To add Integration Admin Roles, search for or select one or more roles from the Integration Admin options, and then click on the Apply button. The responsibility of the Integration Admin includes configuring crawling and profiling settings for the connector, as well as deleting connectors, schemas, or data objects. To add Security and Governance Admin roles, search for or select one or more roles from the list, and then click on the Apply button. The security and Governance Admin is responsible for: Configure role permissions for the connector and its associated data objects. Add admins to set permissions for roles on the connector and its associated data objects. Update governance roles. Create custom fields. Develop Service Request templates for the connector. Create Approval workflows for the templates
No of archive objects*	It is the number of last modifications made in the metadata data of a dataset at Remote/source. By default, the number of archive objects is set to disable mode. Click on the Archive toggle button and enter the number of objects you wish to archive
Select Bridge	With the OvalEdge Bridge component, any cloud-hosted server can connect with any on-premise or public cloud data sources without modifying firewall rules. A bridge provides real-time control that makes it easy to manage data movement between any source and destination. For more information, refer to Bridge Overview.

4. Click on the Validate button to validate the connection details.

5. Click on the Save button to save the connection. Alternatively, you can also directly click on the Save & Configure button that displays the Connection Settings pop-up window to configure the settings for the selected Connector. The Save & Configure button is displayed only for the Connectors for which the settings configuration is required.

Note: * (asterisk) indicates the mandatory field required to create a connection. Once the connection is validated and saved, it will be displayed on the Connectors home page.
Note: You can either save the connection details first, or you can validate the connection first and then save it.

Connection Validation Errors

S.No.	Error Message(s)	Description
1	Failed to establish a connection; please check the credentials	Invalid credentials

Note: For any Connector issues, please contact our Customer Support team.

Connector Settings

Once the connection is established successfully, various settings are provided to fetch and analyze the information from the data source.

The connection settings include Crawler, Profiler, Data Access, Query Policies, Access Instruction, Business Glossary Settings, and Others.

Connector Settings	Description
Crawler	Crawler settings are configured to connect to a data source and collect and catalog all the data elements in the form of metadata. Check out the crawler options to set the crawler's behavior in the Crawler & Profiler Settings.
Profiler	The process of gathering statistics and informative summaries about the connected data source(s). Statistics can help assess the quality of data sources before using them for analysis. Profiling is always optional; crawling can be run without profiling also. For more information, refer to Crawler & Profiler Settings.
Data Access	The Data Access Authorization is included in the crawler-specific connector settings to ensure that the right user is accessing the query sheet and queries in the data catalog. Here the system validates the user credentials and allows that particular user to access the query sheet and queries in the data catalog.
Query Policies	It restricts the use of the selected query types based on your role.
Business Glossary Settings	The Business Glossary setting provides flexibility and control over how they view and manage term association within the context of a business glossary at the connector level.
Access Instruction	Access Instruction allows the data owner to instruct other users on using the objects in the application.
Others	The Send Metadata Changes Notifications option is used to set the change notification about the metadata changes of the data objects. You can use the toggle button to set the Default Governance Roles (Steward, Owner Custodian, etc.) From the drop-down menu, you can select the role and team to receive the notification of metadata changes.

Note: The include and exclude regex functionalities are not applicable for functions and procedures, as these are not available in MongoDB. For more information, refer to the Connector Settings.

The Crawling of Schema(s)

You can use the Crawl/Profile option, which allows you to select the specific schemas for the following operations: crawl, profile, crawl & profile, or profile unprofiled. For any scheduled crawlers and profilers, the defined run date and time are displayed to set.

Navigate to the Connectors page, and click on the Crawl/Profile option.
Select the required Schema(s).
Click on the Run button that gathers all metadata from the connected source into OvalEdge Data Catalog.

Note: For more information on Scheduling, refer to Scheduling Connector

Additional Information

Standalone Authentication

MongoDB standalone refers to a deployment of MongoDB that runs on a single machine or server. To set up a MongoDB standalone deployment, you need to download and install MongoDB on your machine, and then start the MongoDB server. Once the server is running, you can interact with MongoDB.

Fields	Description
Authentication	Standalone
Server	It is an IP address of the Mongo DB standalone machine.
Port	It is the Port number of Mongo DB when connecting to a standalone server.

Service Authentication

MongoDB Atlas is the official cloud-based MongoDB service provided by MongoDB, Inc. With MongoDB Atlas, you can easily deploy, operate, and scale your MongoDB database in the cloud.

M1-1

Fields	Description
Authentication	Service
Server	It is an IP address of the Mongo DB standalone machine.