Couchbase Connector

Couchbase is a distributed NoSQL cloud database that delivers unmatched versatility, performance, scalability, and financial value for all of your cloud, mobile, on-premises, hybrid, distributed cloud, and edge computing applications. 

OvalEdge uses the REST APIs to connect to the data source, which allows users to Crawl and Profile data objects (Buckets, Scopes, and Collections).

Couchbase consists of  

• Buckets: A Couchbase cluster stores the data in buckets. A bucket is a virtual container that logically groups records within a cluster. A bucket is a data store to store and retrieve information. 
• Scopes: When a new data bucket is created by default, a Scope is created to store. It is a mechanism for grouping or organizing multiple collections in it. 
• Collections: A collection can contain a set of documents within a bucket.

Connector Capabilities 

The following is the list of objects and data types supported by the Couchbase connector:

Functionality	Supported Data Objects
Crawler	Tables and Columns
Profiler	Table Profiling: Row count, Column count, and View sample data Column Profiling: Min, Max, Null count, distinct, top 50 values Full Profiling

Prerequisites 

The following are the prerequisites required for establishing a connection between the source system and the OvalEdge application. 

1. SDK Kit Details
2. Set up a Service account with proper permissions
3. Configure environment variables (Optional)

SDK Kit Details

OvalEdge uses SDK - Software Development Kit to connect to Couchbase:

SDK Kit	Version	Details
Couchbase Java SDK	3.4.2	The Couchbase Java client allows applications to access a Couchbase database. It offers synchronous APIs and reactive and asynchronous equivalents to maximize flexibility and performance.

 Service Account with Minimum Permissions

The following are the minimum privileges required for a service account user to crawl and profile the data.

Operation	Minimum Access Permission
Connection Validation	Read-Only, and Query Select on all buckets
Crawling	Read-Only, and Query Select on all buckets
Profiling	Read-Only, and Query Select on all buckets

Establish Environment Variables (Optional)

This section describes the settings or instructions that users should be aware of prior to establishing a connection. If your environments have been configured, skip this step.

configure Environment Names

The Environment Names allow users to select the environment configured for the specific connector from the dropdown list in the Add Connector pop-up window. You might want to consider crawling the same schema in both stage and production environments for consistency. The typical environments for crawling are PROD, STG, or Temporary, and may also include QA or other environments. Additionally, crawling a temporary environment can be useful for schema comparisons, which can later be deleted, especially during application upgrade assistance. 

Steps to Configure the Environment

1. Navigate to Administration | Configuration. 
2. Select the Connector tab.
3. Find the Key name “connector.environment”.
4. Enter the desired environment values (UAT, QA, or PROD) in the value column. 
5. Click ✔ to save. 

Establish a connection

To establish a Couchbase Connection:

1. Log into the OvalEdge application
2. In the left menu, click on the Administration module name, and click on the Connectors sub-module name. The Connectors Information page is displayed.
3. Click on + New Connector. The Add Connector pop-up window is displayed.
4. Select the connection type as Couchbase. The Add Connector pop-up with Couchbase-specific details is displayed.
   
   
   

   Fields	Details
   Connection Type*	The selected connection type ‘Couchbase’ is displayed by default.  If required, the dropdown menu allows the user to change the connector type and based on the selection of the connection type, the fields associated with the selected connection type are displayed.
   License Add-Ons*	All the connectors will have a Base Connector License by default that allows you to crawl and profile to obtain the metadata and statistical information from a datasource.  OvalEdge supports various License Add-Ons based on the connector’s functionality requirements. Select the Auto Lineage Add-On license that enables the automatic construction of the Lineage of data objects for a connector with the Lineage feature.  Select the Data Quality Add-On license to identify, report, and resolve the data quality issues for a connector whose data supports data quality, using DQ Rules/functions, Anomaly detection, Reports, and more. Select the Data Access Add-On license that will enforce connector access via OvalEdge with Remote Data Access Management (RDAM) feature enabled
   Connection Name*	Enter a Connection name for Couchbase. Users can specify a connection name to identify the Couchbase connection in OvalEdge. Example: Couchbase_db.
   Environment	The environment dropdown menu allows you to select the environment configured for the connector from the dropdown list. For example, PROD, or STG (based on the configured items in the OvalEdge configuration for the connector.environment). The purpose of the environment field is to help you identify which connector is connecting what type of system environment (Production, STG, or QA).   Note: The steps to set up environment variables in explained in the prerequisite section.
   Server*	Host or IP details for Couchbase Server
   Username*	Enter the Service Account Username to connect to the Couchbase server.  Note: Sometimes, this field may be auto-filled by the web browser with the current OvalEdge user login. Please enter the source system Service Account name
   Password*	Enter the Service Account  password required to connect to the Couchbase server
   Use Analytics Query Service	Users can choose ‘Yes’ or ‘No’ if they want to use Analytics Query Service (snapshot of data) for crawling.
   Connection String	Set the Connection string toggle button to automatic to get the details automatically from the credentials provided. Alternatively, you can manually enter the string. Example: couchbase://{server}
   Default Governance Roles*	The admin will select a specific user or a  team from the governance roles (Steward, Custodian, Owner) that get assigned to the data asset.  The dropdown list displays all the configurable roles (single user or a team) as per the configurations made in the OvalEdge Security > Governance Roles.
   Admin Roles	Select the required admin roles for this connector. To add Integration Admin Roles, search for or select one or more roles from the Integration Admin options, and then click on the Apply button.  The responsibility of the Integration Admin includes configuring crawling and profiling settings for the connector, as well as deleting connectors, schemas, or data objects. To add Security and Governance Admin roles, search for or select one or more roles from the list, and then click on the Apply button.  The security and Governance Admin is responsible for: Configure role permissions for the connector and its associated data objects. Add admins to set permissions for roles on the connector and its associated data objects. Update governance roles. Create custom fields. Develop Service Request templates for the connector. Create Approval workflows for the templates.
   Select Bridge	With the OvalEdge Bridge component, any cloud-hosted server can connect with any on-premise or public cloud data sources without modifying firewall rules. A bridge provides real-time control that makes it easy to manage data movement between any source and destination. For more information, refer to Bridge Overview.

5. Click on the button to validate the connection details.
6. Click on the button to save the connection.  Alternatively, the user can also directly click on the button that displays the Connection Settings pop-up window to configure the settings for the selected Connector. The Save & Configure button is displayed only for the Connectors for which the settings configuration is required. 

Connection Validation Errors 

S.No.	Error Message(s)	Description
1	Failed to establish a connection; please check the credentials.	Username and Password validation.

Connector Settings 

Once the connection is validated successfully, various settings are provided to retrieve and display the information from the data source. 

Connection Settings	Description
Crawler	Crawler settings are configured to connect to a data source and collect and catalog all the data elements in the form of metadata. Check out the crawler options to set the crawler's behavior in the  Crawler & Profiler Settings.
Profiler	The process of gathering statistics and informative summaries about the connected data source(s). Statistics can help assess the data source's quality before using it in an analysis. Profiling is always optional; crawling can be run without profiling also. For more information, refer to Crawler & Profiler Settings.
Access Instruction	Access Instruction allows the data owner to instruct other users on using the objects in the application.
Others	The Send Metadata Changes Notifications option is used to set the change notification about the metadata changes of the data objects. Users can use the toggle button to set the Default Governance Roles (Steward, Owner, Custodian, etc.)  From the drop-down menu, the user can select the role and team to receive the notification of metadata changes.

The Crawling of Schema(s)

The user can use the Crawl/Profile option, which allows the user to select the specific schemas for the following operations: crawl, profile, crawl & profile, or profile unprofiled. For any scheduled crawlers and profilers, the defined run date and time are displayed to set.

1. Navigate to the Connectors page, and click Crawl/Profile option.
2. Select the required Schema(s). 
3. Click on the Run button that gathers all metadata from the connected source into OvalEdge Data Catalog. 

---

Copyright © 2023, OvalEdge LLC, Peachtree Corners GA USA