Knowledge Base

Connectors

How does Bridge Facilitate the Connectivity with Connectors?

As part of the cloud offering from OvalEdge, customers can start using the platform as a hosted service without worrying about connecting to the data sources which are securely hosted on a different cloud (from where OvalEdge is hosted) or on premise.

With the OvalEdge Bridge component, any cloud-hosted server can connect with any on-premise or public cloud data sources without modifying firewall rules. Since the client component of the bridge is installed within the customer infrastructure, the customer only needs to whitelist the IP address of the data sources and respective ports in the bridge client for secured connectivity. This allows the OvalEdge platform to perform the metadata related activities while the Bridge Component acts as a secured VPN between OvalEdge and the data sources.

Installation Architecture Diagram 01 (1)

OvalEdge Bridge offers the following features:

The OvalEdge platform can establish up to 10 bridges to client data sources.
A bridge can host multiple connections like Oracle, MySQL, Postgres, etc.
HTTPS is used to secure communication between the OvalEdge platform and the OvalEdge client (deployed/installed in client infrastructure)
In the course of creating a bridge, the OvalEdge platform generates an SSL- enabled key store and makes it available for download.
The communication between the bridge and OvalEdge server is a pull-based communication from the bridge, which keeps polling the server frequently for any jobs (crawling, query execution, etc.)

Installation

The Bridge Software installation is accomplished using the installation components available for download in the OvalEdge server, it is a one-time process performed for a specific bridge at the client location.

Prerequisites

The following are the prerequisites for the VM required for installing the Bridge Software:

Operating Systems: Windows 2019/2022 or Linux (Any Flavor)
Recommended Hardware: SSD: 100GB, RAM: 8GB, vCPUs: 2
Software: Java: 8.0
Whitelist the bridge Client IP in all the required connector data sources.
Outbound Open Port (9443) for the bridge client to communicate with the bridge server.

Bridge Installation Steps

To install the Bridge Software in the VM, complete the following tasks:

Download the Bridge Artifacts from OvalEdge application.
Install and Register Bridge on the on-premise system.

Download the Bridge Artifacts

Login into the OvalEdge application.
Navigate to the Administration > Crawler module, and the Crawler Information page is displayed.
Click on the Create Bridge button. The Bridge Details page is displayed.
Click on the + icon. The Add Bridge pop-up window is displayed.
By default, the Next Available Bridge is displayed, enter the Bridge Location Public IP Address and click on the Add button. A pop-up window with the Bridge ID, Security Code, Download of Bridge Secure Connectivity Pack, and Bridge Installation Software is displayed.
Make a note of the Security Code, which needs to be mentioned in one of the pages of the Bridge Installation process.
Download the Bridge Secure Connectivity Pack to your desktop.
Download the Bridge Installation Software to your desktop.

Note: The size of Bridge Installation Software is around 1 GB
Click on the Ok button, and the new bridge is created and displayed in the Bridge page.

Install and Register Bridge on the on-premise system

Locate where the Bridge Installation JAVA Jar file is downloaded and click on the JAVA Jar file icon. The installation process starts, and the OvalEdge Bridge Software pop-up window is displayed.
Click on the Next button, and enter the location path where the Bridge Software needs to be installed.

Note: It is better to provide the location path other than the C drive path.
Click on the Next button, and enter the Bridge Registration details.
In the OvalEdge Bridge Registration page, enter the Security Code, which was noted at the beginning of adding the bridge.
Enter the IP Address of the computer on which the Bridge Software is getting installed.
Enter the location path of the Bridge Secure Connectivity Pack, which is saved on your computer.
Enter other required information like User Name, Email, and Phone Number and click on the Next button.
Click on the Next button, and Bridge Secure Connectivity Pack is picked from the mentioned location and starts the processing.
Once the Processing is completed, the Successful message the OvalEdge Bridge Software Installed Successfully on your computer is displayed.

Installing Bridge as a Service

OvalEdge offers Bridge functionality as part of the cloud offering that allows customers to use the platform as a hosted service without connecting to client data sources. Bridge installation in a client machine is managed manually by commands, which results in a server-down issue when the machine stops. To overcome this, OvalEdge is providing Bridge as a service in the client machine.

This document describes the steps to install Bridge as a service on a windows machine.

Prerequisites

It is required to install the below software on your system to install Bridge as a service on a windows machine.

Maven 3.3+
Java 1.8
Apache NiFi

The steps to install Bridge as a service on the windows machine are

Downloading the file (Build)
Installation of Bridge
Uninstallation of the bridge.

Build

It is required to download the Apache NiFi zip file from the link https://github.com/devananda880/BridgeAsService.

Click on the Download ZIP, to download the file. Once the file is downloaded, unzip the file.
Open a command prompt and run the mvn install command.
Once the command runs successfully, BUILD SUCCESS message will be displayed on the screen resulting in the creation of an output directory $root/Dist.

Installation of Bridge as a Service for Windows

Copy the nifi-windowsservice.VERSION.jar file as mentioned in the below screenshot and place it in NiFi's /lib/bootstrap directory.
Copy the information from the /dist/bin and paste it into NiFi's /bin folder.
Go to the NiFi's /bin folder and execute bin/install_service.bat. The service will show up in the service control manager as "Apache NiFi," with a service name of "nifi-service."
Go to services to ensure that ‘nifi-service’ is running as expected.

Note:
1. NiFi will be running on the 9443 port.
2. To run the service as a user other than the system administrator, it is required to set the JAVA_HOME environment variable, which will be displayed to the new user as a system variable. You may need to grant permissions (full access) to service run-as user to the entire nifi installation folder.

Creating Connections

To create a connection to On-Premise Data Sources using the OvalEdge Bridge, follow the below steps:

Navigate to the Administration > Crawler module, and the Crawler Information page is displayed.
In the Crawler Information page, click on the New Connection button. The Manage Connection with Search Connector pop-up window is displayed.
In the Manage Connection pop-up window, select the connection type as MySQL. The Manage Connection with MySQL-specific details pop-up window is displayed.
Select the Bridge from the Bride dropdown (all the bridges added in the Bridge page are displayed in the Bridge dropdown), enter the required connection details, and click on the Validate button.
The entered connection details are validated and the Save and Save & Configure buttons are enabled.
Click on the Save button to establish the connection, or you can also directly click on the Save & Configure button to establish the connection and configure the connection settings. When you click on the Save & Configure button, the Connection Settings pop-up window is displayed, where you can configure the connection settings for the selected Connector.
Once the connection is established successfully, the connection details are displayed in the Crawler Information page with the Bridge ID and Bridge Name.

Uninstallation of Bridge as a Service

If you wish to uninstall the Bridge as a service, follow the below steps:

Execute sc delete nifi-service Windows service manager commands.
Run /bin/nifi-service.exe //DS//nifi-service command.

Note: When NiFi starts up, it unpacks all jars, which may take some time. If you issue the stop command during this time, it may take a while for the service to stop. The apache daemon may even terminate the process if it is taking too long, but rest assured, the shutdown has been properly initiated.