This document provides guidance on migrating from previous versions to the latest Release6.3.3 version.
Prerequisites for Migration
Before initiating the migration process, ensure that the following prerequisites are met:
Supporting Application Configurations
-
Upgrade Infrastructure (Optional)
Ensure that the applications and the operating system support the upgraded amount of RAM and have sufficient space on the new SSD for optimal performance.
-
Upgrade Database Version
Upgrade the database to the below-mentioned versions for software performance and security. The previous versions of MySQL are no longer supported.
- Upgrade MySQL Version to 8.0.36 or higher. For more information, refer to
- Upgrade MariaDB Version to 10.6.16+ or 10.11.6+.
Pre Upgradation
Take Database Backup
Ensure a comprehensive backup of the Database (MySQL or MariaDB) is taken.
Please take a look at the following document for backup instructions.
Table Row Count Validation
Validate the row count of all database entities to confirm the success of the migration process.
Detailed instructions are available in the provided document.
Externalizing oasis.properties
- Externalizing oasis.properties on Windows involves configuring Tomcat as a service to enable external accessibility.
For more information, refer to the document below:
Externalizing Oasis Properties for Tomcat in Windows. - Externalizing oasis.properties on Linux: On Linux, the externalization of oasis.properties requires a strategic approach to key settings.
For more information, refer to the document below:
Externalizing Oasis Properties for Tomcat on Linux.
Externalizing log4j.properties
Externalizing of log4j.properties is mandatory to overcome the logs writing into Catalina. For some of the customers, the space accumulates in the Tomcat folder.
Refer to the document for detailed steps on Externalizing log4j.properties
Upgrade or Install ElasticSearch
Upgrade ElasticSearch to 7.17.x for improved search capabilities within the OvalEdge. Refer to the document for more information.
Upgrade Java Version (optional)
It is recommended that Java be upgraded to version 8U_422 for optimal performance and compatibility.
For more information, refer to Upgrading Java on Windows
Upgrade Tomcat Service Version
It is recommended that Tomcat be upgraded to version 9.0.x.
For more information, refer to
Upgrading Tomcat from 9.0.x to 9.0.95 with Step-by-Step Instructions.
Application Backups
Before commencing the migration, execute the following backup and validation procedures:
Take Application War, and Jars Backup
Take a backup of War files from Apache Tomcat > Webapps and Jars (csp libs). This step is critical to safeguard against any potential disruptions during the migration process.
Take a Backup of the Shared NFS Folder
Take a backup of the files present in the shared NFS repository.
Tomcat Server Backup & Preparation
Take a backup of the Tomcat server. Clearing these folders helps create a clean environment for the upcoming migration, minimizing the risk of conflicts. In the Tomcat server, empty the following folders:
- Webapps folder
- Temp folders (excluding Jar files). Please ensure that jar files are not removed by accident.
- Logs folders
System Settings Backup
Take the backup of the system settings table from the OvalEdge database in XLSX format to ensure the configured default settings are available to reflect in the new version.
For detailed instructions on taking table backups to CSV/XLSX, refer to the MySQL Table Backup to CSV/XLSX Guide
Migration Process
For VM, the migration process involves downloading release artifacts links provided in the email. The procedure involves sequential tasks, such as downloading the war from the given link, configuring oasis.properties, and running the Tomcat service to reflect the latest changes. Furthermore, executing advanced jobs is necessary to observe the latest changes specific to certain functionalities.
War Replacement
- Download the war files from the provided email link.
- Once downloaded, go to the Tomcat directory, find the "Webapps'' folder, and copy the downloaded latest war file.
- This ensures the integration of the latest functionalities, improvements, and gap resolutions addressed in Release6.3.3 so that the OvalEdge application can function properly.
Connectors & Lineage/Jars Upgrade
- Download the Jar files using the link provided in the email.
- Replace the existing csplib / jars within the External Jars Directory > third_party_jars.
- Ensure that the csp library (csp-lib-6.3.3-RELEASE-with-dependencies.jar) for standard connectors is updated.
- Ensure that the “oe-lineage.jar” is updated.
- Update the csp library for big data connectors (client specific)
Advanced Job/Jars
- Download the Jar files using the link provided in the email.
- Replace the existing csplib / jars within the External Jars Directory > third_party_jars.
- Ensure that the “oe-advancejob.jar” is updated.
Setting oasis.properties in Tomcat
Within the VM, users can customize the oasis.properties in the External Property Directory file to meet their specific requirements, including database connectivity, security, authentication (SSO), Elasticsearch, management of external dependencies, and configuration storage in key vaults.
Note: The oasis.properties file now includes a new configuration parameter, ldap.additional.userSearchBase=, specifically for organizations using LDAP authentication with multiple or additional user search bases. This parameter allows administrators to define an alternative search base for locating user information within the LDAP directory.
For more information, refer to the 6.2 to 6.3.3 changes in oasis.properties file
Governance App Configurations
These configurations are crucial to redirect users to governance-related applications using email or direct links. Refer to the document to configure the Governance application. Governance App Configuration
Tomcat Service Restart
Restart the Tomcat services to apply and update the changes and initiate the migration successfully.
Post Migration Checks
Tables & Row Count Validation
Conduct a thorough verification of data row counts in the post-migration environment.
Ensure that the actual counts match the pre-upgradation counts to identify and rectify discrepancies. This will serve as a key indicator of a successful migration.
The SQL scripts can be used to validate row and table counts.
Download the scripts from the following S3 link: Tables and Row Count Validation Scripts.
Verify Application
Verify that the latest enhancements and features are successfully deployed in the application post-migration.
Run the Advance Jobs
This involves running the Advance Jobs, which are essential processes for ensuring the proper functioning of OvalEdge in its upgraded state.
For more information, please refer to the Appendix A below.
Verify System Settings
The migration process does not mandate adjustments to any system settings. However, it's important to note that for Release6.3.3, new system settings have been introduced, some have been deprecated, and others have undergone modifications. For more information, please refer to the Appendix A below.
Verify Data Row Counts
Conduct a thorough verification of data row counts in the post-migration environment.
Ensure that the actual counts match the pre-migration counts to identify and rectify any discrepancies. This is a key indicator of a successful migration.
Appendix A: Advanced Jobs
After migrating to Release6.3.3, the following advanced jobs must be run
Run the same advanced job that was run for Release6.3 migration and additionally run the below listed advanced jobs associated with Release6.3.3.
A1: 6.3.3 Advanced Job Checklist
|
Advanced Job Name |
Description |
|
HDFS to HIVE Lineage |
This job establishes lineage from HDFS to Hive based on the location of Hive external tables, as specified in the custom fields of the Hive table. |
|
Update Ovalsight Details |
This advanced job is designed to update Ovalsight details and should be run for the first time following a migration. |
|
License Issue |
This advanced job, related to migration, aims to add additional attributes to certain existing connectors, fetch unique values for the connectors, and update the license accordingly. |
Appendix B: New System Settings Added in Release6.3.3
The latest release introduces new configurations that provide users with even greater control over the application's behavior. The newly added configurations are shown below.
|
Key |
Description |
Impacted Modules in OvalEdge |
|
ms.teams.integration.type |
Configure Microsoft Teams integration method for sending notifications. Parameters: The default value is Webhook. If you have updated Microsoft Teams to a version that no longer supports webhooks, switch the integration method to Workflows and set the value to Workflow. |
This setting impacts notifications |
|
connections.order.destination.lineage |
Configure to define the connection order for destination tables based on which the schema, tables, or data objects are selected and displayed in priority order while executing a Snowflake to DocumentDB Lineage advance job. Parameters: The default value is empty. Enter the order of the connection ID separated by commas. |
It impacts the lineage between Snowflake and DocumentDB when built using an advanced job |
|
hide.report.count.in.reportgroupfilter |
Configure to disable the display of the total Report count of a Report Group in the Report Group filter in the Data Catalog grid. Parameters: If set to True, the count will be shown. If set to False, the count will be hidden. |
This impacts the Report Group filter in the Data Catalog List Page for Reports |
Copyright © 2024, OvalEdge LLC, Peachtree Corners, GA USA