OvalEdge Migration Process
      Back to home
      1. Knowledge Base
      2. Architecture, Security & Releases
      3. OvalEdge Migration Process

      Release6.3 Migration Process for VM Model

      This article provides guidance on migrating from previous versions to the latest 6.3 version.

      Prerequisites for Migration

      Before initiating the migration process, ensure that the following prerequisites are met:

      Supporting Application Configurations 

      • Upgrade Infrastructure (Optional)

        Ensure that the applications and the operating system support the upgraded amount of RAM and have sufficient space on the new SSD for optimal performance.

      • Upgrade Database Version

        Upgrade the database to the below-mentioned versions for software performance and security. The previous versions of MySQL are no longer supported. 

      • Pre Upgradation

        • Take Database Backup

          Ensure a comprehensive backup of the Database (MySQL or MariaDB) is taken. Refer to the following document for backup instructions. For more information, OvalEdgeDB Backup Process

        • Table Row Count Validation

          Validate the row count of all database entities to confirm the success of the migration process. Detailed instructions are available in the provided document. DB Migration Checklist

      Externalizing oasis.properties

        • Externalizing log4j.properties

          Externalizing of log4j.properties is mandatory to overcome the logs writing into Catalina. For some of the customers, the space accumulates in the Tomcat folder. Refer to the document for detailed steps on Externalizing log4j.properties

        • Upgrade Java Version (optional)

        It is recommended that Java be upgraded to version 8U_xxx for optimal performance and compatibility. For more information, refer to Upgrading JRE and JDK on Windows.

        • Upgrade Tomcat Service Version

        It is recommended that Tomcat be upgraded to version 9.0.x. For more information, refer to Upgrading Tomcat from 9.0.x to 9.0.83 with Step-by-Step Instructions.

        Application Backups

        Before commencing the migration, execute the following backup and validation procedures:

        • Take Application War, and Jars Backup

          Take a backup of War files from Apache Tomcat > Webapps and Jars (csp libs). This step is critical to safeguard against any potential disruptions during the migration process.

        • Take a Backup of the Shared NFS Folder

          Take a backup of the files present in the shared NFS repository.

        • Tomcat Server Backup & Preparation

          Take a backup of the Tomcat server. Clearing these folders helps create a clean environment for the upcoming migration, minimizing the risk of conflicts. In the Tomcat server, empty the following folders:

          • Webapps folder

          • Temp folders (excluding Jar files). Please ensure that jar files are not removed by accident.

          • Logs folders 

          • System Settings Backup

            Take the backup of the system settings table from the ovaledge database in XLSX format to ensure the configured default settings are available to reflect in the new version.

          Migration Process

          For VM, the migration process involves downloading release artifacts links provided in the email. The procedure involves sequential tasks, such as downloading the war from the given link, configuring oasis.properties, and running the Tomcat service to reflect the latest changes. Furthermore, executing advanced jobs is necessary to observe the latest changes specific to certain functionalities. 

          War Replacement

          1. Download the war files from the provided email link. 
          2. Once downloaded, go to the Tomcat directory, find the "Webapps'' folder, and copy the downloaded latest war file. 
          3. This ensures the integration of the latest functionalities, improvements, and gap resolutions addressed in Release6.3 so that the OvalEdge application can function properly.

          Connectors & Lineage/Jars Upgrade

          1. Download the Jar files using the link provided in the email.
          2. Replace the existing csplib / jars within the External Jars Directory > third_party_jars. 
            1. Ensure that the csp library for standard connectors is updated.
            2. Ensure that the “oe-lineage jar” is updated.
            3. Update the csp library for big data connectors (client specific)

          Setting oasis.properties in Tomcat  

          Within the VM, users can customize the oasis.properties in the External Property Directory file to meet their specific requirements, including database connectivity, security, authentication (SSO), Elasticsearch, management of external dependencies, and configuration storage in key vaults. 

          oasis.properties to Migrate from 6.0 to 6.3

          Please make the changes below if oasis.properties is set up to read from an external location or if properties are configured in environment variables for Docker environments.

          The “Keys” listed below are removed in Release6.1

          • saml-metadata-type=url
          • entity-base-islb=false
          • entity-base-protocol=https
          • entity-base-host=localhost
          • entity-base-port=443
          • entity-base-contextpath=/ovaledge
          • entity-base-port-in-url=false
          • spring.security.oauth2.tenantId=tenantId

          There are few modifications to the existing values of keys

          Key

          Previous Value

          New Value

          Note: The parameters below should be updated if moving from MySql to Mariadb.

          driverClassName

          com.mysql.cj.jdbc.Driver

          org.mariadb.jdbc.Driver (for MariaDB)

          url

          jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

          jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

          read.url

          jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

          jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

          Note: The parameters below need to be verified and updated in the oasis.properties with the new values, whether MySQL or Mariadb.

          db.max.connections

          40

          30

          read.db.max.connections

          60

          40

          spring.session.db.max.connections

          40

          60

          There are a few modifications to the existing keys, as mentioned in the following table

          Existing Keys

          Renamed As

          azure.secretname.jdbcstring=jdbcstringurl

          secret.key.jdbcstring=jdbcstringurl

          azure.secretname.username=username

          secret.key.username=username

          azure.secretname.password=password

          secret.key.password=password

          azure.secretname.read.jdbcstring=readjdbcstringurl

          secret.key.read.jdbcstring=readjdbcstringurl

          azure.secretname.encryptdecryptkey=encryptdecryptkey

          secret.key.encryptdecryptkey=encryptdecryptkey

          azure.secretname.eshost=eshost

          secret.key.eshost=eshost

          azure.secetname.esport=esport

          secret.key.esport=esport

          azure.secretname.esprotocol=esprotocol

          secret.key.esprotocol=esprotocol

          azure.secretname.esusername=esusername

          secret.key.esusername=esusername

          azure.secretname.espassword=espassword

          secret.key.espassword=espassword

          The “Keys” listed below are newly introduced in Release 6.1.

          New Key

          Value

          aws-secrets

          <true/false>

          aws-secretregion

          <aws region where the secret name exists>

          aws-secretname

          <secret name from which the parameters are read>

          hikari.connectionTimeout.in.seconds

          50

          hikari.idleTimeout.in.minutes

          2

          hikari.validationTimeout.in.seconds

          50

          hikari.leakDetectionThreshold.in.seconds

          30

          hikari.cachePrepStmts

          true

          hikari.prepStmtCacheSize

          250

          hikari.prepStmtCacheSqlLimit

          2048

          hikari.useServerPrepStmts

          true

          hikari.useLocalSessionState

          true

          hikari.rewriteBatchedStatements

          true

          hikari.cacheResultSetMetadata

          true

          hikari.cacheServerConfiguration

          true

          hikari.elideSetAutoCommits

          true

          Note: Refer to the AWS Secrets Manager Configurations article to learn more about the newly added keys.

          oasis.properties to Migrate from 6.2 to 6.3

          Please note that the following configurations are included in the Master DB Instance and the Read Replica Instance Sections.

          Master DB Instance Section

          # Mysql 8.0.x Driver and Url for Azure Entra ID authentication

          #driverClassName=com.mysql.cj.jdbc.Driver

          #url=jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&allowPublicKeyRetrieval=true&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin


          # MariaDB 10.x.x Driver and Url for Azure Entra ID authentication

          #driverClassName=org.mariadb.jdbc.Driver

          #url=jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin


          Read Replica Instance Section

          # MySQL 8.0.x Url for Azure Entra ID authentication

          #read.url=jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&allowPublicKeyRetrieval=true&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin


          # MariaDB 10.x.x Url for Azure Entra ID authentication

          #read.url=jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&allowPublicKeyRetrieval=true&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin

          Tomcat Service Restart

          Restart the Tomcat services to apply and update the changes and initiate the migration successfully.

          Post Migration Checks 

          Tables & Row Count Validation

          Conduct a thorough verification of data row counts in the post-migration environment.

          Ensure that the actual counts match the pre-upgradation counts to identify and rectify any discrepancies. This will serve as a key indicator of a successful migration. 

          Verify Application

          Verify that the latest enhancements and features are successfully deployed in the application post-migration.

          Run the Migration Advance Jobs

          This involves running the Migration Advance Jobs, which are essential processes for ensuring the proper functioning of OvalEdge in its upgraded state. Detailed instructions are in Appendix A of the migration documentation.

          Verify System Settings

          The migration process does not mandate adjustments to any system settings. However, it's important to note that for Release6.3, new system settings have been introduced, some have been deprecated, and others have undergone modifications. For more information, please refer to the appendices below.

          Verify Data Row Counts

          Conduct a thorough verification of data row counts in the post-migration environment.

          Ensure that the actual counts match the pre-migration counts to identify and rectify any discrepancies. This is a key indicator of a successful migration. 

          Appendix A: Migration Advanced Jobs

          While migrating to Release6.3, the following advanced jobs must be run

          Run the same advanced job that was run for Release6.2 migration and additionally run the below listed advanced job to migrate to Release6.3. 

          A1: 6.3 Migration Advanced Job Checklist 

          Advanced Job Name

          Module Name

          Description

          INSERT_NOTIFICATION_PREFERENCE

          User Notification Preferences Manager

          This job enables the addition of new modules and submodules in the notification preference table of the My Profile section for all users. 

          METADATA_CURATION_SCORES_TO_DENORMALIZED_TABLES

          Meta Data Curation Scores 

          This migration job helps to calculate the metadata curation score for existing objects in version 6.2 and saves the metadata curation score values in denormalized tables(ext) associated with the objects.

          UPDATE_CODE_SORTORDER

          Helpful Resources

          This job helps to migrate helpful resources from 6.2 to 6.3. It lets users move the previously added helpful resources using the drag-and-drop option in the updated version.

          Note: This job sorts the order of existing helpful resources for customers migrating from 6.1/6.2 to 6.3. A fresh installation does not require this job.

          RestoreBusinessGlossaryObjectsSourceInfo

          Business Glossary

          This job helps to migrate source information for associated objects in the business glossary for previously added objects.

          Note: 

          • From Release6.3, the source information for associated objects is automatically generated.
          • This job is used to restore the BG source info for the customers migrating from 6.1/6.2 to 6.3. A fresh installation does not require this job.

          QUESTIONWALL_MIGRATION

          QuestionWall Migration

          This job helps to migrate the questions and answers (replies) previously found in Question Wall > Inbox to Question Wall > General Wall.

          LOAD_DATA

          Data Stories Migration

          This Advanced Job is designed to transfer Data Stories from one environment to another. It lets users download Data Stories from the source environment and upload them to the target environment using the same job. 

          Attributes: 

          Attr1: Specify ""Upload"" or ""Download"" based on the action. For Downloads, use Attr1 = ""Download""; for Uploads, use Attr1 = ""Upload"". 

          Attr2 - Enter File Server Connection Id (optional).

          Attr3 - Enter the file path. 

          Attr4 - Enter Story Zone Names or IDs with comma-separated values (optional)

          Attr5 - Enter Story Names or IDs with Comma-separated values (optional).

          POWERBI_MIGRATION

          Reports

          This job helps move the existing report columns and their curated information to their related report columns, dataset columns, and page columns, which in turn move to report columns.

          Note: This job must run if the client has a PowerBI connection created in versions before 6.3.

          FILL_CRON_AND_ASSOCIATION_DESC

          Schedule 

          This job helps to include the filter icon options for the Schedule > Cron type column in the administration and improves the search functionality in the Data Catalog > Codes > Associations > Associate objects column.

          PROJECT_STATUS_SYNC

          Projects

          This advanced job syncs old and new project statuses and stats in the updated version, which is required for migrating the current application to version 6.0.

          Attributes: No attributes needed.

          Appendix B: New System Settings Added in Release6.3

          The latest release introduces new configurations that provide users with even greater control over the application's behavior. The newly added configurations are shown below.

          Key

          Description

          assetmanager.virusscan.enabled

          Configure to enable virus scan for the uploaded files.

          Parameters:

          The default value is False.

          If set to True, virus scan on files is enabled.

          If set to False, virus scan on files is disabled.

          assetmanager.storage.service

          Configure to upload a file from the Governance Apps and store it in the mentioned storage service.

          Parameters:

          The default value is Empty.

          Enter the storage service name.

          assetmanager.storage.connId

          Configure to enter the connection ID of the storage service.

          Parameters:

          The default value is Empty.

          Enter the connection ID.

          assetmanager.storage.service.bucketname

          Configure to enter the bucket name of the storage service.

          Parameters:

          The default value is Empty.

          Enter the storage service bucket name.

          oe.crawl.excludeTables

          Configure to hide the tables.

          Parameters:

          The default value is Empty.

          Enter the table names separated by a comma.

          smtp.oauth2.tenantid

          Enter Tenant ID for OAuth Token Validation and SMTP Server Authentication.

          Parameters: 

          By default, this setting is Empty.

          Enter a tenant ID in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          smtp.oauth2.clientid

          Enter Client ID for OAuth Token Validation and SMTP Server Authentication.

          Parameter: 

          By default, this setting is Empty.

          Enter a Client ID in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          smtp.oauth2.clientsecret

          Enter Client Secret for OAuth Token Validation and SMTP Server Authentication.

          Parameter: 

          By default, this setting is Empty. 

          Enter a Client Secret in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          smtp.oauth2.cloud.name

          Enter Cloud Name for OAuth Token Validation and SMTP Server Authentication.

          Parameter: 

          By default, this setting is Empty. 

          Enter a Cloud Name in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          Copyright © 2024, OvalEdge LLC, Peachtree Corners, GA USA