OvalEdge Migration Process
      Back to home
      1. Knowledge Base
      2. Architecture, Security & Releases
      3. OvalEdge Migration Process

      Release6.3 Migration Process for VM Model

      This article provides guidance on migrating from previous versions to the latest 6.3 version.

      Prerequisites for Migration

      Before initiating the migration process, ensure that the following prerequisites are met:

      Supporting Application Configurations 

      • Upgrade Infrastructure (Optional)

      Ensure that the applications and the operating system support the upgraded amount of RAM and have sufficient space on the new SSD for optimal performance.

      • Upgrade Database Version

      Upgrade the database to the below-mentioned versions for software performance and security. The previous versions of MySQL are no longer supported. 

      • Pre Upgradation

        • Take Database Backup

          Ensure a comprehensive backup of the Database (MySQL or MariaDB) is taken. Refer to the following document for backup instructions. For more information, OvalEdgeDB Backup Process

        • Table Row Count Validation

          Validate the row count of all database entities to confirm the success of the migration process. Detailed instructions are available in the provided document. DB Migration Checklist

      • Externalizing oasis.properties

        Externalizing oasis.properties on Windows involves configuring Tomcat as a service to enable external accessibility. For more information, refer to the document below: Externalizing Oasis Properties for Tomcat in Windows.

        Externalizing oasis.properties on Linux: On Linux, the externalization of oasis.properties requires a strategic approach to key settings.  

      • Externalizing log4j.properties

        Externalizing of log4j.properties is mandatory to overcome the logs writing into Catalina. For some of the customers, the space accumulates in the Tomcat folder. Refer to the document for detailed steps on Externalizing log4j.properties

      • Upgrade Java Version (optional)

      It is recommended that Java be upgraded to version 8U_xxx for optimal performance and compatibility. For more information, refer to Upgrading JRE and JDK on Windows.

      • Upgrade Tomcat Service Version

      It is recommended that Tomcat be upgraded to version 9.0.x. For more information, refer to Upgrading Tomcat from 9.0.x to 9.0.83 with Step-by-Step Instructions.

      Application Backups

      Before commencing the migration, execute the following backup and validation procedures:

      • Take Application War, and Jars Backup

        Take a backup of War files from Apache Tomcat > Webapps and Jars (csp libs). This step is critical to safeguard against any potential disruptions during the migration process.

      • Take a Backup of the Shared NFS Folder

        Take a backup of the files present in the shared NFS repository.

      • Tomcat Server Backup & Preparation

        Take a backup of the Tomcat server. Clearing these folders helps create a clean environment for the upcoming migration, minimizing the risk of conflicts. In the Tomcat server, empty the following folders:

        • Webapps folder

        • Temp folders (excluding Jar files). Please ensure that jar files are not removed by accident.

        • Logs folders 

        • System Settings Backup

          Take the backup of the system settings table from the ovaledge database in XLSX format to ensure the configured default settings are available to reflect in the new version.

        Migration Process

        For VM, the migration process involves downloading release artifacts links provided in the email. The procedure involves sequential tasks, such as downloading the war from the given link, configuring oasis.properties, and running the Tomcat service to reflect the latest changes. Furthermore, executing advanced jobs is necessary to observe the latest changes specific to certain functionalities. 

        War Replacement

        1. Download the war files from the provided email link. 
        2. Once downloaded, go to the Tomcat directory, find the "Webapps'' folder, and copy the downloaded latest war file. 
        3. This ensures the integration of the latest functionalities, improvements, and gap resolutions addressed in Release6.3 so that the OvalEdge application can function properly.

        Connectors & Lineage/Jars Upgrade

        1. Download the Jar files using the link provided in the email.
        2. Replace the existing csplib / jars within the External Jars Directory > third_party_jars. 
          1. Ensure that the csp library for standard connectors is updated.
          2. Ensure that the “oe-lineage jar” is updated.
          3. Update the csp library for big data connectors (client specific)

        Setting oasis.properties in Tomcat  

        Within the VM, users can customize the oasis.properties in the External Property Directory file to meet their specific requirements, including database connectivity, security, authentication (SSO), Elasticsearch, management of external dependencies, and configuration storage in key vaults. 

        oasis.properties to Migrate from 6.0 to 6.3

        Please make the changes below if oasis.properties is set up to read from an external location or if properties are configured in environment variables for Docker environments.

        The “Keys” listed below are removed in Release6.1

        • saml-metadata-type=url
        • entity-base-islb=false
        • entity-base-protocol=https
        • entity-base-host=localhost
        • entity-base-port=443
        • entity-base-contextpath=/ovaledge
        • entity-base-port-in-url=false
        • spring.security.oauth2.tenantId=tenantId

        There are few modifications to the existing values of keys

        Key

        Previous Value

        New Value

        Note: The parameters below should be updated if moving from MySql to Mariadb.

        driverClassName

        com.mysql.cj.jdbc.Driver

        org.mariadb.jdbc.Driver (for MariaDB)

        url

        jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

        jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

        read.url

        jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

        jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false

        Note: The parameters below need to be verified and updated in the oasis.properties with the new values, whether MySQL or Mariadb.

        db.max.connections

        40

        30

        read.db.max.connections

        60

        40

        spring.session.db.max.connections

        40

        60

        There are a few modifications to the existing keys, as mentioned in the following table

        Existing Keys

        Renamed As

        azure.secretname.jdbcstring=jdbcstringurl

        secret.key.jdbcstring=jdbcstringurl

        azure.secretname.username=username

        secret.key.username=username

        azure.secretname.password=password

        secret.key.password=password

        azure.secretname.read.jdbcstring=readjdbcstringurl

        secret.key.read.jdbcstring=readjdbcstringurl

        azure.secretname.encryptdecryptkey=encryptdecryptkey

        secret.key.encryptdecryptkey=encryptdecryptkey

        azure.secretname.eshost=eshost

        secret.key.eshost=eshost

        azure.secetname.esport=esport

        secret.key.esport=esport

        azure.secretname.esprotocol=esprotocol

        secret.key.esprotocol=esprotocol

        azure.secretname.esusername=esusername

        secret.key.esusername=esusername

        azure.secretname.espassword=espassword

        secret.key.espassword=espassword

        The “Keys” listed below are newly introduced in Release 6.1.

        New Key

        Value

        aws-secrets

        <true/false>

        aws-secretregion

        <aws region where the secret name exists>

        aws-secretname

        <secret name from which the parameters are read>

        hikari.connectionTimeout.in.seconds

        50

        hikari.idleTimeout.in.minutes

        2

        hikari.validationTimeout.in.seconds

        50

        hikari.leakDetectionThreshold.in.seconds

        30

        hikari.cachePrepStmts

        true

        hikari.prepStmtCacheSize

        250

        hikari.prepStmtCacheSqlLimit

        2048

        hikari.useServerPrepStmts

        true

        hikari.useLocalSessionState

        true

        hikari.rewriteBatchedStatements

        true

        hikari.cacheResultSetMetadata

        true

        hikari.cacheServerConfiguration

        true

        hikari.elideSetAutoCommits

        true

        Note: Refer to the AWS Secrets Manager Configurations article to learn more about the newly added keys.

        oasis.properties to Migrate from 6.2 to 6.3

        Please note that the following configurations are included in the Master DB Instance and the Read Replica Instance Sections.

        Master DB Instance Section

        # Mysql 8.0.x Driver and Url for Azure Entra ID authentication

        #driverClassName=com.mysql.cj.jdbc.Driver

        #url=jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&allowPublicKeyRetrieval=true&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin


        # MariaDB 10.x.x Driver and Url for Azure Entra ID authentication

        #driverClassName=org.mariadb.jdbc.Driver

        #url=jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin


        Read Replica Instance Section

        # MySQL 8.0.x Url for Azure Entra ID authentication

        #read.url=jdbc:mysql://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&allowPublicKeyRetrieval=true&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin


        # MariaDB 10.x.x Url for Azure Entra ID authentication

        #read.url=jdbc:mariadb://localhost:3306/ovaledgedb?useUnicode=true&character_set_server=utf8mb4&useSSL=false&allowPublicKeyRetrieval=true&authenticationPlugins=com.azure.identity.extensions.jdbc.mysql.AzureMysqlAuthenticationPlugin

        Tomcat Service Restart

        Restart the Tomcat services to apply and update the changes and initiate the migration successfully.

        Post Migration Checks 

        Tables & Row Count Validation

        Conduct a thorough verification of data row counts in the post-migration environment.

        Ensure that the actual counts match the pre-upgradation counts to identify and rectify any discrepancies. This will serve as a key indicator of a successful migration. 

        Verify Application

        Verify that the latest enhancements and features are successfully deployed in the application post-migration.

        Run the Migration Advance Jobs

        This involves running the Migration Advance Jobs, which are essential processes for ensuring the proper functioning of OvalEdge in its upgraded state. Detailed instructions are in Appendix A of the migration documentation.

        Verify System Settings

        The migration process does not mandate adjustments to any system settings. However, it's important to note that for Release6.3, new system settings have been introduced, some have been deprecated, and others have undergone modifications. For more information, please refer to the appendices below.

        Verify Data Row Counts

        Conduct a thorough verification of data row counts in the post-migration environment.

        Ensure that the actual counts match the pre-migration counts to identify and rectify any discrepancies. This is a key indicator of a successful migration. 

        Appendix A: Migration Advanced Jobs

        While migrating to Release6.3, the following advanced jobs must be run

        Run the same advanced job that was run for Release6.2 migration and additionally run the below listed advanced job to migrate to Release6.3. 

        A1: 6.3 Migration Advanced Job Checklist 

        Advanced Job Name

        Module Name

        Description

        INSERT_NOTIFICATION_PREFERENCE

        User Notification Preferences Manager

        This job enables the addition of new modules and submodules in the notification preference table of the My Profile section for all users. 

        METADATA_CURATION_SCORES_TO_DENORMALIZED_TABLES

        Meta Data Curation Scores 

        This migration job helps to calculate the metadata curation score for existing objects in version 6.2 and saves the metadata curation score values in denormalized tables(ext) associated with the objects.

        UPDATE_CODE_SORTORDER

        Helpful Resources

        This job helps to migrate helpful resources from 6.2 to 6.3. It lets users move the previously added helpful resources using the drag-and-drop option in the updated version.

        Note: This job sorts the order of existing helpful resources for customers migrating from 6.1/6.2 to 6.3. A fresh installation does not require this job.

        RestoreBusinessGlossaryObjectsSourceInfo

        Business Glossary

        This job helps to migrate source information for associated objects in the business glossary for previously added objects.

        Note: 

        • From Release6.3, the source information for associated objects is automatically generated.
        • This job is used to restore the BG source info for the customers migrating from 6.1/6.2 to 6.3. A fresh installation does not require this job.

        QUESTIONWALL_MIGRATION

        QuestionWall Migration

        This job helps to migrate the questions and answers (replies) previously found in Question Wall > Inbox to Question Wall > General Wall.

        LOAD_DATA

        Data Stories Migration

        This Advanced Job is designed to transfer Data Stories from one environment to another. It lets users download Data Stories from the source environment and upload them to the target environment using the same job. 

        Attributes: 

        Attr1: Specify ""Upload"" or ""Download"" based on the action. For Downloads, use Attr1 = ""Download""; for Uploads, use Attr1 = ""Upload"". 

        Attr2 - Enter File Server Connection Id (optional).

        Attr3 - Enter the file path. 

        Attr4 - Enter Story Zone Names or IDs with comma-separated values (optional)

        Attr5 - Enter Story Names or IDs with Comma-separated values (optional).

        POWERBI_MIGRATION

        Reports

        This job helps move the existing report columns and their curated information to their related report columns, dataset columns, and page columns, which in turn move to report columns.

        Note: This job must run if the client has a PowerBI connection created in versions before 6.3.

        FILL_CRON_AND_ASSOCIATION_DESC

        Schedule 

        This job helps to include the filter icon options for the Schedule > Cron type column in the administration and improves the search functionality in the Data Catalog > Codes > Associations > Associate objects column.

        PROJECT_STATUS_SYNC

        Projects

        This advanced job syncs old and new project statuses and stats in the updated version, which is required for migrating the current application to version 6.0.

        Attributes: No attributes needed.

        Appendix B: New System Settings Added in Release6.3

        The latest release introduces new configurations that provide users with even greater control over the application's behavior. The newly added configurations are shown below.

        Key

        Description

        assetmanager.virusscan.enabled

        Configure to enable virus scan for the uploaded files.

        Parameters:

        The default value is False.

        If set to True, virus scan on files is enabled.

        If set to False, virus scan on files is disabled.

        assetmanager.storage.service

        Configure to upload a file from the Governance Apps and store it in the mentioned storage service.

        Parameters:

        The default value is Empty.

        Enter the storage service name.

        assetmanager.storage.connId

        Configure to enter the connection ID of the storage service.

        Parameters:

        The default value is Empty.

        Enter the connection ID.

        assetmanager.storage.service.bucketname

        Configure to enter the bucket name of the storage service.

        Parameters:

        The default value is Empty.

        Enter the storage service bucket name.

        oe.crawl.excludeTables

        Configure to hide the tables.

        Parameters:

        The default value is Empty.

        Enter the table names separated by a comma.

        smtp.oauth2.tenantid

        Enter Tenant ID for OAuth Token Validation and SMTP Server Authentication.

        Parameters: 

        By default, this setting is Empty.

        Enter a tenant ID in the field provided to validate an OAuth token and set up SMTP Server Authentication.

        smtp.oauth2.clientid

        Enter Client ID for OAuth Token Validation and SMTP Server Authentication.

        Parameter: 

        By default, this setting is Empty.

        Enter a Client ID in the field provided to validate an OAuth token and set up SMTP Server Authentication.

        smtp.oauth2.clientsecret

        Enter Client Secret for OAuth Token Validation and SMTP Server Authentication.

        Parameter: 

        By default, this setting is Empty. 

        Enter a Client Secret in the field provided to validate an OAuth token and set up SMTP Server Authentication.

        smtp.oauth2.cloud.name

        Enter Cloud Name for OAuth Token Validation and SMTP Server Authentication.

        Parameter: 

        By default, this setting is Empty. 

        Enter a Cloud Name in the field provided to validate an OAuth token and set up SMTP Server Authentication.

        Copyright © 2024, OvalEdge LLC, Peachtree Corners, GA USA