OvalEdge Releases
      Back to home
      1. Knowledge Base
      2. Architecture, Security & Releases
      3. OvalEdge Releases

      Release6.3

      OvalEdge Release6.3 introduces major enhancements to the Question Wall, AI Curation, Data Access Management, Governance Apps, Data Classification Recommendation, Data Quality Rules Recommendations, Domain Relationship, Data Lake, and Folder Analysis Dashboard.

      Minor improvements were also made to the Tags, Data Catalog, Service Desk, Data Quality, and Administration modules, and critical bugs across various modules have been addressed.  

      Release Type   

      Release Version

      Build

      <Release. Build Number. Release Stamp>

      Build Date
      Minor Release Release6.3 Release6.3.6300.ae28af1 12 June, 2024

      Major Enhancements

      Question Wall

      The Question Wall allows users to submit questions, enabling open communication and knowledge sharing within the organization. Subject Matter Experts in relevant areas will address the questions. 

      The Question Wall consists of My Walls, Community Walls, Rooms, and Walls.

      • My Walls: This section provides easy access to discussions and questions, and users can control their interactions.
      • Community Walls: It acts as a central hub, displaying all the available Rooms and Walls that hold discussions and questions within a collaborative platform.
      • Rooms: A space to create walls. A room can have many walls.
      • Wall: A wall is a section of a room where questions are asked and discussions take place.

      AI Curation

      The Curation by AI feature generates business, technical, or detailed descriptions for objects and terms based on the provided questions and answers.

      • AI Prompt:  Administrators can add questions in the AI Prompt repository.
      • Unique Questions for Each Object Type: AI Prompt offers questions for each object type. Object types include Data Catalog Objects and Business Glossary Domains and terms.
      • Global or Connector Level: Administrators can add questions to the objects at the Global level (All Connector or All Domains) or Connector Level (Specific Connector or Specific Domain) based on the selected type of description (Business or Technical or Detailed).
      • Assigning Users: Assigned Governance Roles or  Administrators can add, edit, and assign questions to other users (basic application users with minimum Meta-Read permissions on the object) for answers.
      • Notifications for Collaboration: The system notifies relevant assignees of question assignments or deletions.
      • Generating Descriptions: Users with Meta-Write permissions on the object can create descriptions manually if all questions are unanswered or generate descriptions automatically based on the questions answered.

      Metadata Curation

      A metadata curation score helps assess the quality, completeness, and accuracy of metadata and data elements in the application. This score reflects how well metadata is managed, considering factors like metadata name clarity, relevance, and accuracy of descriptions (both business and technical), presence of associated terms and tags, assignment of governance roles, use of custom fields for additional context, and overall metadata quality across different sections like Data Catalog > Object Types and Business Glossary > Terms.

      Data Access Management 

      Support for Tableau Cloud has been added for Data Access Management. The main features include:

      • Management of groups, users, and permissions at the site level
      • Simplified access to view permissions granted on projects, sub-projects, and various types of reports like workbooks, data sources, and views to site groups and users
      • Audit logs for all the access management actions managed through OvalEdge
      • Synchronizing Tableau's Projects and Reports permissions with OvalEdge Security

      Report Curation in Data Catalog

      • The Report Columns Table summary has been improved by adding the following new columns for metadata.
        • Table (Only Power BI - Dataset): Displays the Semantic tables associated with the Semantic Columns.
        • Attribute Type (Only Power BI - Dataset): Displays whether a Semantic Column is a Measure or a Dimension.
        • Subreport (All Reports): Displays the Subreport associated with the Report Columns.
        • Subreport Type (All Reports): Displays the Subreport type of the associated Subreport.
        • Favorites: Users can mark a column as a favorite. Favorite report columns will appear at the top of the report columns list on the summary page.
      • A separate tab for Report Previewing is available to provide a focused view, allowing users to access and understand the underlying data and structure of the Reports.
      • The Subreport tab enhances user understanding of report hierarchies.
      • For Power BI, the Report Summary displays Report Columns and Semantic Columns separately.
      • The Power BI Report summary in the Data Catalog incorporates additional metadata fields for datasets, such as Measures, Dimensions, and Table Count.
      • The report summary now displays the Sub-report count for all the reports.

      Data Classification Recommendation

      The Data Classification Recommendation enables users to create AI models providing recommendations for data objects. Users can create, run, edit, and delete these models and schedule them as needed. 

      Each AI model offers recommendations that users can accept or reject. Users can assess the AI model's performance through AI Model Metrics and restore rejected recommendations. 

      In the Business Glossary > Term Summary > Recommendations, users can view recommendations from multiple models. If a term does not have recommendations, users can navigate to Data Classification Recommendation to create a new model for the selected term.

      Data Quality Rule Recommendations

      The Data Quality Rule Recommendations enable users to create rule models providing recommendations for data objects (Table or File). Users can manage and organize these recommendation models.

      Each rule model offers recommendations that users can either accept or ignore. Users can assess each model's performance through Rule Model Metrics and define rule behavior for models using Rule Settings.

      Governance Apps

      The Governance Apps module has been enhanced with the following features:

      1. Form Building: Users can now design forms using drag-and-drop functionality. They can edit multiple fields simultaneously, save the app, and share it using a link.
      2. Upload: The new upload feature allows users to upload various file types, including images and documents (up to 25 MB).
      3. Download & Upload Forms: Users can use the ‘Download’ button to download any form in JSON format and ‘Upload’ existing JSON forms directly into the app.
      4. Responsive Design: Every form automatically adapts for optimal viewing on desktops, mobiles, and tablets, ensuring a smooth user experience across all devices.
      5. Governance App Permissions: Users with Meta-Write permissions for a specific Governance App can edit the app's name, description, and governance roles.

      Domain Relationship Dashboard

      The Domain Relationship Dashboard allows users to view connections between different domains. When users click on the dotted line connecting two domains, they can view the Domain-Domain Relationship diagram. This diagram shows how terms within those domains are connected to each other. To explore a specific term's relationships further, users can click on the term's name, which displays a Term-Relationship diagram, illustrating how that term is connected to data objects and other terms.

      Data Lake Dashboard

      The Data Lake Dashboard has been enhanced with the following features:

      1. Folder Details Tiles: Displays the last folder level, total folders, empty folders, and total file count.
      2. Top 10 File Formats Donut Chart: This chart displays the top 10 file formats with the most files in the selected connector. It shows file names, total files, and percentages. Other file types can be viewed by clicking the "View All" button. A central indicator shows the total number of file types in the connector.
      3. File Size Range Analysis Donut Chart: It categorizes file sizes into five ranges (<100KB, 100KB-1MB, 1MB-10MB, 10MB-100MB, >100MB) within the selected connector. Clicking on a chart segment shows all files within that size range. A central indicator shows the total folder size for the selected connector.
      4. Folder Modification Summary Bar Graph: Displays the last modification dates of subfolders within the selected connector, organized by quarters (Q1, Q2, Q3, Q4). Hovering over a bar shows the folder count for each quarter, and clicking a bar provides a detailed view of folders modified within that quarter.
      5. File Modification Summary Bar Graph: Displays the last modification dates of files within the selected connector, organized by quarters (Q1, Q2, Q3, Q4). Hovering over a bar shows the file count for each quarter, and clicking a bar provides a detailed view of files modified within that quarter.

      Folder Analysis Dashboard

      The Folder Analysis Dashboard has been enhanced with the following features:

      1. Folder Details Tiles: Displays the folder's level, total subfolders, empty subfolders, last folder level, and total file count.
      2. Top 10 File Formats Donut Chart: This chart displays the top 10 file formats with the most files in the selected folder. It shows file names, total files, and percentages. Other file types can be viewed by clicking the "View All" button. A central indicator shows the total number of file types in the folder.
      3. File Size Range Analysis Donut Chart: It categorizes file sizes into five ranges (<100KB, 100KB-1MB, 1MB-10MB, 10MB-100MB, >100MB) within the selected folder. Clicking on a chart segment shows all files within that size range. A central indicator shows the total folder size for the selected folder.
      4. Folder Modification Summary Bar Graph: Displays the last modification dates of subfolders within the selected folder, organized by quarters (Q1, Q2, Q3, Q4). Hovering over a bar shows the folder count for each quarter, and clicking a bar provides a detailed view of folders modified within that quarter.
      5. File Modification Summary Bar Graph: Displays the last modification dates of files within the selected folder, organized by quarters (Q1, Q2, Q3, Q4). Hovering over a bar shows the file count for each quarter, and clicking a bar provides a detailed view of files modified within that quarter.

      Minor Enhancements & Bug Fixes

      Home

      • In the Home at Helpful Resources widget, users can now rearrange the sequence of resources such as tutorials, user manuals, FAQs, best practices, case studies, and other information.
      • The Edit Profile pop-up includes a new validate button for the Microsoft Teams Webhook and Google Chat Webhook URLs, which enables the logged-in user to validate the URLs.
      • Email notifications now include a "View Conversation" option. Clicking this button directly opens the corresponding collaboration message within the user account inbox.
      • Users can integrate Google Chat and Microsoft Teams using webhook links in the My Profile section. For security, these webhook URLs are now masked.

      Tags

      • The Tags List View now provides a customizable display of tags and relevant information.
        • Configure Views allows users to personalize how tags are displayed by prioritizing relevant columns and customizing the layout.
        • System View is the default view. It provides a standardized perspective with a pre-configured set of columns and cannot be edited or deleted.
        • My Views & Public Views allow users to create custom views for personal use and explore views created by others within the application.
      • Authors and Viewer users can now download a complete list of tags from the list view within a folder. This file includes parent tags, tag names, and description details.
      • The "Add Object" functionality is integrated into Tags Associations to assign the required Data Objects.

      Data Catalog

      • In the Tables Summary > Entity Relationships > Tabular, a sort option is added to the “Related Table Row Count” column.
      • In the Table Summary, the profiling status was previously limited to "Full" and "Not Profiled." Now, the statuses have been enhanced to include "Profiled (Partial Success)," "Profiled (Failed)," "Sample Profiled (Success)," "Sample Profiled (Partial Success)," and "Sample Profiled (Failed)."
      • In the Table Columns, File Columns, Report Columns, and API Attributes, a "Clear Search" icon is included for the column name text box, where users can clear the entered text.
      • In Files and File Columns (List View), a File Type filter column is added to help users filter data based on the file types. 
      • In the Data Catalog > Tabular Lineage, all users, including viewers, can now download the Table Lineage. 
      • Users can now view the source details (LMDF/API/UI) of the Terms, Tags, or Data Quality Rules through the Source Column in the List View.

      Business Glossary

      • In the Term Summary > AI Settings, users can run a "Regular Expression" (E.g., hello. *) to find the required data pattern of objects associated with a Term, where the user needs to add the required data pattern and AI boost score to boost the smart score when the regex matches.
      • Users can personalize their Business Glossary experience by setting the default view to a Tree or List view. This configuration is accessible through Administration > System Settings > Others > Key (default.view.businessglossary) > Value (Tree or List)
      • All User Licenses, including Viewer, can now "Suggest a Term" and "Suggest a Tag.” This functionality is available with existing Terms and Tags in the Data Catalog, Business Glossary, Governance Catalog, Data Stories, and Data Quality Rules.
      • In Domain Security, changing the default association preference for any domain will no longer affect the existing terms. However, the new preference will be applied whenever a new term is created under that domain.
      • The Business Glossary list view now includes a "Number of Associated Objects" column, which shows the number of objects linked to each term.
      • In the Business Glossary, the following permissions for the Business Users are removed:
        • The Edit access for Business Description, Tags, and Governance Roles has been removed from the List View.
        • The Edit access for tags has been removed from the term summary.
        • The Add Tags and Remove Tags options are disabled from the nine dots.

      Note:

      • Change Management
        Users can create and schedule AI models through Data Classification Recommendations and accept and reject them on the term summary page. As the approach to generating recommendations is changed in the latest release, the previous recommendations generated will be lost. Users can accept or reject the recommendations before migrating to Release6.3

      Data Stories

      • In Data Stories, the story reordering is now limited to administrators.
      • Users can now export and import data stories from different environments.

      Projects 

      In Projects, Viewer can now perform the following tasks:

      • Create a project.
      • Change any Project member to be a Project Owner.
      • Add objects/tasks to the project.
      • Add or remove users from the project.

      Service Desk

      • The Service Desk now offers dedicated templates:
        • API Metadata Change Request 
        • API Attribute Metadata Change Request 
        • API Data Access Request
        • Data Quality Issue 
      • A common Service Desk template is available to report data quality issues for generic object types. The object type can be selected in the template itself, reducing the number of templates to be managed. The generic object type can be Schema, Table, Table Column, File, File Column, Report, Report Column, Codes, API, or API attributes.
      • Service Desk introduces new roles for enhanced data governance:
        • Parent Steward
        • Category Steward
        • Domain Steward
        • Story Author
        • Project Admin

      These roles can be assigned by Admin and Security & Governance Admin (SGA) users.

      • The SLA Management is enhanced with two new fields:
        • Pending Time: Displays the time remaining for the current approver to act on an SLA-related request.
        • Consolidated SLA: This shows the overall time remaining for all approvers involved in the SLA workflow.
      • A new "SLA Status" filter option is included to identify requests marked as “Near Expiry,” “Not Exceeded,” “Exceeded,” and “None.” 
      • The Service Desk now allows Metadata Change Requests for all Data Catalog objects (except Codes) and Business Glossary. 
      • The Service Desk allows custom actions and statuses to be added directly to templates. The custom actions and status are configured in Administration > System Settings. 
      • When creating a term-related request, if the category steward is not defined, the request is routed to the Domain steward.
      • The approver can now either approve or send back the request for revision by selecting "Refer Back" to the previous approvers from the action menu (Analyse, Verify, etc.). All actions, including approvals and revisions, are recorded and can be viewed in the history.

      Governance Catalog

      • In the Governance Apps, users can now download the campaign response table of a specific Governance App, including Open and Collaborative surveys.
      • In the Governance Apps, users will now receive a notification message when they complete a survey and hit the submit button.

      Data Quality

      • New Data Quality Function: Currency Validation
        • Function Description: Currency Validation (FC) ensures that all values in a file column match the selected currency type, such as dollars or rupees. It checks each value to ensure it fits the chosen currency format, helping users sort and analyze their data accurately.
        • Supported Object Types: File Columns.
        • Supported Data Types: Integer, Decimal, and String.
        • Supported Connectors: S3, NFS, and Azure Data Lake.
        • Input: The Currency Validation (FC) function requires an input where the user will select the choice of currency from the drop-down.
        • Output: The function's output can be one of the following: Success, Failure, Undetermined, Not Executed, or Execution Failed. On Failure, the rule shows the Passed Count, Failed Count, Total Count, and Failed Values.
      • Users can now create and update Data Quality Functions for Tables, Table Columns, Files, and File Columns through the Load Metadata from Files (LMDF) templates.
      • Users can now schedule DQRs regardless of their current status (Draft or Active).
      • In the Data Quality Rules > Associated Objects, the Advanced Settings consisting of Rule Query and Failed Value Query are now supported for Files and File Columns.
      • Data Quality Remediation Center (DQRC) now supports Tables, Files, and File Columns.
      • The new option, Avoid Duplicate Failures, allows users to determine whether to report duplicate failures to the DQRC.
        • Selecting "Yes" skips reporting duplicate entries, simplifying the remediation process for unique issues.
        • Selecting "No" (default) maintains the current behavior of reporting all failures, including duplicates, for detailed review.
      • The Data Quality Remediation Center now provides a customizable list view for failed values associated with the data quality rules.
        • Configure Views allows users to personalize how failed values are displayed by prioritizing relevant columns and customizing the layout.
        • System View is the default view. It provides a standardized perspective with a pre-configured set of columns and cannot be edited or deleted.
        • My Views & Public Views allow users to create custom views for personal use and explore views created by others within the application.
      • SalesForce tables and table columns now support the following Data Quality Functions:
        • For Tables: Data Density, Null Data Density, and Total Rows.
        • For Table Columns: Average, Distinct Values, Minimum Value, Maximum Value, Null Values, Sum, Data Density, Is Unique Column, Empty Count Range, and Not Empty Count Range.
      • The Data Quality Score Dashboard now features score calculations with formulas and references for the Overall Score, DQ Score, Child Score, Profile Score, and Service Request Score in the pop-up.

      • The system setting for data quality has been improved by adding an option to configure the data quality function query for the file columns. Furthermore, in the dimensions, a delete option has been added for custom-defined dimensions, allowing users to delete dimensions that are no longer in use.

      File Manager

      • OvalEdge supports multiple file types for data crawling and profiling. In addition to existing formats like CSV, JSON, and Parquet, users can now work with image files (JPEG, JPG, PNG, GIF) and PDFs.
      • In Folder Analysis, a new column labeled 'Sub Folder Count' has been added, showing the count of subfolders within each folder. Additionally, search and sort functionality have been incorporated into this column.
      • In the File Explorer, previously, users used to run the folder "OvalSight" from the first level on multiple folders. This has been enhanced, and now users can run it on a selected folder at the first level and restrict it to the second level. Additionally, the name has been changed from "Run Folder Analysis" to "Run Folder OvalSight."

      Query Sheet 

      • Previously, only users who created or edited specific codes could edit their code names in the Query Sheet. A new enhancement has been implemented so all Meta-Write users can rename the codes created via the Query Sheet.

      Advanced Tools

      • Users can now use the configuration sheet's CCF (Custom Code Fields) options to add, modify, or delete custom fields through Load MetaData from Files.
      • The Lineage Intelligence Tool for Power BI now includes a "Report Group" column, which allows users to compare reports and quickly identify their group. 
      • The Build Auto Lineage now displays the "Last Run Date" within its statistical information for Reports and ETL connectors.
      • In the Load Metadata from Files > User Template, usernames can now include letters and numbers (alphanumeric). The minimum length remains three characters, with at least one mandatory number.
      • Build Auto Lineage: Users can create lineage without source code queries for components with table or table column information.
      • Lineage Maintenance: If users try to add the same column to a table, an error message "column name duplicate” is now displayed.
      • Lineage Maintenance: The last updated column is used to show the user ID instead of the user's full name. This has been improved, and now the last updated column displays the user's full name.
      • Build Auto Lineage logs for the DBT Connector now use color-coded text for quicker identification of errors and warnings during lineage building:
        • Errors: Displayed in red for immediate attention.
        • Warnings: Highlighted in orange for potential issues.
        • Information: Remains in black for regular updates.
      • In Build Auto Lineage, for Alteryx Connector, new workflow components have been added to workflows, and problems with lineage and query parsing have been fixed by adding the new components (AlteryxDynamicInput, SharepointInputEngine, Py, AlteryxDirectory, AlteryxDateTime, AlteryxRegEx, AlteryxLockInStreamOut, and AlteryxLockInSelect) to the workflow.
      • In Build Auto Lineage > DBT Connector, job logs previously displayed all text in black, making it difficult to distinguish errors and warnings. This functionality has been enhanced. Now, job logs display information in highlight mode for better visibility and clarity, showing errors in red, warnings in orange, and regular information in black.

      OvalEdge APIs

      • The new API tag "api/table/certify" allows the certification process for various data objects (tables, columns, reports, files) through the API.

      Administration

      Connectors

      • In the Connectors, the Crawl/Profile button is enabled/disabled based on the Connector Health.
        • If the Connector Health is green, the Crawl/Profile button is enabled, allowing users to perform crawling and profiling operations. 
        • If the Connector Health is red or gray, the Crawl/Profile button is disabled, restricting users from performing crawling and profiling operations. 
      • This release includes the new AWS DMS (Database Migration Service) connector, a cloud service that allows users to migrate relational databases, data warehouses, NoSQL databases, and other types of data stores. The SDK driver connects to the data source to crawl and build lineage. The following are the characteristics of the connector:
        • Type: ETL
        • Crawl: Supported
        • Profile: Not Supported
        • Lineage: Supported
        • Data Quality: Not Supported
        • Query Execution: Not Supported
        • Bridge: Supported
      • For the Delta Lake Unity Catalog connector, after crawling and building auto-lineage, the source code for “Views” is now excluded.
      • When Tables, Reports, Files, Procedures, or other objects are deleted in the source system, they are automatically deactivated in OvalEdge after the next crawl. 
      • Reporting Connectors now uses distinct icons for temporary and resident tables.
      • In the Connector's Business Glossary Settings, the “12Y” text field option has been replaced with the "Glossary-Catalog Sync Options".
      • For the Snowflake connector, the “Crawler” settings are enhanced to allow users to select and add Stages, Pipes, and Tasks.
      • For the QlikSense and QlikView Reports, users can now view the count of child reports in the Data Catalog Reports Summary.
      • For the Dremio connector, users can retrieve all the previous Dremio connector settings and parameters along with the migration process.
      • For Oracle Connector, the automatic lineage for database tables now captures related synonyms. These synonyms are accessible within the Data Catalog under the Table Summary > Synonyms.
      • Users can now view the terms associated with report columns from the OvalEdge application and their object certification status directly on the Power BI source. Additionally, users can navigate to the selected Term summary associated with the Reports/Report Columns from the Power BI source page.
      • OvalEdge now supports crawling Talend packages not only with owner credentials but also with organization/repo details.
      • In the Connectors Bridge, when the server bridge is updated with the latest NAR, the previous NAR file is automatically backed up in the client's environment.

      Data Access

      • For Snowflake and Redshift, when the data access admin changes a particular user's details, like username and password, at the instance or connector level, the source privileges assigned to that user will remain intact for performing the actions at the instance and connector levels, respectively. 
      • For Snowflake, the roles and permissions assigned to schemas will now apply to its associated tables and views.
      • For Snowflake, when users update permissions at the schema level, the permissions are now automatically updated for the associated tables.
      • For Redshift and Snowflake: The Filter option is added to the Row Access Policies column for Tables.
      • For Redshift: At the instance level, a download option is added for Groups to download the data file.
      • For Redshift, when permissions are removed for a user/group/role in Tables, a pop-up will appear asking whether to revoke all permissions on table columns. If confirmed, all permissions on table columns will be revoked. The audit log will show any unsuccessful delete attempts or successful deletions.
      • For Redshift, with the 'Sync OvalEdge Masking Policies to Redshift Policies' option enabled, removing a masking policy for a table column in Administration > Security will now remove it from Data Access Management > Table Columns and the source system.
      • For Redshift: Users could not remove permissions at the table column level when trying to remove them at the table level. This functionality has been enhanced, and now users can remove permissions at the table column level when they try to remove them at the table level and view the audit history for table columns regarding any deleted actions.
      • For Tableau, a new column named "ovaledge user" has been added to the Site Users tab.
      • For Tableau Connector, the two new columns, "Last Modified Date" and "Tableau Created Date," are added to the Reports and Projects tab.
      • For Snowflake, when users update permissions at the schema level, the permissions for its associated tables are now updated automatically.
      • Data Access Admins and assigned users will now receive notifications for any actions taken within the Data Access Management (DAM) system, both at the instance and connector levels.
      • Crawling jobs are triggered whenever the user crawls at the instance and connector levels, providing a complete list of logs that includes:
        • Instance Level: Roles, Groups, User privileges, and its summary.
        • Connector Level: Database and Schema permissions, table permissions, associated policies, and its summary.

      Job Workflow

      • The “Fetch Query Log” option is added to the Job Workflow to schedule query logs and build lineage.

      Users & Roles

      • In Users and Roles, deleting a user now involves selecting "Delete/Deactivate" from a dropdown menu instead of a toggle button. Users must also choose the user's name from another dropdown to transfer responsibilities before deletion.
      • In Users & Roles > Teams, when users created a Role, a Team was automatically generated along with it. This functionality has been enhanced, and automatic Team creation has been removed from Role creation; users can now create Teams manually. A new column, Team Description, has been added, allowing users to input the required Team description.

      Security

      • Users can now view the list of Tags and Terms associated with the cataloged Schemas, Tables, Table Columns, Reports, APIs, and API Attributes. 
      • In Reports, a new column, "Type," has been added to filter and display the reports based on their Report Type.

      Advanced Jobs

      • Earlier, running Advanced Jobs required deploying the entire OvalEdge application. But now, the advanced jobs are operated as a separate module with a dedicated library located in a third-party jar.

      Custom Fields 

      The new metadata fields have been added to the System Fields and can be viewed from the Data Catalog object summary.

      • The Last Modified By (Catalog Details) and Created By (Crawl & Profile Details) are added to the Schema summary.
      • The reference Count field is added to all the object summaries except for Databases.

      System Settings 

      • In Notifications System Settings, previously, only administrators would receive a notification when the "Test SMTP" button was clicked. But now, all account users with author licenses and role admin privileges will receive a notification.

      Audit Trails

      • Data Objects
        • The Data Objects view shows the data objects' Origin as Crawl/ Manual/Virtual.
        • The 'Inserted' option in the 'Action' filter is renamed 'Inserted/Created' to reflect virtual object creation, and the action for deleting virtual objects is shown as 'Deleted.'
        • The 'Source' column displays 'Data Catalog' for creating virtual and manual objects and '0' for deletions.
      • Data Quality 
        • Audit trails now record all activities performed within Data Quality Rules (DQRs). This includes details like:
          • DQR Name
          • Rule Type
          • Object Type
          • Functions
          • Dimensions
          • User Actions (e.g., Rule Created, Deleted, Object Type Changed (From & To), Function Changed (From & To), etc.)
          • Updated By 
          • Updated On
        • Governance Catalog
          • Users can now view audit event changes related to Terms and Domains.
            • Audit Trails for Terms: The audit trails for Terms include updates on:
              • Manage Data Association Preferences:
              • Copy Title to Catalog (checked/unchecked)
              • Copy Business Description to Catalog (checked/unchecked)
              • Copy Steward_New to Catalog
              • Copy Custodian_New to Catalog
              • Copy Owner_New to Catalog
              • Copy Curator to Catalog
              • Copy Governance Role 4, 5 & 6 to Catalog
              • Restricted (checked/unchecked)
              • Masked (checked/unchecked)
              • Show Classification in Catalog (checked/unchecked)
            • Audit Trails for Domains: In addition to the above, audit trails for Domains include updates on classifications:
              • Added
              • Added and checked
              • Checked/unchecked
              • Modified to new classification name
              • Deleted
          • Audit trails now track creation, editing, and deletion activities for virtual objects within the Data Catalog

          Others

          • Earlier, users could connect to the OvalEdge database using MySQL Authentication only, but now, they can connect using Microsoft Entra Authentication via Azure MySQL from the oasis.properties file.
          • The authentication type for Azure SQL has been changed from "MySQL Authentication Only" to "Microsoft Entra Authentication Only," which does not require entering a password.
          • Earlier, individual queries were required for each profile attribute and column to retrieve distinct, min, max, not null, etc. With DuckDB's Summarize query option, all these values can now be obtained in a single query.
          • During scheduled profiling, users can now set specific regex patterns for tables in different Attributes (for example, if the table name begins with an alphabet).
          • Users can now dynamically change the log level to other statuses, such as debug, info, warning, and error at the Package/Class level.
          • Users can request access only for certified data objects through an access cart or service desk. This setting can be configured by navigating to Administration > System Settings > Others, then selecting the key "datacatalog.object.access.request.support.mode," choosing "Certified Objects only.”

          New Advanced Jobs

          Advanced Job Name

          Module Name

          Description

          INSERT_NOTIFICATION_PREFERENCE

          User Notification Preferences Manager

          This job enables the addition of new modules and submodules in the notification preference table of the My Profile section for all users. 

          METADATA_CURATION_SCORES_TO_DENORMALIZED_TABLES

          Meta Data Curation Scores 

          This migration job helps to calculate the metadata curation score for existing objects in version 6.2 and saves the metadata curation score values in denormalized tables(ext) associated with the objects.

          UPDATE_CODE_SORTORDER

          Helpful Resources

          This job helps to migrate helpful resources from 6.2 to 6.3. It lets users move the previously added helpful resources using the drag-and-drop option in the updated version.

          LOAD_BD_TD_DESC

          Data Catalog

          This job helps to associate business and technical descriptions with tables and table columns.

          Attribute:

          File Path: Enter the file path. 

          Note: The file must be in .csv format.

          FILTER_BASED_PROFILING

          Filter Based Profiling

          This job will profile tables based on their names, starting with specific letters provided in the inputs. This job will run for each cycle (provided as per the schedule) based on the connection ID and schema name.

          Attributes: 

          Enter connectionId and schemaName (Ex:- 1000: schemaName): Please input the connection ID and schema name in the format: 1000: schemaName (without spaces between the colon symbol).

          Cycle 1: Specify the starting letters of the tables to be profiled to lessen the profiling workload.

          ASSOSCIATE_TAGS_CONNECTOR_OBJECTS

          Associate Tags To Connector Objects

          This job assists in associating tags to chosen connector objects.

          Attributes:

          Connector IDs: Input connector IDs separated by commas.

          Object Types: Input a list of object types separated by commas.

          Tag IDs: Input tag IDs separated by commas.

          OBIEE_LINEAGE

          OBIEE Lineage Advanced Job

          This job creates lineage in the OBIEE Connector system within the Repository Data Metadata files (RPD).

          Attributes: 

          OBIEE Connector Id - The ID for the established connection.

          RPD File Name - Specify the name of the Repository Data file.

          RestoreBusinessGlossaryObjectsSourceInfo

          Business Glossary

          This job helps to migrate source information for associated objects in the business glossary for previously added objects.

          Note: From release 6.3, the source information for associated objects is automatically generated.

          QUESTIONWALL_MIGRATION

          QuestionWall Migration

          This job helps to migrate the questions and answers (replies) previously found in Question Wall > Inbox to Question Wall > General Wall.

          LOAD_DATA

          Data Stories Migration

          This Advanced Job is designed to transfer Data Stories from one environment to another. It lets users download Data Stories from the source environment and upload them to the target environment using the same job. 

          Attributes: 

          Attr1: Specify ""Upload"" or ""Download"" based on the action. For Downloads, use Attr1 = ""Download""; for Uploads, use Attr1 = ""Upload"". 

          Attr2 - Enter File Server Connection Id (optional).

          Attr3 - Enter the file path. 

          Attr4 - Enter Story Zone Names or IDs with comma-separated values (optional)

          Attr5 - Enter Story Names or IDs with Comma separated values (optional).

          POWERBI_MIGRATION

          Reports

          This job helps move the existing report columns and their curated information to their related report columns, dataset columns, and page columns, which in turn move to report columns.

          FILL_CRON_AND_ASSOCIATION_DESC

          Schedule 

          This job helps to include the filter icon options for the Schedule > Cron type column in the administration and improves the search functionality in the Data Catalog > Codes > Associations > Associate objects column.

          ENCRYPT_USER_WEBHOOK_URL

          My Profile

          This job will encrypt all users' webhook URLs for Google Chat and Microsoft Teams.

          New System Settings

          Key

          Description

          assetmanager.virusscan.enabled

          Configure to enable virus scan for the uploaded files.

          Parameters:

          The default value is False.

          If set to True, virus scan on files is enabled.

          If set to False, virus scan on files is disabled.

          assetmanager.storage.service

          Configure to upload a file from the Governance Apps and store it in the mentioned storage service.

          Parameters:

          The default value is Empty.

          Enter the storage service name.

          assetmanager.storage.connId

          Configure to enter the connection ID of the storage service.

          Parameters:

          The default value is Empty.

          Enter the connection ID.

          assetmanager.storage.service.bucketname

          Configure to enter the bucket name of the storage service.

          Parameters:

          The default value is Empty.

          Enter the storage service bucket name.

          oe.crawl.excludeTables

          Configure to hide the tables.

          Parameters:

          The default value is Empty.

          Enter the table names separated by a comma.

          smtp.oauth2.tenantid

          Enter Tenant ID for OAuth Token Validation and SMTP Server Authentication.

          Parameters: 

          By default, this setting is Empty.

          Enter a tenant ID in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          smtp.oauth2.clientid

          Enter Client ID for OAuth Token Validation and SMTP Server Authentication.

          Parameter: 

          By default, this setting is Empty.

          Enter a Client ID in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          smtp.oauth2.clientsecret

          Enter Client Secret for OAuth Token Validation and SMTP Server Authentication.

          Parameter: 

          By default, this setting is Empty. 

          Enter a Client Secret in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          smtp.oauth2.cloud.name

          Enter Cloud Name for OAuth Token Validation and SMTP Server Authentication.

          Parameter: 

          By default, this setting is Empty. 

          Enter a Cloud Name in the field provided to validate an OAuth token and set up SMTP Server Authentication.

          Bug Fixes

          Global Search

          • When users tried to add tables or reports to the access cart from the Global/Elastic search, they could not. To fix this, update the system settings:
            • Go to System Settings > Others.
            • Search for datacatalog.object.access.request.support.mode and select "Certified Objects only".
            • Search for access.cart.support.object.types and insert "Table and Reports".

          Now, users can add certified tables or reports from the Elastic search to the access cart. A success message appears for certified items: Success, the certified table or report is added to the access cart. An error message appears for non-certified items: The table or report is not certified and unable to be added to the access cart. 

          My Profile 

          • In the Inbox > Governance Catalog, the issue related to the notifications generated through the collaboration for an RDM (Reference Data Management) object not displaying details and hyperlinks has been resolved.
          • In the Notifications section, the count used to show 0 when there were no notifications, which was misleading. This issue has been fixed by removing the count when there are no notifications.

          Business Glossary 

          • The mismatch between the Date and Code Custom fields information in the Term Summary and the Business Glossary Configure View has been resolved. The same information is displayed in the Term Summary and the Business Glossary Configure View.
          • The issue with tracking the history of changes made to a specific domain description in the Business Glossary has been resolved, and users can now view the history of changes made to domain descriptions.
          • The issue regarding updating the Governance Roles in the Term summary by its Domain or Term-level steward has been resolved.  Now, they can update Governance Roles from its summary page.
          • In the Business Glossary, users could not view details such as Associated By, Associated Method, and Associated Date for existing objects. However, these details were visible for newly added objects, indicating their source (e.g., API, Manual, LMDF, etc.). For existing objects, the source information was defaulted to "Manual." This issue is resolved by executing the advanced "Restore Business Glossary SourceInfo" job.
          • In the Business Glossary, assigned stewardship terms weren't visible under "My Resources" > "My Permissions" > "Steward" > "Terms." This issue has been resolved, allowing stewards to view the assigned terms.

          Data Catalog

          • In Data Catalog > Data Objects > Lineage, previously, after removing caution, the status became null for all downstream objects. This issue has been fixed. Now, when caution is removed, the status is set back to the previous value.
          • The issue with the "Send Messages" alert pop-up in the Data Catalog object's summary has been resolved. Users can now send alerts to Owners, Stewards, or Top Users.
          • In the Data Catalog > Table Columns > Column Lineage > Tabular View, the issue of displaying the wrong source column at the table level has been resolved.
          • In the Data Catalog, the issue of Table API not saving numeric custom fields when the value is zero is resolved.
          • In the Data Catalog > Table / Table Columns, technical and source descriptions are now obtained from the remote source during crawling, eliminating null values for Oracle Connectors.
          • In the Data Catalog > Table Summary Page > Lineage > Tabular, when a user with MRDW permission who doesn't have access to the Jobs module tries to download lineage in tabular format, they get a message saying, "Download Job for Lineage Report is submitted with Job Id." This issue has been resolved by displaying the message "Download Job for Lineage Report is submitted with Job Id. You can access the file once the job is finished by clicking the link sent in the notification."
          • The permission sink issue between the Data Objects and the Users and Roles is resolved, and authorized users can view Tables and Views.
          • In the Data Catalog, when users add certified tables or reports to the access cart from the summary page, no success message appears.
            • To fix this, update the system settings: Go to System Settings > Others.
            • Search for datacatalog.object.access.request.support.mode and select "Certified Objects only".
            • Search for access.cart.support.object.types and insert "Table and Reports".
            Now, users see a success message when adding certified tables or reports to the access cart. An error message appears for non-certified items.

          Governance Catalog

          • In the GDQ, stewards assigned to GDQ Results with “No Data Access” permission could view the results. This has been fixed, and stewards are now restricted from viewing the results.
          • In the GDPR ROPA, users could not view all the Term-related column details (Domain, Category, Subcategory, and PII Terms) in the "Processing Activities" downloaded Excel sheet. This issue has been resolved, and users can now view all the details in the downloaded Excel sheet.

          Data Quality 

          • The SSN Format Check function previously displayed an incorrect Fail Count in Data Quality Rules. This issue is fixed, and now the accurate failed count is displayed.
          • In the Data Quality Rules Summary, when "Send Alert on Failure" was enabled for a rule, notifications were sent for each data object instead of the rule execution level. This issue has been fixed; now, notifications are sent only for actions related to the rule execution levels.
          • In the Data Quality > Rules Summary, an issue was encountered where enabling "Send Alert on Failure" for a rule led to job failures. As a result, job logs displayed a "Null exception" instead of helpful information. This issue has been fixed, and jobs are running as expected with accurate log details.
          • In Data Quality Rules, when users tried to execute the "Should have Unique Values" function on Dremio Objects, it showed a failure. This has been fixed, and now the function executes as expected on Dremio Objects.

          Jobs

          • In the Jobs section, users faced difficulty returning to the main page from the Jobs summary page because there was no back button. They had to refresh the page instead. This issue has been resolved by adding a back button to the jobs summary page.
          • In the Job Logs, error messages related to the Deltalake and Unity Catalog connectors were incorrectly displayed. This issue has been fixed, and the Job Logs now display the correct messages as expected.

          Query Sheet

          • In the Query Sheet, when running a query, only the associations for tables were displayed, while table columns were missing. This problem has been fixed, and now the associated table columns are also shown.  

          Advanced Tools 

          • Earlier, users could delete the crawled source objects from the application via Load Metadata from Files. However, this issue has been resolved, and users can no longer delete them.
          • In Build Auto Lineage, some codes failed to parse when users tried to build auto lineage. This issue has been resolved, and users can now successfully build lineage.
          • In Build Auto Lineage > SSAS on-prem, previously, files in sub-folders weren't being fetched in the application. This problem has been fixed, and now users can fetch files from sub-folders as well.
          • In Build Auto Lineage, the connector's name collides with the source code loaded. This issue has been fixed by displaying part of the connector name with the full name in the tooltip.
          • In Build Auto Lineage, custodians couldn't build lineage and received an error stating, "Only OE_Admin" can perform this action. This issue has been resolved, and custodians can now build lineage as intended.
          • In Build Auto Lineage, for OBIEE Connector, a new functionality has been added in the correct query screen for data sets. Now, there is a "replace" option available for queries that are parse failed.
          • In Lineage Maintenance, the Code Type filter previously displayed all the data types, but now it only shows the user’s live data types.
          • In Load Metadata from Files > User Template, the user name field previously accepted only numeric values, causing errors in job logs during updates. This issue has been resolved. The user name field now accepts both alphabetic and numeric characters.

          Note: The username must contain at least 3 characters and include at least 1 numeric value.

          In Ovaledge APIs > Swagger, the bearer token issue has now been resolved. 

          To authorize the Ovaledge APIs, enter the bearer token. Ex: "JWT bearer string." 

          Note: To obtain the bearer string, 

          1. Users must download the API credentials from the 'My Profile' section. 
          2. Users should visit 'api/users' on the OvalEdge API's Swagger page and try using the username and user secret string from the downloaded API Credentials file. Upon execution, the bearer key can be seen in the response body. 

          Administration

          • Data Access Management for the Redshift Connector: 
            • Previously, when a user attempted to delete roles/users/groups at the instance level, they were deleted from the instance but remained visible at the connector level. This has been improved; now, deleting them at the instance level also removes them from the connector level.
            • There was an issue where roles, groups, or users were randomly added without a specific order. This issue has been fixed, and roles have been added first, followed by groups and users.
          • For Redshift and Snowflake: Previously in Inbox > Data Access Management, all notifications were received from the administrator, regardless of the sender's name. This issue has been resolved, and notifications have been received from the currently logged-in user.
          • In the PostgreSQL schema, the triggers were displayed more than once due to event manipulation. All event manipulations are now grouped, and the triggers appear only once.
          • The "Discover Relationships" advanced job for Tables, which automatically generates Primary and Foreign Key relationships, can now be deleted.
          • For the Talend connector, the issue of validating the connection through the bridge has been resolved.
          • In Service Desk Templates, cloning a template and changing the JSON in additional settings causes an error when saving. Running the basic advanced job "Servicedesk V1" fixes this problem.
          • In the Connector Settings > Profiler > Profiler Settings, there was an issue where choosing the profile type "Random Sample" would disable the "Calculate Relationships" option, preventing the display of relationships in the Data Catalog. This issue has been resolved. When the "Random Sample" profile type is selected, the "Calculate Relationships" option is enabled to calculate the relationships and display them in the Data Catalog.
          • In the Security, 'ovaledge.role.admin' can add or remove role-based access to certain modules or sub-modules to authorized roles. Previously, certain unauthorized roles could still see the Connector Security tab even after removing role-based access for Connector Security. This issue has been resolved, and now, if role-based access is removed, any unauthorized role or user cannot see the Connector Security tab.
          • In Security, configuring the 4th Governance Role as "Chief Data Steward" was preventing users from viewing and updating the role from the Business Glossary or Data Catalog. There was also an issue where all assigned Governance Responsibilities for a category under a domain would disappear automatically after a specific duration. These issues have been resolved. Users can now view and update the configured governance role, and all configured Governance Responsibilities will persist without disappearing unexpectedly.

          Copyright © 2024, OvalEdge LLC, Peachtree Corners, GA USA