Release6.2.1

This release focuses on improvements to Data Access Management, Data Catalog, Data Quality, and bug fixes across various functionalities.

Major Enhancement

Data Access Management

Support for Amazon Redshift has been added for Data Access Management. The main features include:

• Management of roles, groups, users, and their respective permissions at the account level.
• Access Management of databases, schemas, tables, and table columns for roles, groups, and users across multiple connectors created for connecting to Redshift systems.
• Masking and row-level security policy management for tables and columns.
• Audit logs for all the access management actions managed through OvalEdge. 

The access management support includes the following modes:

• OvalEdge follows security policies defined in OvalEdge and pushes them to the Redshift system
• OvalEdge syncs the security policies defined in the Redshift system and follows them in the OvalEdge system. 

Bug Fixes & Minor Enhancements

• Home 
  • Helpful Resources: An issue has been resolved where backspace wouldn't delete the entire link while editing a resource link in the "Update Helpful Resource" settings. Users can now fully remove links using backspace.
• Tags 
  • Tag Summary: The breadcrumb "This tag has multiple parents" on the Tags > Tag Summary page is displayed incorrectly, even if only one associated parent tag exists. This is fixed. The breadcrumb message now displays only when multiple parent tags are associated with a tag.

• Data Catalog
  • Enhanced Lineage Download: Users can now download lineage information for tables and table columns in the Data Catalog. This functionality provides a tabular format for lineage data.
  • Schema Ownership: Although multiple owners contributed to the same schema while crawling the Data Pipeline connector, only single-owner details are fetched and displayed. Now, schema ownership management is improved by introducing custom fields (Owners) at the schema level. These fields allow teams to designate multiple owners.
  • Improved Technical Descriptions: Technical descriptions within schemas retrieved from the Data Pipeline source system are now more concise. The redundant "AND" from "AND producer AND consumer" within the description has been removed and replaced with "Producer Consumer" for better readability.
  • Report Lineage Fix: Previously, rebuilding lineage for a report sometimes caused existing report columns to become inactive while adding a new column. This made it appear that associated terms and descriptions were lost.
    • Fix 1: Prevent Inactivation: This fix ensures existing report columns remain active after lineage rebuild.
    • Fix 2: Reactivate Inactivated Columns: An "Advanced Job: Report Column Term Association" has been created to identify and reactivate any report columns unintentionally deactivated during past lineage builds.
  • Report Column Formula Visibility: The "Formula" field was previously missing from Report Column statistics in the Data Catalog. This fix enables users to view and manage the "Formula" field. 
  • Configuration: To enable/disable the "Formula" field, navigate to Administration > Custom Fields. Select Object Type: Report Column, then Global > Text Custom Field > System Fields > Formula.

• • Table Summary - Enhanced Column Notes: Searching and editing Column Notes within the Table Summary wasn't possible. This release addresses these limitations by enabling users to:
    • Edit: Modify existing Column Notes.
    • Search: Locate specific Column Notes using the search functionality.
  • Enhanced Salesforce Report Filtering: Salesforce reports could not be filtered based on their lineage. This fix enables users to filter Salesforce reports with or without lineage information, providing greater flexibility for data exploration.

• Business Glossary - Terms
  • Masked Term: In the Business Glossary, when a term was created and enabled with the "Masked" option, and specific users and roles were assigned to view the masked data, there was an issue where in the Data Catalog, under "Table Columns > Data," the authorized users and roles could not view the data in an unmasked format. However, this issue has now been resolved. Assigned users and roles can now correctly view the data in an unmasked format as intended.
  • Masked & Restricted Term: Enabling both "Masked" and "Restricted" options for a term in the Business Glossary prevented details like authorized roles, users, and policy names from showing in the Data Catalog's "Column Security" section for table columns. This is fixed. Now, all required details are displayed when both options are enabled.
  • Max and Min Values: In the Business Glossary, when a term was created and enabled with the "Masked" option, there was an issue where in the Data Catalog, under Table Columns > Statistics > User Activities,  the Maximum (Max) and Minimum (Min) Values are showing in unmasked format. However, this issue has now been resolved. The Max and Min Values are now correctly masked as intended.
  • Column Security: In the Business Glossary, when a term was created and enabled with the "Masked" option, and specific users and roles were assigned to view the masked data, there was an issue where in the Data Catalog, under "Table Columns > Columns > Statistics > Column Security," the Masked option is disabled, and details like authorized roles, users, and policy names were missing. This issue has now been resolved. The Masked option is enabled, and authorized roles, users, and policy names are displayed in the column security section.
  • Email Column: Previously, there was an issue in the Business Glossary where enabling the "Masked" option for a term, such as for masking email addresses to show only the last four characters, would display correctly in the Data Catalog under "Table Columns > Data > Email Column." However, when the file containing the masked data was downloaded, it only showed three characters as masked instead of the intended four. This issue has been addressed and resolved. The application display and the downloaded file consistently show the same masked characters.
  • Masked Column User Selection: In Administration > Security > Table Columns, after selecting Mask Column in the Assign Masking Policy, the user names did not appear in the "Allowed Users" section when choosing an existing policy. This issue has been resolved. User names now appear correctly when selecting an existing policy for masked columns.
  • Masked Column Values in Profile Comparison: Enabling "Mask Column" for a Table Column in Administration didn't mask minimum and maximum values in the "Compare Profile Results" section of Advanced Tools. The masked values appeared in only a single profile interval instead of two. This is fixed, and minimum and maximum values are now correctly masked and displayed in both profile intervals.
  • Restricted Term Functionality: Enabling the "Restricted" option for a term in the Business Glossary didn't apply the restriction as intended in the Data Catalog ("Table Columns > Columns") and Advanced Tools ("Compare Profile Results"). This is fixed. Column restrictions are now enforced correctly for both table and file columns.
  • Masked Term Policy Name: Creating a masked term in the Business Glossary with a time stamp masking policy (including policy name, masking scheme, and allowed users/roles) didn't display the correct policy name in the Data Catalog's "Table Columns > Columns > Statistics" section. This is fixed. The intended policy name is now displayed accurately in the Data Catalog.
  • Masked Term Users and Policy Name: Creating a masked term in the Business Glossary with a boolean masking policy (including policy name, masking scheme, and allowed users/roles) didn't display the users and policy name in the Data Catalog's "Table Columns > Columns > Statistics" section. This is fixed. The intended users and policy names are now displayed accurately in the Data Catalog.
  • Masked Term - Top, Max, and Min Values: Masked terms in the Business Glossary displayed unmasked top values, minimum (Min), and maximum (Max) values in the Data Catalog's "Table Columns > Columns > Statistics > Column Security" section. This is fixed. These values are now correctly masked as intended.
  • Masked Term - Max and Min Values: Masked terms in the Business Glossary displayed unmasked minimum (Min) and maximum (Max) values in the Data Catalog's " File Columns > Columns > Statistics > Column Security" section. This is fixed. These values are now correctly masked as intended.
  • File Manager: Masked terms in the Business Glossary displayed the unmasked data in the File Manager > Table View. This issue is resolved. The specified column values for masking are now correctly masked within the File Manager, ensuring data privacy as intended.
  • Allowed Users: In the Business Glossary, when a term was created and enabled with the "Masked" option and specific users and roles were assigned to view the masked data, there was an issue where in the Data Catalog, under "File Columns > Data is the masked format for authorized users instead of unmasked format. This issue has now been resolved. The authorized users can view the data in the Data Catalog, File Column.
  • Add Roles: In the Administration section under Security > Table Columns, when the "Restrict Column" option was selected, users encountered an error message when repeatedly clicking on the Add Icon beside the username in the Roles section while assigning roles under the Authorized Roles section. This issue has been resolved. Users can now assign roles repeatedly by clicking the Add Icon without encountering error messages.
• File Manager
  • CIFS Connector "Folder Analysis" Visibility Fixed: Users can now access the "Folder Analysis" tab for CIFS connectors in the File Manager. This tab displays files and folders associated with the CIFS connection.

• Governance Catalog

• ROPA Visibility Control: Previously, ROPA was always visible in the application. This update introduces enhanced security by allowing administrators to control ROPA visibility. Users with administrative privileges can enable or disable ROPA through Administration > System Settings > Others. Simply set the "hide.ROPA" key to "Yes" to hide ROPA or "No" to keep it visible.

• Data Quality Rules 

• • The Data Quality reporting is enhanced for a better user experience:

    • Object Execution Results: The Input and Success Criteria columns were removed, and Undetermined Objects and Execution Objects were added to the DQ Rule Report. 
    • Rule Execution: Streamlined reporting by moving Undetermined Objects and Execution Failed Objects from Summary to the DQ Rule Report. 
    • Results Summary: An eye Icon beside the Results Value is included in the Object and Rule Execution summary for quick access to the Results Summary.
  • Enhanced Term Search: Users can now search for Terms when associating them with data objects in Data Quality Rules. This improvement simplifies linking relevant terms for more efficient data management.

• • Data Quality Organization Dashboard Enhancements: The Data Quality Organization Dashboard now features a more user-friendly interface with improved navigation.

• • Clearer Filter Options: The "Organization" and "Departments" dropdown lists have been replaced with "Select Domain" and "Select Category" for better clarity.

• • Resolved Functionality Issues: All previously reported issues with drop-downs, drill-downs, count accuracy and back-button functions have been addressed, ensuring a smooth and reliable user experience.

• Jobs

• • Users can now download logs for Custom Field Migration Jobs within OvalEdge Jobs. This fix provides a convenient way to review and analyze job execution details.

• Advanced Tools
  • Table Column Metadata Management: OvalEdge allows users to streamline metadata management for table columns through the "Load Metadata from Files" feature in Advanced Tools. This update expands functionality to include:
    • Tag Assignment: Easily associate relevant tags with table columns for improved organization and discoverability.
    • Term: Link appropriate business terms to table columns.
    • Governance Role Designation: Assign governance roles to table columns, ensuring proper access controls and data security.
  • An issue is resolved where the "Build Auto Lineage" function displayed an empty status after successful lineage generation. Additionally, the unprocessed code count remained at “0” despite successful processing. Lineage build status is now accurately reflected.
  • Accurate Tag Assignment: In  Load Metadata from Files > Select Object Name (Tag), When loading metadata from Excel files for tags, Non-Assignable Tags were incorrectly created as Assignable Tags in OvalEdge. This issue is now resolved. Tags are created with their configuration in the Excel file.
  • Source/Destination Object Handling: Previously, users needed to manually rearrange JSON files when a data object acted as both source and destination within the same connection. This could be time-consuming and prone to errors. This is fixed. Users can now leverage the connection hostname from the JSON file. OvalEdge automatically compares this hostname with its connection list to determine the correct connection ID for the source and destination object. This streamlines the process and eliminates the need for manual file manipulation.
  • Lineage Maintenance - Report Headers: In Advanced Tools > Lineage Maintenance, the source and destination headers are displayed incorrectly when users select reports under the object type. This inconsistency could lead to confusion when interpreting data lineage. This issue is resolved. The source and destination headers now display accurately for reports, ensuring clear and consistent visualization of data lineage relationships.
  • Adding Reports & Report Columns: Previously, the "add" action in Advanced Tools > Load Metadata From Files > Reports & Report Columns wasn't functional, limiting the ability to add new reports and report columns. This is fixed. Users can now successfully use the "add" action.
  • Business Glossary API Fix: Previously, the Business Glossary API (GET method) in Advanced Tools could not retrieve OvalEdge Terms data, specifically details related to APIs and API Columns. This issue is resolved. Users can now successfully retrieve the required OvalEdge Terms data using the GET method within the Business Glossary API.

• Administration > Data Access Management
  • System-Defined User Management: Previously, OvalEdge allowed managing system-defined users (e.g., deleting and changing passwords). This issue is fixed, and users are restricted by managing system-defined users.
  • Redshift RDAM Fix - Sorting: Previously, sorting users by clicking the "Users" field header in Administration > Data Access Management > Redshift RDAM wasn't functional. This issue is resolved. Users can now effectively sort the user list by clicking the "Users" field header.

• Administration > Advanced Jobs
  • Bulk Delta Lake Connector Creation: The Delta Lake connector setup is now streamlined in the Administration> Advanced Jobs. Upload an encrypted file path and its key in the "Load Connector Details" attributes. Run the advanced job, and OvalEdge automatically creates and validates Delta Lake connections in bulk.
• Administration > Connectors
  • Expanded .NET Framework Support: The SSAS On-Prem connector now supports both .NET 3.0 and 6.0 versions.
  • Expanded Power BI Connector Functionality: Source descriptions are now visible in Reports and SubReports within the Data Object Summary.
  • Improved Lineage Support: The SSAS/SSAS On-Prem connector now includes a new field called "Deployment File Path." This field accepts a JSON file path with a .asdatabase extension and is used for lineage tracking.
    • To build lineage for SSAS models:
      • Deployment File Path: Specify the path to the JSON file containing the model definition.
      • Auto Lineage (Optional): Enable the "Auto Lineage" option for automated lineage generation.
      • Crawler Settings (Optional): In the connector settings (accessible via the three dots menu), ensure the "Views," "Procedures," and "Source Code" checkboxes are selected within the crawler settings. This enables the crawler to capture relevant data sources for lineage purposes.
  • Enhanced Talend Connector Support: OvalEdge now supports a wider range of Talend packages and components, providing greater flexibility for data integration tasks. Additionally, Lineage can now be successfully built for Talend components containing queries.
  • Fixed an issue preventing lineage building for the DataStage connector. Users can now generate lineage information for their DataStage jobs without encountering errors.
  • Fixed parsing errors encountered when building lineage for the Dremio Connector in "Build Auto Lineage." Users can now successfully generate lineage for the Dremio Connector.
  • Power BI Report Deletion Notification Optimization: Deleting a Power BI report name (referred to as "Domain" within OvalEdge) from the remote source triggered two notifications: one in-app ("System Alerts") and a separate email. This release streamlines the process by sending a single notification, improving user experience.
  • Power BI On-Prem Connector Fix: Building lineage for Power BI On-Prem reports failed if the report connected to a different source than the configured connector. This issue is resolved. Users can now successfully build lineage for Power BI On-Prem reports.
  • Databricks Connector Fix - Memory Issue Resolved: Users encountered a "Java heap space" error when attempting to crawl the Databricks connector. This issue has been fixed, and users can now crawl the Databricks connector successfully.
  • Dremio Connector Improvement - Version Support: Users were limited to adding Dremio connector versions up to version 20 in Administration > Connectors. This functionality has been improved. Users can now add Dremio connectors with versions up to 24.
  • Azure Synapse Connector Fix: Users encountered "Error in crawling" issues when attempting to crawl data for the Azure Synapse Connector in Administration > Connectors. This functionality has been improved. Users can now successfully crawl data for the Azure Synapse Connector, ensuring accurate data lineage and analysis capabilities.
  • Salesforce Connector Fix: Table Description Retrieval: Previously, users encountered issues retrieving table descriptions while crawling the Salesforce connector. This issue is resolved. Users can now access the table description in the Data Catalog under "Tables > Source Description" after successfully crawling the Salesforce connector. 
  • Alteryx Connector Fix: Users encountered issues building lineage for Alteryx workflows in Administration > Connectors. This limited visibility into data flow within Alteryx processes. This functionality is now improved. Users can successfully build lineage for Alteryx workflows. Additionally, detailed component information is displayed in the Associations tab, providing a clearer understanding of how data moves through Alteryx workflows.

Copyright © 2024, OvalEdge LLC, Peachtree Corners, GA USA