OvalEdge Releases
      Back to home
      1. Knowledge Base
      2. Architecture, Security & Releases
      3. OvalEdge Releases

      Release 5.2.10

      About OvalEdge 5.2.10

      The OvalEdge 5.2.10 is the latest release enhanced with the new Data Custodian Role, which represents the IT sidetracking of data metrics, including data lineage, overseeing data storage, and tracking the data usage. This release includes the Entity Delete Framework that allows the users with system administrator privileges to remove specific database connections, schemas, domains, and archiving or transferring roles. 

      Along with the existing functionality enhancements in Home, Data Catalog, Business Glossary, Data Stories, Governance Catalog, My Resources, Jobs, Advanced Tools, and Administration modules.

      • The Elasticsearch Filter is improvised to display the search results specific to data objects. Each data object is enabled with a set of search filters to simplify the search results further.
      • The Data Catalog and Business Glossary main data grids are improvised with the customizable view functionality. The user can select the desired columns to be displayed in the Data Catalog and Business Glossary main data grids.
      • The Sankey Lineage Diagram is enhanced to a multi-level horizontal level to display data flow among the objects at a higher level and can trace a single object through the flow.
      • The Projects module is redesigned to display the list of all tasks assigned to the logged-in user with Kanban and Table view formats with the different Project statuses displayed in separate columns. 

      In this release, the critical and significant bugs associated with the Home, Data Catalog, Business Glossary, Data Stories, Governance Catalog, My Resources, Jobs, Advanced Tools, Administration, and Connectors are fixed and working as expected.

      Release Type                                                         Release Date

      Service Release                                                   02 June, 2022

      Release Version 5.2.10 

      Improvements

      • Data Catalog
        In the Table Columns earlier, the Technical Description was used to get updated based on the view popularity score and used to state whether the column is significant or not. Now, this process is enhanced to manually enter the Technical description associated with the Table Column.
      • Query Sheet
        In the Query Sheet, now the users can develop the queries associated with the DB2 connection and view the results in the Data Catalog - Queries.
      • Advanced Tools
        In the Advanced Tools earlier, the Temp Lineage Correction used to take a long time to fetch the recommendation for the Temp Tables. The current functionality is enhanced to run the job based on the user action and show the recommendations associated with the Temp Tables.
      • Advanced Jobs
        In the Advanced Jobs, the current advanced job associated with the Temp Lineage Merging with #Temp Tables is now enhanced to allow the users to choose the prefix (Ex: @, #) of a Table by which the Lineage needs to be merged.
      • Administration

        Connectors
        • Oracle NetSuite CRM
          The new connector Oracle NetSuite Customer Relationship Management (CRM), helps manage interactions with current and potential customers, partners, and suppliers in a single place. In the OvalEdge application, the Oracle NetSuite CRM connector allows users to crawl and sample profile business entities like Accounts, Customers, Sales, etc. 
        • Microsoft Event Hub
          The new connector Microsoft Event Hub is a fully managed, real-time data ingestion service that streams millions of events per second from any source to build dynamic data pipelines and immediately respond to business challenges. In the OvalEdge application, the Microsoft Event Hubs connector allows the users to crawl the metadata, profile the sample data, and build lineage to view the movement of the crawled data.
        • In the Crawler, for Microstrategy connection, users can now view the Top Users Stats, i.e., the user actions performed on the Reports information by running an advanced job and viewing the details in the Data Catalog - Reports.
        • In the Crawler, for SSRS connection, the Lineage used to be built for only datasets of that particular report whenever the data is crawled. The current functionality is enhanced, and the Lineage is built for the shared datasets.
        • In the Crawler, for the Redshift connection, the system used to crawl only views associated with that particular connection. The existing functionality is enhanced to crawl Latent Binding Views, i.e., the special view types available in RedShift.
        • In Crawler, for the Databricks connection, the existing connection validation is enhanced by providing access token type authentication. The user can validate the connection by providing the token generated at the Databricks source.
        • In the Crawler, for the DENODO connection, the existing Lineage functionality is enhanced by implementing the Custom Wrapper, which is used as a data source by writing a JAVA wrapper in case of building the Lineage.
      • OvalEdge API
        In Crawler, once after establishing the connection, instead of configuring and updating the Crawler and Profile settings manually, the new API “Update Crawler and Profile Rules” is developed to update the Crawler and Profile settings for all the Connectors by calling the respective API.

      Bug Fixes

      1. Advanced Tools
        1. In the Advanced Tools, while building the Lineage for the Tableau reports, the system is throwing an error due to Null objects, now this issue is fixed, and the Null Checks are added while performing the Lineage Job.
        2. In the Advanced Tools > Data & Metadata Changes, for the Reports Connection, the issue related to displaying the version count for all the Reports in the Report Group instead of the modified Report is now fixed and working as expected.
      2. Administration
        1. In the Crawler, for the Qlikview connection in the Data Catalog - Reports, whenever the data is crawled, the inactive sheets are getting considered, and the sheets count was not showing appropriately, but now this issue is fixed, and the sheet count in the Reports are showing correctly.
        2. In the Crawler, for the Snowflake and PostgreSQL connections earlier, the system used to crawl the materialized views as the direct views and used display in the Data Catalog - Queries with which it was getting difficult for users to discriminate the views which one are direct or materialized. Now, this issue is fixed by fetching the definition and the columns associated with materialized views and displaying them under the Data Catalog - Database - Schema - Views.
        3. In the Crawler, for SSIS connection, in the Data Catalog - Queries - Association earlier, the Table Columns used to not link, and it was getting difficult for users to understand the Lineage Column Mapping, but now this issue is fixed, and the Table Columns are also associated with the Dataset.
        4. In the Crawler, the issue related to validating the connection after entering the required details and the spaces issue associated with the Schema Names is now fixed and working as expected for the DB2 connection.
        5. In the Crawler, the issue related to parsing getting failed while building the lineage for Views is now fixed and able to parse the query for the DB2 connection.
        6. In the Crawler, the issue related to the display of Profile options and settings is now fixed and made disabled for the AWS Glue ETL connection.
        7. In the Crawler, the issue related to the Linage building for the derived views and base views is now fixed by updating the query to retrieve the base views for the DENODO connection.
        8. In the Crawler, the issue related to parsing getting failed while building the lineage for the Materialized Views with a couple of “Create Statements” is now fixed and able to parse the query for the Redshift connection.
        9. In the Crawler, the issue of getting an error while profiling the data is now fixed and working as expected for the Kafka connection.
        10. In the Crawler, the issue of getting an error while exporting the multiple parse failed queries is now fixed and working as expected for the Teradata connection.
        11. In the Crawler, once after Crawling and Building the Lineage, the issue related to the formation of the Lineage for the Reports is now fixed and working as expected for the Power BI connection.
        12. In the Crawler, once after Crawling and Building the Lineage, the issue related to the Lineage formation is now fixed and working as expected for the Power BI On-Premises connection.
        13. In the Crawler, the issue of getting an error while profiling the data is now fixed and working as expected for the HubSpot connection.
        14. In the Crawler, the issue related to validating the connection after entering the required details is now fixed and working as expected for the SSAS and SSAS On-Premise connections.
        15. In the Crawler, the issue of getting an error while profiling the data is now fixed and working as expected for the SSAS connection.
        16. In the Crawler, the issue with fetching the columns from the SSAS Tabular Database into the OvalEdge application for all the Schemas using .Net API is now fixed and working as expected.
        17. In the Crawler, for the PostgreSQL connection in the Data Catalog - Table Columns, whenever the data is crawled, the issue related to the displaying of Primary Key related information in the Technical Description is now fixed, and the Primary Key associated details are displayed in the Statistics page at the Key field.

      Release Version 5.2.9

      Improvements

      • Data Catalog
        • In the  Data Catalog module > specific object detailed page > clicking upon the Lineage tab (upon crawling is done and lineage is built), hovering on to a specific object in the lineage page > three dots(…) with Copy to Clipboard and View Association options are clickable and highlighted in blue color.
        • In the Data Catalog module, when the user associates any tags/terms governance roles are updated only on Tables/Files/Reports and updates on the Table Column/File Column/Reports Column. When a user updates the governance roles on the Tables/Files/Reports, the respective columns should update.
      • Dashboard
        • The Data Lineage main page is now provided with a header title in the top left of the page.
        • In the Dashboard > Favorite Reports > a search filter is integrated with the top right of the page to search from All reports.
      • Governance Catalog
        • Servicedesk
          • When a user creates a new Service Desk request for getting access to the schema. In the service request template, the users select a schema from the dropdown and now the system displays the schema list as ‘connection name. schema name’. For example. SQL Server is the name of the connection, and dbo is the name of the schema, it displays as “sqlserver.dbo”.
          • The Servicedesk home page displays the list of Waiting, My Request, and All Request for service requests raised, and on the main page, it displays the summary with a clickable option that will navigate to the service request templates.
      • Advanced Tools
        In the Log Viewer Page, add Session-Id, Request Id, and User Id column fields are added. In All Logs, a Clear Logs button is enabled to keep 10k logs and remove the rest from the table entries.
      • Administration 
        • Crawler 
          • From the Crawler setting page, the profiler tab has been removed for the Reporting & File types connectors.
            • Reporting connector: Power BI, Tableau, SSRS SQL,  SISENSE, Microstrategy, Looker.
            • File Connectors: NFS, S3, Azure Data lake, SFTP, Google Drive Connector, GCS Google Cloud Storage, HDFS Kerberos, HDFS non-Kerberos, DSEFS Connector.
          • In the Crawler page, the crawl/profile page is enhanced with the Schedule feature with date & time. The user can schedule the crawl time by providing the information such as Date, time, and frequency of crawling as month/year/week/day.
          • In the Job Workflow page, at the time of adding a new job, the user can schedule the crawling time for multiple jobs in a single run. 
        • Users & Roles
          The roles page is enhanced with a column ‘Created on’ to show when a user role is created and ‘Updated on’ to display the date when details of any users were last updated. The Created Date column provides the date on which the Role (Eg: OE_ADMIN, OE_PUBLIC, OE_CSM) is created and the Last Updated column provides the date on which update is recently done in the Roles.
        • Manage Servicedesk
          In the Manage Service Desk, when the user edits the service desk template, the name of the template selected is displayed at the top of the template.  
        • Advanced Job 
          • The Advanced Job - Upload Business Glossary and associate Term CLS - now the Steward and Reviewer fields (also Teams) are now associated at the Category level and at the Domain level the admin is by default set for Steward and Reviewer. The custom field Frequency column is now changed to Frequency in Months.
          • The Advanced Job - Load Domain and Business Glossary Terms and Associate to Columns - now the Steward and Reviewer fields (also Teams) are now associated at the Category level, and at the Domain level, the admin is by default set for Steward and Reviewer. Additional Detail Tab and Previously Assigned Group columns are added to the excel sheet.

      Release Version 5.2.8

      Features

      1. Governance Roles
        The OvalEdge Governance Roles are enhanced with the new Data Custodian Role, which represents the IT sidetracking of data metrics, including data lineage, overseeing data storage, and tracking the data usage, data governance, and are responsible for more technical responsibilities. 
        The Custodian is often someone on the BI/Data Services team and works closely with the Data Steward to share and prioritize backlogs, agree on tech debt retirements, and move users to self-service areas.
        Along with the Data Custodian role, the OvalEdge application has labels for Owner, Steward, Custodian, and has the ability to define more labels according to new roles (Governance Roles 4,5, and 6) for additional responsibilities required for integrity and security of data usage. 
      2. Delete Framework
        The OvalEdge Entity Delete Framework allows the users with system administrator privileges to remove specific database connections, schemas, domains, and archiving or transferring roles. The purpose of this framework is to follow a pattern when the OE admin deletes an entity in the application. This pattern involves deleting all child objects iteratively, transferring all responsibilities, and archiving (if required) before deleting the entity.  
      3. Elasticsearch Filter
        The Elasticsearch Filter functionality is improvised to display the search results specific to data objects like Databases/Tables/Table Columns/Files/File Columns/Reports/Report Columns/Queries/Business Glossary/Tags/Projects/Data Stories/Service Requests. Each data object is enabled with a set of search filters to simplify the search results further. 
        • The Advanced Filter is allowed to refine the search results by applying AND/OR/NOT operations next to the search filters.
        • The new highlight option is available next to the data object title to display how the search keyword is related and matched to the data object.
        • The Popularity Score/Endorsement Score /Relevance Score is also available for every search result.
        • The Global Search Filter with exact search term functionality is enhanced to fetch the precise search results by including qualifiers (& for AND,  II for OR, - for NOT) in the search filter.
      4. Customizable  View
        The Data Catalog and Business Glossary main data grids are improvised with the customizable view functionality. The user can select the desired columns to be displayed in the Data Catalog and Business Glossary main data grids. Here, the user can create multiple views for a page, add a name and description to the view, and specify the view to be Private/Public/System.
      5. Entity Relationship Diagrams 

        Entity Relationship shows the complete logical relationship structure among tables and their attributes in a database using Rectangles and connecting lines as links between them. The ER Diagrams of the table can be viewed by scrolling the bar to the Graphical view. 

        • The Main Table along with Linked Table columns are displayed in respective colored Table Title headers and hyperlinks to navigate to the respective Table Detail page. Additionally, the Primary Key or Foreign Key associated with the Table columns is shown using the key icon.
        • The Similarity Score and Join Score are displayed on the right side along with the Main and Linked Table column details.
        • The different Table Column relationships, including One-to-One, One-to-Many, Many-to-One, and Many-to-Many can be viewed.
        • Zoom in Zoom out functionality for the ER Diagrams is enabled for more readability and visualization. It provides an option to drag the tables left/right/top/bottom to clear visuals of the Table relations.
      6. Multi-Level Sankey
        The Sankey Lineage Diagram is enhanced to a multi-level horizontal level to display data flow among the objects at a higher level and can trace a single object through the flow. Users can now dig deep into the lineage relationships from Schema level to table and column levels. Apart from the existing viewing capabilities, viewing data lineage graphs has made it user-friendly with the following features:
        • Schema Level Movement: Displays the data flow among the data assets (Tables/Files/Reports) in the application.
        • Table Level Movement: Displays the relationships among a specific object to different other tables.
        • Column Level Movement: Displays the relationships of each Table to different columns in other tables.
        • Zoom In Zoom Out:  This allows users to zoom in and zoom out the Sankey diagram at the Schema/Table/Column level and also permits them to drag the nodes as per their convenience and understand the links better.
      7. Projects
        The Projects module is redesigned to display the list of all tasks assigned to the logged-in user with Kanban and Table view formats with the different Project statuses displayed in separate columns. Users can create more Project statuses along with the predefined statuses, drag and drop the objects from one status to another, add transitions to the Project Tasks, and change the multiple project tasks at one instance. The different project status counts for To Do, In Progress, and Done are displayed using the color bar for better visualization.
      8. Data Catalog
        In the Data Catalog, all data objects - Table/Table Columns/File/File Columns/Report/Report Columns/ Queries are made downloadable for the Customize View selected in the excel format.
        The existing user interface of the Data Catalog is improvised to apply filters on the selected columns, and apply sorting to display the results in alphabetical order from A-Z/Z-A, and also supports the multi-level sorting where the sorting can be performed individually to the columns without impacting other columns.
      9. Business Glossary 
        The Business Glossary module is enhanced to allow the users to update the governance roles (Owner, Steward, Custodian, Governance Roles 4, Governance Roles 5, and Governance Roles 6) for a term and manage the default governance roles through the term and assign the terms on the object (s) or data set. 
        If a user updates the governance roles, the Audit Trail - Data Objects records the object on which the user got updated. For example, the owner of a table called 'customerorder' was named 'Harry', later admin users could update this user name to 'Robert', in this case, User Harry would be recorded as deletion, and User 'Robert' recorded as an insertion. Users can be either individuals or teams. 
      10. Advanced Tools
        In the Advanced Tools module, the Load Metadata from Files template is improvised, and the Object Names are included with the following fields:

        Object Name

        Field Name

        Schema 

        Default Governance Roles

        Table 

        Notify, Last Modified Date, and Default Governance Roles

        Table Column

        Default Governance Roles

        File 

        Notify, Last Modified Date, and Governance Roles

        File Column

        Last Modified Date, and Default Governance Roles

        Report

        Notify, Last Modified Date, Certification, and Governance Roles

        Report Column

        Last Modified Date, and Default Governance Roles

        Business Glossary

        The Business Glossary Template an excel sheet is downloaded in 4 separate sheets with respective Business Glossary fields.

        • Terms
        • Term Relationships
        • Term Objects
        • Reference Data

        Tags

        The Tags Template is downloaded in an excel sheet with 3 separate sections with respective Business Glossary fields.

        • Tags
        • Tag relationship Hierarchy
        • Reference Data

        Dataset Correction

        • Lineage Status

        • Manual Lineage

         

      11. Administration
        In the Crawler module, the Manage Connection is improvised and categorized into Connection  Settings and Default Governance Roles. Here, by default, the governance roles and users configured in the system are displayed, where the user can select a specific user or a team for that particular connector. 

      Improvements

      • Home
        • In the Home page when a tag is selected, the Tag Description, Created By user, and Created Date details are displayed at the top of the page.
        • The Filter is provided to refine the search process by selecting specific data objects. Users can now select all the data objects in the filter options by entering an object name, and the search filter results are shown.
        • The Tooltip feature is enabled to the Table Plugin, hovering over any element the user can view the brief information about the element.
        • All the Date fields in the Table plugin are made filterable, and the user can choose to select the date from the calendar pop-up.
        • In the Helpful Resources, earlier users were only able to save the resource links. Now, the Functionality is enhanced where the resource links are made clickable, and users can now navigate to the resource links page directly.
        • While performing any validation, saving, or crawling, any error message usually pops up on the screen. Users can now Copy the error message to the clipboard to view the full details of the error and figure out what the error is about.
        • The Bookmark feature is enhanced to organize the Bookmark as per the user requirements. Here the Bookmark names can be edited/deleted/unpin from the Bookmark list. 

          Elasticsearch Filter
          • In the Home module, the Elasticsearch Filter is improvised to support the search based on the Custom Fields, and when a user searches an Object using Tag, then the search results are now refined to include all the Objects associated with the Tags.
          • In the Elasticsearch page, a new pagination box is provided in the bottom left of the page to show the current page count and the total number of pages, and users can use the right and left arrows to navigate to the left/right pages.
      • Data Catalog
        • In the Data Catalog > Tables page, an additional Column count column is added, similar to the row count to the main table.
        • In the Data Catalog > Tables page, when a table with no data is downloaded then the downloaded file now displays the names of the column related to the table.
        • In the Data Catalog > Table > Lineage > Edit Lineage > the Map column via AI is now enabled for File columns and Report columns.
        • In the Data Catalog > Reports page, the search Audit Report is updated to show the most searched terms for 12 weeks.
        • The Object Type column is added to the respective object table page to display the Object Type from the Empty/Table/View options.
        • In the Data Catalog, users can now certify an object with Certify/Caution/Violation/Inactive/None options from a specific object and can endorse data objects at Column level - Table Columns/File Columns/Report Columns.

          Reports
          • The new report “Role Wise Access on the Objects” is developed to display the user's role in having access to the various data objects like Schema, Table, File, and Reports.
          • The new report “Duplicate Count” is developed to display the duplicate count of objects in the schema based on the connections, and these duplicates can be seen in Tables and File.
      • Business Glossary
        • In the Business Glossary, the AI Recommendation is improvised to display the data score and pattern score for the object type file column similar to the Table Column object. Here, the File Column object associated with a term displays a recommendation based on the 'Smart Score', 'Name,' 'Data,' and 'Pattern'. 
        • Users can now download Business Glossary Term data along with custom field details, associated objects to the Term Details, and Description.
        • The Custom Fields are now provided with the View History option to display the logs of updates with the Username, Date & Time. 
        • Users can now delete multiple terms from the Business Glossary main page by selecting multiple objects.
        • Users can now download all the tables and columns within each table and their descriptions at an instance from the Data Catalog main page.
      • Data Stories
        • The Data Stories is enhanced with the use of the @ notation feature to access the object and users list as listed below:
          • @ Users (owner, steward, owner, and other governance roles)
          • @ Objects( Schema/Table/TableColum/Report/ReportColumn/File/FileColumn, Business Glossary)
          • @  in the collaboration message tool
          • @ in the content page to pull any data objects
      • Projects
        • In the Project Detailed page, the breadcrumbs are provided in the top left of the screen to show the navigation path from the Project module to a specific project detail page.
        • In the All Projects page, additional columns are added to the main table that includes Added by, Added on, Update By, Updated on, Completed By, Completed On columns, and by default, the 1000 objects are displayed.
      • Governance Catalog
        • In the Data Quality, DQR Objects page, the Results Stats Query and Failed Data Query are now provided with the tooltips to view the detailed descriptions.
        • In the Certification Policy page, the Policy Description is now made as an editable field and displayed in a separate pop-up window.
        • In the Data Classification, additional category and subcategory fields are included, and if the domain is selected, pertaining category and subcategory options are displayed.
        • In the Data Classification > Business Glossary Page > Domain Column, a filter option is enabled for the users to refine the search based on the keyword entered.

          Note: 

          Governance Catalog > Service Desk

          For the migration customers, if a customer wants to raise a service request (Content change) on a data object having both TERM and DAG then the customer needs to manually remove one of the TERM or DAG on the data object otherwise, the service request may not be approved as per the new design i.e. the customer is allowed to add only a TERM or a DAG but not both.

      • My Resource
        • In the My Resources, My Profile page, the Add Slack ID and Add Phone Number fields are now made clickable and allow the user to edit the profile details.
        • In the My Resource, the Watchlist Notification is now integrated with the overall application to receive notifications associated with the Metadata and Remote Metadata modifications. Here, whenever the data objects in the watchlist undergo a significant percentage of data change, the users get notifications for the newly created custom fields or update or delete content in the custom fields.
      • Jobs
        • In Jobs, the main table, the Duration and Size columns, are added for the users to view the duration and size of every job.
        • The Job Log Value is enhanced with a search filter that allows filtering by timestamp and date for the list of jobs executed.
      • Advanced Tools
        • In the Load Metadata From Files, the DQR template is improvised to support uploading the SQL functions in the query.
        • In the Load Meta Data Files, Data Quality Rules are improvised to upload 100+rules for DQR SQL functions for both existing and non-existing queries.
        • In the OvalEdge API, the new API (/api/term/searchTerms) is included to fetch the list of all terms associated with all the objects in the OE application.
      • Administration
        • Users & Roles
          • The add role page is implemented with the ‘Role description’ column that gives a brief description of a role. It is also provided with an edit option to update the role description further.
          • The bulk deletion option is provided to delete one or more roles in the list at one instant.
        • Security
          • In the Security module, the existing Domain main page is enhanced to delete multiple domain users at one instant.
          • In the Security, Reports tab, a new Schema column is added to the table.
        • Manage Servicedesk
          • The Manage Servicedesk data grid is included with a column called ‘Status’ and ‘Approval Status’ for a custom template. When a custom template is created, by default status of the custom template is set as ‘Published.’ The Approval Status indicates whether the template is active or not. Here, whenever the templates are no longer necessary and need to hide from the list, they can be set as ‘Inactive’ mode.
          • A user can change the approval status of the custom or system-defined template from the ‘Edit Service Request template’ (Custom or System) and cannot make it to an ‘Inactive’ status when a custom template is in the process of approval with a Servicedesk ticket. 
        • Advanced Job
          • In the Advanced Job page, the Last Run Date and Status columns are added to the table.
        • Audit Trails
          • In the Audit Trails page, the Audit Trail title is provided in the header for easy reference of the page.
          • In Audit Trails, Catalog > Terms, the complete Term value is displayed in specific Term fields along with domain name, category name, subcategory name, and term name.
          • In the Audit Trails, the existing Catalog page is improvised and included with the ‘Custom Field’ tab to capture the value changes done to the custom field type, and the Object Name field also displays the name of the object along with a hyperlink clicking upon which user navigates to the specific objects page.
          • In Audit Trails, Catalog > Tags > the DAG tags are now shown using a superscript.

      Copyright © 2019, OvalEdge LLC, Peachtree Corners GA USA