OvalEdge Releases
      Back to home
      1. Knowledge Base
      2. Architecture, Security & Releases
      3. OvalEdge Releases

      Release6.0

      The OvalEdge 6.0 is a major release that includes the following features:

      1. New UI/UX:
        1. The new UI/UX makes data governance a more aesthetic and easy-to-use experience.
        2. The lightweight look and feel of the application give an all-around pleasant experience for the user.
        3. The user interaction with the application is enhanced by making all the actionable items very interactive and provided with color schemes, help icons, and clickable links on every significant data element.
        4. Customizable interfaces are provided for users to organize the data grid columns according to their specific needs.
        5. The segregation of associations based on object type is provided for the users to easily discover the objects from tables.
        6. Users are provided a better sense of control over the interface by prioritizing elements of information and breaking down information into sections using expandable and collapsible accordions and also have the ability to drill down into details for the targeted content using quick links.
      2. Service Desk: The existing Service Desk template is customized to display the Connection Name, Connection type, and Object type for Data Catalog Objects and Domain, Category, and Subcategory for the Governance Catalog Objects.
      3. OvalEdge on Cloud: The cloud version of OvalEdge is available for the 6.0 version. Users can now access the application through AWS's secured cloud infrastructure.
      4. OvalEdge Bridge: It is a new platform that enables the cloud-hosted OvalEdge server to connect with on-premise data sources without changing the firewalls.
      5. Global Search Enhancements: OvalEdge is integrated with Elasticsearch to refine the filtering of data objects. The left panel field filters are provided to refine specific object-type search results. Also, an Advanced search filter is enabled for every object type with multi-level granularity filtering to the Data Catalog using the AND/OR operators.
      6. Service Desk Enhancements: OvalEdge supports multi-object service requests, workflow templatization, and an access cart for centralized requests.
      7. Crawler | Access Instruction: In the Crawler Setting, the access instruction is a new option provided to allow the Crawler Admin to write the instruction relevant to the data source/connector that has been crawled.
      8. New Connectors: Many new connectors are included in the OvalEdge existing connectors list.
        1. Amazon DocumentDB
        2. Azure Data Factory
        3. Cosmos DB 
        4. DBT
        5. GitHub
        6. Google Drive
        7. HubSpot
        8. Okta
      9. Performance Optimization: The application performance has been optimized significantly to make it highly responsive.
      10. Security Improvements: Authorization/Permissions, Sensitive Data Exposure, IDOR (Indirect Object Reference), Privilege Escalation Exploit, SQL Injection Attacks, Persistent XSS, Cross-site Scripting (XSS)/Reflected XSS, and Cross-site Request Forgery (CSRF) Token Attacks Security Vulnerabilities are addressed in this release.
      11. Bugs Fixes: In this release, all the critical and significant bugs associated with functionalities like Home, Data Catalog, Business Glossary, Data Stories, Governance Catalog, My Resources, Jobs, Advanced Tools, Administration, and Connectors are fixed and working as expected.

      Release Type                                                         Release Date

      Major Release                                                         15 November, 2022

      Features

      New User Interface/User Experience

      The responsive design and interactive interface make the data discovery process more efficient across the different modules in the application.

      1. Home
        1. The welcome page allows users to quickly onboard and begin using the following features of the applications:
          1. End-to-End Data Governance - Data Access, Data Quality, Business Glossary, and Data Stories.
          2. Self Servicing Tools - Impact Analysis, Lineage Maintenance, Service Desk, Load Metadata from Files, Data Classification and Query Sheet.
          3. Data Catalog - APIs, Tables, Files, and Reports.
        2. The home page enhances accessibility to the most commonly used application features relevant to the logged-in user. The top and frequently used catalog objects pertinent to the user are displayed in separate widgets. The action items are clearly established, aiding users to quickly get started with essential tasks and navigate to targeted information.
        3. The header menu is enhanced with new clickable icons like My Access Cart, Default Projects, and Service Desk to drill down to details of that specific section.
        4. The service requests pending from the user are now notified through the home page.
        5. The helpful resources section is provided for users to access important information directly from the home screen easily.
        6. A graph on the home page gives quick visual feedback about the number of data objects updated.
        7. The tags and top domains sections are newly included on the home page to discover tags and domains using the cards and clickable links.
      2. Tags
        1. Tags are now displayed as a separate module to simplify the process of data object grouping for the users.
        2. The custom view feature offers users the flexibility they need to manage data.
        3. The newly added tree view helps users quickly look at the hierarchically organized terms under respective parent, child, and subordinate child tags.
        4. The Tags detail page is categorized into different sections with expandable and collapsible sections (accordions) that helps the users to make quick decisions by referring to the targeted data element.
      3. Business Glossary
        1. The Business Glossary is now integrated with the Tree View (default) option that helps users to have a glance at the terms organized hierarchically with terms under respective domains, categories, and sub-categories.
        2. By using the Search Filter enabled in the left panel, users can search for terms, sort the term list alphabetically, and collapse all/expand all accordions to view only domain names or domain names along with categories, subcategories, and terms
        3. The View Details option is provided to drill down into details of targeted terms.
        4. Existing Term Details pages are categorized into various sections with expandable and collapsible sections (accordions) so that the user can browse to the targeted data elements clutter-free.
      4. Data Catalog
        1. The Data Catalog detailed page layout is now made more user-friendly, and the icons are rearranged for better reach of the information.
      5. My Resources
        1. The My Resources-Inbox is provided with the Pending and Cleared tab, to avoid confusion of the read and unread message/notification.
        2. My Projects tasks are removed from the Projects and added under My resources to easily navigate the information related to a particular User.
      6. Compare Schema
        1. The Compare Schema interface is improvised with a new layout of the tabular view with Column Change Summary and Column Change Detail tabs that provides the comparison overview results more effectively.
      7. Users & Roles
        1. The Column Security Policies is newly implemented to get an overview of the policies created in the application to safeguard the data. Similarly, the Governance Catalog is newly added to the Audit Trail, which provides the audit trail data for Tags, Terms, and Domains. 

      New Search 

      All columns with search functionality are equipped with the conditional search option. Users can search for the needed information using various conditions, such as Contains, Doesn't Contain, Doesn't Equal, Empty, Ends With, Equals, Is not Empty, and Starts With.

      Deletion Framework

      The OvalEdge Deletion Framework allows the users with system administrator privileges to remove specific users by transferring their roles to configured users defined by the admin. This pattern involves deleting all child objects iteratively, transferring all responsibilities, and archiving (if required) before deleting the user. 

      SaaS Offering or Cloud Easy Deployment

      The OvalEdge application now provides a SaaS distribution model in which the application can be deployed to any cloud provider and makes it available to end-users with a single click. Here the standard(manual) installation and configurations required for setting up the OvalEdge application are skipped. 

      With the SaaS model, OvalEdge applications can be deployed with one click and customers will be given a URL to access them. The SaaS offering will not incur any additional cost, as it will be included in the OvalEdge license itself.

      The OvalEdge takes the responsibility of hosting, maintaining, and deploying the application to separate VM for every customer and ensures that one customer's data is not shared with other customers. 

      The OvalEdge also follows the ISO Standards and all the security protocols to secure the customer data and reduces the effort for application setup, maintenance, and upgrades. As of now, this process is supported only for AWS.

      OvalEdge Bridge Development for Cloud Offering

      As part of the cloud offering from OvalEdge, customers can start using the platform as a hosted service without worrying about connecting to client data sources. With the OvalEdge Bridge component, any cloud-hosted server can connect with any on-premise or public cloud data sources without modifying firewall rules (since the client component of the bridge is installed within the client infrastructure, the client only needs to whitelist the IP address and port of the client).

      OvalEdge Bridge offers the following features:

      1. The OvalEdge platform can establish up to 10 bridges to client data sources.
      2. A bridge can host multiple connections like Oracle, MySQL, Postgres, etc.
      3. HTTPS is used to secure communication between the OvalEdge platform and the OvalEdge client (deployed/installed in client infrastructure)
      4. In the course of creating a bridge, the OvalEdge platform generates an SSL-enabled key store and makes it available for download.
      5. The communication between the bridge and server is a pull-based communication from the bridge, which keeps polling the server frequently for any jobs (crawling, query execution, etc).

      Performance Optimization

      The application performance has been optimized significantly to make it highly responsive and the latency for all the landing pages in the OvalEdge application is targeted to be achieved in less than 2 seconds by normalizing & splitting the queries, using cache memory, pre-fetching the data, reducing the number of synchronize jobs, and adding read replicas (managing the user requests).

      Earlier, the loading time for the Data Catalog page with 3 Million data objects was around 2 minutes and 20 seconds, and now it is efficient to load the same data in less than 2 seconds. 

      The sample OvalEdge performance report before 6.0 

      In the above bar graph and statistics:

      1. The green, yellow, and orange bars represent the average response time taken for 100 users with 7983 requests was around 26 seconds.
      2. The red bar represents failed user requests. 
      3. Out of a Total of 7983 User Requests - 69% Passed and 31% Failed. 

       The sample OvalEdge performance report after 6.0

      In the above bar graph and statistics:

      1. The green and yellow bars represent the average response time taken for 1000 users with 239310 requests is around 0.4 seconds.
      2. Out of a Total of 239310 User Requests - 100% Passed and 0% Failed. 

      Remote Data Access Management 

      1. As part of RDAM Sync functionality, OvalEdge (OE) Roles and OE Users are created from Remote Roles and Remote Users where the Business License is set to the newly created OE Roles and OE Users.
      2. In the Users & Roles Management, the new tab “Remote Policies” is added to view the source system (Snowflake, S3, etc) access policies, and the new column “Source” is included in both Users and Roles Tab to display the information about the source from which the Users and Roles are added. Ex: OvalEdge, AD, SAML, OAUTH, REMOTE, OVALEDGE API

      Global Search Enhancements

      Global Search is now refined with multiple filtering options such as

      1. Left Panel Field Search: The left panel helps refine search results on specific object-type fields by selecting the preferred fields. Users can now search for Certified data objects specific to an object type using the Certified checkbox option enabled in the left panel search field.
      2. Search using Qualifiers: The Global search filter works on the exact search term functionality. Users can fetch accurate search results for the keyword entered by including qualifiers (& for AND, II for OR, - for NOT) in the search filter.
      3. Advanced Search: Advanced search filter is enabled for every object type with multi-level granularity filtering to the data catalog using the AND/OR operators.
      4. It facilitates the users to refine the search results by applying filters on top of the search results.
      5. In the search filter, the AND search operator is applied to Tags/Terms/Synonyms fields, and OR search operator is applied to fields like Connection Name/Schema Name/Status/Certify fields.
      6. The Data Highlight icon is provided for every object to display how the search keyword is related and matched to the data object attribute in a pop-up window.
      7. The Relevance Score is used to display how closely the data objects match the search keyword. It uses a 1-100 scale in which "1" means the data object search keyword is much less relevant than other data objects, and "100" means the data object is much more relevant. It is calculated using (Elasticsearch Score + Synonym Score) ✖ Popularity Score.

      Business Glossary Enhancements

      The Business Glossary is now enhanced with multiple action buttons listed below:

      1. Suggest a Term - The Suggest a Term is improvised, where if a user suggests a term, a new service request is raised and sent for the approval process, and upon fulfillment of the request, a term is created in the Draft status. 
      2. View Details - The View Details is a new button on the Tree View page that helps the users view the complete details of the selected Term.
      3. Request Publish - The Request Publish is the new button on the Term Detail page that helps the users to publish the draft status terms. When a user clicks on the Request Publish button, a new service request is raised and notified to the configured approvers who are set in the approval workflow process.
      4. Publish - The Publish option is now enabled only for Admin/Steward and the Final Approvers, who are set in the approval workflow process. The Admin/Steward users can now directly Publish a Term without raising a service request from the Term Detailed page.
      5. Manage Data Associations
        The Manage Associations Data and Manage Governance Roles sections are merged into one section and labeled as Manage Data Associations, which includes:
        1. Copy Title to catalog
        2. Copy Business Description to Catalog
        3. Copy Steward to Catalog
        4. Copy Custodian to Catalog
        5. Copy Owner to Catalog
        6. Copy Role4 to Catalog
        7. Copy GovernanceRole5 to Catalog
        8. Copy GovernanceRole6 to Catalog
        9. Restricted
        10. Masked
        11. Show Classification in Dictionary
      6. The Manage Classifications section is now renamed to Classifications.

      Service Desk Enhancements

      My Access Cart is a feature implemented to create Service Requests on multiple data objects in a single instance. To add the data objects to the My Access Cart, the My Access Cart project should be set as a default project to add the selected objects to the Cart.  Data objects can be added to the My Access Cart Project from various modules (Data Catalog) in the application.

      My Access Cart UI is integrated with the following components:

      1. Listing of multiple (Service Request) SRs, each containing multiple objects grouped by the algorithm in the left panel.
      2. Display the respective SR form in the right-side panel for the SR selected in the left panel based on the Request Type, Object Type, Connection Type/Connection/Domain/Category/Sub-Category.
      3. Display the Service Request ID in the SR card after submitting the Service Request.

      Build Auto Lineage Enhancement

      The Build Auto Lineage module is enhanced with the new feature to Building Lineage for new/changed queries. Whenever any changes are made in a query/or a new query is added within a remote database, it will allow you to build lineage for the new or changed query after crawling the database. 

      Endorsement Rating 

      Raise a Flag: A red flag is a warning given to notify other users suggesting that there is a potential issue with the data quality of the data object. If the users raise a red flag on a specific data object, the data moving downstream to any tables, files, and reports will be marked with a red flag. The red flag helps other users become aware of the issue.

      1. Users with Meta-Read access permissions can also raise a flag, but it is mandatory to provide comments when a Red flag is raised.
      2. The steward is notified of the same. Users with Meta-Write, Steward, and Governance roles (Steward, Owner, Custodian) users of the data object can remove the red flag. Also, the user who created the flag can remove the flag by clicking on the Remove the Flag button in the same pop-up window.

      New Connectors

      1. Amazon Document DB is a scalable, highly durable, and fully managed database service. In the OvalEdge application, the Amazon Document DB connector allows the users to crawl and sample profile the data.
      2. Azure Data Factory is a cloud-based ETL and data integration service that allows users to create data-driven workflows for orchestrating data movement and transforming data at scale. In the OvalEdge application, the Azure Data Factory connector allows the users to crawl Datasets, Dataflows, Pipelines, Activities, and Linked Services and build the Lineage.
      3. Cosmos DB is a cloud-based NoSQL database and globally distributed database service that allows users to manage data across data centers worldwide. In the OvalEdge application, the Cosmos DB connector allows the users to crawl the database objects.
      4. Data Building Tool (DBT) helps data analysts and engineers to transform data in their warehouses by writing select statements simply. DBT performs the transformation part of ETL but doesn't support extraction and loading. In the OvalEdge application, the DBT connector  allows the users to crawl datasets, i.e., JOBS and RUNS, and build the Lineage.
      5. HubSpot is a CRM platform with all the tools and integrations required for marketing, sales, content management, and customer service. In the OvalEdge application, the HubSpot connector allows the users to crawl the metadata and profile the sample data.
      6. Okta is an enterprise-grade identity management service built for the cloud but compatible with many on-premises applications. With Okta, IT can manage any employee's access to any application or device. OKTA features include Provisioning, Single Sign-On (SSO), Active Directory (AD) and LDAP integration, the centralized de-provisioning of users, multi-factor authentication (MFA), mobile identity management, and flexible policies for organization security and control. In the OvalEdge application, the Okta connector allows the user to crawl Users and User Groups from Okta for specified applications on Okta.
      7. Github provides source code management (SCM) and distributed version control. In the OvalEdge application, the Github connector allows the users to crawl all the files in the Github repositories.
      8. Google Drive allows sharing or transferring files and folders between users. In the OvalEdge application, the Google Drive connector allows the users to crawl the database objects and profile the sample data.
      9. Microsoft Dynamics 365 is a cloud-based service that provides a set of intelligent applications that seamlessly combine the capabilities of CRM business software and ERP systems. In the OvalEdge application, the Microsoft Dynamics 365 connector allows the users to crawl the CRM objects and profile the sample data.

      Improvements

      Global Search

      1. In Global Search | All Tabs - A Certified objects filter is provided in the All Tabs left search panel to view only certified objects in the search results.
      2. The Global Search > Service Desk Tab > the search results now display the status associated with the service tickets like Resolved, Reopened, Pending, etc.
      3. The Global Search results allow users to download the Business Description and Technical Description.
      4. In Global Search, an Add to Projects icon is enabled for Tables/Table Columns/File/File Columns data objects to add the specific data object to the default project.

      Known Limitation: Search results obtained through the Advanced Search option and then filtered cannot be bookmarked. Whenever a bookmark is saved for the above, the Advanced Search values are ignored. 

      In the next release, this limitation will be addressed.

      My Access Cart

      1. In the My Access Cart | Tables, the Type column is enhanced to filter the table type using Table, View, API, and Materialized View options.
      2. In My Access Cart > In every object type tab > Add Table or Add File, clickable buttons are provided in the bottom left to add data objects to the Access Cart.

      Masking Policy

      We improved the existing masking policy so that whenever Column-level Security Masking policies conflict with Term Masking policies, the Term Masking Policy takes precedence. Also, once Term level masking is applied, the security level masking enabled on the data objects gets disabled. 

      Below is a list of different Masking Policy scenarios that change depending on the Term status (Initial Draft,  Published, Draft).

      1. Column level Masking enabled and Term in Draft /Published state without masking - Masking Policy at the column level is enabled.
      2. Column level Masking enabled, and Term in initial draft status, i.e. (a newly created awaiting to be published) - Applies column-level Masking policy.
      3. Column level Masking enabled and Term is in Published status - Masking Policy at Term level is applied to the data object.
      4. Column level Masking enabled and Term moved to Draft status - Masking Policy at Term level will be intact even if the Term is transferred to the Draft state.
      5. Column level Masking enabled and Term deleted, or Masking Policy removed from the Term - Data will not be masked, and the column security masking is also removed.
      6. Suppose the existing Term is replaced with another Term that is in Draft status. In that case, It removes all the masking policies applied to the data object, even if any column security exists.

      Data Catalog

      1. In the Data Catalog | Synonyms, we can capture the Oracle synonyms from the 6.0 version.
      2. In the Data Catalog | Catalog Query, there was no provision to execute the Catalog Query without saving the modified query. But now, it is improvised where the user can change the query and test the result without affecting the original query.
      3. In the Data Catalog | Tables, the “Download” functionality is enhanced to download multiple tables' data and column details. Here, when the user selects the Download option, a pop window is displayed to choose whether the selected table data needs to be downloaded with the column details or not.
      4. In the Data Catalog | Table Column, the Quality Index of a data object (Table column) is displayed for query level (Example: Boolean/Stats/Failed). However, the Quality Index score is displayed at the column level.
      5. In the Data Catalog | Lineage | Code/Query, the new Search bar is implemented to search for the specific SQL query.
      6. For the ETL Connection, now the images available in the ETL associated database instance are fetched and displayed in the Data Catalog > Data Objects > Data tab.
      7. In the Data Catalog | Reports, a new reporting framework is developed for the following:
        1. To generate user access reports for each data object like Table, Table Columns, File, File Columns, Report, and Report Columns.
        2. To generate a report to get the Term count associated with each domain.
        3. To generate a report to check the status of all the tickets a user raised.
        4. To generate a report to get the statistics on Terms with their respective published and draft statuses.
        5. To generate a self-service analytics dashboard for the objects associated with a project where the bar graph will display the number of objects associated with the projects available in the OvalEdge application.

      Business Glossary

      1. In the Business Glossary, specific Term details can now be downloaded using the Nine Dots through the Download Description option.
      2. In the Business Glossary page, under the customize view option, the code custom field has been added with a filter.
      3. In the Business Glossary Users can download bulk terms from the List view using the Download Terms option in Nine Dots.
      4. In the Business Glossary, the “Download Term Details” functionality is enhanced to download all the additional fields' data, and if the additional fields' data are not existing, then the system will not display the additional fields' columns in the downloaded excel sheet.
      5. In the Business Glossary, the “Download Term Details” functionality is enhanced to download the terms with the configured custom fields. If the custom fields' data are not existing, then the system will not display the custom fields column in the downloaded excel sheet.
      6. If a new Term is created (suggest a Term), and when the Term completes the Approval WorkFlow and gets Published then the creator of that particular Term gets notified.

      File Manager

      In File Manager, the Data Lake page is improvised by including the Last Modified Date and Time, Created By columns, and at the Folder Level the new options Connection Name, File Type, Created By, and Last Modified Date and Time are added to analyze the data more efficiently.

      Projects

      1. In the Projects,  All Projects are enhanced with Board View (Kanban) and List View. In Board View the projects are listed under four predefined project statuses New, To Do, In Progress, and Completed
        Note: My Project Tasks are made into a separate sub-module and moved to My Resources.
      2. In the Projects, a user can now invite individual users and users based on roles to projects.
      3. In Projects, now the Assignee, Status, Object Type, Object Name, Project Notes, Created By, and Last Updated By are enabled with filters to refine the search.

      Dashboard

      The Data Quality Dashboard is enhanced to display the graph for different dimensions, and the summary of the data quality index scores for each data object like Schema, Table, and Table Columns at their dimensions levels in the user-defined format where the users can select the hierarchy of objects to populate the dashboard for different dimensions.

      Governance Catalog

      1. The DQR is enhanced to show the first 50 values of the failed data asset on the Execution Result page and by default, the Ticket option is enabled on the summary page. Here, the existing DQR summary page is also enhanced to display the following information when a DQR is created and executed.
        1. Execution Information: DQR execution details such as Last Status, Last Run Time, and Rule Object.
        2. Functions: The function applied on a rule with success range value and its dimension are displayed.
        3. Violation Message: A message is displayed when the rule is violated.
      2. In the Service Desk | Description box, the link of the DQR | Execution result page for that particular object is provided for the user to know why a DQR ticket has been created so that the user can simply open a link and click on the View icon to view failed rows and take necessary analysis correction steps.
      3. The Service Desk tickets in all the tabs (Waiting My Approval, My Created Requests, All Requests) are downloadable. A download icon is enabled in the bottom right corner to download Service Ticket(s).
      4. In the Service Desk, if the raised SR is rejected, the user who raised the Service Request can ‘Reopen’ the service request ticket and edit and submit the request.
      5. In the Service Desk, an Object Name column is newly added to enter the Object name details.
      6. In the Service Desk | Service Templates, The data objects on which Access permissions are needed are grouped based on Service Request Template Id, Connection type, Object Type, Object Id,  and Configured approvers (Owner, Steward, Custodian, Governance Roles).
      7. In the Service Desk, the Service Templates related to the Snowflake, S3, and Oracle connection are enhanced to clone the existing templates and customize them as per the user requirements
      8. In the Service Desk | Access Requests, users can now select multiple objects and raise Service Requests for multiple objects in one user flow and will be able to track the approvals for the same holistically.
      9. In Service Desk, the download icon is provided in the bottom right corner in all the Waiting My Approval, My Created Requests, and All Requests tabs to download the service requests into XLXS format.

      My Resources

      1. In the OvalEdge application, the profile details of the user are now editable for the users logged in through the SAML SSO Authentication process.
      2. In the My Watch List, the Table Columns tab is implemented to view the watch-listed Table columns.

      Advanced Tools

      1. In the Load Metadata from Files, Table Column Template, the Column Data Type should not be a mandatory field.
      2. In the Load Metadata from Files | DQR object Type, an additional Dimension column is added to the Template to apply an accurate rule to the Query.
      3. In the Load Metadata from Files, while uploading a file from the Load Metadata page and in the File Manager, for the NFS Connector, while uploading a File or Folder to the NFS path, now the system verifies whether the File Scanner is enabled or not. 
      4. If enabled, the file is sent to the scan service to identify whether any malicious content exists in the file. If malicious content exists, the user is sent an email notification related to that file, and the same notification is displayed in the Alert Messages.
      5. In the Load Metadata from Files, the Business Glossary Template is enhanced to display the excel sheet based on the domain names. Each domain name sheet contains the additional fields data and custom fields data, data classification, and business description details for the term.
      6. In the Load Metadata from Files | DQR, Additional Associations are displayed when a user uploads a file for Custom DQR through Custom SQL.
      7. In the Load Metadata from Files, a Report ID column is added to the Reports data object template to add Report Id.
      8. In Temp Lineage Correction, a new pop-up displays the Temp Schema objects dropdown to select a schema. Additionally, a Run Now button is enabled in the top right to run the recommendation job.
      9. The new API named "saveConnection" is being developed for creating a new connection. As a result, when the "saveConnection" API is executed, the "getConnectionParm" API is invoked, where the user must provide the mandatory details needed to create a new connection.
      10. The new File API is developed to catalog and profile the files from the client database.
      11. The new Tag API is developed to delete unwanted tags in the OvalEdge application.
      12. The new APIs related to the Projects are developed to fetch the following details:
        1. Get Project API: This API is used to fetch the details of the project with the Project ID, Project Name, Description, Created By, Created Date, Default Project, and Last Modified Date information.
        2. GetProjectByUsingName API: This API is used to fetch the details of the project with the Project Name.
        3. Delete Project API: This API is used to provide the ability for the super administrator to delete the projects. Here the super administrator can delete the projects created by anyone, and the remaining users can delete their individual projects.
        4. Adding a Project: This API is used to build the new project.
        5. GetAssociatedObjectByProjectName: This API is used to fetch the data asset details associated with a particular project.
      13. The Message API is enhanced to get the list of messages based on the object types oeschema, oetable, oefile, oechart from the client database.
      14. In the Impact Analysis Summary, the Object Type column now displays whether the reports added to the impact analysis are Published reports or Embedded reports.

      Administration

      1. In the Crawler, while adding the Athena Connector in the Output S3 Folder path, the system was not validating whether the path was valid. However, after adding the connection while crawling the database, it was throwing an error for the invalid path. Now it is enhanced to validate the path while adding the connection itself.
      2. In the Crawler | Setting, a new Lineage tab is implemented for the Report and ETL connectors to configure multiple dialects (by Selecting Source Server Type for lineage) and connections priority lists to pick the tables to build lineage.
      3. In the Crawler | RDAM for the Databricks(Deltalake), the Databricks have a sub-group concept, i.e., a group can also have sub-groups attached, OvalEdge implemented a new table called “remoterolesrelation” to save this parent-child relationship in a separate table for now. When crawling, all remote role relations to the OvalEdge, identifying removed relations in the latest crawl and deleting such.  So, OvalEdge implemented a new table called “remoterolesrelation” with the following columns
        • Table → remoterolesrelation
        • Remoterolesrelationid
        • Remoteroleid
        • Parentroleid
        • Connectioninfoid
        • Createdate
        • Createdby
      4. In the Job Workflow, when a user schedules a job, the system will now trigger a notification mail to the setup users stating the job workflow results along with an attachment in .xlsx format.
      5. In the Security | Approval Workflow, now the user can select a Team to approve the request.
      6. In the Security | Approval Workflow, the approval workflow is enhanced to support the selection of team stewardship from the dropdown list on the Approval workflow page.  
      7. For example, a "Service Desk" team handles customer service desk requests. From the approval workflow page, team stewardship can be selected as '#servicedesk.'
      8. In the Security | Approval Workflow, the existing Configure Approvers is improvised to include the Team for the Approval Workflow.
      9. In the Security, the enable/disable option is enhanced to hide the Administration, Dashboard, and Certification Policy modules in the OvalEdge application for Business Users.
      10. In the Security | Domains, the Configure Classifications are improvised to add 20 Classifications under a specific Domain.
      11. In the Security > Folders, while creating a new folder, the Apply Security section provides additional security by setting User/Role Permission, Meta Permission, and Data Permissions for the folder. Additionally, the user can add additional roles with the help of the Add Role Access option.
      12. In the Configuration | Lineage, a new key “is.correctquery.required” is configured where you can change the value of ‘is.correctquery.required’ to false to prevent the Query sheet from being displayed in the Advanced Tools > Build Auto Lineage module.
      13. In the Configurations, the system template for External Access Requests are now made editable by setting up configurations. Here, a new configuration is added to Configurations > Others > servicedesk.system.template.edit.enable, and when set to True the system helps you to add more fields to the External Access Request template fields.
      14. In Manage Servicedesk | Add New Service Request Templates | Custom Template Fields, while creating a custom template, the following field types can be selected from the dropdown list.
        • OvalEdge License Types - Business Users and Author & Analytical Users OvalEdge Users and Roles
        • Users & Roles - Both OvalEdge and Remote Users and Roles
      15. In the Manage Servicedesk, the Access Service Request Templates are improvised by adding the Requested For field to help the users raise service requests on behalf of other users.
      16. In the Manage Servicedesk, while creating the Custom Service Request template, the Field Visibility option is provided for the different field types that are selected for creating the template with three levels:
        • Creation - It enables users to view the field while raising a service request in the template.
        • Approval - It enables the configured approvers to view the field at the Approval Desk.
        • Creation & Approval - It is used to display the field in the template while raising a service request as well as at the approval desk.

      Advanced Jobs

      1. The following advanced jobs have been added and improved for various user actions:
        1. Create Team Using Roles - It is used to create teams with the names of roles available in the OvalEdge Application. Users with particular roles will be assigned to their respective Teams.
          With this feature, users can select roles in the collaboration messages using the Teams option in the dropdown to notify the users included in that Team.
        2. Advance Job for Remove staging temp tables starts with # for a Connection - It is used to eliminate temp tables that start with the # Symbol.  Advance Job for Remove staging temp tables starts with that particular #symbol for a Connection ID. 
          Note: Only the Staging table name must be eliminated from the lineage if it is linked to templineage.
        3. Same Table name and Column name lineage - It is used to build the lineage between two table names by not considering the suffix and prefix of the table names.
        4. Same Column name lineage - It is used to build the lineage between two connections by considering the same column names.
        5. Transactional Data Changes - It is used to display the table columns, in which data is added, deleted, or modified based on the profiling results.
        6. Load OE Reports - It is used to get a report of the number of objects associated with a particular Term.
        7. MRN API Execution - It is used to execute the API and parse the JSON to identify the source columns used for creating the table and columns in the Lineage Manual Building.
        8. Load JdEdwards manually - It is used to update the metadata for the data objects available in the JdEdward tool.
        9. Load Schema For Any Connection - It is used to download specific table information in JSON format.
        10. CDB Request Job: It is used to capture the DQ Results of objects.
        11. Load Users into OvalEdge: It is used to load users from an external .csv file into OvalEdge.
        12. QlikSense Labels: It is used to generate the labels for the tables available in the QlikSense connector. The labels will be generated in the array format separated by a comma.
        13. TableauRemoteDataUpdate: It is used to update Tableau Remote Data with configurations or modifications (Workbook Name and Tags) made in the OvalEdge application.
        14. Load Dataid Details - The existing job is improvised to run when either of the attributes like Impacted Table, Impacted Column, or Impact Notes-1 are provided.
        15. Upload Business Glossary and associate Terms CLS - The existing job is enhanced to include the Category and Custom field columns in the template.
        16. Catalog Creator: The existing job is improvised to create tags for report groups.

      Connectors

      1. Credential Manager functionality associated with DB2, Snowflake, and Salesforce connections is improvised by allowing users to choose between Database, HashiCrop, or AWS Secret Manager for connection validation.
        • When the Credential Manager is selected as the Database, then the Username and Password associated with the Database must be entered.
        • When the Credential Manager is selected as the HashiCrop, then the Key Value and Password generated in the HashiCrop server must be entered.
        • When the Credential Manager is selected as the AWS Secret Manager, then the Key Value and Password generated in the AWS Secret Manager must be entered.
      2. The  API connector is enhanced to pull the metadata from the existing YAML files and helps the users to crawl and catalog the APIs. To connect to the API Connector, the user must place all the required YAML (API) files in one folder where all the OvalEdge-related configuration files exist.
      3. The Denodo connector is enhanced to build the lineage for the tables created using custom wrapper objects. Here the custom wrappers are associated with a custom data source where a class (JAVA) implementing the wrappers of the custom wrapper type must be specified.
      4. The Vertica connector is enhanced with Build Auto Lineage functionality, where the lineage for multiple data objects is built automatically.
      5. For the SAP BO connector, the date custom fields are modified from Text to Date format.
      6. The Kafka connector is improvised to download the Kafka schema in JSON format.
      7. In the PowerBI connector, a login credential has been implemented to view the reports.
      8. In the Azure Cosmos DB connector, the managed connection pop window is improvised to establish a connection using the connection string.
      9. The SSRS connector is enhanced to crawl the SSRS Reports existing in SSRS Database, which is on the Linux/Unix environment, through the Windows authentication process.
      10. In the HDFS connector, API is required to catalog and profiles from given text files instead of Advanced Jobs.
      11. In the Tableau connector, the Crawler setting is provided with the Delta Crawl option. Reports (by default enabled), as Delta Crawl. This crawling will get report/report columns only when the remote report is changed (It will compare the remote report date with the local date). If the Delta crawl option is, Enabled - It will crawl as a Delta crawl, and if Disabled - It will crawl as a full crawl.
      12. The AWS Glue connector now provides support for crawling Jobs, Workflow, Triggers, Crawlers, and lineage building for the above entities. The AWS Glue entities are created as datasets such as Jobs, Workflows, Crawlers, and Triggers. It works on a job's script and builds the lineage for it whereas the crawlers, triggers, and workflows would extract the information of entities involved and Build the associations accordingly.
      13. The SAP HANA connector is enhanced with the Incremental Crawl option to crawl the new objects (Table, Columns) without deactivating the existing data assets.
      14. In the Snowflake connector, as an alternative to basic authentication (such as username and password), Snowflake's connection security now supports Key Pair Authentication (RSA). To use this authentication method, you must have a minimum of 2048-bit RSA key pair generated by using OpenSSL.
        • In the case of a public key, the connection can be established by simply entering the Key details in the Private Key field.
        • In the case of a private key (encrypted), the connection can be established by entering the Key details and the parsed value associated with the Key in the Private Key Password field.
      15. The Snowflake connector RDAM is improvised to set a default OE Role for the same users who are existing in both OvalEdge and Snowflake. Here, the Policy management is also enhanced to include Column Level and Table Level Policy schema. The policy applied on the column level is known as the Masking policy and the policy applied at the table level is called the Row Access Policy. 
        The Masking Policy is enhanced with an additional masking scheme:
        1. Mask Numbers
        2. Mask with Default Time
        3. Mask with Default Date
        4. Masked Column in Remote
        Row Access Policy
        The concept of row-level policy is a masking scheme for the row level of the table. For this, you may navigate to Security > Database. Click on the Edit icon in the column of "available User or Roles". It will navigate to the page of Adding row access policy.  It will display the list of Policy names and their SQL statement.
      16. The SSIS connector now displays all associates at all SQL levels and deletes all temporary lineages when a lineage is rebuilt.

      Copyright © 2022, OvalEdge LLC, Peachtree Corners GA USA