Crawler settings comprises of the following:
- Crawler Options
- Crawler Rules
Crawler Options
If the crawl settings option is selected, Admin gets to choose from the following:
- Users & Roles: Discovery of users and their roles from the source system and importing those into OvalEdge. Currently, this is enabled only for Oracle connections.
- Procedures, Functions & Views Source Code: The crawling will discover the procedures, functions, views source code within the schemas selected, and bring them into OvalEdge.
- Tables, Views, and Columns: Discovery of tables, views, and Columns and bringing them into OvalEdge. This is the Default option for crawling.
- Relationship: Determining relationships between the tables of selected schema and bringing this information into OvalEdge.
- Permissions: The crawling will discover the users and their roles with their type of remote access permission provided to the users of OvalEdge.
- Additional Crawl: While accessing remote tables, in case, if the user does not have permissions, an additional crawl option is used. Using alternate query metadata of those tables will be fetched.
For reports, crawler options are:
- Reports: This crawling will discover all the reports and bring them into OvalEdge.
- Report columns: This crawling will discover all the report columns existing in the reports and bring them into OvalEdge.
- Delta Crawl: This crawling will get report/report columns only when the remote report is changed (i.e. we will compare remote report date with local date).
-
- Enabled - It will crawl as Delta crawl
- Disabled - It will crawl as full crawl.
Note: In full crawl, we will get all reports / report columns present in the remote data irrespective of any changes in remote.
Crawler Rules
There are also Crawler Rules that can be configured to provide more refined results through regular expressions written against the source system to search for matching character sequences against patterns specified by the same in the code. These options include:
- Include Table Regex
- Exclude Table Regex
- Include Column Regex
- Exclude Column Regex
- Include Procedure & Function Regex
- Exclude Procedure & Function Regex
When setting up the regex rules the user will be able to write rules that will either include and/or exclude tables, views, columns, procedures, and functions that start with, end with, or have middle characters as defined in the rule.
Note: Non-Regex include/exclude it is supported for startswith, endswith using %.
Example: If a user only wants schemas that start with "a" then the user needs to write "a(.*)" in the Include Schema Regex field. After that they need to run the crawl job, then the results for the schemas that start only with character "a” are displayed.
Example: If a user only wants schemas that ends with "e" then the user needs to write "(.*)e" in the Include Schema Regex field. After that they need to run the crawl job, then the results for the schemas that end only with character "e” are displayed.
Example: If a user only wants to exclude schemas that start with "a" then the user needs to write "a(.*)" in the Exclude Schema Regex field. After that they need to run the crawl job, then the results for the schemas without a character "a” at the starting of the schema name are displayed.
Example: If a user only wants to exclude schemas that ends with "e" then the user needs to write "(.*)e" in the Exclude Schema Regex field. After that they need to run the crawl job, then the results for the schemas without a character "e” at the end of the schema name are displayed.
NOTE: Above examples are on the table level, functions & procedures, and columns are executed the same way.