How to Schedule Connectors

In this article, you’ll learn how to set up job workflows for scheduling crawls in OvalEdge.

Introduction

Scheduling crawls is a way of automating the crawling process at the most convenient time for your organization. There are various ways to schedule crawls in OvalEdge.

Scheduling Recrawls

Crawling can be scheduled for specific times. Go to the ‘Administration’ tab in the left panel menu and select ‘Connectors’ from the dropdown. Select the connection you wish to schedule in the ‘Crawler’ screen. Next, click on the ‘Crawl/profile’ button on the far right of the screen.

Under ‘Schema Name,’ select the schemas you wish to crawl or crawl/profile. In the crawler options, you have the ability to schedule a crawl, a crawl and profile, just a profile, or update an existing profile. Below, select the ‘Schedule’ option and set up the scheduling frequency.

Click ‘Schedule,’ and a pop-up will appear stating that the operation was successful.

If you need to see the job, navigate to the ‘Administration’ tab and select ‘Schedule’ from the drop-down. This area is organized by name, but you can sort the ‘Changed On’ column to bring the job you just created to the top.

Scheduling Recrawls through Job Workflows

To automate this process to run at scheduled intervals, navigate to the ‘Administration’ tab and select ‘Job Workflow’ from the drop-down. Select the ‘add’ icon in the Job Workflow screen to add a new job workflow.

On the next page, you'll need to select the type of job you wish to schedule from the drop-down menu. You’ll notice the same selections we saw from ‘crawler options’ exist in the ‘Select Type’ dropdown.

For this example, let's use a ‘Crawler.’ All of the Crawler connection names are visible. Your workflow is created when you drag and drop the various objects and their associated action. Upon completion of one, the next job workflow action will commence. If you start more than one workflow stream, they will all begin at the scheduled time. This is how you create a multi-thread workflow.

Once you have added all the workflows, give the job workflow a name and click save. I have made my selections, and now I will save the job using the save icon. To schedule the workflow, select the ‘Schedule’ icon and make the selections to set the job’s cadence. Click the ‘Schedule’ button to save it.

Add Experts/Observers by clicking on the icon directly left of the nine dots icon and save the selections by choosing ‘add experts.’ When scheduling jobs, it is important to add individuals who will take action when notified of any job failures. We recommend having more than one person. These individuals will be notified through their profile settings via email.

To select the notification types you wish your experts to receive, go to the nine dots icon and select ‘notifications.’ You can choose notifications for errors, success, and partial success. Save any changes. Check with your experts to see which notifications they would like to receive.

Additional resources

OvalEdge Academy

Connector scheduling: