Data Orchestration

This article details the data orchestration process, how to modify the configuration of data orchestration, how to launch data orchestration manually, and how to interpret the details of a data orchestration.

What is data orchestration?

Data orchestration is a sequence of flows that performs data integration, consolidation, computation, and assigned automations to get your data ready for searching and graphing. Data orchestration collects, organizes, and standardizes your data from all sources to provide a consolidated view of the data imported into the Brinqa Platform. While data integration maps the data retrieved from your connectors to data models, data orchestration generates metrics and calculates risk scores on your data to help you assess and prioritize your risks. Data orchestration and data integration work together to get your data ready for searching and visualization.

As shown in Figure 1, data integration retrieves the source data models (SDM) from the connector and then maps them to the unified data models (UDM). After consolidation, computation and automation run to generate risk scores and metrics on your assets and findings, as well as update your existing exception requests, reports, and tickets.

Figure 1. The data orchestration process.

Navigate to Administration on the upper-right corner and under Data, click Orchestration to see the run history of your data orchestrations, launch your data orchestration manually, or modify the data orchestration settings.

Change the data orchestration settings

To modify the schedule of data orchestration, select an action in the event of data integration failure, or notify users of successful or failed orchestrations, follow these steps:

1. Navigate to Administration > Data > Orchestration.

2. Click Edit and modify the following fields:

   • Schedule: By default, the data orchestration runs once per day. You can change how often the data orchestration runs, what time and day it runs, and the time zone.

   • Failure handling: Indicate what action to take upon a failed data integration or automation in the data orchestration process. By default, data orchestration continues to run and collect data from data integrations or automations when one of them fails. You can also stop the data orchestration altogether. Click the drop-down and select Stop orchestration to do so.

   • (Optional) Enable notifications. If needed, select Enable notifications and specify the roles or users to notify when the data orchestration runs successfully, fails, or runs successfuly when a data integration or automation was skipped due to a failure.

3. Click Update.

Launch a data orchestration

Data orchestration runs daily at the scheduled time of your choice. To manually launch data orchestration, follow these steps:

1. Navigate to Administration > Data > Orchestration.

2. Click Launch.

Data orchestration may take a considerable amount of time and system resources to process depending on how many integrations you have configured and the scope of those integrations. If a data orchestration is in progress, you must cancel it before launching a new run.

Data orchestration details and logs

To see the logs or details of each data orchestration run, navigate to Administration > Data > Orchestration, hold your pointer over the data orchestration entry and click View logs or Details.

The Details page contains information on the steps and flows involved in the data orchestration.

Click the caret menu to expand the entry and view details. If a flow encounters an error and fails, you can see which step has failed.

The View logs option directs you to the Logs page containing logs generated for that specific data orchestration run. See Application Event Logs for additional information.

note

Brinqa removes both successful and failed flows after 28 days.