Data Orchestration

This article details the data orchestration process, how to modify the configuration of data orchestration, how to launch data orchestration manually, and how to interpret the details of a data orchestration.

What is data orchestration?

Data orchestration is a sequence of flows that performs data integration, lifecycle management, consolidation, computation, and assigned automations to get your data ready for searching and graphing. Data orchestration collects, organizes, and standardizes your data from all sources to provide a consolidated view of the data imported into the Brinqa Platform. While data integration maps the data retrieved from your connectors to data models, data orchestration generates metrics and calculates risk scores on your data to help you assess and prioritize your risks. Data orchestration and data integration work together to get your data ready for searching and visualization.

As shown in Figure 1, data integration creates the source data models (SDM) and then maps them to the unified data models (UDM). After consolidation, computation runs to generate risk scores and metrics on your assets and findings, as well as update your existing remediation requests, reports, and tickets. Brinqa Platform then takes a snapshot of the processed data and stores it in the Data Warehouse. Additionally, flexible workflows called automations are set up to run across different stages.

data orchestration diagram

Figure 1. The data orchestration process.

The following table outlines the various stages in data orchestration:

Stage	Description
Data Integration	Imports data from various sources through a streamlined and automated method.
Data Lifecycle	Enables you to designate data as inactive if it hasn't been updated within a set timeframe. Inactive data can then be scheduled for automatic purging, ensuring that only relevant and current data is retained in the Brinqa Platform.
Data Consolidation	Combines data from multiple sources into a unified format, eliminating duplicates and enriching the data.
Data Computation	Updates data by performing certain actions such as executing scripts, launching syncs, or sending notifications.
Data Storage	Processes Analytics Source and takes a snapshot of the unified data to stores in Data Warehouse.

Change the data orchestration settings

To modify the schedule of data orchestration, select an action in the event of data integration failure, or notify users of successful or failed orchestrations, follow these steps:

Navigate to Administration > Data > Orchestration.
Click Edit and modify the following fields:
- Schedule: By default, the data orchestration runs once per day. You can change how often the data orchestration runs, what time and day it runs, and the time zone.
- Failure handling: Indicate what action to take upon a failed data integration or automation in the data orchestration process. By default, data orchestration continues to run and collect data from data integrations or automations when one of them fails. You can also stop the data orchestration altogether. Click the drop-down and select Stop orchestration to do so.
- (Optional) Enable notifications. If needed, select Enable notifications and specify the roles or users to notify when the data orchestration runs successfully, fails, or runs successfully when a data integration or automation was skipped due to a failure.
Click Update.

Launch a data orchestration

Data orchestration runs daily at the scheduled time of your choice. To manually launch data orchestration, follow these steps:

Navigate to Administration > Data > Orchestration.
Click Launch.

Data orchestration may take a considerable amount of time and system resources to process depending on how many integrations you have configured and the scope of those integrations. If a data orchestration is in progress, you must cancel it before launching a new run.