Import Your Data through Brinqa Connect

This tutorial guides you through the steps of importing data into the Brinqa Platform using Brinqa Connect.

Prerequisites

Before using Brinqa Connect, you must have these prerequisites in place:

1. A running instance of Brinqa Connect.

   Cloud-based customers can access Brinqa Connect directly, whereas on-premises customers must download and install it first. After Brinqa Connect is configured, you can access it by entering https://<brinqa-platform-url>/connect/ into your web browser. Replace <brinqa-platform-url> with the URL to your Brinqa Platform.

2. A Brinqa Connect user with the appropriate role assigned.

   Brinqa Connect provides three different roles to protect your data: admin, egress, and ingress.

   Role	Permissions
   ADMIN	Users can read or write data, and create configuration.
   EGRESS	Users can read data.
   INGRESS	Users can write data.

   For the purpose of this tutorial, you need a user with either the ADMIN or INGRESS role.

3. Brinqa Connect user credentials.

   Brinqa Connect uses distinct credentials for authentication, separate from the Brinqa Platform credentials. To obtain your Brinqa Connect credentials, please contact Brinqa Support or the Brinqa team overseeing your deployment. Customers with on-premises installations can generate the credentials during the user creation process.

Import your data

Using Brinqa Connect, You can ingest your unstructured data into the Brinqa Platform with these steps:

• Authenticate with Brinqa Connect

• Push your data into Brinqa Connect

• Create an integration to import your data

• Add attribute mapping for your data

Authenticate with Brinqa Connect

Brinqa Connect supports the OpenAPI schema and user interface (UI) for a developer friendly experience. With support for OpenAPI, developers can generate a client in their desired language. The sandbox UI is available at the relative URL /connect, which provides a simple interface to try the API endpoints. Alternatively, you can interact with the endpoints using any API platform you prefer. For this tutorial, we'll use the Brinqa Connect sandbox.

Before using any of the Brinqa Connect API endpoints, you must provide authentication. To do so, follow these steps:

1. Enter https://<brinqa-platform-url>/connect/ into your web browser.

   If Brinqa Connect is running, the page displays the Brinqa Connect API with the Authentication setting:

   

2. Enter the username and password of the Brinqa Connect user gathered in prerequisites, and click SET.

3. After successful authentication, you can explore and test the API endpoints using the TRY buttons.

Push your data into Brinqa Connect

The Brinqa Connect API supports different endpoints for sending data, either synchronously or asynchronously. In this tutorial, we'll use the POST /api/v1/data/{name} endpoint. To do so, follow these steps:

1. In the Brinqa Connect sandbox, click Send semi-structured data.

   

2. In the name field, type "host".

   The 'name' parameter is a string that describes the nature of your data. In this case, we are going to create a host.

3. In the namespace field, type "test".

   The 'namespace' parameter is a string that separates your data into groups. If not set, the default value is 'global'.

   tip

   Make a note of these two parameters. You'll need to use them later when creating an integration.

4. In the EXAMPLE field, paste in this sample code and click TRY:

   {
      "attributes": {
        "os": "Microsoft Windows 11 Enterprise Build 22000",
        "display_name": "Sample Server for Brinqa Connect",
        "date_created": "2024-01-01 20:30:00",
        "hostname": "sampleServer",
        "fqdn": "sampleServer.myDomain.com",
        "tags": [
          "Windows",
          "Server"
        ],
        "external": false,
        "status": "Online"
     }
   } 

   This request creates a Windows Server host with attributes such as display_name, fqdn, hostname, and tags.

5. Brinqa Connect creates the record and sends the following response:

   

   If needed, you can click the CURL tab and copy the command to use elsewhere.

Notice that there is a date in the example above, "date_created": "2024-01-01 20:30:00". When your data contain Date attributes, it's crucial to define their formats. Otherwise, they are treated as String attributes. The endpoint to use in this context is POST /api/v1/config. To define a Date format, follow these steps:

1. Click Create type definition.

2. In the EXAMPLE field, paste in this sample code and click TRY.

   Please pay attention to the dateFieldConfig section, where you can specify type, formats, and timezones for each Date attribute.

   You can use title to designate a friendly name for representing your data in the Brinqa UI.

   {
     "id": {
        "namespace": "test",
        "name": "host"
     },
     "title": "Host",
     "description": "This configuration is for the test host in the Brinqa Connect import tutorial.",
     "uidFields": [
        "fqdn"
     ],
     "dateFieldConfig": {
        "date_created": {
           "type": "DATE_TIME",
           "formats": ["yyyy-MM-dd HH:mm:ss"],
           "timezones": ["UTC"]
        }
     }
   }
   

3. You should receive a Response Status: 201 with no message.

Create an integration to import your data

After successfully ingesting data into Brinqa Connect, the next step is to bring that data into the Brinqa Platform, which you can achieve by creating a data integration with the Brinqa Connect connector. To do so, follow these steps:

1. Log in to your Brinqa Platform as a system administrator.

2. Navigate to Integrations > Sources and click Create.

3. Fill out the general information:

   • Title: Enter a title for this integration, e.g.: "My data through Brinqa Connect".

   • Connector: Select Brinqa Connect from the drop-down.

   • Server: Keep as is, which is 'Local server'.

   • Description: If desired, provide a description for the integration.

4. Complete the required connection settings:

   • Connect URL: Enter the URL for your Brinqa Connect installation, e.g.: https://<your-brinqa-platform-url>/connect/.

   • Username and Password: Enter the credentials of the Brinqa Connect user gathered in prerequisites.

   • Page size: Keep as is, which gets up to 1000 records per API request.

   • Namespace: Enter the 'namespace' parameter specified when pushing data into Brinqa Connect. In this case, enter "test".

5. Click Next to test the connection and save the configuration.

   The connector tries to access the Brinqa Connect API. If the connection is successful, more options display.

6. In Types, the title that you specify when pushing data into Brinqa Connect is listed. Click the checkbox next to 'Host' to select it, which selects all the attributes for import.

   

7. Keep the remaining settings as they are and click Create.

   The page reloads and you should see your integration listed. If you do not see it, use Search to locate it.

8. Click the title of your integration to open its details. Click Sync to manually start the data integration.

   

   Copy the value of Source data for usage in the next step. Observe that it consists of the title of your integration followed by the type you've selected.

9. After the sync has completed, a log entry displays showing the number of imported records.

   

Add attribute mapping for your data

Unlike the other Integration+ connectors, the Brinqa Connect connector doesn't know how to map your unstructured data to Brinqa's unified data models (UDM). Therefore, you must specify the mappings yourself. To do so, follow these steps:

1. Navigate to Administration > Data > Models.

2. Navigate to the data model to which you want to map your data. For this tutorial, search for Host and click it.

3. Click Consolidation.

4. Click the Sources field and type the initial characters of the source data name from the previous step, that is "My Data Through Brinqa Connect host". Select the full name as it appears.

   tip

   The name is case-sensitive.

5. Go to the Identifiers section and select an identifier for your data. An identifier is a piece of information that uniquely identifies your data. See data consolidation for more information.

   1. Click HOSTNAME, and then click the + sign at the last row.

   2. In the new row, click Data model and type "My". Click the full name to select it.

   3. Click Attribute Name and type "hostname". Click it to select the attribute.

      tip

      Attribute Name refers to the attributes you specify when pushing data into Brinqa Connect. These attributes are case-sensitive.

   

6. Go to the Attribute mappings section to add your mappings.

   1. Click DNS names, and then click the + sign at the last row.

      A new row is added with the Data model field set to your source data name.

   2. Click Source attribute and type "fqdn". Click it to select the attribute.

      tip

      Source attributes are the attributes you specify when pushing data into Brinqa Connect. The attributes are case-sensitive.

   3. Repeat the steps to add more mappings. For this tutorial, add the following mappings:

      • Hostnames ~ hostname
      • First seen ~ date_created
      • Name ~ display_name
      • Operating system ~ os
      • Tags ~ tags

   4. Go to the bottom of the page and click Update.

7. You can wait for the daily orchestration to process your data, or run a sub-flow to see the result right away. For the latter, click Flows in the left menu.

8. Click Host consolidation flow, click Launch, and then click Launch again.

   This process runs data consolidation and incorporates your data with the Host UDM.

9. After the flow has completed, navigate to Inventory > Assets > Hosts.

10. Search for your data using the name or any of the attributes that you've mapped. In this case, type 'Sample' in the Search field and press Enter.

Click the name to verify the details of your host.