CSV Connector

File

Overview

The CSV Connector ingests data from delimited flat files (CSV, TSV, or any character-delimited format) that are placed in a configured directory on the file system. Rather than calling a remote API, the connector reads files that match a configurable path pattern, parses each row into a record, and emits the records to Brinqa as connector objects.

The connector is schema-flexible: the set of attributes for each record is derived dynamically from the header row of the CSV file. Columns can be typed (string, integer, long, double, boolean, or date/time), treated as multi-value, or merged across multiple rows. The target data model that the parsed records are mapped to is itself configurable (it defaults to Record), so the same connector can feed Host, Vulnerability, or any other Brinqa data model depending on the source file's contents.

Files are discovered, parsed, and then optionally renamed/moved once processed. Incremental syncs only reprocess files whose last-modified timestamp is newer than the last successful run.

Data retrieved from CSV Connector

Connector Object	Required	Maps to Data Model
Record (configurable target)	Yes	Record (configurable target)

Model relationships

note

For detailed steps on how to view the data retrieved from CSV Connector in the Brinqa Platform, see How to view your data.

Setup

Connection settings

When setting up a data integration, select CSV Connector from the Connector dropdown and provide the following:

Setting	Required	Default	Description
Data directory	Yes	(none)	The fully qualified path to data directory. Wildcards are allowed in the path, e.g. /feed/vendors/vendor_.csv or /feed/mobile/mobile_device.*
Target	Yes	Record	The target data model name to be map the provided data, e.g. Host, Vulnerability etc; default to 'Record'
Identifier fields	No	(none)	(Optional) Comma separated list of fields that will be used as identifiers. The declared order is important. Example usage: a,b
Unique fields	No	(none)	(Optional) Comma separated list of fields when combined represents a unique row. Example usage: a,b
Numeric fields	No	(none)	(Optional) Comma separated list of field=type that contain numbers. Example usage: Severity=i,Risk=d,Count=l
Boolean fields	No	(none)	(Optional) Comma separated list of fields that contain boolean values. Example usage: Active, Internet Facing
Date fields	No	(none)	(Optional) Comma separated list of field=date-format that contain dates. Example usage: LastUpdated=MM/dd/yyyy
Multi-value fields	No	(none)	(Optional) Comma separated list of field=delimiter that contain multiple values. Example usage: High=:, Low=;
Multi-row fields	No	(none)	(Optional) Comma separated list fields that contain values spanning across multiple rows. Unique fields must be specified for this to work as expected.
File encoding	No	OS default charset	The encoding of the file such as UTF-8.
Text Qualifier	No	"	Used to determine the beginning and ending of a field.
Field delimiter	No	,	Delimiter for each of the fields.
EOL characters	No	CRLF	Characters that represent EOL for example \n
Failure threshold	No	-1	Acceptable number of failed rows before processing is halted where -1 is infinite.
Max age	No	(inherited)	Maximum number of days a file will be retained, any value less than 0 indicates forever. Zero indicates not to keep the file.
Max files	No	(inherited)	Maximum number of files to retain. Less than 0 indicates all and 0 indicates none.
Post process	No	true	Rename or move the file after its processed.

Authentication

This is a file-based connector. It does not connect to a remote service and therefore requires no credentials, tokens, OAuth flow, or API endpoints. Access is controlled entirely by the file system: the Brinqa agent running the connector must have read access to the configured data directory (and write access if post-processing/renaming of processed files is enabled).

Ingestion works as follows:

Step	Behavior
Discover	Enumerate files under the configured Data directory that match the path pattern (wildcards supported, e.g. /feed/vendors/vendor_*.csv).
Filter	During an incremental sync, only include files whose lastModified timestamp is greater than the sync token from the previous run and that are not already suffixed .processed.
Parse	Read the header row to build the schema, then parse each data row using the configured delimiter, text qualifier, encoding, and end-of-line symbols.
Type / transform	Apply per-column cell processors for numeric, date, boolean, multi-value, and multi-row columns.
Emit	Build a connector object per record (per merged record when multi-row fields are used) and hand it to the sync handler.
Post-process	If enabled, rename processed files with a timestamped .processed (or .failed) suffix and apply retention (max age / max number of files).

Sync Behavior

The connector supports incremental (delta) syncs. It maintains a sync token between runs and applies it as an incremental timestamp filter, so each run re-processes only the records that changed after the previous sync. The initial run retrieves the complete data set; later runs are incremental. The specific timestamp field applied to each object is documented under that object's Sync Duration Parameter.

How to obtain CSV Connector credentials

Obtain the required credentials (Data directory (dataDirectory), Target (objectClassType)) from your CSV Connector administrator or the CSV Connector admin console, then enter them in the connection settings above.

note

When multiple CSV files match a wildcard, the connector processes them by last-modified time from oldest to newest, so the most recent data takes precedence when there are duplicates.

tip

When Post process is enabled, each ingested CSV file is renamed with a .processed suffix so it is not re-ingested on the next sync.

Data & mappings

Attribute mappings

Expand the sections below to view the mappings between the source and the Brinqa data model attributes:

Record (configurable target)

Source Field Name	SDM Attribute
---	---
Derived from unique fields or row content	Uid
Source Field	Attribute Name
Source file lastModified timestamp	Modified
Source file name (without extension)	File Name
Sync run timestamp	Last Captured
The CSV column with the same name	(one attribute per CSV header)

Operations & API

Operations & API

Expand each connector object to see its operation options, delta-sync behavior, and the API it uses. See connector operation options for how to apply operation options (keys and values are case-sensitive).

Record (configurable target)

Operation options

This object does not support any operation options.

Delta sync

Supported. The connector performs an incremental (delta) sync via the since sync token, filtering on lastModified.

API

• Type: File system (delimited flat files — no remote API) · Endpoint: Local file system path matching the configured **Data directory** pattern (wildcards supported, e.g.
• Default filters: During incremental syncs, only files whose lastModified timestamp is greater than the previous run's sync token and that are not already suffixed .processed are included

Changelog

Changelog

The CSV Connector connector has undergone the following changes:

Version	Description	Migration Steps
3.0.13	Bug Fixes - Syncs that use multi-row fields no longer fail with a ClassNotFoundException referencing an internal framework class (KryoSerializationFactory). The connector now serializes records through the framework's public serialization API while combining multi-row records, so the sync runs regardless of the agent's classloader sandboxing.	N/A
3.0.12	Maintenance release. Refreshed the connector icon; no functional changes.	N/A
3.0.11	New Features - Each ingested record now carries a Last Captured timestamp recording when the sync run that produced it took place, making it easier to distinguish the file's own modified date from when Brinqa last read it.	N/A
3.0.10	Bug Fixes - Field type mappings (numeric, date, boolean, multi-value, etc.) are now rebuilt from scratch each time the connector is initialized. Previously, stale mappings from a prior configuration could linger and be applied to a subsequent run, causing columns to be parsed with the wrong type.	N/A
3.0.9	Improvements - Date-typed columns are now emitted as true date/time values rather than raw numeric epoch values, so date fields land in Brinqa as proper dates.	• Configured date fields: Columns declared in the Date fields setting now produce date/time values instead of numeric (epoch) values. Any target model attribute previously mapped to these columns as a number should be remapped to a date/time type — Action: re-sync so the model picks up the corrected type.
3.0.8	Maintenance release. No customer-facing changes.	N/A
3.0.7	Maintenance release. Removed an unused embedded database dependency, reducing the connector's footprint. No functional changes.	N/A
3.0.6	New Features - Date fields can now be configured to parse epoch timestamps directly. Formats beginning with epochm/milli are read as epoch milliseconds, and epochs/sec/unix as epoch seconds, in addition to the existing pattern-based date formats. Bug Fixes - Resolved a null-pointer error that could occur while parsing certain date values.	N/A
3.0.5	Improvements - Columns literally named uid are now automatically excluded from both the schema and the ingested data, with a warning, because uid is a reserved identifier. This prevents a source column from colliding with the connector's own unique identifier.	N/A
3.0.4	Bug Fixes - Invalid or unrecognized values in Boolean fields are now treated as false rather than failing the row, so a malformed boolean cell no longer halts file processing.	N/A
3.0.3	New Features - Added a Boolean fields configuration option, letting you designate columns whose values should be ingested as true/false booleans (e.g. Active, Internet Facing).	N/A
3.0.2	Maintenance release. No customer-facing changes.	N/A
3.0.1	Maintenance release. Updated to the latest connector framework; no functional changes.	N/A
3.0.0	Overview The CSV Connector ingests data from delimited flat files (CSV, TSV, or any character-delimited format) placed in a configured directory on the file system. Instead of calling a remote API, it discovers files matching a configurable path pattern, parses each row into a record using the configured delimiter, text qualifier, encoding, and end-of-line characters, and emits the records to Brinqa. The schema is derived dynamically from each file's header row, and columns can be typed (string, integer, long, double, boolean, or date), treated as multi-value, or merged across multiple rows. Category: File Models	N/A