CSV Connector

File

Overview

The CSV Connector ingests data from delimited flat files (CSV, TSV, or any character-delimited format) that are placed in a configured directory on the file system. Rather than calling a remote API, the connector reads files that match a configurable path pattern, parses each row into a record, and emits the records to Brinqa as connector objects.

The connector is schema-flexible: the set of attributes for each record is derived dynamically from the header row of the CSV file. Columns can be typed (string, integer, long, double, boolean, or date/time), treated as multi-value, or merged across multiple rows. The target data model that the parsed records are mapped to is itself configurable (it defaults to Record), so the same connector can feed Host, Vulnerability, or any other Brinqa data model depending on the source file's contents.

Files are discovered, parsed, and then optionally renamed/moved once processed. Incremental syncs only reprocess files whose last-modified timestamp is newer than the last successful run.

Data retrieved from CSV Connector

Connector Object	Required	Maps to Data Model
Record (configurable target)	Yes	Record (configurable target)

Model relationships

note

For detailed steps on how to view the data retrieved from CSV Connector in the Brinqa Platform, see How to view your data.

Setup

Connection settings

When setting up a data integration, select CSV Connector from the Connector dropdown and provide the following:

Setting	Required	Default	Description
Data directory (dataDirectory)	Yes	(none)	Fully qualified path to the data directory. Wildcards are allowed in the path, e.g. /feed/vendors/vendor_.csv or /feed/mobile/mobile_device.*.
Target (objectClassType)	Yes	Record	The target data model name the provided data maps to, e.g. Host, Vulnerability. Becomes the connector object class type.
Identifier fields (identifierFields)	No	(none)	Comma-separated list of fields used as identifiers. The declared order is significant. Example: a,b.
Unique fields (uniqueFields)	No	(none)	Comma-separated list of fields whose combined values represent a unique row (used to build the UID and to merge multi-row records). Example: a,b.
Numeric fields (numericFields)	No	(none)	Comma-separated list of field=type for numeric columns, where type is i (int), l (long), or d (double). Example: Severity=i,Risk=d,Count=l.
Boolean fields (booleanFields)	No	(none)	Comma-separated list of fields that contain boolean values. Truthy values include 1,true,t,yes,y; unparseable values fall back to false. Example: Active, Internet Facing.
Date fields (dateFields)	No	(none)	Comma-separated list of field=date-format for date columns. Supports Joda/DateTimeFormat patterns, ISO date/time when no format is given, and the keywords epochMillis/milli, epochSeconds/sec/unix. Example: LastUpdated=MM/dd/yyyy.
Multi-value fields (multiValueFields)	No	(none)	Comma-separated list of field=delimiter for columns holding multiple values. Each value is split on the given delimiter (default ,) and any per-column type processor is applied to each value. Example: High=:, Low=;.
Multi-row fields (multiRowFields)	No	(none)	Comma-separated list of fields whose values span multiple rows; rows sharing the same unique id are merged and these columns accumulate all values. Requires uniqueFields to be set.
File encoding (encoding)	No	OS default charset	Character encoding used to read the file, such as UTF-8.
Text Qualifier (textQualifier)	No	"	Character used to mark the beginning and end of a field. Must not equal the field delimiter.
Field delimiter (fieldDelimiter)	No	,	Delimiter separating fields. Use a value containing TAB for tab-separated files.
EOL characters (endOfLineSymbols)	No	CRLF	End-of-line symbols, expressed using CR and/or LF (e.g. CRLF).
Failure threshold (acceptableNumberOfFailedRows)	No	-1	Number of failed rows tolerated before processing is halted; -1 means unlimited.
Max age (maxAgeInDays)	No	(inherited)	Maximum number of days a processed file is retained. A value less than 0 means keep forever; 0 means do not keep the file.
Max files (maxNumberOfFiles)	No	(inherited)	Maximum number of processed files to retain. Less than 0 means retain all; 0 means retain none.
Post process (renameProcessed)	No	true	When enabled, files are renamed/moved after processing with a timestamped .processed (or .failed) suffix.

Authentication

This is a file-based connector. It does not connect to a remote service and therefore requires no credentials, tokens, OAuth flow, or API endpoints. Access is controlled entirely by the file system: the Brinqa agent running the connector must have read access to the configured data directory (and write access if post-processing/renaming of processed files is enabled).

Ingestion works as follows:

Step	Behavior
Discover	Enumerate files under the configured Data directory that match the path pattern (wildcards supported, e.g. /feed/vendors/vendor_*.csv).
Filter	During an incremental sync, only include files whose lastModified timestamp is greater than the sync token from the previous run and that are not already suffixed .processed.
Parse	Read the header row to build the schema, then parse each data row using the configured delimiter, text qualifier, encoding, and end-of-line symbols.
Type / transform	Apply per-column cell processors for numeric, date, boolean, multi-value, and multi-row columns.
Emit	Build a connector object per record (per merged record when multi-row fields are used) and hand it to the sync handler.
Post-process	If enabled, rename processed files with a timestamped .processed (or .failed) suffix and apply retention (max age / max number of files).

Sync Behavior

The connector supports incremental (delta) syncs. It maintains a sync token between runs and applies it as an incremental timestamp filter, so each run re-processes only the records that changed after the previous sync. The initial run retrieves the complete data set; later runs are incremental. The specific timestamp field applied to each object is documented under that object's Sync Duration Parameter.

How to obtain CSV Connector credentials

Obtain the required credentials (Data directory (dataDirectory), Target (objectClassType)) from your CSV Connector administrator or the CSV Connector admin console, then enter them in the connection settings above.

note

When multiple CSV files match a wildcard, the connector processes them by last-modified time from oldest to newest, so the most recent data takes precedence when there are duplicates.

tip

When Post process is enabled, each ingested CSV file is renamed with a .processed suffix so it is not re-ingested on the next sync.

Data & mappings

Attribute mappings

Expand the sections below to view the mappings between the source and the Brinqa data model attributes:

Record (configurable target)

Source Field Name	SDM Attribute
---	---
Derived from unique fields or row content	Uid
Source Field	Attribute Name
Source file lastModified timestamp	Modified
Source file name (without extension)	File Name
Sync run timestamp	Last Captured
The CSV column with the same name	(one attribute per CSV header)

Operations & API

Operations & API

Expand each connector object to see its operation options, delta-sync behavior, and the API it uses. See connector operation options for how to apply operation options (keys and values are case-sensitive).

Record (configurable target)

Operation options

This object does not support any operation options.

Delta sync

Supported.

API

• Type: File system (delimited flat files — no remote API) · Endpoint: Local file system path matching the configured **Data directory** pattern (wildcards supported, e.g.
• Default filters: During incremental syncs, only files whose lastModified timestamp is greater than the previous run's sync token and that are not already suffixed .processed are included

Changelog

Changelog

The CSV Connector connector has undergone the following changes:

Version	Description	Migration Steps
3.0.12	Maintenance release. Refreshed the connector icon; no functional changes.	N/A
3.0.11	New Features - Each ingested record now carries a Last Captured timestamp recording when the sync run that produced it took place, making it easier to distinguish the file's own modified date from when Brinqa last read it.	N/A
3.0.10	Bug Fixes - Field type mappings (numeric, date, boolean, multi-value, etc.) are now rebuilt from scratch each time the connector is initialized. Previously, stale mappings from a prior configuration could linger and be applied to a subsequent run, causing columns to be parsed with the wrong type.	N/A
3.0.9	Improvements - Date-typed columns are now emitted as true date/time values rather than raw numeric epoch values, so date fields land in Brinqa as proper dates.	• Configured date fields: Columns declared in the Date fields setting now produce date/time values instead of numeric (epoch) values. Any target model attribute previously mapped to these columns as a number should be remapped to a date/time type — Action: re-sync so the model picks up the corrected type.
3.0.8	Maintenance release. No customer-facing changes.	N/A
3.0.7	Maintenance release. Removed an unused embedded database dependency, reducing the connector's footprint. No functional changes.	N/A
3.0.6	New Features - Date fields can now be configured to parse epoch timestamps directly. Formats beginning with epochm/milli are read as epoch milliseconds, and epochs/sec/unix as epoch seconds, in addition to the existing pattern-based date formats. Bug Fixes - Resolved a null-pointer error that could occur while parsing certain date values.	N/A
3.0.5	Improvements - Columns literally named uid are now automatically excluded from both the schema and the ingested data, with a warning, because uid is a reserved identifier. This prevents a source column from colliding with the connector's own unique identifier.	N/A
3.0.4	Bug Fixes - Invalid or unrecognized values in Boolean fields are now treated as false rather than failing the row, so a malformed boolean cell no longer halts file processing.	N/A
3.0.3	New Features - Added a Boolean fields configuration option, letting you designate columns whose values should be ingested as true/false booleans (e.g. Active, Internet Facing).	N/A
3.0.2	Maintenance release. No customer-facing changes.	N/A
3.0.1	Maintenance release. Updated to the latest connector framework; no functional changes.	N/A
3.0.0	Overview The CSV Connector ingests data from delimited flat files (CSV, TSV, or any character-delimited format) placed in a configured directory on the file system. Instead of calling a remote API, it discovers files matching a configurable path pattern, parses each row into a record using the configured delimiter, text qualifier, encoding, and end-of-line characters, and emits the records to Brinqa. The schema is derived dynamically from each file's header row, and columns can be typed (string, integer, long, double, boolean, or date), treated as multi-value, or merged across multiple rows. Category: File Models	N/A