Amazon DynamoDB

Amazon Web Services

Overview

Integrates with Amazon DynamoDB to synchronize the contents of DynamoDB tables into the Brinqa platform. Because DynamoDB is schemaless, this connector is generic: each configured table is exposed as its own Brinqa object class (named after the table), and that class's schema is discovered dynamically by scanning a sample of the table's items and inferring an attribute per item field. Table rows are read with a parallel (segmented) Scan and each item becomes one Brinqa object.

Data retrieved from Amazon DynamoDB

Connector Object	Required	Maps to Data Model
Table	Yes	Table

note

For detailed steps on how to view the data retrieved from Amazon DynamoDB in the Brinqa Platform, see How to view your data.

Setup

Connection settings

When setting up a data integration, select Amazon DynamoDB from the Connector dropdown and provide the following:

Setting	Required	Default	Description
Access key ID	No	—	AWS user access key ID, if not specified, the connector consults the default credentials provider chain to try and determine the \
Secret access key	No	—	AWS user secret access key, if not specified, the connector consults the default credentials provider chain to try and determine the \
Default region	No	us-east-1	Default AWS region, if not specified, the connector consults the default region provider chain to try and determine the region to use.
Table names	No	—	Optional list of table names, if not specified, the connector will retrieve all available tables.
Filter expression	No	—	Optional filter expression to limit the table scan items.
Unique fields	No	—	Optional comma-separated list of fields when combined represents a unique row, if not specified, the connector will use all data present in the table record.
Date fields	No	—	Optional comma-separated list of fields that contain dates.
Page size	No	—	The maximum number of items to retrieve per operation. Also, if the processed dataset size exceeds 1 MB\
Parallel requests	No	min(4, available processors)	Maximum number of parallel requests
Total segments	No	256	The total number of segments for a parallel scan as recommended by AWS, defaults to 256.

Authentication

Authentication uses the AWS SDK for Java v2 DynamoDbClient, which signs every request with AWS SigV4 using credentials resolved as follows:

1. Static access keys — if accessKey / secretKey are configured, they are used directly.
2. Default credentials provider chain — if either key is blank, the connector falls back to the AWS default credentials provider chain (environment variables, ~/.aws/credentials, container role, or EC2 instance metadata).

The client is built for the configured region (when set). This sub-connector does not support STS AssumeRole.

Required AWS API permissions

Permission	Purpose
dynamodb:ListTables	Enumerate table names when the tables property is not set (schema/type discovery)
dynamodb:Scan	Sample items for dynamic schema discovery and read table content during sync

Sync Behavior

Each sync is a full scan of the configured DynamoDB table; the connector does not apply an incremental date filter by default. It performs a delta sync only if the operator supplies a custom filterExpression that references the auto-bound :since placeholder against a customer-chosen attribute (there is no built-in date field).

---

How to obtain Amazon DynamoDB credentials

Create an IAM user for Amazon DynamoDB access

For the Amazon DynamoDB connector to interact with the Amazon DynamoDB API, you must provide specific AWS credentials and permissions. To do so, follow these steps:

1. Log in to your organization's AWS Management Console as an administrator.

2. Navigate to the Identity and Access Management (IAM) dashboard.

3. From the navigation pane under Access management, click Users, and then click Create user.

4. Provide a User name, leave the Provide user access to AWS Management Console option unchecked, and then click Next.

5. Click the Attach policies directly option, search for and select the AmazonDynamoReadOnlyAccess permission.

   

6. Click Next and then click Create user.

   The Users page displays and the new IAM user is available in the Users table.

Note: If you do not have permissions to create a new IAM user, contact your AWS administrator. For additional information, see AWS documentation.

Obtain Amazon DynamoDB access keys

After you have created an IAM user, you can generate the access keys that are required for the Amazon DynamoDB connector to access the AWS SDK. To generate the access keys required for the integration, follow these steps:

1. Log in to your organization's AWS Management Console as an administrator.

2. Navigate to the IAM dashboard.

3. From the navigation pane under Access management, click Users.

4. Choose the IAM user you created in the earlier steps.

5. Click the Security credentials tab and then click Create access key.

6. Select the Application running outside AWS use case and then click Next.

7. Provide a description and then click Create access key.

   

   Your new access keys display. You cannot view the keys again after this. Copy and save them to a safe and secure location.

Note: If you do not have permissions to create access keys, contact your AWS administrator. For additional information, see AWS documentation.

Data & mappings

Attribute mappings

Expand the sections below to view the mappings between the source and the Brinqa data model attributes:

Table

Source Field Name	SDM Attribute
derived key (see below)	UID
DynamoDB item field of the same name	<field name>

Operations & API

Operations & API

Expand each connector object to see its operation options, delta-sync behavior, and the API it uses. See connector operation options for how to apply operation options (keys and values are case-sensitive).

Table

Operation options

Option	Type	Default	Description
filterExpression	DynamoDB filter expression applied to each Scan.	filterExpression configuration property
:<name>	Binds an expression attribute value referenced in the filter expression (string). :since is auto-bound to the sync token instant.	none
#<name>	Binds an expression attribute name referenced in the filter expression.	none

Delta sync

Not supported. The connector performs a full sync of Table on every run and applies no incremental date filter.

API

• Type: AWS SDK for Java v2 (DynamoDbClient)

Changelog

Changelog

The Amazon DynamoDB connector has undergone the following changes:

Version	Description	Migration Steps
3.2.2	Bug Fixes - Table syncs no longer fail when a configured unique field has no value. Previously, if a unique field existed on an item but carried no value, the sync could stop with an error and import nothing from that table. Such items now sync successfully. Because items that share the same unique field values resolve to the same record identity, the connector now logs a warning whenever a unique field has no value, helping you spot unique field configurations that may not uniquely identify each item.	N/A
3.2.1	No changes in this release.	N/A
3.2.0	No changes in this release.	N/A
3.1.0	Improvements - Sync failures are now visible. Previously, certain AWS API errors (throttling, server errors, transient network failures, validation errors) could be silently swallowed during multi-region sync, producing empty results without indicating a problem. These now surface as sync failures with a clear log entry detailing the HTTP status, AWS error code, request id, and service name. Per-region permission gaps (HTTP 401 / 403) still allow the sync to continue across other regions, but are now visible in logs.	N/A
3.0.20	No changes in this release.	N/A
3.0.19	New Features - Configurable scan parallelism. A new Total segments setting lets you control how DynamoDB divides each table into segments for parallel scanning (up to 2048). Combined with the existing Parallel requests setting, this allows large tables to be synced faster by spreading the workload across more concurrent scan operations. Improvements - More reliable parallel table scans. The table scan engine was reworked so that all configured segments are scanned concurrently with a managed thread pool. Errors raised mid-scan now stop the sync cleanly and propagate the underlying AWS error instead of being lost, making failed syncs easier to diagnose.	N/A
3.0.18	Dependency Upgrades - Updated the AWS SDK to a newer release for improved compatibility and reliability when communicating with DynamoDB.	N/A
3.0.17	No changes in this release.	N/A
3.0.16	No changes in this release.	N/A
3.0.15	No changes in this release.	N/A
3.0.14	No changes in this release.	N/A
3.0.13	Dependency Upgrades - Updated the AWS SDK to a newer release for improved compatibility and reliability when communicating with DynamoDB.	N/A
3.0.12	New Features - Unique fields. You can now specify which table attributes uniquely identify each item. When set, the connector builds each record's identity from those fields instead of hashing the entire item, giving you stable, predictable record identities across syncs. - Date fields. You can now designate table attributes that should be interpreted as dates. Values in these fields are parsed and synced as proper date/time values rather than plain text or numbers, enabling correct date-based filtering and reporting.	• Tables: Configuring Unique fields changes how each item's identity is derived, so previously synced records will be re-keyed. Designating Date fields changes the data type of those attributes from text/number to date. — Action: re-sync the connector after changing either setting so records are rebuilt with the new identities and attribute types.
3.0.11	No changes in this release.	N/A
3.0.10	No changes in this release.	N/A
3.0.9	No changes in this release.	N/A
3.0.8	No changes in this release.	N/A
3.0.7	No changes in this release.	N/A
3.0.6	No changes in this release.	N/A
3.0.5	No changes in this release.	N/A
3.0.4	Overview The Amazon DynamoDB connector integrates with Amazon Web Services to synchronize the contents of your DynamoDB tables, discovering each table's attributes dynamically and importing its items as records. Category: Amazon Web Services Models	N/A