Amazon S3

Amazon Web Services

Overview

Integrates with Amazon S3 to inventory the S3 buckets in the configured AWS accounts. For each bucket the connector reads ownership, region, tags, and the full public-access surface — the bucket policy, the bucket ACL, and the S3 Block Public Access (PAB) configuration — and consolidates them into a single effective-public verdict so downstream analysis can flag internet-exposed buckets. The connector enumerates buckets via the global S3 ListBuckets API and emits one Brinqa object per bucket.

Data retrieved from Amazon S3

Connector Object	Required	Maps to Data Model
S3Bucket	Yes	Cloud Resource

note

For detailed steps on how to view the data retrieved from Amazon S3 in the Brinqa Platform, see How to view your data.

Setup

Connection settings

When setting up a data integration, select Amazon S3 from the Connector dropdown and provide the following:

Setting	Required	Default	Description
Access key ID	No	—	AWS user access key ID, if not specified, the connector consults the default credentials provider chain to try and determine the \
Secret access key	No	—	AWS user secret access key, if not specified, the connector consults the default credentials provider chain to try and determine the \
Default region	No	us-east-1	Default AWS region for the S3 client. Per-bucket operations follow regional redirects automatically.
Assume role	No	— (empty)	Amazon Resource Name for the role to assume.
External ID	No	—	Optional STS external ID, sent when assuming the role named in the Assume role \
Session duration	No	900	Assume role session duration in seconds.
Page size	No	100	The maximum number of results to retrieve per operation.
Parallel requests	No	min(4, CPU cores)	Maximum number of parallel requests.

Authentication

Authentication is shared across the AWS connector bundle via the AwsConfiguration base class. The S3 sub-connector uses the AWS SDK for Java v2 S3Client, which signs every request with AWS SigV4 using credentials resolved through the following chain (in order):

1. STS AssumeRole — if assumeRoleARN is configured, the connector assumes that role (using the credentials below as the principal) and uses the resulting temporary session credentials. Set externalId as well if the role's trust policy enforces an ExternalId condition; it is sent on the AssumeRole call and is required for cross-account confused-deputy protection.
2. Static access keys — if accessKey / secretKey are configured, they are used directly.
3. Default credentials provider chain — environment variables, ~/.aws/credentials, container role, or EC2 instance metadata.

Assumed-role sessions use the configurable sessionDuration (default 900 s) with a unique session name per sync. assumeRoleARN may be a comma-separated list to sync multiple accounts in one run. ListBuckets is a global call, so a single S3Client is built per account (per assumed role) using the configured default region; the client has cross-region access enabled so per-bucket calls transparently follow regional redirects to each bucket's home region.

Choosing an authentication path

The list above is the order the connector resolves credentials in. In terms of what to configure, three paths are supported:

Path	Configure	When to use
STS AssumeRole (recommended)	assumeRoleARN, plus externalId if the trust policy requires one	Cross-account access, and the only path that supports an ExternalId condition. Still needs a principal, from either of the paths below.
Default credentials provider chain (fully keyless)	nothing	The platform runs inside your AWS account on an EC2 instance profile or an EKS IRSA role. No long-lived keys exist anywhere.
Static access keys (optional)	accessKey, secretKey	The platform runs outside your AWS account and no role is available. Long-lived credentials — prefer either path above.

sts:AssumeRole is required only when assumeRoleARN is set.

Required AWS API permissions

Permission	Purpose
s3:ListAllMyBuckets	List all buckets in the account (ListBuckets)
s3:GetBucketLocation	Resolve each bucket's home region
s3:GetBucketPolicy	Read the bucket policy (public-policy detection)
s3:GetBucketAcl	Read the bucket ACL (public-grantee detection)
s3:GetBucketPublicAccessBlock	Read the Block Public Access configuration
s3:GetBucketTagging	Read bucket tags
sts:AssumeRole	Only when assumeRoleARN is set

Sync Behavior

Each sync is a full sync. Every run enumerates all resources through the AWS APIs; the connector does not maintain a sync token or apply an incremental since date filter.

---

How to obtain Amazon S3 credentials

Create an IAM user for AWS access

For the Amazon S3 connector to interact with the AWS SDK and retrieve bucket data, you must provide specific AWS credentials and permissions. To create an IAM user, follow these steps:

1. Log in to your organization's AWS Management Console as an administrator.

2. Navigate to the Identity and Access Management (IAM) dashboard.

3. From the navigation pane under Access management, click Users, and then click Create user.

4. Provide a User name, leave the Provide user access to AWS Management Console option unchecked, and then click Next.

5. Click the Attach policies directly option and then click Create policy.

6. Click the JSON tab and paste the following minimum required policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:ListAllMyBuckets",
           "s3:GetBucketLocation",
           "s3:GetBucketPolicy",
           "s3:GetBucketAcl",
           "s3:GetBucketPublicAccessBlock",
           "s3:GetBucketTagging"
         ],
         "Resource": "*"
       }
     ]
   }

7. Click Next, provide a name for the policy, and then click Create policy.

8. Back on the Add permissions page, search for and select the policy you just created, and then click Next.

9. Click Create user.

Note: If you do not have permissions to create a new IAM user, contact your AWS administrator. For additional information, see AWS documentation.

Data & mappings

Attribute mappings

Expand the sections below to view the mappings between the source and the Brinqa data model attributes:

S3Bucket

Source Field Name	SDM Attribute
"arn:aws:s3:::" + Bucket.name	UID
"arn:aws:s3:::" + Bucket.name	BUCKET_ARN
Bucket.creationDate	FIRST_SEEN
Bucket.name	NAME
Bucket.name	BUCKET_NAME
constant AWS	CLOUD_PROVIDER
derived	HAS_POLICY
derived	PAB_FULLY_BLOCKED
derived from ACL grantee URIs	ACL_HAS_PUBLIC_GRANTEE
derived from policy JSON	POLICY_HAS_PRINCIPAL_STAR
derived — see below	IS_EFFECTIVELY_PUBLIC
GetBucketAclResponse.grants()	ACL_GRANTS (multivalued)
GetBucketLocation.locationConstraintAsString()	REGION
GetBucketPolicyResponse.policy	BUCKET_POLICY
GetBucketTaggingResponse.tagSet()	TAGS (multivalued)
ListBucketsResponse.owner().displayName	OWNER_DISPLAY_NAME
ListBucketsResponse.owner().id	OWNER_ID
PublicAccessBlockConfiguration.blockPublicAcls	BLOCK_PUBLIC_ACLS
PublicAccessBlockConfiguration.blockPublicPolicy	BLOCK_PUBLIC_POLICY
PublicAccessBlockConfiguration.ignorePublicAcls	IGNORE_PUBLIC_ACLS
PublicAccessBlockConfiguration.restrictPublicBuckets	RESTRICT_PUBLIC_BUCKETS
sync timestamp (Instant.now())	LAST_CAPTURED

Operations & API

Operations & API

Expand each connector object to see its operation options, delta-sync behavior, and the API it uses. See connector operation options for how to apply operation options (keys and values are case-sensitive).

S3Bucket

Operation options

This object does not support any operation options.

Delta sync

Not supported. The connector performs a full sync of S3Bucket on every run and applies no incremental date filter.

API

• Type: AWS SDK for Java v2 (S3Client) · Endpoint: s3:ListBuckets

Changelog

Changelog

The Amazon S3 connector has undergone the following changes:

Version	Description	Migration Steps
3.2.3	New Features - Roles that require an External ID can now be used. A new optional External ID setting is sent when assuming the role named in the connector's Assume role setting, so roles whose trust policy enforces an ExternalId condition — the standard protection against cross-account confused-deputy access — can now be assumed. Existing configurations are unaffected: leave it blank and the role is assumed exactly as before. The connector's authentication documentation now also describes all three supported paths — assuming a role, running fully keyless from an EC2 instance profile or EKS IRSA role, and static access keys. Bug Fixes - The Parallel requests setting is now labeled on the configuration screen. The setting was shown without its proper name or help text. It now displays its label and description. The setting's behavior and default are unchanged.	N/A
3.2.2	No changes in this release.	N/A
3.2.1	No changes in this release.	N/A
3.2.0	No changes in this release.	N/A
3.1.0	New Features - New connector — syncs S3 buckets with a consolidated "effectively public" flag combining bucket policy, ACL, and Public Access Block analysis. Improvements - Sync failures are now visible. Previously, certain AWS API errors (throttling, server errors, transient network failures, validation errors) could be silently swallowed during multi-region sync, producing empty results without indicating a problem. These now surface as sync failures with a clear log entry detailing the HTTP status, AWS error code, request id, and service name. Per-region permission gaps (HTTP 401 / 403) still allow the sync to continue across other regions, but are now visible in logs.	N/A