Amazon S3

Amazon Web Services

Overview

Integrates with Amazon S3 to inventory the S3 buckets in the configured AWS accounts. For each bucket the connector reads ownership, region, tags, and the full public-access surface — the bucket policy, the bucket ACL, and the S3 Block Public Access (PAB) configuration — and consolidates them into a single effective-public verdict so downstream analysis can flag internet-exposed buckets. The connector enumerates buckets via the global S3 ListBuckets API and emits one Brinqa object per bucket.

Data retrieved from Amazon S3

Connector Object	Required	Maps to Data Model
S3Bucket	Yes	Cloud Resource

note

For detailed steps on how to view the data retrieved from Amazon S3 in the Brinqa Platform, see How to view your data.

Setup

Connection settings

When setting up a data integration, select Amazon S3 from the Connector dropdown and provide the following:

Setting	Required	Default	Description
Access key ID	No	—	AWS user access key ID, if not specified, the connector consults the default credentials provider chain to try and determine the \
Secret access key	No	—	AWS user secret access key, if not specified, the connector consults the default credentials provider chain to try and determine the \
Default region	No	us-east-1	Default AWS region for the S3 client. Per-bucket operations follow regional redirects automatically.
Assume role	No	— (empty)	Amazon Resource Name for the role to assume.
Session duration	No	900	Assume role session duration in seconds.
Page size	No	100	The maximum number of results to retrieve per operation.

Authentication

Authentication is shared across the AWS connector bundle via the AwsConfiguration base class. The S3 sub-connector uses the AWS SDK for Java v2 S3Client, which signs every request with AWS SigV4 using credentials resolved through the following chain (in order):

1. STS AssumeRole — if assumeRoleARN is configured, the connector assumes that role (using the credentials below as the principal) and uses the resulting temporary session credentials.
2. Static access keys — if accessKey / secretKey are configured, they are used directly.
3. Default credentials provider chain — environment variables, ~/.aws/credentials, container role, or EC2 instance metadata.

Assumed-role sessions use the configurable sessionDuration (default 900 s) with a unique session name per sync. assumeRoleARN may be a comma-separated list to sync multiple accounts in one run. ListBuckets is a global call, so a single S3Client is built per account (per assumed role) using the configured default region; the client has cross-region access enabled so per-bucket calls transparently follow regional redirects to each bucket's home region.

Required AWS API permissions

Permission	Purpose
s3:ListAllMyBuckets	List all buckets in the account (ListBuckets)
s3:GetBucketLocation	Resolve each bucket's home region
s3:GetBucketPolicy	Read the bucket policy (public-policy detection)
s3:GetBucketAcl	Read the bucket ACL (public-grantee detection)
s3:GetBucketPublicAccessBlock	Read the Block Public Access configuration
s3:GetBucketTagging	Read bucket tags
sts:AssumeRole	Only when assumeRoleARN is set

How to obtain Amazon S3 credentials

Create an IAM user for AWS access

For the Amazon S3 connector to interact with the AWS SDK and retrieve bucket data, you must provide specific AWS credentials and permissions. To create an IAM user, follow these steps:

1. Log in to your organization's AWS Management Console as an administrator.

2. Navigate to the Identity and Access Management (IAM) dashboard.

3. From the navigation pane under Access management, click Users, and then click Create user.

4. Provide a User name, leave the Provide user access to AWS Management Console option unchecked, and then click Next.

5. Click the Attach policies directly option and then click Create policy.

6. Click the JSON tab and paste the following minimum required policy:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "s3:ListAllMyBuckets",
           "s3:GetBucketLocation",
           "s3:GetBucketPolicy",
           "s3:GetBucketAcl",
           "s3:GetBucketPublicAccessBlock",
           "s3:GetBucketTagging"
         ],
         "Resource": "*"
       }
     ]
   }

7. Click Next, provide a name for the policy, and then click Create policy.

8. Back on the Add permissions page, search for and select the policy you just created, and then click Next.

9. Click Create user.

Note: If you do not have permissions to create a new IAM user, contact your AWS administrator. For additional information, see AWS documentation.

Data & mappings

Attribute mappings

Expand the sections below to view the mappings between the source and the Brinqa data model attributes:

S3Bucket

Source Field Name	SDM Attribute
"arn:aws:s3:::" + Bucket.name	UID
"arn:aws:s3:::" + Bucket.name	BUCKET_ARN
Bucket.creationDate	FIRST_SEEN
Bucket.name	NAME
Bucket.name	BUCKET_NAME
constant AWS	CLOUD_PROVIDER
derived	HAS_POLICY
derived	PAB_FULLY_BLOCKED
derived from ACL grantee URIs	ACL_HAS_PUBLIC_GRANTEE
derived from policy JSON	POLICY_HAS_PRINCIPAL_STAR
derived — see below	IS_EFFECTIVELY_PUBLIC
GetBucketAclResponse.grants()	ACL_GRANTS (multivalued)
GetBucketLocation.locationConstraintAsString()	REGION
GetBucketPolicyResponse.policy	BUCKET_POLICY
GetBucketTaggingResponse.tagSet()	TAGS (multivalued)
ListBucketsResponse.owner().displayName	OWNER_DISPLAY_NAME
ListBucketsResponse.owner().id	OWNER_ID
PublicAccessBlockConfiguration.blockPublicAcls	BLOCK_PUBLIC_ACLS
PublicAccessBlockConfiguration.blockPublicPolicy	BLOCK_PUBLIC_POLICY
PublicAccessBlockConfiguration.ignorePublicAcls	IGNORE_PUBLIC_ACLS
PublicAccessBlockConfiguration.restrictPublicBuckets	RESTRICT_PUBLIC_BUCKETS
sync timestamp (Instant.now())	LAST_CAPTURED

Operations & API

Operations & API

Expand each connector object to see its operation options, delta-sync behavior, and the API it uses. See connector operation options for how to apply operation options (keys and values are case-sensitive).

S3Bucket

Operation options

This object does not support any operation options.

Delta sync

The connector README does not document sync behavior for this object.

API

• Type: AWS SDK for Java v2 (S3Client) · Endpoint: s3:ListBuckets

Changelog

Changelog

The Amazon S3 connector has undergone the following changes:

Version	Description	Migration Steps
3.2.1	No changes in this release.	N/A
3.2.0	No changes in this release.	N/A
3.1.0	New Features - New connector — syncs S3 buckets with a consolidated "effectively public" flag combining bucket policy, ACL, and Public Access Block analysis. Improvements - Sync failures are now visible. Previously, certain AWS API errors (throttling, server errors, transient network failures, validation errors) could be silently swallowed during multi-region sync, producing empty results without indicating a problem. These now surface as sync failures with a clear log entry detailing the HTTP status, AWS error code, request id, and service name. Per-region permission gaps (HTTP 401 / 403) still allow the sync to continue across other regions, but are now visible in logs.	N/A