Skip to content

Add opt-in path-style S3 addressing via MCAP_S3_FORCE_PATH_STYLE - #1600

Open
alexhilgart wants to merge 1 commit into
foxglove:mainfrom
alexhilgart:feat/s3-force-path-style
Open

Add opt-in path-style S3 addressing via MCAP_S3_FORCE_PATH_STYLE#1600
alexhilgart wants to merge 1 commit into
foxglove:mainfrom
alexhilgart:feat/s3-force-path-style

Conversation

@alexhilgart

@alexhilgart alexhilgart commented Mar 13, 2026

Copy link
Copy Markdown

Changelog

Add optional path-style S3 addressing support via the MCAP_S3_FORCE_PATH_STYLE environment variable for S3-compatible storage endpoints.

Docs

This PR updates the CLI documentation to include the new MCAP_S3_FORCE_PATH_STYLE environment variable.

Description

Some S3-compatible object storage systems (such as MinIO, Ceph, or custom S3 gateways) require path-style addressing:


https://endpoint/bucket/key

instead of the default virtual-host style:


https://bucket.endpoint/key

The MCAP CLI currently constructs its S3 client using:

s3.NewFromConfig(cfg)

which causes the AWS SDK for Go v2 to use virtual-host style addressing by default. When used with S3-compatible endpoints that do not support bucket subdomains, this can result in DNS errors such as:

lookup <bucket>.<endpoint>: no such host

This PR adds an optional environment variable:

MCAP_S3_FORCE_PATH_STYLE=true

When this variable is set, the S3 client is configured with:

o.UsePathStyle = true

This allows the CLI to work with S3-compatible object stores that require path-style addressing, while preserving the current default behavior for AWS S3 users.

The change is intentionally minimal:

  • No behavior changes unless the environment variable is set
  • No refactoring of existing reader logic
  • Applies to both anonymous and authenticated S3 client initialization paths

Documentation has also been updated in website/docs/guides/cli.md.

Manual testing

Built the CLI locally and verified the behavior against an S3-compatible endpoint.

Without the environment variable, the CLI attempted requests using virtual-host style addressing:

https://bucket.endpoint/key

With the environment variable enabled:

MCAP_S3_FORCE_PATH_STYLE=true

the CLI correctly used path-style addressing:

https://endpoint/bucket/key

and successfully read the remote MCAP file.

BeforeAfter

MCAP CLI always used virtual-host style addressing when accessing S3:

https://bucket.endpoint/key

This caused DNS failures when the endpoint did not support bucket subdomains.

Path-style addressing can be enabled when needed:

MCAP_S3_FORCE_PATH_STYLE=true

Resulting requests:

https://endpoint/bucket/key

This allows the CLI to work with S3-compatible object stores that require path-style addressing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant