feat(go-server): add DuckDB security profiles - #1151
Draft
derekperkins wants to merge 39 commits into
Draft
Conversation
Validate every parsed function and operator against an opt-in exact-name policy while preserving the existing blocklist and fail-closed exec behavior.
Preserve an explicitly empty allowlist as deny-all while keeping an omitted flag backward compatible.
Describe exact matching, deny-all semantics, parser-visible helpers, and the version-specific external-resource vectors the syntactic policy cannot resolve.
Verify allowlist denials remain HTTP 403 responses and generic command authorization cannot bypass restricted exec.
derekperkins
commented
Aug 13, 2026
derekperkins
left a comment
Collaborator
Author
There was a problem hiding this comment.
Reviewed the three commits against codex/function-allowlist, so #1150's contents are out of scope here.
This is in good shape. I spent most of the review trying to get out of catalog-only and local-files and could not:
glob,read_text,read_blob,read_csvoutside the grant — blocked, includingglob('/*')and<allowed>/../<other>/secret.txttraversal.SET,SET SESSION,SET GLOBAL,RESET, andPRAGMAforms of every profile-owned setting — all rejected bylock_configuration.LOAD,INSTALL,FORCE INSTALL— blocked, includingLOADof an extension file planted inside an allowed directory, which was the escalation path I most expected to work.- File-backed primary — the automatic grant is the database file and its
.walonly; neighbours, directory globs,ATTACH, andCOPYinto that directory are all blocked. temp_directory=''genuinely prevents spilling rather than silently falling back to a default.
Verified locally on 316ae55 (Go 1.26.5, darwin/arm64, DuckDB 1.5.5): go build, go test -count=1, go test -race -count=1, go vet (all -tags=duckdb_arrow) and golangci-lint run are clean.
Four comments: one code issue that is latent today but silent and total if it is ever hit, the startup exit code, and two documentation points — one of which is asking you to claim a stronger guarantee than the text currently does.
— Claude Opus 5
Reject source-qualified function calls while preserving DuckDB's parser-generated main-qualified helper nodes, and report each distinct allowlist violation once.
Document explicit-empty and repeated flag behavior, macro and replacement-scan limits, and the exact DuckDB scheme mapping source.
Route process exit through run so configuration and startup failures signal failure without bypassing deferred connector and database cleanup.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 13, 2026 16:43
316ae55 to
7ab8f93
Compare
Match written qualifier prefixes without assuming DuckDB preserves the source function token, while avoiding synthesized helpers over qualified columns.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 13, 2026 16:59
7ab8f93 to
3e7e935
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 02:03
3e7e935 to
d4aa901
Compare
Function qualifiers do not establish function identity because unqualified calls can resolve through macros and the search path. Match DuckDB's serialized leaf function name like the existing blocklist and document the trusted-catalog requirement.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 02:16
d4aa901 to
6ccaf30
Compare
Keep allowlist and blocklist polarity behind named constructors while sharing AST decoding and fail-closed malformed-node handling.
Aggregate normalized AST function names so repeated policy violations produce one deterministic error with an occurrence count while singleton messages remain unchanged.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 03:18
6ccaf30 to
64c5ea9
Compare
Keep the default authority surface pinned to DuckDB 1.5.5 and fail upgrades on unreviewed catalog collisions, side effects, or volatility.
Keep omitted policy unrestricted, while a configured zero value admits reviewed local computation. Exact include and exclude overrides retain a deny-all escape hatch.
Keep the CLI unrestricted when no policy flag is present, and provide explicit include, exclude, and exact-only controls when the allowlist is enabled.
Explain default activation, exact overrides, copy-returning inventories, and the version-pinned boundary that remains outside resource enforcement.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 05:04
64c5ea9 to
a2042eb
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 05:09
a2042eb to
02a298a
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 05:23
02a298a to
5360642
Compare
Audit every DuckDB 1.5.5 internal macro so fixed local-compute wrappers are admitted while dynamic dispatch, metadata, identity, time, and delay functions remain excluded.
Keep query policy enforcement separate from the reviewed DuckDB name sets so extension bundles can grow without overwhelming the query package API.
Provide opt-in Spatial and Parquet inventories without changing the reviewed defaults. The helpers authorize names only and deliberately leave resource argument policy to a separate layer.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
2 times, most recently
from
August 14, 2026 06:03
7cfc602 to
de36865
Compare
derekperkins
force-pushed
the
codex/function-allowlist
branch
from
August 14, 2026 06:05
37fd919 to
1ef7c94
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 06:06
de36865 to
fde7ccd
Compare
Represent each DuckDB 1.5.5 core extension with immutable compute, elevated, and complete function inventories so callers can compose policies without expanding query package APIs.
Audit every DuckDB 1.5.5 core extension against its pinned source or runtime and partition callable names into compute and elevated groups. Keep uncertain proprietary MotherDuck names elevated.
Include every reviewed core-extension compute group in the default function policy while keeping resource, stateful, volatile, and source-uncertain names opt-in.
Explain the compute, elevated, and complete inventories for every DuckDB 1.5.5 core extension, including source limitations and the update workflow.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 07:01
fde7ccd to
96185b3
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 11:52
96185b3 to
5aae6db
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 14:57
5aae6db to
1da1fb2
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 15:27
1da1fb2 to
6217ca6
Compare
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 15:52
6217ca6 to
47e8da3
Compare
A full DuckDB 1.5.5 extension pass found the UI URL function missing from the reviewed elevated group. Record the runtime discovery workflow for initial inventories and future upgrades.
Resolve strict profiles before opening the connector and complete DuckDB's one-way configuration lock before constructing the query layer. Keep compatibility behavior as the default.
Exercise compatibility, catalog-only, and local-files behavior against DuckDB. Cover source-derived URI schemes, replacement scans, writes, attachments, symlinks, exact paths, and concurrent physical connections.
Describe the three common configuration bundles, their read and write semantics, and DuckDB's primary-database exception. Record version-specific URI coverage and the remaining filesystem and process-isolation limits.
Create independent one-time initializers for each connector so reusing resolved profile settings cannot leave a separate DuckDB instance unlocked.
Pin DuckDB 1.5.5's exact primary-database sidecar grant and document the repeated-read cost of disabling the external file cache.
derekperkins
force-pushed
the
codex/duckdb-hardening
branch
from
August 14, 2026 16:52
47e8da3 to
c8abd68
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
compat,catalog-only, andlocal-filessecurity profiles to the Go server binaryProfile contracts
compatis the default and preserves existing DuckDB and--load-extensionsbehavior.catalog-onlydisables external access outside DuckDB's primary-database internals, disables spill files and dynamic extension behavior, and locks the configuration.local-filesadds repeatable--allowed-directoryand--allowed-pathcapabilities tocatalog-only. Directory and exact-path grants are read/write capabilities, includingCOPYandATTACH.The strict profiles reject
--load-extensions, URI and network-share database paths, and operator-supplied DSN values for profile-owned settings. They remain CLI-local rather than introducing a public cross-package profile API.DuckDB automatically retains access to its file-backed primary database and related WAL, checkpoint, and recovery paths. Stable symlink escapes are denied, but mutable filesystem state still requires OS or container isolation. Statically linked and core extensions remain available.
The URI regression matrix covers the source-derived
http,https,s3,s3a,s3n,gcs,gs,r2,hf,azure,az, andabfssprefixes from the DuckDB 1.5.5 extension-prefix table. This is version-specific best-effort coverage; the nativeenable_external_accesssetting is the enforcement boundary.Commit structure
feat(go-server): add DuckDB security profilestest(go-server): cover DuckDB security profilesdocs(go-server): document security profilesfix(go-server): scope profile initialization per connectordocs(go-server): clarify hardening profile tradeoffsfix(go-server): compose hardening with allowlist defaultsThe former standalone startup-exit fix is now inherited from #1150 after the restack, so it was dropped rather than duplicated.
Validation
go test -count=1 -tags=duckdb_arrow ./...go test -race -count=1 -tags=duckdb_arrow ./...go test -count=20 -tags=duckdb_arrow . -run '^(TestCatalogOnlyProfileInitializesDistinctConnectors|TestSecurityProfileInitializerIsConcurrent|TestLocalFilesProfileInitializesConcurrentConnections)$'go build -tags=duckdb_arrow ./...go vet -tags=duckdb_arrow ./...golangci-lint rungit diff --check origin/codex/function-allowlist...HEADStack
This PR is intentionally separate from and based on #1150, which adds the function allowlist. The database profiles provide the resource boundary that a syntactic function policy cannot infer from string arguments.