[#12457] feat(secret): Add load*WithResolvedProperties for catalog/schema/fileset - #12458
Open
lasdf1234 wants to merge 7 commits into
Open
[#12457] feat(secret): Add load*WithResolvedProperties for catalog/schema/fileset#12458lasdf1234 wants to merge 7 commits into
lasdf1234 wants to merge 7 commits into
Conversation
…talog/schema/fileset Co-authored-by: Cursor <cursoragent@cursor.com>
…s OpenAPI path Co-authored-by: Cursor <cursoragent@cursor.com>
…options Use loadCatalogResolvedProperties when building Lance table storage options so secret URNs are replaced with plaintext. Co-authored-by: Cursor <cursoragent@cursor.com>
Keep the implementation PR focused; design discussion can stay in the issue. Co-authored-by: Cursor <cursoragent@cursor.com>
lasdf1234
changed the title
…edProperties Replace Map-based load*ResolvedProperties with Catalog/Schema/Fileset returning load*WithResolvedProperties so callers get plaintext via properties() in one load. REST uses optional ?view=resolved on existing load endpoints; remove dedicated /properties + PropertyMapResponse. Wire Spark/Flink/Trino/Lance/Iceberg to the new API. Co-authored-by: Cursor <cursoragent@cursor.com>
…d path - Fix CI: auth expression method signatures, Lance CatalogOperator stub, Trino mock loadCatalogWithResolvedProperties - Emit redacted Load* events for resolved loads (no plaintext in payload) - Wire Java/Python GVFS property merge to resolved load APIs - Add FilesetOperationDispatcher loadWithResolvedProperties e2e test - Align SupportsCatalogs default UOE; document CatalogInfo limits - Update catalog docs for view=resolved vs credential vending Co-authored-by: Cursor <cursoragent@cursor.com>
Stub loadCatalogWithResolvedProperties in Lance mode-parsing unit tests, and backfill the default in-use property on catalog resolved loads so describeNamespace matches BaseCatalog#properties(). Co-authored-by: Cursor <cursoragent@cursor.com>
Code Coverage Report
Files |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Add an explicit API so connectors and REST services can load catalog/schema/fileset with secret URNs resolved to plaintext in
properties(), without changing defaultloadCatalogomit behavior.SecretPropertyUtils.buildResolvedProperties+ dispatcherload*WithResolvedPropertiesreturning Catalog/Schema/Fileset?view=resolvedon existing load endpointsExample:
Why are the changes needed?
Fix: #12457
Entity secrets (epic #12297) persist URNs in properties; default HTTP load omits those keys. Connectors and Lance/IRC need plaintext for connection config without weakening the safe default load path or duplicating credential vending.
Does this PR introduce any user-facing change?
loadCatalogWithResolvedProperties/loadSchemaWithResolvedProperties/loadFilesetWithResolvedPropertiesview=resolvedloadCatalog().properties()semantics unchangedHow was this patch tested?
./gradlew :core:test --tests 'org.apache.gravitino.secret.TestSecretPropertyUtils' -PskipITs?view=resolved