Add BigLake configuration validation for catalog creation and updates

[davidchaava]

Summary

Adds fail-fast validation for BigLake Iceberg REST external catalog configurations during catalog creation and update.

Validation is scoped to GCP-authenticated Iceberg REST catalogs that target the official BigLake endpoint, https://biglake.googleapis.com/iceberg/v1/restcatalog. Other GCP-authenticated Iceberg REST catalogs retain their existing behavior.

Changes

• Validates the official BigLake endpoint and supported remote catalog identifiers, including bl://projects/<project-number>/catalogs/<catalog-id>.
• Requires a valid header.x-goog-user-project quota-project header.
• Preserves compatibility with existing CLI-created catalogs that store the quota-project header in catalog properties.
• Updates the CLI to place header.x-goog-user-project in connectionConfigInfo.properties for new GCP Iceberg REST catalogs.
• Validates GCS storage locations and the GCS service account when credential vending is enabled.
• Applies the same validation rules to catalog creation and update.
• Adds unit, management-service, and CLI regression coverage.

Validation

• ./gradlew :polaris-runtime-service:test --tests org.apache.polaris.service.admin.BigLakeCatalogValidatorTest
• pytest tests/test_catalogs_command.py::TestCatalogsCommand::test_external_catalog_gcp -q
• Live Polaris to BigLake namespace-listing smoke test using GCP Application Default Credentials.

Related to #5195

[davidchaava]

Hi @sungwy ,
Could you please take a look at this PR when you have a chance?

We would especially appreciate your feedback from the GCP/BigLake compatibility perspective.

Thank you.

[sungwy]

This only accepts the user-managed SA form, so it rejects the Compute-default (…-compute@developer.gserviceaccount.com) and App-Engine-default (@appspot.gserviceaccount.com) SAs which are both valid impersonation targets. GCP's generateAccessToken accepts any SA email, and Polaris' own GcpCredentialsStorageIntegration only checks presence, so this check is stricter than both. Could we accept any valid SA email, or, if the restriction's intentional, reword the error (those defaults are Google SA emails)? WDYT

[davidchaava]

Good catch! That restriction wasn’t intentional. I’ve relaxed the check so it no longer assumes the user-managed service account format, and added coverage for Compute Engine and App Engine default service accounts. GCP remains the source of truth for whether the account exists and can be impersonated.
Thanks!