[MINOR] docs(fileset): Restructure the fileset catalog pages around a runnable path by yuqi1129 · Pull Request #12434 · apache/gravitino

yuqi1129 · 2026-08-12T06:47:05Z

What changes were proposed in this pull request?

Restructures the fileset catalog documentation along a single axis: shared reference material lives in exactly one place, while each per-backend page keeps a complete runnable path.

fileset-catalog.md becomes the shared reference and gains a storage-backend table (bundle jar, location scheme, backend properties) in place of a bare list of links.
The five per-backend pages are generated from one template so they stay strictly parallel. Each merges the three former property tables (catalog, Java client, Python client) into one — which also makes the hyphen-vs-underscore difference visible at a glance — and drops the deprecated filesystem-providers / default-filesystem-provider rows that were listed as required.
Each page uses one consistent set of names end to end, so the examples can be run in order.

Corrections made along the way:

A missing comma in the Python catalog properties dict.
A schema example that loaded hive_catalog instead of the catalog it had just created.
SparkSession.builder chains with no line continuations, which could not be run as written.
A stray quote in the Python GVFS example.
hadoop dfs replaced with hadoop fs.
The Java client section linking to the Python client's anchor.
Four dead anchors left by the earlier manage-* page split: #catalog-operations, #schema-operations, #placeholder and #manage-filesets-across-multiple-clusters.
Tencent Cloud COS missing from the index page and from the GVFS page.

This is an alternative take on #12399. That PR removes the duplication between the five per-backend pages, which is real — 116 lines are byte-identical across all five of them. The difference here is where the removed procedure lands: this PR keeps it on each backend page rather than consolidating it away, on the view that a page a newcomer follows to get something running is worth duplicating, while property semantics and inheritance rules are not.

Why are the changes needed?

The per-backend pages could not be followed end to end. Names changed between steps, several code samples did not parse, and deprecated properties were marked as required. Anyone new to filesets had to reconstruct a working setup from three pages.

Does this PR introduce any user-facing change?

Documentation only. No property keys or APIs change.

How was this patch tested?

Checked code-fence, Tabs/TabItem and table-column balance across every touched page.
Verified that every intra-doc link and anchor into and out of the touched pages resolves.
Compiled every Python snippet on the touched pages.

… runnable path Split the fileset documentation along one axis: shared reference material lives in exactly one place, while the per-backend pages each keep a complete runnable path so a newcomer never has to assemble a working setup from three pages. - fileset-catalog.md is now the shared reference and carries a storage-backend table (bundle jar, location scheme, backend properties) instead of a bare list of links. - The five per-backend pages are regenerated from one template so they stay strictly parallel. Each merges the three former property tables (catalog, Java client, Python client) into one, which also makes the hyphen-vs-underscore difference visible at a glance, and drops the deprecated filesystem-providers / default-filesystem-provider rows that were listed as required. - Each page now uses one consistent set of names end to end, so the examples can be run in order rather than needing rewrites between steps. Fixes along the way: - Python catalog properties dict was missing a comma. - The schema example loaded "hive_catalog" instead of the catalog it just created. - SparkSession.builder chains had no line continuations and could not be run. - A stray quote in the Python GVFS example. - "hadoop dfs" replaced with "hadoop fs". - The Java client section linked to the Python client's anchor. - Dead anchors into manage-fileset-metadata-using-gravitino.md left behind by the earlier manage-* page split (catalog-operations, schema-operations, placeholder, manage-filesets-across-multiple-clusters). - Tencent Cloud COS was missing from the index page and from the GVFS page.

Copilot

Pull request overview

This PR restructures the fileset catalog documentation to separate shared reference material (properties/semantics/inheritance) from per-backend pages that each provide a complete runnable “start-to-finish” path (bundle jar → catalog/schema/fileset → access via GVFS clients/Spark/Hadoop/Python/pandas), and updates cross-doc links/anchors accordingly.

Changes:

Reworked fileset-catalog.md into the shared reference page and added a backend summary table (bundle jar, URI scheme, backend properties).
Rebuilt each cloud-backend page (S3/GCS/ADLS/OSS/COS) into a consistent runnable template with merged property tables and aligned examples.
Updated GVFS and related docs to reference the new per-backend property sections and fixed several anchors/wording issues.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
docs/how-to-use-gvfs.md	Updates GVFS docs to point to per-backend property sections and refreshes multi-cluster wording.
docs/hive-catalog-with-cloud-storage.md	Fixes catalog-operations link target to the correct management doc.
docs/fileset-catalog.md	Converts to shared reference and adds a backend summary table plus updated ops/placeholder links.
docs/fileset-catalog-with-s3.md	Rewrites S3 page as an end-to-end runnable guide with unified property table and updated examples.
docs/fileset-catalog-with-oss.md	Rewrites OSS page as an end-to-end runnable guide with unified property table and updated examples.
docs/fileset-catalog-with-gcs.md	Rewrites GCS page as an end-to-end runnable guide with unified property table and updated examples.
docs/fileset-catalog-with-cos.md	Rewrites COS page as an end-to-end runnable guide with unified property table and updated examples.
docs/fileset-catalog-with-adls.md	Rewrites ADLS page as an end-to-end runnable guide with unified property table and updated examples.
docs/fileset-catalog-index.md	Updates cloud-backend index section to include COS and emphasize runnable per-backend pages.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…ly come from

…e backend pages

github-actions · 2026-08-12T10:14:56Z

Code Coverage Report

Overall Project	68.75% `+0.2%`	🟢
Files changed	86.97%	🟢

Module	Coverage
aliyun	1.72%	🔴
api	49.31%	🟢
authorization-common	85.96%	🟢
aws	42.04%	🟢
azure	2.47%	🔴
catalog-common	9.92%	🔴
catalog-fileset	79.77%	🟢
catalog-glue	68.95%	🟢
catalog-hive	82.96%	🟢
catalog-jdbc-common	45.7%	🟢
catalog-jdbc-doris	81.8%	🟢
catalog-jdbc-mysql	79.33%	🟢
catalog-jdbc-postgresql	83.39%	🟢
catalog-jdbc-starrocks	79.16%	🟢
catalog-kafka	77.01%	🟢
catalog-lakehouse-generic	59.18%	🟢
catalog-lakehouse-hudi	79.1%	🟢
catalog-lakehouse-iceberg	85.86%	🟢
catalog-lakehouse-paimon	84.23%	🟢
catalog-model	77.72%	🟢
cli	44.48%	🟢
client-java	78.56%	🟢
common	52.75%	🟢
core	83.63% `+0.25%`	🟢
filesystem-hadoop3	77.28%	🟢
flink	0.0%	🔴
flink-common	48.68%	🟢
flink-runtime	0.0%	🔴
gcp	14.12%	🔴
hadoop-auth	68.0%	🟢
hadoop-common	12.7%	🔴
hive-metastore-common	53.4%	🟢
iceberg-aliyun-bundle	0.0%	🔴
iceberg-common	64.75%	🟢
iceberg-rest-server	75.1%	🟢
idp-basic	86.02%	🟢
integration-test-common	0.0%	🔴
jobs	62.92%	🟢
lance-common	31.75%	🔴
lance-rest-server	63.47%	🟢
lineage	53.02%	🟢
optimizer	83.24%	🟢
optimizer-api	21.95%	🔴
server	87.25%	🟢
server-common	79.41% `-1.1%`	🟢
spark	28.57%	🔴
spark-common	45.89%	🟢
tencent	69.84%	🟢
trino-connector	40.29%	🟢

Files

Module	File	Coverage
core	EntityCacheChangeLogListener.java	100.0%	🟢
	Configs.java	98.47%	🟢
	EntityChangeLogPoller.java	94.06%	🟢
	EntityChangeLogCleaner.java	91.53%	🟢
	CatalogChangeLogListener.java	88.37%	🟢
	RelationalEntityStore.java	77.29%	🟢
	EntityChangeLogListener.java	0.0%	🔴
server-common	JcasbinChangeListener.java	55.91%	🔴

jerryshao · 2026-08-17T02:12:18Z

@diqiu50 @mchades please take a review. We need to get this merged today.

Copilot

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (19)

docs/fileset-catalog-with-cos.md:460

The Python GVFS implementation has no cosn storage type or COS storage handler (gvfs_storage_handler.py only registers HDFS, local, GCS, S3A, OSS, and ABS). This example therefore fails when the resolved fileset location is cosn://...; pandas and the later Python credential-vending example fail for the same reason. Keep the unsupported-data-plane note instead, or add COS support to the Python client before documenting these paths.

fs = gvfs.GravitinoVirtualFileSystem(server_uri="http://localhost:8090",
                                     metalake_name="metalake",
                                     options=options)
fs.ls("gvfs://fileset/cos_catalog/cos_schema/example_fileset/")

docs/how-to-use-gvfs.md:445

This Python-client note refers to the Java/Hadoop key spelling. The property immediately above is fs_path_config_<name>, so users following this sentence would supply an unrecognized option.

of the different clusters, using the `fs.path.config.<name>` properties described above.

docs/fileset-catalog-with-cos.md:494

This link has an empty fragment and only navigates to the top of credential-vending.md; that document currently has no COS section. Point to a real COS credential section after adding one, or remove the backend-specific link.

[COS credentials](./security/credential-vending.md#) for the properties

docs/fileset-catalog-with-s3.md:537

The vending configuration creates s3_catalog_with_vending, but this access example still targets s3_catalog. As a result it exercises the original catalog rather than the configured s3-token provider (and the new catalog has no schema/fileset). Make the vending section configure and access the same complete catalog path.

Path filesetPath = new Path("gvfs://fileset/s3_catalog/s3_schema/example_fileset/new_dir");

docs/fileset-catalog-with-oss.md:529

The vending configuration creates oss_catalog_with_vending, but this access example still targets oss_catalog. It therefore exercises the original catalog rather than the configured oss-token provider, while the newly created catalog has no schema/fileset. Make the vending section configure and access one complete catalog path.

Path filesetPath = new Path("gvfs://fileset/oss_catalog/oss_schema/example_fileset/new_dir");

docs/fileset-catalog-with-gcs.md:495

The vending configuration creates gcs_catalog_with_vending, but this access example still targets gcs_catalog; the configured catalog also has no schema or fileset. Thus the runnable sequence never accesses the catalog it just configured. Make the vending section configure and access the same complete catalog path.

Path filesetPath = new Path("gvfs://fileset/gcs_catalog/gcs_schema/example_fileset/new_dir");

docs/fileset-catalog-with-adls.md:523

The vending configuration creates adls_catalog_with_vending, but this access example still targets adls_catalog. It therefore does not exercise the configured adls-token provider, and the new catalog has no schema/fileset. Make the vending section configure and access one complete catalog path.

Path filesetPath = new Path("gvfs://fileset/adls_catalog/adls_schema/example_fileset/new_dir");

docs/fileset-catalog-with-cos.md:536

The vending configuration creates cos_catalog_with_vending, but this access example still targets cos_catalog. It therefore does not exercise the configured cos-secret-key provider, and the new catalog has no schema/fileset. Make the vending section configure and access one complete catalog path.

Path filesetPath = new Path("gvfs://fileset/cos_catalog/cos_schema/example_fileset/new_dir");

docs/fileset-catalog-with-s3.md:479

Spark does not create a file named part-00000.csv here; DataFrameWriter.csv emits a generated name such as part-00000-<uuid>-c000.csv. This read therefore fails after running the preceding Spark example. Discover the generated part-*.csv file (for example via fsspec/GVFS globbing) before passing it to pandas.

ds = pd.read_csv("gvfs://fileset/s3_catalog/s3_schema/example_fileset/people/part-00000.csv",
                 storage_options=storage_options)

docs/fileset-catalog-with-oss.md:471

Spark does not create a file named part-00000.csv here; DataFrameWriter.csv emits a generated name such as part-00000-<uuid>-c000.csv. This read therefore fails after running the preceding Spark example. Discover the generated part-*.csv file before passing it to pandas.

ds = pd.read_csv("gvfs://fileset/oss_catalog/oss_schema/example_fileset/people/part-00000.csv",
                 storage_options=storage_options)

docs/fileset-catalog-with-gcs.md:450

Spark does not create a file named part-00000.csv here; DataFrameWriter.csv emits a generated name such as part-00000-<uuid>-c000.csv. This read therefore fails after running the preceding Spark example. Discover the generated part-*.csv file before passing it to pandas.

ds = pd.read_csv("gvfs://fileset/gcs_catalog/gcs_schema/example_fileset/people/part-00000.csv",
                 storage_options=storage_options)

docs/fileset-catalog-with-adls.md:464

Spark does not create a file named part-00000.csv here; DataFrameWriter.csv emits a generated name such as part-00000-<uuid>-c000.csv. This read therefore fails after running the preceding Spark example. Discover the generated part-*.csv file before passing it to pandas.

ds = pd.read_csv("gvfs://fileset/adls_catalog/adls_schema/example_fileset/people/part-00000.csv",
                 storage_options=storage_options)

docs/fileset-catalog-with-cos.md:482

Even once COS support exists in Python GVFS, Spark does not create part-00000.csv; it emits a generated name such as part-00000-<uuid>-c000.csv. This read would therefore still fail after the preceding Spark example. Discover the generated part-*.csv file before passing it to pandas.

ds = pd.read_csv("gvfs://fileset/cos_catalog/cos_schema/example_fileset/people/part-00000.csv",
                 storage_options=storage_options)

docs/fileset-catalog-with-s3.md:247

This overstates the jar requirement: the Python GVFS and pandas clients below are fsspec-based and do not use Java jars. Limit this statement to Java/Hadoop-based clients so Python users are not instructed to install irrelevant artifacts.

Every client needs `gravitino-filesystem-hadoop3-runtime`, which is published on Maven Central,
plus the Amazon S3 filesystem implementation. Only the latter differs by environment:

docs/fileset-catalog-with-oss.md:238

This overstates the jar requirement: the Python GVFS and pandas clients below are fsspec-based and do not use Java jars. Limit this statement to Java/Hadoop-based clients so Python users are not instructed to install irrelevant artifacts.

Every client needs `gravitino-filesystem-hadoop3-runtime`, which is published on Maven Central,
plus the Alibaba Cloud OSS filesystem implementation. Only the latter differs by environment:

docs/fileset-catalog-with-gcs.md:235

This overstates the jar requirement: the Python GVFS and pandas clients below are fsspec-based and do not use Java jars. Limit this statement to Java/Hadoop-based clients so Python users are not instructed to install irrelevant artifacts.

Every client needs `gravitino-filesystem-hadoop3-runtime`, which is published on Maven Central,
plus the Google Cloud Storage filesystem implementation. Only the latter differs by environment:

docs/fileset-catalog-with-adls.md:239

This overstates the jar requirement: the Python GVFS and pandas clients below are fsspec-based and do not use Java jars. Limit this statement to Java/Hadoop-based clients so Python users are not instructed to install irrelevant artifacts.

Every client needs `gravitino-filesystem-hadoop3-runtime`, which is published on Maven Central,
plus the Azure Data Lake Storage filesystem implementation. Only the latter differs by environment:

docs/fileset-catalog-with-cos.md:245

This overstates the jar requirement: Python GVFS/pandas clients do not use Java jars (and currently do not support COS at all). Limit this statement to Java/Hadoop-based clients.

Every client needs `gravitino-filesystem-hadoop3-runtime`, which is published on Maven Central,
plus the Tencent Cloud COS filesystem implementation. Only the latter differs by environment:

docs/fileset-catalog-with-s3.md:495

This list omits the supported aws-irsa provider that the consolidated property table now lists and credential-vending.md documents. Include it here so the backend page does not contradict its own reference table.

The supported providers are `s3-token`, which vends a short-lived STS token, and
`s3-secret-key`, which vends the static access key configured on the catalog. The example below uses
`s3-token`.

jerryshao

Two issues found on the COS page while reviewing the runnable-path restructuring.

jerryshao · 2026-08-17T02:29:55Z

 ```

-### Access the Fileset with the GVFS Python Client / Pandas
+### GVFS Python client


This section removes the previous (accurate) warning that the GVFS Python client does not ship a COS storage handler, and replaces it with a working-looking fs.ls("gvfs://fileset/cos_catalog/...") / pandas.read_csv("gvfs://...") example.

I checked clients/client-python/gravitino/filesystem/gvfs_storage_handler.py: StorageType only defines HDFS, LOCAL, GCS, S3A, OSS, ABS — there's no COS/cosn entry, and get_storage_handler_by_path raises GravitinoRuntimeException("Storage type doesn't support now. Path:...") for any unmatched prefix. Following this example as written will throw at runtime, not produce the shown output. Since the goal of this PR is a page that can be run end to end, this section looks like it should keep the old "not supported" note (or the Python client actually needs COS support added first).

Restore the word that COS does not support Python GVFS/pandas.

jerryshao · 2026-08-17T02:29:55Z

+With credential vending the catalog holds the Tencent Cloud COS credentials and the Gravitino server hands
+out a credential per request, so clients never hold cloud keys of their own. See
+[Credential Vending](./security/credential-vending.md) for the general mechanism and
+[COS credentials](./security/credential-vending.md#) for the properties


[COS credentials](./security/credential-vending.md#) — the anchor is empty (#). docs/security/credential-vending.md has sections for S3, OSS, ADLS and GCS, but no COS section at all, so there's no target this could point to yet. Either add a COS section there, or drop this link/leave a TODO instead of a dead anchor.

Removed the dead link.

yuqi1129 · 2026-08-17T02:39:04Z

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

Suppressed comments (19)

pyhton GVFS does not support COS
Yes, it's indeed a problem
Yes, the link is a dead
Yes, please correct it
yes, please correct it
yes, please correct it
yes, please correct it
yes, please correct it
It's just an example
It's just an example
It's just an example
It's just an example
It's just an example
Yeah, you are right
Yeah, you are right
Yeah, you are right
Yeah, you are right
Yeah, you are right
Yeas, aws-irsa, but it only support file-based only.

I will modify them as in the replies above.

Copilot AI lite review requested due to automatic review settings August 12, 2026 06:47

Copilot started reviewing on behalf of yuqi1129 August 12, 2026 06:47 View session

Copilot AI reviewed Aug 12, 2026

View reviewed changes

Comment thread docs/how-to-use-gvfs.md Outdated

Comment thread docs/how-to-use-gvfs.md

Comment thread docs/fileset-catalog-with-cos.md Outdated

yuqi1129 added 4 commits August 12, 2026 15:06

[MINOR] docs(fileset): Align the markdown table columns

c678053

[MINOR] docs(fileset): Attribute the client jars to where they actual…

ad80a51

…ly come from

[MINOR] docs(fileset): Restore the reference material trimmed from th…

0219e52

…e backend pages

[MINOR] docs(fileset): Keep the original page titles

114cd40

yuqi1129 mentioned this pull request Aug 12, 2026

Docs: consolidate the fileset pages #12399

Open

yuqi1129 added the branch-1.3 Automatically cherry-pick commit to branch-1.3 label Aug 13, 2026

yuqi1129 self-assigned this Aug 13, 2026

yuqi1129 requested a review from diqiu50 August 13, 2026 12:07

jerryshao requested a balanced review from Copilot August 17, 2026 02:09

Copilot started reviewing on behalf of jerryshao August 17, 2026 02:10 View session

jerryshao requested a review from mchades August 17, 2026 02:12

Copilot AI reviewed Aug 17, 2026

View reviewed changes

jerryshao reviewed Aug 17, 2026

View reviewed changes

[MINOR] docs(fileset): Address review feedback

d21b321

Navigation Menu

Uh oh!

[MINOR] docs(fileset): Restructure the fileset catalog pages around a runnable path - #12434

[MINOR] docs(fileset): Restructure the fileset catalog pages around a runnable path#12434
yuqi1129 wants to merge 6 commits into
apache:mainfrom
yuqi1129:docs/fileset-docs-runnable-consolidation

yuqi1129 commented Aug 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 12, 2026 •

edited

Loading

Uh oh!

jerryshao commented Aug 17, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

jerryshao left a comment

Uh oh!

jerryshao Aug 17, 2026

Uh oh!

yuqi1129 Aug 17, 2026

Uh oh!

jerryshao Aug 17, 2026

Uh oh!

yuqi1129 Aug 17, 2026

Uh oh!

yuqi1129 commented Aug 17, 2026

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

yuqi1129 commented Aug 12, 2026

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Aug 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Coverage Report

Uh oh!

jerryshao commented Aug 17, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

jerryshao left a comment

Choose a reason for hiding this comment

Uh oh!

jerryshao Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

yuqi1129 Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

jerryshao Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

yuqi1129 Aug 17, 2026

Choose a reason for hiding this comment

Uh oh!

yuqi1129 commented Aug 17, 2026

Pull request overview

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions Bot commented Aug 12, 2026 •

edited

Loading