[#12342] improvement(core): add OCC for namespace hierarchy - #12350
[#12342] improvement(core): add OCC for namespace hierarchy#12350yuqi1129 wants to merge 12 commits into
Conversation
Code Coverage Report
Files
|
7955cfc to
472a791
Compare
There was a problem hiding this comment.
🟡 Not ready to approve
Service-level tests are missing for some new OptimisticLockException conflict paths (notably catalog/schema alter conflicts and metalake stale delete), leaving the new OCC contract partially unverified end-to-end.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.
Pull request overview
This PR introduces optimistic concurrency control (OCC) for alter and delete operations across the namespace hierarchy (metalake, catalog, schema) in the relational metadata store, using version CAS (compare-and-swap) and reporting stale operations via OptimisticLockException.
Changes:
- Increment
current_version/last_versionon every successful alter and enforce CAS updates viaWHERE ... current_version = ?. - Enforce CAS deletes of the root entity (by id + expected current version) before running cascade cleanup; schemas CAS-delete the requested schema then authoritatively delete descendants.
- Add/extend tests to validate version increments and stale update/delete row-count behavior.
File summaries
| File | Description |
|---|---|
| core/src/main/java/org/apache/gravitino/storage/relational/utils/POConverters.java | Always increments entity versions on alter PO conversions. |
| core/src/main/java/org/apache/gravitino/storage/relational/service/MetalakeMetaService.java | Throws OptimisticLockException on stale alter/delete; CAS-delete metalake before cascade cleanup. |
| core/src/main/java/org/apache/gravitino/storage/relational/service/CatalogMetaService.java | Throws OptimisticLockException on stale alter/delete; CAS-delete catalog before cascade cleanup. |
| core/src/main/java/org/apache/gravitino/storage/relational/service/SchemaMetaService.java | Throws OptimisticLockException on stale alter/delete; CAS-delete requested schema before descendant cleanup. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/MetalakeMetaSQLProviderFactory.java | Updates provider factory to pass expected version for metalake soft deletes. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/MetalakeMetaMapper.java | Updates mapper signature to soft-delete metalake by id + expected version. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/MetalakeMetaBaseSQLProvider.java | Narrows update CAS predicate to current_version and adds version predicate to soft delete. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/postgresql/MetalakeMetaPostgreSQLProvider.java | PostgreSQL-specific CAS soft delete by version and narrowed update predicate. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/CatalogMetaSQLProviderFactory.java | Updates provider factory to pass expected version for catalog soft deletes. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/CatalogMetaMapper.java | Updates mapper signature to soft-delete catalog by id + expected version. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/CatalogMetaBaseSQLProvider.java | Narrows update CAS predicate to current_version and adds version predicate to soft delete. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/postgresql/CatalogMetaPostgreSQLProvider.java | PostgreSQL-specific CAS soft delete by version and narrowed update predicate. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/SchemaMetaSQLProviderFactory.java | Adds provider factory method for schema soft delete by id + expected version. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/SchemaMetaMapper.java | Adds mapper method for schema soft delete by id + expected version. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/base/SchemaMetaBaseSQLProvider.java | Narrows schema update CAS predicate to current_version and adds versioned soft delete SQL. |
| core/src/main/java/org/apache/gravitino/storage/relational/mapper/provider/postgresql/SchemaMetaPostgreSQLProvider.java | PostgreSQL-specific schema soft delete by id + expected version and narrowed update predicate. |
| core/src/test/java/org/apache/gravitino/storage/relational/utils/TestPOConverters.java | Asserts version increments on PO update conversion. |
| core/src/test/java/org/apache/gravitino/storage/relational/service/TestMetalakeMetaService.java | Adds mapper-level stale update/delete tests and a metalake alter conflict test for OptimisticLockException. |
| core/src/test/java/org/apache/gravitino/storage/relational/service/TestCatalogMetaService.java | Adds mapper-level stale update/delete tests. |
| core/src/test/java/org/apache/gravitino/storage/relational/service/TestSchemaMetaService.java | Adds mapper-level stale update/delete tests (including versioned schema delete). |
Review details
- Files reviewed: 20/20 changed files
- Comments generated: 4
- Review effort level: Lite
We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.
|
Thanks for the suggestion. I understand it as extracting a common OCC SQL template for update/fence/soft-delete operations, with each provider supplying the table name, ID column, and MyBatis parameter names. After reviewing these statements, I think a generic SQL template would have limited benefit here. Besides the table and ID column, the MyBatis parameter paths, batch collection names, entity-specific SET clauses, and database-specific deleted_at expressions also differ. Passing these as string parameters would hide the final SQL, reduce readability, and move some errors to runtime. I would prefer to keep the entity-specific SQL explicit. I agree that exact duplication should be removed. A narrower approach would be to reuse DatabaseTimeSQL for database-time expressions and let PostgreSQL providers inherit SQL that is identical to the base provider, keeping overrides only where the SQL dialect actually differs. If more entities later converge on an identical OCC SQL shape, we can revisit a narrowly scoped helper. Would this narrower cleanup address your concern, or do you have a specific interface/template design in mind? |
I prefer think further. We can give more constraints about table create SQL standard. Some common column must has the fixed name. Some services must have some specific interfaces. For example, we have delete_at column in every table, we have a delete interface. Every storage service should implement it. I think this is possible. We would better have a framework to handle this issue. We should think more about how to iterate our storage framework. Now, we are using AI to generate the code. It seems not bring much burden. |
Thanks for the suggestion. I understand it as extracting a common OCC SQL template for update/fence/soft-delete operations, with each provider supplying the table name, ID column, and MyBatis parameter names. After reviewing these statements, I think a generic SQL template would have limited benefit here. Besides the table and ID column, the MyBatis parameter paths, batch collection names, entity-specific SET clauses, and database-specific deleted_at expressions also differ. Passing these as string parameters would hide the final SQL, reduce readability, and move some errors to runtime. I would prefer to keep the entity-specific SQL explicit. I agree that exact duplication should be removed. A narrower approach would be to reuse DatabaseTimeSQL for database-time expressions and let PostgreSQL providers inherit SQL that is identical to the base provider, keeping overrides only where the SQL dialect actually differs. If more entities later converge on an identical OCC SQL shape, we can revisit a narrowly scoped helper. Would this narrower cleanup address your concern, or do you have a specific interface/template design in mind? I prefer think further. We can give more constraints about table create SQL standard. Some common column must has the fixed name. Some services must have some specific interfaces. For example, we have delete_at column in every table, we have a delete interface. Every storage service should implement it. I think this is possible. We would better have a framework to handle this issue. We should think more about how to iterate our storage framework. Now, we are using AI to generate the code. It seems not bring much burden. I suggest we use another PR to discuss the problem and leave your ideas in as much detail as possible. AI hinted to me that
If you insist on this point, please just create an issue that is targeted at this problem. I think we can do a pure refactor without this PR. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 33 out of 33 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
core/src/test/java/org/apache/gravitino/storage/relational/service/TestSchemaMetaService.java:198
- This concurrency test calls the lock helper directly, so it still passes if a production entity write forgets to acquire the fence—as the uncovered function/table/model-version update paths currently do. Exercise an actual child service operation (for example
insertTable) while the schema delete is paused, and add stale-update/version-write cases so the transactional wiring, rollback, and orphan prevention are verified end to end.
| void lockSchemaForEntityWrite( | ||
| NameIdentifier entityIdentifier, | ||
| Long observedSchemaId, | ||
| Long observedCatalogId, | ||
| Long observedMetalakeId) { |
There was a problem hiding this comment.
Thanks for catching this. This is a valid cross-instance race, but it predates this PR: the existing TreeLock only serializes schema and child operations within one process, and the relational storage layer has never had a complete schema fence for child update and version-write paths.
A complete fix requires auditing table, view, fileset, topic, function, model, and model-version writes; locking both schemas for table moves; failing zero-row child updates inside the transaction; and adding deterministic concurrency coverage for H2, MySQL, and PostgreSQL. That is a substantial change beyond #12342, which is scoped to OCC for the metalake/catalog/schema namespace hierarchy.
I filed #12406 to track the complete schema-to-child write protocol and will address it in a follow-up PR instead of expanding this already large PR.
There was a problem hiding this comment.
The premise is right: lockSchemaForEntityWrite is taken by the child inserts and by the cross-schema table move, not by same-schema updates. I looked into each path you named and confirmed the concrete defects:
TableMetaService.updateTablewrites the newtable_version_inforow outside theupdateResult > 0guard, so a lost CAS still commits an active version row. Since the legacy cleanup only removes rows withdeleted_at > 0, that row is never collected.FunctionMetaService.updateFunctiondiscards the meta update result entirely, so a concurrent delete leaves an active orphan version row and returns success — a silent lost update.insertModelVersiontakes no schema fence at all.
All three are pre-existing on main and sit outside this PR's scope (metalake/catalog/schema OCC), and this PR is already large, so I'm going to fix them in a follow-up issue rather than grow the diff here.
I don't plan to take the broader suggestion of acquiring the shared schema lock at the start of every child write transaction. Same-schema updates are already mutually exclusive with a cascade delete through the per-row CAS (current_version + deleted_at = 0) — the defects above are about what the losing transaction commits, not about missing exclusion. Guarding the losing path (and adding the missing fence to insertModelVersion) is the smaller and more targeted fix.
|
@roryqi @jerryshao |
| .build(); | ||
| try { | ||
| store().put(schemaEntity, true /* overwrite */); | ||
| store().put(schemaEntity, false /* overwrite */); |
There was a problem hiding this comment.
What is the purpose of changing this to false?
There was a problem hiding this comment.
The interface is an explicit create operation, and we need to change it to false to avoid concurrent insertion of data into the database.
| private OptimisticLockException optimisticLockException(NameIdentifier identifier) { | ||
| return new OptimisticLockException( | ||
| "The catalog %s was modified concurrently; retry the operation", identifier); | ||
| } |
There was a problem hiding this comment.
We can define this method in the helper method.
| SessionUtils.getWithoutCommit( | ||
| SchemaMetaMapper.class, mapper -> mapper.softDeleteSchemaMetasWithVersion(schemaPOs)); | ||
| if (deleted != schemaPOs.size()) { | ||
| throw new OptimisticLockException( |
There was a problem hiding this comment.
It is better to call a helper method if defined, rather than calling a class constructuor.
| private OptimisticLockException optimisticLockException(NameIdentifier identifier) { | ||
| return new OptimisticLockException( | ||
| "The metalake %s was modified concurrently; retry the operation", identifier); | ||
| } |
There was a problem hiding this comment.
You should fix lots of duplications.
There was a problem hiding this comment.
Strictly, it's not duplicated code and is only used for the metalake. Anyway, we need to refine it.
| int updated = | ||
| SessionUtils.getWithoutCommit( | ||
| SchemaMetaMapper.class, | ||
| mapper -> | ||
| ops.updatePO( | ||
| mapper, | ||
| POConverters.updateSchemaPOWithVersion(oldSchemaPO, newEntity), | ||
| oldSchemaPO)); |
There was a problem hiding this comment.
This one and L477, L496, and CatalogMetaService:253/429 are the same problem.
The observation is accurate: these fences validate the immediate parent row by id + name, and an ancestor rename does not change that row.
This is intentional. In Gravitino, entity identity is the ID, and rename preserves identity — the request resolved metalakeId/catalogId, and after the rename those ids still denote the same entity, so the write lands on exactly the entity the caller resolved, just reachable under a new name. Nothing is orphaned and no update is lost. The anomaly this PR is closing is the different one where the parent is deleted (or replaced by a same-named entity with a new id) and a child write would survive it — that case is fenced by the id + existence check.
There was a problem hiding this comment.
The observation is accurate — this CAS locks and validates the schema row only, so neither a catalog nor a metalake rename is fenced here. This is intentional.
Rename in Gravitino is identity-preserving: it changes only the name column of the renamed row, and the ids this request already resolved (metalakeId/catalogId/schemaId) keep denoting the same entities. So the write lands on exactly the entity the caller resolved, just one that is now reachable under a different path — nothing is orphaned and no update is lost. The anomaly this PR closes is the other one: the parent is deleted, or replaced by a same-named entity with a fresh id, and a stale child write survives it. That case is caught by the id + existence checks these fences already do.
Making a stale fully qualified path fail would require shared locks on the whole ancestor chain, root-to-leaf, on every write in the hierarchy — a metalake row read on every catalog/schema/table/model write. That is a much larger change than this PR, and it buys strictness rather than integrity, so I'd rather track it separately if we decide we want strict path semantics.
| private void lockCatalogForSchemaDelete(NameIdentifier identifier, SchemaPO observedSchemaPO) { | ||
| CatalogPO currentCatalogPO = | ||
| SessionUtils.getWithoutCommit( | ||
| CatalogMetaMapper.class, | ||
| mapper -> mapper.selectCatalogMetaByIdForUpdate(observedSchemaPO.getCatalogId())); |
There was a problem hiding this comment.
Correct — the delete fence takes the catalog row and validates its name and metalakeId, neither of which changes when the metalake is renamed. This is intentional.
Rename in Gravitino is identity-preserving: it changes only the name column of the renamed row, and the ids this request already resolved (metalakeId/catalogId/schemaId) keep denoting the same entities. So the write lands on exactly the entity the caller resolved, just one that is now reachable under a different path — nothing is orphaned and no update is lost. The anomaly this PR closes is the other one: the parent is deleted, or replaced by a same-named entity with a fresh id, and a stale child write survives it. That case is caught by the id + existence checks these fences already do.
Making a stale fully qualified path fail would require shared locks on the whole ancestor chain, root-to-leaf, on every write in the hierarchy — a metalake row read on every catalog/schema/table/model write. That is a much larger change than this PR, and it buys strictness rather than integrity, so I'd rather track it separately if we decide we want strict path semantics.
| private void lockCatalogForSchemaCreate( | ||
| CatalogPO observedCatalogPO, boolean createsImplicitAncestors) { | ||
| CatalogPO currentCatalogPO = | ||
| SessionUtils.getWithoutCommit( | ||
| CatalogMetaMapper.class, |
There was a problem hiding this comment.
Correct — the create fence revalidates the catalog row, and a metalake rename leaves that row and its metalakeId untouched. This is intentional.
Rename in Gravitino is identity-preserving: it changes only the name column of the renamed row, and the ids this request already resolved (metalakeId/catalogId/schemaId) keep denoting the same entities. So the write lands on exactly the entity the caller resolved, just one that is now reachable under a different path — nothing is orphaned and no update is lost. The anomaly this PR closes is the other one: the parent is deleted, or replaced by a same-named entity with a fresh id, and a stale child write survives it. That case is caught by the id + existence checks these fences already do.
Making a stale fully qualified path fail would require shared locks on the whole ancestor chain, root-to-leaf, on every write in the hierarchy — a metalake row read on every catalog/schema/table/model write. That is a much larger change than this PR, and it buys strictness rather than integrity, so I'd rather track it separately if we decide we want strict path semantics.
| int updated = | ||
| SessionUtils.getWithoutCommit( | ||
| CatalogMetaMapper.class, | ||
| mapper -> | ||
| mapper.updateCatalogMeta( | ||
| POConverters.updateCatalogPOWithVersion( | ||
| oldCatalogPO, newEntity, oldCatalogPO.getMetalakeId()), | ||
| oldCatalogPO)); |
There was a problem hiding this comment.
Correct — this CAS validates the catalog row and its version only, and a metalake rename changes neither. This is intentional.
Rename in Gravitino is identity-preserving: it changes only the name column of the renamed row, and the ids this request already resolved (metalakeId/catalogId/schemaId) keep denoting the same entities. So the write lands on exactly the entity the caller resolved, just one that is now reachable under a different path — nothing is orphaned and no update is lost. The anomaly this PR closes is the other one: the parent is deleted, or replaced by a same-named entity with a fresh id, and a stale child write survives it. That case is caught by the id + existence checks these fences already do.
Making a stale fully qualified path fail would require shared locks on the whole ancestor chain, root-to-leaf, on every write in the hierarchy — a metalake row read on every catalog/schema/table/model write. That is a much larger change than this PR, and it buys strictness rather than integrity, so I'd rather track it separately if we decide we want strict path semantics.
| private void deleteCatalogWithVersion(NameIdentifier identifier, CatalogPO observedCatalogPO) { | ||
| int deleted = | ||
| SessionUtils.getWithoutCommit( | ||
| CatalogMetaMapper.class, | ||
| mapper -> | ||
| mapper.softDeleteCatalogMetasByCatalogId( | ||
| observedCatalogPO.getCatalogId(), observedCatalogPO.getCurrentVersion())); |
There was a problem hiding this comment.
Correct — the catalog delete CAS checks the catalog row and its version, which a metalake rename leaves unchanged. This is intentional.
Rename in Gravitino is identity-preserving: it changes only the name column of the renamed row, and the ids this request already resolved (metalakeId/catalogId/schemaId) keep denoting the same entities. So the write lands on exactly the entity the caller resolved, just one that is now reachable under a different path — nothing is orphaned and no update is lost. The anomaly this PR closes is the other one: the parent is deleted, or replaced by a same-named entity with a fresh id, and a stale child write survives it. That case is caught by the id + existence checks these fences already do.
Making a stale fully qualified path fail would require shared locks on the whole ancestor chain, root-to-leaf, on every write in the hierarchy — a metalake row read on every catalog/schema/table/model write. That is a much larger change than this PR, and it buys strictness rather than integrity, so I'd rather track it separately if we decide we want strict path semantics.
| CatalogMetaMapper.class, | ||
| mapper -> | ||
| createsImplicitAncestors | ||
| ? mapper.selectCatalogMetaByIdForUpdate(observedCatalogPO.getCatalogId()) |
There was a problem hiding this comment.
Worth flagging as a design tradeoff: any hierarchical schema create (one that materializes implicit ancestors) takes an exclusive lock on the whole catalog row, which will block all other schema creates under that catalog — not just ones touching the same ancestor path — for the duration of the transaction. That's the right call for correctness, but it means catalogs with heavy concurrent hierarchical schema creation will serialize on this lock. Worth calling out explicitly in the PR description as an accepted throughput tradeoff.
There was a problem hiding this comment.
You are right. When a create needs to make implicit ancestors, we take an exclusive lock on the catalog row, so all other schema creates under the same catalog have to wait, even if they use a different path.
We need the exclusive lock here because two concurrent creates can both see that an ancestor is missing and both insert it. Under MySQL REPEATABLE READ, a shared lock is not enough to stop this.
So this is an accepted tradeoff: correctness first, less concurrency for hierarchical creates. I will add this to the PR description.
If it becomes a problem later, we can make the lock smaller: only lock the ancestor rows we really need to create, and use the unique constraint plus a retry instead of locking the whole catalog row. I can do that in a follow-up issue.
There was a problem hiding this comment.
You are right. When a create needs to make implicit ancestors, we take
an exclusive lock on the catalog row, so all other schema creates under
the same catalog have to wait, even if they use a different path.
We need the exclusive lock here because two concurrent creates can both
see that an ancestor is missing and both insert it. Under MySQL
REPEATABLE READ, a shared lock is not enough to stop this.
So this is an accepted tradeoff: correctness first, less concurrency for
hierarchical creates. I will add this to the PR description.
If it becomes a problem later, we can make the lock smaller: only lock
the ancestor rows we really need to create, and use the unique
constraint plus a retry instead of locking the whole catalog row. I can
do that in a follow-up issue.
|
|
||
| return store.delete(ident, EntityType.METALAKE, true); | ||
| } catch (NoSuchMetalakeException e) { | ||
| } catch (NoSuchMetalakeException | NoSuchEntityException e) { |
There was a problem hiding this comment.
CatalogManager.dropCatalog's equivalent new catch block also invalidates catalogCache before returning false (to discard the now-stale cache entry). This branch doesn't do the same for whatever metalake-level cache exists — is that intentional (metalakes aren't cached the same way) or a gap?
There was a problem hiding this comment.
First, catalogCache is not an entity cache. It caches CatalogWrapper objects, which hold the live catalog instance and its class loader. A metalake has nothing like this, and MetalakeManager has no cache field
at all. The "preload all metalakes" comment in the constructor talks about the entity cache in the store, not a cache in the manager.
Second, the entity cache in the store is already invalidated for us. RelationalEntityStore.delete() calls cache.invalidate(ident, entityType) in a finally block, so it runs both when the delete succeeds and when
the entity is already gone. So there is no stale entry left for this branch to clean up.
Also, in this method the NoSuchMetalakeException / NoSuchEntityException mostly comes from metalakeInUse() and store.list(), which run before we reach the delete.
There was a problem hiding this comment.
This is intentional, not a gap. There are two reasons.
First, catalogCache is not an entity cache. It caches CatalogWrapper
objects, which hold the live catalog instance and its class loader. A
metalake has nothing like this, and MetalakeManager has no cache field
at all. The "preload all metalakes" comment in the constructor talks
about the entity cache in the store, not a cache in the manager.
Second, the entity cache in the store is already invalidated for us.
RelationalEntityStore.delete() calls cache.invalidate(ident, entityType)
in a finally block, so it runs both when the delete succeeds and when
the entity is already gone. So there is no stale entry left for this
branch to clean up.
Also, in this method the NoSuchMetalakeException / NoSuchEntityException
mostly comes from metalakeInUse() and store.list(), which run before
we reach the delete.
|
I really would suggest you split PRs into small one, so that we can have a better review. |
I will adopt it for the remaining few PRs. This one is small initially, and getting larger and larger after several round of reviews. |
Move the duplicated concurrent-modification messages into two shared factories on ExceptionUtils and drop the per-service private optimisticLockException helpers.
…ace-hierarchy Signed-off-by: yuqi <yuqi@datastrato.com>
|
Per review feedback that this PR is too large, it has been split into three smaller PRs, in dependency order:
The three branches stacked together produce a tree identical to this PR's head ( Keeping this PR open for the discussion context; it will be closed once the three land. |
|
Close it temporarily. |
What changes were proposed in this pull request?
Add database-backed optimistic concurrency control and transaction boundaries for metalakes, catalogs, and schemas.
Accepted tradeoff: a hierarchical schema create that materializes implicit ancestors takes an exclusive lock on the catalog row, so every other schema create under that catalog waits until that transaction ends, even when it touches a different ancestor path. The exclusive lock is needed because two concurrent creates can both find the same ancestor missing and both insert it, and a shared lock does not prevent that under MySQL REPEATABLE READ. Catalogs with heavy concurrent hierarchical schema creation will therefore serialize on this lock. If it becomes a bottleneck, a narrower fence — locking only the ancestor rows being created and relying on the unique constraint plus a retry — can be done in a follow-up.
This PR builds on the shared conflict response introduced by #12349.
Why are the changes needed?
Managed namespace operations previously consisted of multiple independent reads and writes. Concurrent alter, create, and delete requests could overwrite newer metadata, create children below a deleted parent, leave view/function rows orphaned, or run partial cascade cleanup. Overlapping hierarchical schema drops could also acquire descendant row locks in different orders.
Fix: #12342
Does this PR introduce any user-facing change?
Concurrent managed namespace version conflicts are reported as HTTP 409. If the observed entity was deleted or renamed away, alter reports not found and drop preserves its idempotent false result. A managed schema create that loses a concurrent same-name create returns
SchemaAlreadyExistsExceptioninstead of overwriting the winner.How was this patch tested?
./gradlew :core:check -PskipITs -PskipDockerTests=trueenv dockerTest=true ./gradlew :core:test --tests 'org.apache.gravitino.storage.relational.service.TestSchemaMetaService' --tests 'org.apache.gravitino.storage.relational.service.TestCatalogMetaService' --tests 'org.apache.gravitino.storage.relational.service.TestMetalakeMetaService' -PskipITs -PskipDockerTests=false(H2, MySQL 8.0, PostgreSQL 13)./gradlew :catalogs:catalog-model:test --tests 'org.apache.gravtitino.catalog.model.TestModelCatalogOperations' -PskipITs -PskipDockerTests=true./gradlew :core:javadoc