diff --git a/docs/apache-hive-catalog.md b/docs/apache-hive-catalog.md index 9e96e6cd53b..59763b05a11 100644 --- a/docs/apache-hive-catalog.md +++ b/docs/apache-hive-catalog.md @@ -80,7 +80,7 @@ When using Gravitino authorization for Hive with Apache Ranger, see the [Authori ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `s3-access-key-id`, `s3-secret-access-key`, `oss-access-key-id`, and `oss-secret-access-key` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -103,7 +103,7 @@ The following table lists predefined schema properties for the Hive database. Ad ### Schema Operations -see [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations). +see [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations). ## Table @@ -159,7 +159,7 @@ The following table lists the data types mapped from the Hive catalog to Graviti | `uniontype` | `union` | :::info -1. The data types other than listed above are mapped to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)** that represents an unresolvable data type from the Hive catalog. +1. The data types other than listed above are mapped to Gravitino **[External Type](./tables-and-views.md#external-type)** that represents an unresolvable data type from the Hive catalog. 2. Using the `struct` data type with field comments will throw an error, as it does not work for Hive tables (see [HIVE-26593](https://issues.apache.org/jira/browse/HIVE-26593)). ::: diff --git a/docs/aws-glue-catalog.md b/docs/aws-glue-catalog.md index e9d7a6895dd..590f5bae2e1 100644 --- a/docs/aws-glue-catalog.md +++ b/docs/aws-glue-catalog.md @@ -52,7 +52,7 @@ Besides the [common catalog properties](./gravitino-server-config.md#catalog-pro ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `aws-access-key-id` and `aws-secret-access-key` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -70,7 +70,7 @@ The Glue catalog defines no predefined schema properties beyond `comment`. Addit ### Schema Operations -See [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations). +See [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations). ## Table @@ -133,7 +133,7 @@ The following table lists the data types mapped from the Glue catalog to Graviti | `uniontype` | `union` | :::info -Data types not listed above map to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)**, which represents an unresolvable data type from the Glue catalog. +Data types not listed above map to Gravitino **[External Type](./tables-and-views.md#external-type)**, which represents an unresolvable data type from the Glue catalog. ::: ### Table Properties diff --git a/docs/expression.md b/docs/expression.md index a7b5e506b34..89624970202 100644 --- a/docs/expression.md +++ b/docs/expression.md @@ -11,7 +11,7 @@ import TabItem from '@theme/TabItem'; ## Introduction -This page introduces the expression system of Apache Gravitino. Expressions are vital component of metadata definition, through expressions, you can define [default values](./manage-relational-metadata-using-gravitino.md#table-column-default-value) for columns, function arguments for [function partitioning](./table-partitioning-distribution-sort-order-indexes.md#table-partitioning), [bucketing](./table-partitioning-distribution-sort-order-indexes.md#table-distribution), and sort term of [sort ordering](./table-partitioning-distribution-sort-order-indexes.md#sort-ordering) in tables. +This page introduces the expression system of Apache Gravitino. Expressions are vital component of metadata definition, through expressions, you can define [default values](./tables-and-views.md#table-column-default-value) for columns, function arguments for [function partitioning](./table-partitioning-distribution-sort-order-indexes.md#table-partitioning), [bucketing](./table-partitioning-distribution-sort-order-indexes.md#table-distribution), and sort term of [sort ordering](./table-partitioning-distribution-sort-order-indexes.md#sort-ordering) in tables. Gravitino expression system divides expressions into three basic parts: field reference, literal, and function. Function expressions can contain field references, literals, and other function expressions. ## Field Reference diff --git a/docs/jdbc-clickhouse-catalog.md b/docs/jdbc-clickhouse-catalog.md index c39128cc863..2371ea2bcca 100644 --- a/docs/jdbc-clickhouse-catalog.md +++ b/docs/jdbc-clickhouse-catalog.md @@ -107,7 +107,7 @@ Catalog catalog = -See [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for other catalog operations. +See [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for other catalog operations. ## Schema @@ -161,7 +161,7 @@ Schema schema = catalog.asTableCatalog() -See [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more schema operations. +See [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more schema operations. ## Table @@ -199,7 +199,7 @@ See [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-us | `BOOLEAN` | `Bool` | | `UUID` | `UUID` | -Other ClickHouse types are exposed as [External Type](./manage-relational-metadata-using-gravitino.md#external-type). +Other ClickHouse types are exposed as [External Type](./tables-and-views.md#external-type). ### Table Properties diff --git a/docs/jdbc-doris-catalog.md b/docs/jdbc-doris-catalog.md index d36c5d0e173..ab36c916e70 100644 --- a/docs/jdbc-doris-catalog.md +++ b/docs/jdbc-doris-catalog.md @@ -27,7 +27,7 @@ Gravitino saves some system information in schema and table comments, like - Gravitino catalog corresponds to the Doris instance. - Supports metadata management of Doris (1.2.x, 3.0.x, 4.0.x). - Supports table index (PRIMARY_KEY, UNIQUE_KEY, INVERTED, BITMAP (legacy), ANN/VECTOR). -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). +- Supports [column default value](./tables-and-views.md#table-column-default-value). ### Catalog Properties @@ -81,7 +81,7 @@ Returning null for DATETIME type precision. Driver version: mysql-connector-java ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -102,7 +102,7 @@ Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden ### Schema Operations Refer to -[Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +[Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table @@ -110,7 +110,7 @@ Refer to - Gravitino's table concept corresponds to the Doris table. - Supports index. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). +- Supports [column default value](./tables-and-views.md#table-column-default-value). #### Table Column Types @@ -139,7 +139,7 @@ Refer to | `ExternalType("hll")` | `HLL` | Doris doesn't support Gravitino `Fixed` `Timestamp_tz` `IntervalDay` `IntervalYear` `Union` `UUID` type. -The data types other than those listed above are mapped to Gravitino's **[Unparsed Type](./manage-relational-metadata-using-gravitino.md#unparsed-type)** that represents an unresolvable data type. +The data types other than those listed above are mapped to Gravitino's **[Unparsed Type](./tables-and-views.md#unparsed-type)** that represents an unresolvable data type. :::note Doris `array`, `map`, and `struct` types are loaded as `ExternalType` with the full type string preserved (e.g. `array`). They are not resolved into Gravitino native composite types (`ListType`, `MapType`, `StructType`). The type identifier in `ExternalType` is always lowercase (e.g. `"json"`, not `"JSON"`), matching Doris JDBC metadata behavior. diff --git a/docs/jdbc-hologres-catalog.md b/docs/jdbc-hologres-catalog.md index 0b67eb5a83f..a4412dbd47a 100644 --- a/docs/jdbc-hologres-catalog.md +++ b/docs/jdbc-hologres-catalog.md @@ -29,10 +29,10 @@ Gravitino saves some system information in schema and table comment, like `(From - Supports metadata management of Hologres. - Supports DDL operation for Hologres schemas and tables. - Supports table index (PRIMARY KEY in CREATE TABLE). -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). +- Supports [column default value](./tables-and-views.md#table-column-default-value). - Supports LIST partitioning (physical and logical partition tables). - Supports Hologres-specific table properties via `WITH` clause (orientation, clustering_key, distribution_key, etc.). -- Does not support [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment). +- Does not support [auto-increment](./tables-and-views.md#table-column-auto-increment). ### Catalog Properties @@ -59,7 +59,7 @@ Hologres uses the PostgreSQL JDBC Driver (version 42.3.2 or later recommended). ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -80,7 +80,7 @@ Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden ### Schema Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table @@ -89,10 +89,10 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada - Gravitino's table concept corresponds to the Hologres table. - Supports DDL operation for Hologres tables. - Supports PRIMARY KEY index in CREATE TABLE. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). +- Supports [column default value](./tables-and-views.md#table-column-default-value). - Supports expression columns via DEFAULT expressions (note: Gravitino maps these as column default values, not as true generated/computed columns in the Hologres sense). - Supports LIST partitioning (physical and logical). -- Does not support [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment). Creating auto-increment columns is rejected in both CREATE TABLE and ALTER TABLE. +- Does not support [auto-increment](./tables-and-views.md#table-column-auto-increment). Creating auto-increment columns is rejected in both CREATE TABLE and ALTER TABLE. ### Table Properties @@ -145,7 +145,7 @@ Hologres-specific table properties are set via the `WITH` clause during CREATE T :::info - Hologres does not support precision syntax for `TIMESTAMP`/`TIMESTAMPTZ` (e.g., `timestamptz(6)` is invalid), so the type converter always emits the base type without precision. - Array element types must be non-nullable (Hologres limitation). Multidimensional arrays are not supported. -- Types like `json`, `jsonb`, `uuid`, `inet`, `money`, `roaringbitmap` are mapped to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)** with the original type name preserved. +- Types like `json`, `jsonb`, `uuid`, `inet`, `money`, `roaringbitmap` are mapped to Gravitino **[External Type](./tables-and-views.md#external-type)** with the original type name preserved. ::: ### Table Distribution diff --git a/docs/jdbc-mysql-catalog.md b/docs/jdbc-mysql-catalog.md index 2d969912586..6e7c4172ce0 100644 --- a/docs/jdbc-mysql-catalog.md +++ b/docs/jdbc-mysql-catalog.md @@ -27,7 +27,7 @@ Gravitino saves some system information in schema and table comment, like `(From - Supports metadata management of MySQL (5.7, 8.0). - Supports DDL operation for MySQL databases and tables. - Supports table index. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value) and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment). +- Supports [column default value](./tables-and-views.md#table-column-default-value) and [auto-increment](./tables-and-views.md#table-column-auto-increment). - Supports managing MySQL table features through table properties, like using `engine` to set MySQL storage engine. ### Catalog Properties @@ -84,7 +84,7 @@ Returning null for TIMESTAMP type precision. Driver version: mysql-connector-jav ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -105,7 +105,7 @@ Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden ### Schema Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table @@ -114,7 +114,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada - Gravitino's table concept corresponds to the MySQL table. - Supports DDL operation for MySQL tables. - Supports index. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value) and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment).. +- Supports [column default value](./tables-and-views.md#table-column-default-value) and [auto-increment](./tables-and-views.md#table-column-auto-increment).. - Supports managing MySQL table features through table properties, like using `engine` to set MySQL storage engine. ### Table Column Types @@ -144,7 +144,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada :::info MySQL doesn't support Gravitino `Fixed` `Struct` `List` `Map` `IntervalDay` `IntervalYear` `Union` `UUID` type. -Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)** that represents an unresolvable data type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./tables-and-views.md#external-type)** that represents an unresolvable data type. ::: ### Table Column Auto-Increment diff --git a/docs/jdbc-oceanbase-catalog.md b/docs/jdbc-oceanbase-catalog.md index c93e4d8f300..319a2ad3026 100644 --- a/docs/jdbc-oceanbase-catalog.md +++ b/docs/jdbc-oceanbase-catalog.md @@ -28,7 +28,7 @@ Apache Gravitino provides the ability to manage OceanBase metadata. - Supports metadata management of OceanBase (4.x). - Supports DDL operation for OceanBase databases and tables. - Supports table index. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value) and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment). +- Supports [column default value](./tables-and-views.md#table-column-default-value) and [auto-increment](./tables-and-views.md#table-column-auto-increment). ### Catalog Properties @@ -84,7 +84,7 @@ Returning null for TIMESTAMP type precision. Driver version: mysql-connector-jav ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -105,7 +105,7 @@ Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden ### Schema Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table @@ -114,7 +114,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada - Gravitino's table concept corresponds to the OceanBase table. - Supports DDL operation for OceanBase tables. - Supports index. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value) and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment).. +- Supports [column default value](./tables-and-views.md#table-column-default-value) and [auto-increment](./tables-and-views.md#table-column-auto-increment).. ### Table Properties @@ -146,7 +146,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada :::info OceanBase doesn't support Gravitino `Boolean` `Fixed` `Struct` `List` `Map` `IntervalDay` `IntervalYear` `Union` `UUID` type. -Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)** that represents an unresolvable data type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./tables-and-views.md#external-type)** that represents an unresolvable data type. ::: ### Table Column Auto-Increment diff --git a/docs/jdbc-postgresql-catalog.md b/docs/jdbc-postgresql-catalog.md index 68dfbfa1876..28fa2b2c8bf 100644 --- a/docs/jdbc-postgresql-catalog.md +++ b/docs/jdbc-postgresql-catalog.md @@ -27,7 +27,7 @@ Gravitino saves some system information in schema and table comment, like `(From - Supports metadata management of PostgreSQL (12.x, 13.x, 14.x, 15.x, 16.x). - Supports DDL operation for PostgreSQL schemas and tables. - Supports table index. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment). +- Supports [column default value](./tables-and-views.md#table-column-default-value). and [auto-increment](./tables-and-views.md#table-column-auto-increment). ### Catalog Properties @@ -60,7 +60,7 @@ In PostgreSQL, the database corresponds to the Gravitino catalog, and the schema ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -81,7 +81,7 @@ Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden ### Schema Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table @@ -90,7 +90,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada - The Gravitino table corresponds to the PostgreSQL table. - Supports DDL operation for PostgreSQL tables. - Supports index. -- Support [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value) and [auto-increment](./manage-relational-metadata-using-gravitino.md#table-column-auto-increment). +- Support [column default value](./tables-and-views.md#table-column-default-value) and [auto-increment](./tables-and-views.md#table-column-auto-increment). - Doesn't support table property settings. ### Table Column Types @@ -117,7 +117,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada :::info PostgreSQL doesn't support Gravitino `Fixed` `Struct` `Map` `IntervalDay` `IntervalYear` `Union` type. -Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)** that represents an unresolvable data type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./tables-and-views.md#external-type)** that represents an unresolvable data type. ::: ### Table Column Auto-Increment diff --git a/docs/jdbc-starrocks-catalog.md b/docs/jdbc-starrocks-catalog.md index efe8114e7ba..178f8bf4d1e 100644 --- a/docs/jdbc-starrocks-catalog.md +++ b/docs/jdbc-starrocks-catalog.md @@ -26,7 +26,7 @@ Gravitino saves some system information in table comments, like - Gravitino catalog corresponds to the StarRocks instance. - Supports metadata management of StarRocks (3.3.x). -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). +- Supports [column default value](./tables-and-views.md#table-column-default-value). ### Catalog Properties @@ -80,7 +80,7 @@ Returning null for DATETIME type precision. Driver version: mysql-connector-java ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `jdbc-user` and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -101,14 +101,14 @@ As StarRocks can't get thr properties after set, So now we do not support set Sc ### Schema Operations Refer to -[Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +[Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table ### Table Capabilities - Gravitino's table concept corresponds to the StarRocks table. -- Supports [column default value](./manage-relational-metadata-using-gravitino.md#table-column-default-value). +- Supports [column default value](./tables-and-views.md#table-column-default-value). #### Table Column Types @@ -131,7 +131,7 @@ Refer to StarRocks doesn't support Gravitino `Fixed` `Timestamp_tz` `IntervalDay` `IntervalYear` `Union` `UUID` type. -The data types other than those listed above are mapped to Gravitino's **[Unparsed Type](./manage-relational-metadata-using-gravitino.md#unparsed-type)** that represents an unresolvable data type. +The data types other than those listed above are mapped to Gravitino's **[Unparsed Type](./tables-and-views.md#unparsed-type)** that represents an unresolvable data type. :::note Gravitino cannot load StarRocks `array`, `map` and `struct` type correctly, because StarRocks doesn't support these types in JDBC. diff --git a/docs/lakehouse-generic-catalog.md b/docs/lakehouse-generic-catalog.md index 798d8ad0e22..e5471e98319 100644 --- a/docs/lakehouse-generic-catalog.md +++ b/docs/lakehouse-generic-catalog.md @@ -126,7 +126,7 @@ Catalog catalog = gravitinoClient.createCatalog( -Other catalog operations are general with relational catalogs. See [Catalog Operations](./manage-relational-metadata-using-gravitino.md#catalog-operations) for detailed documentation. +Other catalog operations are general with relational catalogs. See [Catalog Operations](./manage-catalogs-and-schemas.md#catalog-operations) for detailed documentation. ## Schema Management @@ -142,7 +142,7 @@ Schema operations follow the same patterns as relational catalogs: - ✅ Delete schemas - ✅ Check schema existence -See [Schema Operations](./manage-relational-metadata-using-gravitino.md#schema-operations) for detailed documentation. +See [Schema Operations](./manage-catalogs-and-schemas.md#schema-operations) for detailed documentation. ### Schema Properties @@ -192,7 +192,7 @@ catalog.asSchemas().createSchema( -For additional operations, refer to [Schema Operations documentation](./manage-relational-metadata-using-gravitino.md#schema-operations). +For additional operations, refer to [Schema Operations documentation](./manage-catalogs-and-schemas.md#schema-operations). ## Table Management diff --git a/docs/lakehouse-hudi-catalog.md b/docs/lakehouse-hudi-catalog.md index 54d8520c64d..c40c0e905c3 100644 --- a/docs/lakehouse-hudi-catalog.md +++ b/docs/lakehouse-hudi-catalog.md @@ -58,7 +58,7 @@ Property name with this prefix passed down to the underlying backend client for ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. ## Schema @@ -73,7 +73,7 @@ Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metada ### Schema Operations Only support read operations: listSchema, loadSchema, and schemaExists. -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table diff --git a/docs/lakehouse-iceberg-catalog.md b/docs/lakehouse-iceberg-catalog.md index ae5f2a34e6d..281c8d86ac4 100644 --- a/docs/lakehouse-iceberg-catalog.md +++ b/docs/lakehouse-iceberg-catalog.md @@ -254,7 +254,7 @@ Gravitino provides the build-in `org.apache.gravitino.iceberg.common.cache.Local ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `s3-access-key-id`, `s3-secret-access-key`, `oss-access-key-id`, and `oss-secret-access-key` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -273,7 +273,7 @@ You could put properties except `comment`. ### Schema Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ### Hierarchical schema @@ -450,7 +450,7 @@ represent an Iceberg `unknown` column in Gravitino. :::info Apache Iceberg doesn't support Gravitino `Varchar` `Fixedchar` `Byte` `Short` `Union` type. -Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./manage-relational-metadata-using-gravitino.md#external-type)** that represents an unresolvable data type. +Meanwhile, the data types other than listed above are mapped to Gravitino **[External Type](./tables-and-views.md#external-type)** that represents an unresolvable data type. ::: ### Table Properties diff --git a/docs/lakehouse-paimon-catalog.md b/docs/lakehouse-paimon-catalog.md index 0247bf8ba66..e398c5e5b44 100644 --- a/docs/lakehouse-paimon-catalog.md +++ b/docs/lakehouse-paimon-catalog.md @@ -96,7 +96,7 @@ Download the corresponding JDBC driver and place it to the `catalogs/lakehouse-p ### Catalog Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#catalog-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#catalog-operations) for more details. :::note Sensitive catalog properties such as `s3-access-key-id`, `s3-secret-access-key`, `jdbc-user`, and `jdbc-password` are hidden from the load catalog response. Use the [credential vending API](security/credential-vending.md) to retrieve them at runtime. @@ -119,7 +119,7 @@ Sensitive catalog properties such as `s3-access-key-id`, `s3-secret-access-key`, ### Schema Operations -Refer to [Manage Relational Metadata Using Gravitino](./manage-relational-metadata-using-gravitino.md#schema-operations) for more details. +Refer to [Manage Catalogs and Schemas](./manage-catalogs-and-schemas.md#schema-operations) for more details. ## Table diff --git a/docs/tables-and-views.md b/docs/tables-and-views.md index af7490aa3a6..e1adb7c9237 100644 --- a/docs/tables-and-views.md +++ b/docs/tables-and-views.md @@ -43,6 +43,101 @@ across catalogs and each provider maps them to its own. Where a provider cannot represent a type, the provider's own page says so. Type mapping is the most common place two catalogs of different providers differ. +#### Table Column Type + +Gravitino supports the following column types. A catalog may support only a subset; see the +provider's page for its type mapping. + +| Type | Java | JSON | +|---------------------------|-------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------| +| Boolean | `Types.BooleanType.get()` | `"boolean"` | +| Byte | `Types.ByteType.get()` | `"byte"` | +| Unsigned Byte | `Types.ByteType.unsigned()` | `"byte unsigned"` | +| Short | `Types.ShortType.get()` | `"short"` | +| Unsigned Short | `Types.ShortType.unsigned()` | `"short unsigned"` | +| Integer | `Types.IntegerType.get()` | `"integer"` | +| Unsigned Integer | `Types.IntegerType.unsigned()` | `"integer unsigned"` | +| Long | `Types.LongType.get()` | `"long"` | +| Unsigned Long | `Types.LongType.unsigned()` | `"long unsigned"` | +| Float | `Types.FloatType.get()` | `"float"` | +| Double | `Types.DoubleType.get()` | `"double"` | +| Decimal(precision, scale) | `Types.DecimalType.of(precision, scale)` | `"decimal(p,s)"` | +| String | `Types.StringType.get()` | `"string"` | +| FixedChar(length) | `Types.FixedCharType.of(length)` | `"char(l)"` | +| VarChar(length) | `Types.VarCharType.of(length)` | `"varchar(l)"` | +| Timestamp | `Types.TimestampType.withoutTimeZone()` | `"timestamp"` | +| Timestamp(p) | `Types.TimestampType.withoutTimeZone(p)` | `"timestamp(p)"` | +| TimestampWithTimezone | `Types.TimestampType.withTimeZone()` | `"timestamp_tz"` | +| TimestampWithTimezone(p) | `Types.TimestampType.withTimeZone(p)` | `"timestamp_tz(p)"` | +| Date | `Types.DateType.get()` | `"date"` | +| Time | `Types.TimeType.get()` | `"time"` | +| Time(p) | `Types.TimeType.of(p)` | `"time(p)"` | +| IntervalToYearMonth | `Types.IntervalYearType.get()` | `"interval_year"` | +| IntervalToDayTime | `Types.IntervalDayType.get()` | `"interval_day"` | +| Fixed(length) | `Types.FixedType.of(length)` | `"fixed(l)"` | +| Binary | `Types.BinaryType.get()` | `"binary"` | +| List | `Types.ListType.of(Types.IntegerType.get(), true)` | `{"type":"list","containsNull":true,"elementType":"integer"}` | +| Map | `Types.MapType.of(Types.StringType.get(), Types.IntegerType.get(), true)` | `{"type":"map","keyType":"string","valueType":"integer","valueContainsNull":true}` | +| Struct | `Types.StructType.of(Types.StructType.Field.of("id", Types.IntegerType.get(), false, null))` | `{"type":"struct","fields":[{"name":"id","type":"integer","nullable":false}]}` | +| Union | `Types.UnionType.of(Types.IntegerType.get(), Types.StringType.get())` | `{"type":"union","types":["integer","string"]}` | +| UUID | `Types.UUIDType.get()` | `"uuid"` | +| Variant | `Types.VariantType.get()` | `"variant"` | +| Null | `Types.NullType.get()` | `"null"` | +| Geometry | `Types.GeometryType.crs84()` | `"geometry"` | +| Geography | `Types.GeographyType.crs84()` | `"geography"` | + +Decimal precision is in the range 1-38, and scale is in the range 0-precision. The optional +precision for time and timestamp types is in the range 0-12. + +##### Null type + +The null type represents a column that holds only null values and whose concrete type is not yet +known. It is intended to be promoted to a concrete type through schema evolution before data is +written. Support is connector-specific. + +##### External type + +An external type represents a catalog type that is not part of the Gravitino type system. It keeps +the external catalog's type string so clients can inspect it without losing information. + +```json +{ + "type": "external", + "catalogString": "user-defined" +} +``` + +```java +String typeString = ((ExternalType) type).catalogString(); +``` + +##### Unparsed type + +An unparsed type preserves forward compatibility when a client does not recognize a type returned +by the server. The client retains the serialized value instead of failing deserialization. + +```json +{ + "type": "unparsed", + "unparsedType": "unknown-type" +} +``` + +```java +String unparsedValue = ((UnparsedType) type).unparsedType(); +``` + +#### Table Column Default Value + +A column default can be a [literal](./expression.md#literal) or an +[expression](./expression.md). The underlying catalog applies it to new rows, and support depends +on the catalog provider. + +#### Table Column Auto-increment + +An auto-increment column asks the underlying catalog to generate values for new rows. Support and +restrictions are provider-specific, so check the provider's table capabilities before enabling it. + ### Table Properties Properties are provider-specific and carry what the source system needs, such as the file format for diff --git a/docs/trino-connector/installation.md b/docs/trino-connector/installation.md index 3f7297aaf8c..d39b200ee32 100644 --- a/docs/trino-connector/installation.md +++ b/docs/trino-connector/installation.md @@ -182,7 +182,7 @@ system See the `gravitino` catalog in the result set. This signifies the successful installation of the Gravitino Trino connector. -Assuming you have created a catalog named `test.jdbc-mysql` in the Gravitino server, or refer to [Create a Catalog](../manage-relational-metadata-using-gravitino.md#create-a-catalog). Then you can use the Trino CLI to connect to the Trino container and run a query like this. +Assuming you have created a catalog named `test.jdbc-mysql` in the Gravitino server, or refer to [Create a Catalog](../manage-catalogs-and-schemas.md#create-a-catalog). Then you can use the Trino CLI to connect to the Trino container and run a query like this. ```text docker exec -it trino-gravitino trino diff --git a/docs/trino-connector/supported-catalog.md b/docs/trino-connector/supported-catalog.md index 66e39fdc36b..80a0380f30f 100644 --- a/docs/trino-connector/supported-catalog.md +++ b/docs/trino-connector/supported-catalog.md @@ -122,7 +122,7 @@ call gravitino.system.alter_catalog( ``` If you need more information about catalog, refer to: -[Create a Catalog](../manage-relational-metadata-using-gravitino.md#create-a-catalog). +[Create a Catalog](../manage-catalogs-and-schemas.md#create-a-catalog). ## Pass Trino Connector Configuration @@ -180,4 +180,4 @@ Hive does not support `TIME` data type. | Map | MAP | | Struct | ROW | -For more about Trino data types, refer to [Trino data types](https://trino.io/docs/current/language/types.html) and Gravitino data types, refer to [Gravitino data types](../manage-relational-metadata-using-gravitino.md#table-column-type). +For more about Trino data types, refer to [Trino data types](https://trino.io/docs/current/language/types.html) and Gravitino data types, refer to [Gravitino data types](../tables-and-views.md#table-column-type).