From cf897eb042658a9afb07a6d3931ddf2f020339b2 Mon Sep 17 00:00:00 2001 From: Scott Trinh Date: Tue, 23 Dec 2025 14:41:15 -0500 Subject: [PATCH 1/6] Add a self-hosted to Gel Cloud migration guide --- docs/cloud/index.rst | 1 + docs/cloud/migrate_from.rst | 173 ++++++++++++++++++++++++++++++++++++ 2 files changed, 174 insertions(+) create mode 100644 docs/cloud/migrate_from.rst diff --git a/docs/cloud/index.rst b/docs/cloud/index.rst index ea68f523a6a..e4512b67e75 100644 --- a/docs/cloud/index.rst +++ b/docs/cloud/index.rst @@ -19,6 +19,7 @@ Cloud deploy/fly deploy/render deploy/railway + migrate_from |Gel| Cloud is a fully managed, effortless cloud database service, diff --git a/docs/cloud/migrate_from.rst b/docs/cloud/migrate_from.rst new file mode 100644 index 00000000000..1b11efb88a6 --- /dev/null +++ b/docs/cloud/migrate_from.rst @@ -0,0 +1,173 @@ +.. _ref_migrate_from: + +========================================== +Migrating from Gel Cloud to Self-Hosted +========================================== + +:edb-alt-title: Migrating from Gel Cloud to Self-Hosted Gel + +|Gel| Cloud is sunsetting at the end of January 2026. To ensure your +applications continue to run smoothly, you should migrate your data to a +self-hosted |Gel| instance. + +This guide outlines the process of migrating your production data. We strongly +recommend performing a "dry run" with your staging or development environment +first to familiarize yourself with the workflow. + + +Phase 1: Preparation +==================== + +1. Spin up your self-hosted deployment +-------------------------------------- + +While you can host |Gel| on any infrastructure that supports Docker or binary +installations, we recommend using a managed cloud provider (such as Fly.io, +AWS, or GCP with managed Postgres) for production reliability. + +See our :ref:`self-hosted deployment guides ` for +step-by-step instructions on deploying to various platforms. + +Ensure your new instance is: + +* Running the same version of |Gel| as your Cloud instance (or newer). +* Configured with a persistent volume for data or connected to a managed + Postgres instance. + +2. Retrieve connection parameters +--------------------------------- + +Once your new instance is live, you need its DSN (Data Source Name) or +individual connection parameters. Each :ref:`deployment guide +` outlines the best way to retrieve the various +connection parameters from your specific setup. You will typically need: + +* **Host**: The domain or IP of your new instance. +* **Port**: Default is ``5656``. +* **User**: Default is |admin|. +* **Password**: The password you set during initialization. +* **TLS CA**: The certificate used to secure the connection (unless using a + public CA or ``--trust-tls-cert``). + +The DSN format is: + +.. code-block:: text + + gel://:@:/ + +All components except the scheme are optional. See :ref:`ref_reference_connection_dsn` +for more details. + + +Phase 2: The Migration (Cutover) +================================ + +To ensure data consistency, you must prevent new writes to your database +during the transfer. + +1. Enable maintenance mode +-------------------------- + +Before touching the data, put your application into maintenance mode. This +ensures that no new records are created in |Gel| Cloud after you've started +the dump. + +* **Web Apps**: Point your load balancer to a static "Maintenance" page. +* **Background Jobs**: Stop all workers, cron jobs, or queues that interact + with the database. + +2. Perform the migration +------------------------ + +You can use the |Gel| CLI to move data directly from your Cloud instance to +your new self-hosted instance. + +.. note:: + + If your self-hosted instance uses a self-signed TLS certificate, you may + need to add ``--tls-security insecure`` to the restore command, or first + retrieve the TLS certificate and set it via :gelenv:`TLS_CA`. + +Option A: Streaming (Recommended) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +You can pipe the output of :gelcmd:`dump` directly into :gelcmd:`restore`. +This is the fastest method as it doesn't require saving a large file to your +local disk. + +.. code-block:: bash + + # Set your Gel Cloud instance as the source and the new DSN as the target + $ gel dump --instance / --all \ + | gel restore --dsn --all + +Option B: File-based (Safer for large DBs) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you have a very large database or a shaky internet connection, saving to a +file first is safer. + +.. code-block:: bash + + # 1. Dump to file + $ gel dump --instance / --all my_database_backup.gel + + # 2. Restore from file + $ gel restore --dsn --all my_database_backup.gel + + +Phase 3: Verification and Go-Live +================================= + +1. Update application environment variables +------------------------------------------- + +Update your application's configuration to point to the new instance. Replace +your :gelenv:`INSTANCE` and :gelenv:`SECRET_KEY` variables with the new +connection details: + +* :gelenv:`DSN`: ``gel://user:password@host:port/branch`` +* :gelenv:`TLS_CA`: The TLS certificate content (if your instance uses a + self-signed certificate) +* :gelenv:`CLIENT_TLS_SECURITY`: Set to ``insecure`` if you need to skip TLS + verification (not recommended for production) + +For local development, you can use :ref:`gel instance link +` to create a named alias for your self-hosted +instance: + +.. code-block:: bash + + $ gel instance link \ + --dsn \ + --trust-tls-cert \ + --non-interactive \ + my_instance + +This allows you to connect using :gelcmd:`-I my_instance` instead of specifying +the full DSN each time. + +2. Sanity check +--------------- + +Before turning off maintenance mode: + +* Run a few :gelcmd:`query` commands against the new instance to verify data + integrity. +* Check that your schema migrated correctly: :gelcmd:`migrate --status`. +* Launch a local instance of your app connected to the new production DB to + ensure connection logic is sound. + +3. Disable maintenance mode +--------------------------- + +Once verified, restart your application servers and background workers. +Monitor your logs closely for any connection or permission errors. + + +Post-Migration Note +=================== + +Once you are 100% certain your data is safe and your app is stable on the new +host, you can de-provision your |Gel| Cloud instance. Remember that all |Gel| +Cloud data will be deleted after the January 2026 deadline. From f2c7c74272430d271b2a26a138e85d4407a3b38d Mon Sep 17 00:00:00 2001 From: Scott Trinh Date: Tue, 23 Dec 2025 14:56:13 -0500 Subject: [PATCH 2/6] A bit of cleanup --- docs/cloud/migrate_from.rst | 23 ++++------------------- 1 file changed, 4 insertions(+), 19 deletions(-) diff --git a/docs/cloud/migrate_from.rst b/docs/cloud/migrate_from.rst index 1b11efb88a6..257fc7a492f 100644 --- a/docs/cloud/migrate_from.rst +++ b/docs/cloud/migrate_from.rst @@ -1,8 +1,8 @@ .. _ref_migrate_from: -========================================== +======================================= Migrating from Gel Cloud to Self-Hosted -========================================== +======================================= :edb-alt-title: Migrating from Gel Cloud to Self-Hosted Gel @@ -123,8 +123,8 @@ Phase 3: Verification and Go-Live ------------------------------------------- Update your application's configuration to point to the new instance. Replace -your :gelenv:`INSTANCE` and :gelenv:`SECRET_KEY` variables with the new -connection details: +the Gel Cloud specific connection environment variables :gelenv:`INSTANCE` and +:gelenv:`SECRET_KEY` variables with the new connection details: * :gelenv:`DSN`: ``gel://user:password@host:port/branch`` * :gelenv:`TLS_CA`: The TLS certificate content (if your instance uses a @@ -132,21 +132,6 @@ connection details: * :gelenv:`CLIENT_TLS_SECURITY`: Set to ``insecure`` if you need to skip TLS verification (not recommended for production) -For local development, you can use :ref:`gel instance link -` to create a named alias for your self-hosted -instance: - -.. code-block:: bash - - $ gel instance link \ - --dsn \ - --trust-tls-cert \ - --non-interactive \ - my_instance - -This allows you to connect using :gelcmd:`-I my_instance` instead of specifying -the full DSN each time. - 2. Sanity check --------------- From 37c3e59501e9362ee78f7b55fc6549e1b4774c15 Mon Sep 17 00:00:00 2001 From: Scott Trinh Date: Tue, 23 Dec 2025 15:00:04 -0500 Subject: [PATCH 3/6] Use directory path instead of file --- docs/cloud/migrate_from.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/cloud/migrate_from.rst b/docs/cloud/migrate_from.rst index 257fc7a492f..c7b6559f074 100644 --- a/docs/cloud/migrate_from.rst +++ b/docs/cloud/migrate_from.rst @@ -109,11 +109,13 @@ file first is safer. .. code-block:: bash - # 1. Dump to file - $ gel dump --instance / --all my_database_backup.gel + # 1. Dump from Gel Cloud to directory + $ gel dump --instance / \ + --all --format=dir \ + production_dump - # 2. Restore from file - $ gel restore --dsn --all my_database_backup.gel + # 2. Restore to self-hosted from dump + $ gel restore --dsn --all production_dump Phase 3: Verification and Go-Live From d4fab92f5f64f7bb726cfd2ab7fdcbaa8f2c4675 Mon Sep 17 00:00:00 2001 From: Scott Trinh Date: Tue, 23 Dec 2025 15:26:26 -0500 Subject: [PATCH 4/6] Fix Gel URI directive usage --- docs/cloud/migrate_from.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/cloud/migrate_from.rst b/docs/cloud/migrate_from.rst index c7b6559f074..1615c0a2a61 100644 --- a/docs/cloud/migrate_from.rst +++ b/docs/cloud/migrate_from.rst @@ -128,7 +128,7 @@ Update your application's configuration to point to the new instance. Replace the Gel Cloud specific connection environment variables :gelenv:`INSTANCE` and :gelenv:`SECRET_KEY` variables with the new connection details: -* :gelenv:`DSN`: ``gel://user:password@host:port/branch`` +* :gelenv:`DSN`: :geluri:`user:password@host:port/branch` * :gelenv:`TLS_CA`: The TLS certificate content (if your instance uses a self-signed certificate) * :gelenv:`CLIENT_TLS_SECURITY`: Set to ``insecure`` if you need to skip TLS From a5b25a2299782c616f0bb4b5a7f4d1f65a04ea8d Mon Sep 17 00:00:00 2001 From: Scott Trinh Date: Tue, 23 Dec 2025 16:48:57 -0500 Subject: [PATCH 5/6] Add connection param guidance (#9133) --- .../running/deployment/aws_aurora_ecs.rst | 125 ++++++++++++---- .../deployment/azure_flexibleserver.rst | 110 ++++++++++++-- .../running/deployment/bare_metal.rst | 99 ++++++++++--- docs/reference/running/deployment/docker.rst | 135 ++++++++++++++++++ docs/reference/running/deployment/fly_io.rst | 88 ++++++++---- docs/reference/running/deployment/gcp.rst | 93 ++++++++---- 6 files changed, 539 insertions(+), 111 deletions(-) diff --git a/docs/reference/running/deployment/aws_aurora_ecs.rst b/docs/reference/running/deployment/aws_aurora_ecs.rst index 6b0d24803e2..361f9f72f27 100644 --- a/docs/reference/running/deployment/aws_aurora_ecs.rst +++ b/docs/reference/running/deployment/aws_aurora_ecs.rst @@ -79,32 +79,6 @@ has been assigned to your Gel instance: .. lint-on -It's often convenient to create an alias for the remote instance using -:gelcmd:`instance link`. - -.. code-block:: bash - - $ gel instance link \ - --trust-tls-cert \ - --dsn gel://admin:@ \ - my_aws_instance - -This aliases the remote instance to ``my_aws_instance`` (this name can be -anything). You can now use the ``-I my_aws_instance`` flag to run CLI commands -against this instance, as with local instances. - -.. note:: - - The command groups :gelcmd:`instance` and :gelcmd:`project` are not - intended to manage production instances. - -.. code-block:: bash - - $ gel -I my_aws_instance - Gel x.x - Type \help for help, \quit to quit. - gel> - To make changes to your Gel deployment like upgrading the Gel version or enabling the UI you can follow the CloudFormation `Updating a stack `_ instructions. Search for @@ -143,6 +117,105 @@ your terminal: /AWSCloudFormation/latest/UserGuide/cfn-whatis-howdoesitwork.html .. _docker-tags: https://hub.docker.com/r/geldata/gel/tags + +Connecting your application +=========================== + +To connect your application to the Gel instance, you'll need to provide +connection parameters. Gel client libraries can be configured using either +a DSN (connection string) or individual environment variables. + +Obtaining connection parameters +------------------------------- + +Your connection requires the following components: + +- **Host**: The ``PublicHostname`` value from the CloudFormation Stack's + ``Outputs`` tab. +- **Port**: ``5656`` (the default Gel port) +- **Username**: |admin| (the default superuser) +- **Password**: The ``SuperUserPassword`` you specified during deployment +- **Branch**: |main| (the default branch) + +Construct the DSN using these values: + +.. code-block:: bash + + $ GEL_DSN="gel://admin:@:5656" + +Obtaining the TLS certificate +----------------------------- + +.. warning:: + + The CloudFormation template does not configure TLS certificates correctly. + We recommend using ``--tls-security insecure`` for testing, but for + production you should use our `helm chart `_ or configure + TLS manually. + +To connect securely, your application needs the server's TLS certificate. +For self-signed certificates, you can retrieve the certificate by connecting +to the instance and extracting it: + +.. code-block:: bash + + $ gel --dsn $GEL_DSN --tls-security insecure \ + query "SELECT sys::get_tls_certificate()" + +Store this certificate and provide it to your application via the +:gelenv:`TLS_CA` or :gelenv:`TLS_CA_FILE` environment variable. + +Using in your application +------------------------- + +Set these environment variables where you deploy your application: + +.. code-block:: bash + + GEL_DSN="gel://admin:@:5656" + # For self-signed certificates: + GEL_CLIENT_TLS_SECURITY=insecure + # Or with a proper TLS certificate: + GEL_TLS_CA="" + +Gel's client libraries will automatically read these environment variables. + +Local development with the CLI +------------------------------ + +To make your remote instance easier to work with during local development, +create an alias using :gelcmd:`instance link`. + +.. note:: + + The command groups :gelcmd:`instance` and :gelcmd:`project` are not + intended to manage production instances. + +.. code-block:: bash + + $ gel instance link \ + --dsn $GEL_DSN \ + --non-interactive \ + --trust-tls-cert \ + my_aws_instance + +You can now refer to the remote instance using the alias ``my_aws_instance``. +Use this alias wherever an instance name is expected: + +.. code-block:: bash + + $ gel -I my_aws_instance + Gel x.x + Type \help for help, \quit to quit. + gel> + +Or apply migrations: + +.. code-block:: bash + + $ gel -I my_aws_instance migrate + + Health Checks ============= diff --git a/docs/reference/running/deployment/azure_flexibleserver.rst b/docs/reference/running/deployment/azure_flexibleserver.rst index c1525cd6248..b5964a88292 100644 --- a/docs/reference/running/deployment/azure_flexibleserver.rst +++ b/docs/reference/running/deployment/azure_flexibleserver.rst @@ -183,32 +183,114 @@ or reboots copy the certificate files and use their contents in the "GEL_SERVER_TLS_CERT=$cert" -To access the Gel instance you've just provisioned on Azure from your local -machine link the instance. +Connecting your application +=========================== + +To connect your application to the Gel instance, you'll need to provide +connection parameters. Gel client libraries can be configured using either +a DSN (connection string) or individual environment variables. + +Obtaining connection parameters +------------------------------- + +Your connection requires the following components: + +- **Host**: The FQDN of your Azure container instance. Retrieve it with: + + .. code-block:: bash + + $ az container list \ + --resource-group $GROUP \ + --query "[?name=='gel-container-group'].ipAddress.fqdn | [0]" \ + --output tsv + +- **Port**: ``5656`` (the default Gel port) +- **Username**: |admin| (the default superuser) +- **Password**: The password you set in the ``$PASSWORD`` variable +- **Branch**: |main| (the default branch) + +Construct the DSN using these values: + +.. code-block:: bash + + $ GEL_HOST=$(az container list \ + --resource-group $GROUP \ + --query "[?name=='gel-container-group'].ipAddress.fqdn | [0]" \ + --output tsv) + $ GEL_DSN="gel://admin:$PASSWORD@$GEL_HOST:5656" + +Obtaining the TLS certificate +----------------------------- + +Since we configured Gel with a self-signed TLS certificate, your application +needs the certificate to connect securely. Retrieve it from the container: + +.. code-block:: bash + + $ az container exec \ + --resource-group $GROUP \ + --name gel-container-group \ + --exec-command "cat /tmp/gel/edbtlscert.pem" \ + | tr -d "\r" > gel-tls-cert.pem + +Alternatively, you can retrieve it using the Gel CLI: .. code-block:: bash - $ printf $PASSWORD | gel instance link \ - --password-from-stdin \ - --non-interactive \ - --trust-tls-cert \ - --host $( \ - az container list \ - --resource-group $GROUP \ - --query "[?name=='gel-container-group'].ipAddress.fqdn | [0]" \ - --output tsv ) \ - azure + $ gel --dsn $GEL_DSN --tls-security insecure \ + query "SELECT sys::get_tls_certificate()" > gel-tls-cert.pem + +Using in your application +------------------------- + +Set these environment variables where you deploy your application: + +.. code-block:: bash + + GEL_DSN="gel://admin:@:5656" + # For self-signed certificates, either trust the cert: + GEL_TLS_CA_FILE="/path/to/gel-tls-cert.pem" + # Or (for development only) disable TLS verification: + GEL_CLIENT_TLS_SECURITY=insecure + +Gel's client libraries will automatically read these environment variables. + +Local development with the CLI +------------------------------ + +To make your remote instance easier to work with during local development, +create an alias using :gelcmd:`instance link`. .. note:: The command groups :gelcmd:`instance` and :gelcmd:`project` are not intended to manage production instances. -You can now connect to your instance. +.. code-block:: bash + + $ printf $PASSWORD | gel instance link \ + --dsn $GEL_DSN \ + --password-from-stdin \ + --non-interactive \ + --trust-tls-cert \ + my_azure_instance + +You can now refer to the remote instance using the alias ``my_azure_instance``. +Use this alias wherever an instance name is expected: + +.. code-block:: bash + + $ gel -I my_azure_instance + Gel x.x + Type \help for help, \quit to quit. + gel> + +Or apply migrations: .. code-block:: bash - $ gel -I azure + $ gel -I my_azure_instance migrate + Health Checks ============= diff --git a/docs/reference/running/deployment/bare_metal.rst b/docs/reference/running/deployment/bare_metal.rst index 7fa86cf4642..d8fa05dbc56 100644 --- a/docs/reference/running/deployment/bare_metal.rst +++ b/docs/reference/running/deployment/bare_metal.rst @@ -165,38 +165,105 @@ You may need to restart the server after changing the listen port or addresses. $ sudo systemctl restart gel-server-6 -Link the instance with the CLI -============================== +Connecting your application +=========================== -The following is an example of linking a bare metal instance that is running on -``localhost``. This command assigns a name to the instance, to make it more -convenient to refer to when running CLI commands. +To connect your application to the Gel instance, you'll need to provide +connection parameters. Gel client libraries can be configured using either +a DSN (connection string) or individual environment variables. + +Obtaining connection parameters +------------------------------- + +Your connection requires the following components: + +- **Host**: The IP address or hostname of your server (e.g., ``localhost``, + ``192.168.1.100``, or ``gel.example.com``) +- **Port**: ``5656`` by default, or the custom port if you changed it with + ``CONFIGURE INSTANCE SET listen_port`` +- **Username**: |admin| (the default superuser) +- **Password**: The password you set with ``ALTER ROLE admin SET password`` +- **Branch**: |main| (the default branch) + +Construct the DSN using these values: .. code-block:: bash - $ gel instance link \ - --host localhost \ - --port 5656 \ - --user admin \ - --branch main \ - --trust-tls-cert \ - bare_metal_instance + $ GEL_DSN="gel://admin:@:5656" + +Obtaining the TLS certificate +----------------------------- -This allows connecting to the instance with its name. +If you configured Gel with ``GEL_SERVER_TLS_CERT_MODE=generate_self_signed``, +your application needs the certificate to connect securely. + +The generated certificate is stored in the data directory. You can find it at: .. code-block:: bash - $ gel -I bare_metal_instance + $ cat /var/lib/gel/6/data/edbtlscert.pem +Alternatively, retrieve it using the Gel CLI: -Upgrading Gel -============= +.. code-block:: bash + + $ gel --dsn $GEL_DSN --tls-security insecure \ + query "SELECT sys::get_tls_certificate()" + +Using in your application +------------------------- + +Set these environment variables where you deploy your application: + +.. code-block:: bash + + GEL_DSN="gel://admin:@:5656" + # For self-signed certificates, provide the CA cert: + GEL_TLS_CA_FILE="/path/to/edbtlscert.pem" + # Or embed the certificate content directly: + GEL_TLS_CA="" + +Gel's client libraries will automatically read these environment variables. + +Local development with the CLI +------------------------------ + +To make your instance easier to work with during local development, +create an alias using :gelcmd:`instance link`. .. note:: The command groups :gelcmd:`instance` and :gelcmd:`project` are not intended to manage production instances. +.. code-block:: bash + + $ gel instance link \ + --dsn $GEL_DSN \ + --non-interactive \ + --trust-tls-cert \ + my_bare_metal_instance + +You can now refer to the instance using the alias ``my_bare_metal_instance``. +Use this alias wherever an instance name is expected: + +.. code-block:: bash + + $ gel -I my_bare_metal_instance + Gel x.x + Type \help for help, \quit to quit. + gel> + +Or apply migrations: + +.. code-block:: bash + + $ gel -I my_bare_metal_instance migrate + + +Upgrading Gel +============= + When you want to upgrade to the newest point release upgrade the package and restart the ``gel-server-6`` unit. diff --git a/docs/reference/running/deployment/docker.rst b/docs/reference/running/deployment/docker.rst index da11b646062..55c8373de54 100644 --- a/docs/reference/running/deployment/docker.rst +++ b/docs/reference/running/deployment/docker.rst @@ -276,6 +276,141 @@ been applied. script files to run on bootstrap is to prepend the filenames with ``01-``, ``02-``, and so on to indicate your desired order of execution. + +Connecting your application +=========================== + +To connect your application to the Gel instance, you'll need to provide +connection parameters. Gel client libraries can be configured using either +a DSN (connection string) or individual environment variables. + +Obtaining connection parameters +------------------------------- + +Your connection requires the following components: + +- **Host**: The container hostname or IP address. In Docker Compose, this is + the service name (e.g., ``gel``). For standalone containers, use + ``localhost`` if on the same host, or the container's IP/hostname. +- **Port**: ``5656`` (the default Gel port, unless remapped with ``-p``) +- **Username**: |admin| (the default superuser) +- **Password**: The value of :gelenv:`SERVER_PASSWORD` you set when starting + the container +- **Branch**: |main| (the default branch) + +Construct the DSN using these values: + +.. code-block:: bash + + $ GEL_DSN="gel://admin:@:5656" + +For a Docker Compose setup with the service named ``gel``: + +.. code-block:: bash + + $ GEL_DSN="gel://admin:secret@gel:5656" + +Obtaining the TLS certificate +----------------------------- + +If you configured Gel with ``GEL_SERVER_TLS_CERT_MODE=generate_self_signed``, +your application needs the certificate to connect securely. + +Retrieve the certificate from the running container: + +.. code-block:: bash + + $ docker exec cat /var/lib/gel/data/edbtlscert.pem + +Or using the Gel utility script: + +.. code-block:: bash + + $ docker exec \ + gel-show-secrets.sh --format=raw GEL_SERVER_TLS_CERT + +Alternatively, retrieve it using the Gel CLI: + +.. code-block:: bash + + $ gel --dsn $GEL_DSN --tls-security insecure \ + query "SELECT sys::get_tls_certificate()" + +If you mounted a persistent volume at :gelenv:`SERVER_DATADIR`, the +certificate is also available at ``/edbtlscert.pem``. + +Using in your application +------------------------- + +Set these environment variables in your application container: + +.. code-block:: yaml + + # docker-compose.yaml example + services: + app: + image: your-app + environment: + GEL_DSN: "gel://admin:secret@gel:5656" + # For self-signed certificates: + GEL_CLIENT_TLS_SECURITY: "insecure" + # Or provide the CA certificate: + # GEL_TLS_CA: "" + +For production, we recommend providing the TLS certificate rather than +disabling TLS verification: + +.. code-block:: yaml + + services: + app: + image: your-app + environment: + GEL_DSN: "gel://admin:${GEL_PASSWORD}@gel:5656" + GEL_TLS_CA_FILE: "/certs/gel-ca.pem" + volumes: + - ./certs:/certs:ro + +Gel's client libraries will automatically read these environment variables. + +Local development with the CLI +------------------------------ + +To make your Gel container easier to work with during local development, +create an alias using :gelcmd:`instance link`. + +.. note:: + + The command groups :gelcmd:`instance` and :gelcmd:`project` are not + intended to manage production instances. + +From your host machine, link to the container: + +.. code-block:: bash + + $ gel instance link \ + --dsn gel://admin:secret@localhost:5656 \ + --non-interactive \ + --trust-tls-cert \ + my_docker_instance + +You can now refer to the instance using the alias ``my_docker_instance``. +Use this alias wherever an instance name is expected: + +.. code-block:: bash + + $ gel -I my_docker_instance + Gel x.x + Type \help for help, \quit to quit. + gel> + +Or apply migrations: + +.. code-block:: bash + + $ gel -I my_docker_instance migrate + + Health Checks ============= diff --git a/docs/reference/running/deployment/fly_io.rst b/docs/reference/running/deployment/fly_io.rst index f34410cc4dd..fdba79b5b19 100644 --- a/docs/reference/running/deployment/fly_io.rst +++ b/docs/reference/running/deployment/fly_io.rst @@ -206,27 +206,32 @@ skip this step). | tr -d '\r' | flyctl secrets import --app $EDB_APP -Connecting to the instance -========================== +Connecting your application +=========================== + +To connect your application to the Gel instance, you'll need to provide +connection parameters. Gel client libraries can be configured using either +a DSN (connection string) or individual environment variables. + +Obtaining connection parameters +------------------------------- -Let's construct the DSN (AKA "connection string") for our instance. DSNs have -the following format: :geluri:`:@:`. We -can construct the DSN with the following components: +Your connection requires the following components: -- ````: the default value — |admin| -- ````: the value we assigned to ``$PASSWORD`` -- ````: the name of your Gel app (stored in the - ``$EDB_APP`` environment variable) suffixed with ``.internal``. Fly uses this - synthetic TLD to simplify inter-app communication. Ex: - ``myorg-gel.internal``. -- ````: ``8080``, which we configured earlier +- **Host (internal)**: ``$EDB_APP.internal`` — Fly uses this synthetic TLD + for inter-app communication (e.g., ``myorg-gel.internal``) +- **Host (external)**: ``$EDB_APP.fly.dev`` — for connections from outside + Fly.io (requires exposing the port, see below) +- **Port**: ``8080``, which we configured earlier with :gelenv:`SERVER_PORT` +- **Username**: |admin| (the default superuser) +- **Password**: The value you assigned to ``$PASSWORD`` +- **Branch**: |main| (the default branch) -We can construct this value and assign it to a new environment variable called -``DSN``. +Construct the DSN for internal Fly.io connections: .. code-block:: bash - $ DSN=gel://admin:$PASSWORD@$EDB_APP.internal:8080 + $ GEL_DSN=gel://admin:$PASSWORD@$EDB_APP.internal:8080 Consider writing it to a file to ensure the DSN looks correct. Remember to delete the file after you're done. (Printing this value to the terminal with @@ -234,10 +239,23 @@ delete the file after you're done. (Printing this value to the terminal with .. code-block:: bash - $ echo $DSN > dsn.txt + $ echo $GEL_DSN > dsn.txt $ open dsn.txt $ rm dsn.txt +Obtaining the TLS certificate +----------------------------- + +If you need secure TLS connections (required for external access), retrieve +the server's TLS certificate: + +.. code-block:: bash + + $ flyctl ssh console -a $EDB_APP \ + -C "gel-show-secrets.sh --format=raw GEL_SERVER_TLS_CERT" + +Save this to a file or set it as a secret in your application. + From a Fly.io app ----------------- @@ -309,10 +327,10 @@ You can securely obtain the certificate content by running: $ flyctl ssh console -a $EDB_APP \ -C "gel-show-secrets.sh --format=raw GEL_SERVER_TLS_CERT" -From your local machine ------------------------ +Local development with the CLI +------------------------------ -To access the Gel instance from local development machine/laptop, install +To access the Gel instance from your local development machine, install the Wireguard `VPN `_ and create a tunnel, as described on Fly's `Private Networking `_ @@ -321,31 +339,41 @@ docs. Once it's up and running, use :gelcmd:`instance link` to create a local alias to the remote instance. +.. note:: + + The command groups :gelcmd:`instance` and :gelcmd:`project` are not + intended to manage production instances. + .. code-block:: bash $ gel instance link \ - --trust-tls-cert \ - --dsn $DSN \ + --dsn $GEL_DSN \ --non-interactive \ - fly - Authenticating to gel://admin@myorg-gel.internal:5656/main + --trust-tls-cert \ + my_fly_instance + Authenticating to gel://admin@myorg-gel.internal:8080/main Successfully linked to remote instance. To connect run: - gel -I fly + gel -I my_fly_instance -You can now run CLI commands against this instance by specifying it by name -with ``-I fly``; for example, to apply migrations: +You can now refer to the remote instance using the alias ``my_fly_instance``. +Use this alias wherever an instance name is expected: -.. note:: +.. code-block:: bash - The command groups :gelcmd:`instance` and :gelcmd:`project` are not - intended to manage production instances. + $ gel -I my_fly_instance + Gel x.x + Type \help for help, \quit to quit. + gel> + +Or apply migrations: .. code-block:: bash - $ gel -I fly migrate + $ gel -I my_fly_instance migrate .. _vpn: https://fly.io/docs/reference/private-networking/#private-network-vpn + Health Checks ============= diff --git a/docs/reference/running/deployment/gcp.rst b/docs/reference/running/deployment/gcp.rst index 6613612c6a8..3d2607ddd28 100644 --- a/docs/reference/running/deployment/gcp.rst +++ b/docs/reference/running/deployment/gcp.rst @@ -198,10 +198,25 @@ Expose Gel $ kubectl expose deploy/gel --type LoadBalancer -Get your instance's DSN -======================= +Connecting your application +=========================== + +To connect your application to the Gel instance, you'll need to provide +connection parameters. Gel client libraries can be configured using either +a DSN (connection string) or individual environment variables. + +Obtaining connection parameters +------------------------------- + +Your connection requires the following components: + +- **Host**: The ``EXTERNAL-IP`` of the LoadBalancer service +- **Port**: ``5656`` (the default Gel port) +- **Username**: |admin| (the default superuser) +- **Password**: The value you assigned to ``$PASSWORD`` +- **Branch**: |main| (the default branch) -Get the public-facing IP address of your database. +Get the public-facing IP address of your database: .. code-block:: bash @@ -209,10 +224,8 @@ Get the public-facing IP address of your database. NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) gel LoadBalancer 5656:30841/TCP - -Copy and paste the ``EXTERNAL-IP`` associated with the service named -``gel``. With this IP address, you can construct your instance's :ref:`DSN -`: +Copy the ``EXTERNAL-IP`` associated with the ``gel`` service and construct +your instance's :ref:`DSN `: .. code-block:: bash @@ -227,8 +240,26 @@ environment. $ echo $GEL_DSN -The resuling DSN can be used to connect to your instance. -To test it, try opening a REPL: +Obtaining the TLS certificate +----------------------------- + +Since we configured Gel with a self-signed TLS certificate, your application +needs the certificate to connect securely. Retrieve it from the running pod: + +.. code-block:: bash + + $ kubectl exec deploy/gel -c=gel -- \ + gel-show-secrets.sh --format=raw GEL_SERVER_TLS_CERT \ + > gel-tls-cert.pem + +Alternatively, retrieve it using the Gel CLI: + +.. code-block:: bash + + $ gel --dsn $GEL_DSN --tls-security insecure \ + query "SELECT sys::get_tls_certificate()" > gel-tls-cert.pem + +Test your connection by opening a REPL: .. code-block:: bash @@ -237,11 +268,11 @@ To test it, try opening a REPL: Type \help for help, \quit to quit. gel> select "hello world!"; -In development --------------- +Local development with the CLI +------------------------------ -To make this instance easier to work with during local development, create an -alias using :gelcmd:`instance link`. +To make your remote instance easier to work with during local development, +create an alias using :gelcmd:`instance link`. .. note:: @@ -255,29 +286,41 @@ alias using :gelcmd:`instance link`. --password-from-stdin \ --non-interactive \ --trust-tls-cert \ - gcp_instance + my_gcp_instance -You can now refer to the remote instance using the alias instance on your -machine called ``gcp_instance``. You can use this alias wherever an instance -name is expected; for instance, you can open a REPL: +You can now refer to the remote instance using the alias ``my_gcp_instance``. +Use this alias wherever an instance name is expected: .. code-block:: bash - $ gel -I gcp_instance + $ gel -I my_gcp_instance + Gel x.x + Type \help for help, \quit to quit. + gel> Or apply migrations: .. code-block:: bash - $ gel -I gcp_instance migrate + $ gel -I my_gcp_instance migrate + +Using in your application +------------------------- + +Set these environment variables where you deploy your application: + +.. code-block:: bash + + GEL_DSN="gel://admin:@:5656" + # For self-signed certificates, provide the CA cert: + GEL_TLS_CA_FILE="/path/to/gel-tls-cert.pem" + # Or embed the certificate content directly: + GEL_TLS_CA="" + # Or (for development only) disable TLS verification: + # GEL_CLIENT_TLS_SECURITY=insecure -In production -------------- +Gel's client libraries will automatically read these environment variables. -To connect to this instance in production, set the :gelenv:`DSN` environment -variable wherever you deploy your application server; Gel's client -libraries read the value of this variable to know how to connect to your -instance. Health Checks ============= From 528d5ace182a994ebc03ccc8053856084c577ac9 Mon Sep 17 00:00:00 2001 From: Scott Trinh Date: Tue, 23 Dec 2025 16:50:06 -0500 Subject: [PATCH 6/6] Remove streaming guidance This only works for a single branch, and it's safer to just have the intermediate file anyway, so let's remove a choice here. --- docs/cloud/migrate_from.rst | 31 ++++++------------------------- 1 file changed, 6 insertions(+), 25 deletions(-) diff --git a/docs/cloud/migrate_from.rst b/docs/cloud/migrate_from.rst index 1615c0a2a61..237a79e0928 100644 --- a/docs/cloud/migrate_from.rst +++ b/docs/cloud/migrate_from.rst @@ -82,31 +82,6 @@ the dump. You can use the |Gel| CLI to move data directly from your Cloud instance to your new self-hosted instance. -.. note:: - - If your self-hosted instance uses a self-signed TLS certificate, you may - need to add ``--tls-security insecure`` to the restore command, or first - retrieve the TLS certificate and set it via :gelenv:`TLS_CA`. - -Option A: Streaming (Recommended) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -You can pipe the output of :gelcmd:`dump` directly into :gelcmd:`restore`. -This is the fastest method as it doesn't require saving a large file to your -local disk. - -.. code-block:: bash - - # Set your Gel Cloud instance as the source and the new DSN as the target - $ gel dump --instance / --all \ - | gel restore --dsn --all - -Option B: File-based (Safer for large DBs) -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If you have a very large database or a shaky internet connection, saving to a -file first is safer. - .. code-block:: bash # 1. Dump from Gel Cloud to directory @@ -117,6 +92,12 @@ file first is safer. # 2. Restore to self-hosted from dump $ gel restore --dsn --all production_dump +.. note:: + + If your self-hosted instance uses a self-signed TLS certificate, you may + need to add ``--tls-security insecure`` to the restore command, or first + retrieve the TLS certificate and set it via :gelenv:`TLS_CA`. + Phase 3: Verification and Go-Live =================================