diff --git a/README.md b/README.md index 89cefae66..c7742abd2 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,56 @@ -

pubky

+

Pubky

- An open protocol for per-public-key backends for censorship resistant web applications. + Homeserver and SDKs for Pubky.

- - -

- - Docs Site - + Docs | - - Rust Client's Docs - + Rust SDK | - | - - JS bindings - + JavaScript SDK

+ GitHub License + GitHub Release + Crates.io Version + npm Version +
+ Telegram Chat Group + Ask DeepWiki
-[![GitHub Release](https://img.shields.io/github/v/release/pubky/pkdns)](https://github.com/pubky/pubky-core/releases/latest/) -[![Crates.io Version](https://img.shields.io/crates/v/pubky)](https://crates.io/crates/pubky) -[![Telegram Chat Group](https://img.shields.io/badge/Chat-Telegram-violet)](https://t.me/pubkycore) -[![GitHub License](https://img.shields.io/github/license/pubky/pubky-core)](https://github.com/pubky/pubky-core/blob/main/LICENSE) -[![Ask DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/pubky/pubky-core) - -> The Web, long centralized, must decentralize; Long decentralized, must centralize. - -## Overview - -Pubky-core combines a [censorship resistant public-key based alternative to DNS](https://pkarr.org) with conventional, tried and tested web technologies, to keep users in control of their identities and data, while enabling developers to build software with as much availability as web apps, without the costs of managing a central database. - -## Features -- Public key based authentication. -- Public key based 3rd party authorization. -- Key-value store through PUT/GET/DELETE HTTP API + pagination. - -## Getting started - -This repository contains a [Homeserver](./pubky-homeserver), and a [SDK](./pubky-sdk) (both Rust and JS wasm bindings). -You can a run a local homeserver using `cargo run` with more instructions in the README. -Check the [Examples](./examples) directory for small feature-focesed examples of how to use the Pubky SDK. +
-### JavaScript -If you prefer to use JavaScript in NodeJs/Browser or any runtime with Wasm support, you can either install from npm [`@synonymdev/pubky`](https://www.npmjs.com/package/@synonymdev/pubky) -or build the bindings yourself: -```bash -cd pubky-sdk/bindings/js/pkg -npm i -npm run build -``` +[Pubky](https://pubky.org) is an open protocol for building censorship-resistant applications where users own their identity, data, and connections. No platform lock-in, no losing everything when a service shuts down. Your keys are your identity, and you choose where your data lives. -#### Testing -There are unit tests for the JavaScript bindings in both NodeJs and headless web browser, but first you need to run a local temporary Homeserver -```bash -npm run testnet -``` -Then in a different terminal window: -```bash -npm test -``` +To learn more about the vision, see [What is Pubky?](https://pubky.org/tldr/), [Censorship Resistance](https://pubky.org/explore/concepts/censorship/), and [Credible Exit](https://pubky.org/explore/concepts/credible-exit/). -### Docker +This repository contains the core infrastructure: a homeserver that stores and serves user data, Rust and JavaScript SDKs for building apps, a local testnet, and examples. -An alternative way to start tinkering with Pubky is to build an isolated container and run it locally. Here is an -example command how to build an image: +## Who Is This For? -```bash -docker build --build-arg TARGETARCH=x86_64 -t pubky:core . -``` +- **Operators**: [Install and run a homeserver](./docs/INSTALL.md) to host user data. +- **App developers**: Use the [SDK](https://pubky.org/explore/pubkycore/sdk/) to build apps that read and write user data on homeservers. Guides coming soon. -A command for running it in an isolated environment with log output: +## What Is a Homeserver? -```bash -docker run -it pubky:core -``` +A Pubky homeserver stores and serves user data. Users choose which homeserver holds their data, and can move to another at any time. The homeserver exposes HTTP APIs for authenticated writes and public reads, and publishes [PKARR](https://github.com/pubky/pkarr) records so other clients can discover where a user's data lives. -Additional optional arguments can be used to run it in the background, but the most important is `--network=host`, which allows the container to access the network and provides an admin endpoint accessible from the host machine. Please refer to the Docker documentation for more detailed options. +- Public-key based sign-up, sign-in and third-party app authorization. +- File storage via HTTP `PUT`, `GET`, `DELETE`, and listing APIs (WebDAV-like). +- PKARR/PKDNS publishing for homeserver discovery. +- Admin and metrics endpoints for operators. -## Links +## Repository Layout -- [Contributors Guide](./CONTRIBUTORS.md) +| Path | Purpose | +| --- | --- | +| [`pubky-homeserver`](./pubky-homeserver) | Homeserver binary and library crate. | +| [`pubky-sdk`](./pubky-sdk) | Rust client for Pubky apps, plus JS/WASM bindings. | +| [`pubky-common`](./pubky-common) | Shared types and helpers used by the SDK and homeserver. | +| [`pubky-testnet`](./pubky-testnet) | Local ephemeral Pubky network for development and tests. | +| [`examples`](./examples) | Rust and JavaScript examples for signup, auth, storage, and requests. | +| [`e2e`](./e2e) | End-to-end tests covering cross-crate workflows. | +| [`docs`](./docs) | Install guides, local development, and testing docs. | diff --git a/docs/DEV_TESTING_GUIDES.md b/docs/DEV_TESTING_GUIDES.md deleted file mode 100644 index ed73d8bb3..000000000 --- a/docs/DEV_TESTING_GUIDES.md +++ /dev/null @@ -1,46 +0,0 @@ -# Developer / Testing Guides - -This documents describes common problems and their solution that developers or tester encounter. - - -## Postgres - -The easiest way to run postgres is with a docker container - -```bash -docker run --name postgres \ - -e POSTGRES_USER=postgres \ - -e POSTGRES_PASSWORD=postgres \ - -e POSTGRES_DB=pubky_homeserver \ - -p 127.0.0.1:5432:5432 \ - -d postgres:18-alpine -``` - -This command creates a postgres container and also automatically creates the `pubky_homeserver` database. Use this connection string in the homeserver config: - -```toml -[general] -database_url = "postgres://postgres:postgres@localhost:5432/pubky_homeserver" -``` - -[pgadmin](https://www.pgadmin.org/) is a great explorer to inspect database values. - -### Create Database Manually - -The docker image creates the database automatically. If you have postgres installed with a different method, you need to create the database manually. - -This can be done with psql. - -```bash -sudo -u postgres psql -c 'create database pubky_homeserver;' -``` - - -## Test Databases - -If compiled with the `testing` feature, `?pubky-test=true` can be added to the database url. -This way, an empheral test database is created and dropped after the test. - -**Example** `postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true` For each test, the homeserver will connect to the -specified database (postgres in this case) and create a new test database. With the `#[pubky_test_utils::test]` macro, the test database is -dropped again after the test completes/panics. diff --git a/docs/GETTING_STARTED.md b/docs/GETTING_STARTED.md deleted file mode 100644 index eeb6775a8..000000000 --- a/docs/GETTING_STARTED.md +++ /dev/null @@ -1,40 +0,0 @@ -# Getting Started - -## Install Binaries - -Downloading and installing an official release binary is recommended for use on mainnet. [Visit the release page on GitHub](https://github.com/pubky/pubky-core/releases) and select the latest version that does not have the "Pre-release" label set (unless you explicitly want to help test a Release Candidate, RC). - -Choose the package that best fits your operating system and system architecture. It is recommended to choose 64bit versions over 32bit ones, if your operating system supports both. - -Extract the package and place the binary (pubky-homeserver or pubky-homeserver.exe on Windows) somewhere where the operating system can find them. - -## Run - -> The homeserver depends on Postgresql as its database backend. Make sure you have it installed. - -Executing the `pubky-homeserver` binary will create the `~/.pubky` application folder. All data related to the homeserver is stored there. A sample `config.toml` will be written to application folder. -It includes all important configuration values. - -For the homeserver to start, it needs the correct `database_url` to connect to the postgres database. - -```toml -# Example database url in ~/.pubky/config.toml -database_url = "postgres://username:password@localhost:5432/pubky_homeserver" -``` - -Make sure to create the `pubky_homeserver` database in postgres so the homeserver can connect to it. -On a local Linux installation, you create the database with this command: - -```bash -sudo -u postgres psql -c 'create database pubky_homeserver;' -``` - -Setting the database_url is enough to run the homeserver locally: - -```bash -# Start the homeserver -./pubky_homeserver -``` - - - diff --git a/docs/INSTALL.md b/docs/INSTALL.md new file mode 100644 index 000000000..e98f55936 --- /dev/null +++ b/docs/INSTALL.md @@ -0,0 +1,328 @@ +# Install and Run Pubky Homeserver + +How to set up and operate a Pubky homeserver on Linux. Commands and package names assume a Debian-based system (Ubuntu, Debian, etc.), adapt as needed for other distributions. + +> **Looking for something else?** +> See [Pubky Testnet](../pubky-testnet/README.md) for running a local development testnet and [Testing](./TESTING.md) for test databases and CI setup. + +## Contents + +- [Install the Homeserver](#install-the-homeserver) + - [Release Binary](#release-binary) | [Build From Source](#build-from-source) ([Cargo](#build-a-binary-with-cargo) | [Docker](#build-a-docker-image)) + - [Initialise the Data Directory](#initialise-the-data-directory) + - [Set Up PostgreSQL](#set-up-postgresql) + - [Docker](#docker-1) | [Native](#native) | [Existing](#existing-instance) + - [Configure the Homeserver with PostgreSQL](#configure-the-homeserver-with-postgresql) + - [Run](#run) +- [Configuration](#configuration) +- [Production Notes](#production-notes) +- [Troubleshooting](#troubleshooting) + + +## Install the Homeserver + +Pick a version and platform from the [Pubky Core releases page](https://github.com/pubky/pubky-core/releases). The commands below use these variables, so set them first: + +```bash +PUBKY_CORE_VERSION=0.x +PUBKY_CORE_PLATFORM=linux-amd64 # or linux-arm64. Alternatively: osx-arm64, osx-amd64, windows-amd64 +``` + +### Release Binary + +Download and extract the archive (requires `curl`; `sudo apt install curl`): + +```bash +curl -LO https://github.com/pubky/pubky-core/releases/download/v${PUBKY_CORE_VERSION}/pubky-core-v${PUBKY_CORE_VERSION}-${PUBKY_CORE_PLATFORM}.tar.gz +tar -xf pubky-core-v${PUBKY_CORE_VERSION}-${PUBKY_CORE_PLATFORM}.tar.gz +``` + +Place the binary on your `PATH`: + +```bash +cp pubky-core-v${PUBKY_CORE_VERSION}-${PUBKY_CORE_PLATFORM}/pubky-homeserver /usr/local/bin +``` + +Verify the install: + +```bash +pubky-homeserver --version +``` + +### Build From Source + +Install build dependencies: + +```bash +sudo apt update && sudo apt install -y build-essential pkg-config libssl-dev git curl +``` + +Clone the repository: + +```bash +git clone https://github.com/pubky/pubky-core.git +cd pubky-core +git checkout v${PUBKY_CORE_VERSION} +``` + +#### Build a binary with Cargo + +Make sure you have the Rust toolchain installed and working. + +
+How to Install the Rust Toolchain + +Quick setup using [rustup](https://rustup.rs/) (recommended): + +```bash +curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh +source ~/.cargo/env +``` + +For other methods, see the [Rust Install Guide](https://rust-lang.org/tools/install/). + +
+ +Build and place the binary on your `PATH`: + +```bash +cargo build --release -p pubky-homeserver +cp ./target/release/pubky-homeserver /usr/local/bin +``` + +Verify the install: + +```bash +pubky-homeserver --version +``` + +#### Build a Docker image + +Requires [Docker Engine](https://docs.docker.com/engine/install/). + +Build the homeserver image using the [Dockerfile](../Dockerfile): + +```bash +docker build --build-arg BUILD_TARGET=homeserver -t pubky-homeserver . +``` + +Verify the image built correctly: + +```bash +docker run --rm pubky-homeserver homeserver --version +``` + +## Initialise the Data Directory + +Create the data directory, default `config.toml`, and server keypair without starting the server or connecting to PostgreSQL: + +```bash +pubky-homeserver init +``` + +With Docker: + +```bash +docker run -it -v ~/.pubky:/root/.pubky pubky-homeserver homeserver init +``` + +> **Note:** The `init` subcommand is available from v0.10 onwards. On v0.9 or earlier, the data directory is created automatically on first run. Start the homeserver once (it will fail if PostgreSQL is not yet configured, but the directory, sample config, and keypair will already be written to `~/.pubky/`). + +This creates `~/.pubky/` with a sample config and a fresh server keypair. To use a different path: + +```bash +pubky-homeserver --data-dir /path/to/pubky-data init +``` + +## Set Up PostgreSQL + +The homeserver requires a running PostgreSQL instance with an empty database. + +### Docker + +Requires [Docker Engine](https://docs.docker.com/engine/install/ubuntu/). + +Start a PostgreSQL container with the `pubky_homeserver` database: + +```bash +docker run --name pubky-postgres \ + -e POSTGRES_USER=postgres \ + -e POSTGRES_PASSWORD=postgres \ + -e POSTGRES_DB=pubky_homeserver \ + -p 127.0.0.1:5432:5432 \ + -v postgres-data:/var/lib/postgresql \ + -d postgres:18 +``` + +### Native + +Install PostgreSQL: + +```bash +sudo apt update && sudo apt install -y postgresql +``` + +Start the server (not needed on systems with `systemd`, where PostgreSQL starts automatically): + +```bash +pg_ctlcluster $(pg_lsclusters -h | awk '{print $1, $2}') start +``` + +Set a password and create the database: + +```bash +sudo -u postgres psql -c "ALTER USER postgres PASSWORD 'postgres';" +sudo -u postgres createdb pubky_homeserver +``` + +Verify the connection: + +```bash +psql "postgres://postgres:postgres@localhost:5432/pubky_homeserver" -c '\conninfo' +``` + +### Existing instance + +Create a database on your existing PostgreSQL instance: + +```bash +createdb -h -U pubky_homeserver +``` + +Verify the connection: + +```bash +psql "postgres://:@:5432/pubky_homeserver" -c '\conninfo' +``` + +## Configure the Homeserver with PostgreSQL + +Uncomment and set `database_url` in `~/.pubky/config.toml`. For the Docker and Native example setup above, it should look like: + +```toml +[general] +database_url = "postgres://postgres:postgres@localhost:5432/pubky_homeserver" +``` + +Here's a handy sed command to edit as above: + +```bash +sed -i 's|^# \[general\]|[general]|; s|^# database_url = .*|database_url = "postgres://postgres:postgres@localhost:5432/pubky_homeserver"|' ~/.pubky/config.toml +``` + +## Run + +Start the homeserver: + +```bash +pubky-homeserver +``` + +With Docker: + +```bash +docker run -it --network=host -v ~/.pubky:/root/.pubky pubky-homeserver homeserver +``` + +Use `--network=host` so the container can reach PostgreSQL on the host and expose its endpoints. The volume mount shares the data directory (config and keypair) with the container. + +The default endpoints are: + +| Endpoint | Default | +| --- | --- | +| Public HTTP API | `http://127.0.0.1:6286` | +| Pubky TLS API | `127.0.0.1:6287` | +| Admin API | `http://127.0.0.1:6288` | + +Standalone homeservers require signup tokens by default. Generate one through the admin API: + +```bash +curl -X GET "http://127.0.0.1:6288/generate_signup_token" \ + -H "X-Admin-Password: admin" +``` + +## Configuration + +Important settings in `config.toml`: + +| Setting | Purpose | +| --- | --- | +| `general.database_url` | PostgreSQL connection string. | +| `drive.icann_listen_socket` | Regular HTTP API listen address. | +| `drive.pubky_listen_socket` | Pubky TLS API listen address. | +| `storage.type` | Storage backend: `file_system`, `google_bucket`, or `in_memory`. | +| `admin.enabled` | Enables the admin API. | +| `admin.listen_socket` | Admin API listen address. | + +Review the full documented sample at [`pubky-homeserver/config.sample.toml`](../pubky-homeserver/config.sample.toml). + +## Production Notes + +Before using a homeserver in production: + +- Back up the homeserver's state: + - The keypair `.pubky/secret` + - Any user data depending on the configured option. For example, by default files are saved in `.pubky/data/files` + - Postgres +- Do not expose the admin or metrics APIs to the public internet. +- Change the default admin password in `[admin].admin_password`. +- Configure `pkdns.public_ip`, `pkdns.icann_domain`, and public ports for your deployment. +- The homeserver exposes two sockets: a **Pubky TLS** socket (`pubky_listen_socket`, default port 6287) and a regular **HTTP** socket (`icann_listen_socket`, default port 6286). Pubky TLS uses PKARR-based TLS and does not need a certificate, so can be exposed directly. The HTTP socket serves browsers and should be put behind a reverse proxy if you need standard HTTPS with a domain certificate. + +## Troubleshooting + +### `database "pubky_homeserver" does not exist` + +Create the database. With a native PostgreSQL install: + +```bash +createdb -h -U pubky_homeserver +``` + +Or if PostgreSQL is running in Docker: + +```bash +docker exec pubky-postgres createdb -U postgres pubky_homeserver +``` + +Or update `[general].database_url` in `~/.pubky/config.toml` to point at an existing database. + +### PostgreSQL Connection Refused + +Check that the `host` and `port` in `general.database_url` match where PostgreSQL is actually listening. + +**1. Is PostgreSQL running?** + +Native install: + +```bash +pg_isready +``` + +Docker: + +```bash +docker exec pubky-postgres pg_isready +``` + +If it reports "no response" then start or restart PostgreSQL. + +**2. Can you connect with the configured credentials?** + +Test the exact connection string from your `config.toml`. With a native install: + +```bash +psql "postgres://postgres:postgres@localhost:5432/pubky_homeserver" -c '\conninfo' +``` + +If PostgreSQL is running in Docker: + +```bash +docker exec pubky-postgres psql -U postgres -d pubky_homeserver -c '\conninfo' +``` + +If this fails with "password authentication failed", check the username and password. If it fails with "connection refused", PostgreSQL may be listening on a different address or port - check `listen_addresses` and `port` in `postgresql.conf`. + +### Invalid Configuration + +Compare your config with [`pubky-homeserver/config.sample.toml`](../pubky-homeserver/config.sample.toml). If the config was generated on first run, the file is safe to edit in place. diff --git a/docs/TESTING.md b/docs/TESTING.md new file mode 100644 index 000000000..e14664255 --- /dev/null +++ b/docs/TESTING.md @@ -0,0 +1,120 @@ +# Testing + +This guide is for contributors running Rust tests, integration tests, or CI jobs. For writing tests with a local testnet (ephemeral or persistent), see the [pubky-testnet README](../pubky-testnet/README.md). + +## PostgreSQL for Tests + +Many homeserver and testnet tests need PostgreSQL. Start a local instance with Docker: + +```bash +docker run --name pubky-postgres \ + -e POSTGRES_USER=postgres \ + -e POSTGRES_PASSWORD=postgres \ + -p 127.0.0.1:5432:5432 \ + -d postgres:18 +``` + +Then run tests with a test connection string: + +```bash +TEST_PUBKY_CONNECTION_STRING='postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true' \ + cargo test -p pubky-homeserver --all-features +``` + +The `?pubky-test=true` parameter tells the test helpers to create an ephemeral `pubky_test_*` database inside the configured PostgreSQL instance. Databases are cleaned up after each test. + +## Automatic Database Cleanup + +Use the Pubky test macro to ensure ephemeral PostgreSQL databases are cleaned up: + +```rust +#[tokio::test] +#[pubky_testnet::test] +async fn my_test() { + // test code +} +``` + +The macro ensures registered test databases are dropped after the test completes or panics. + +## Docker PostgreSQL in Tests + +The `docker-postgres` feature lets tests automatically start a PostgreSQL container via Docker, removing the need for an external postgres instance. It's used by the [Rust examples](../examples/rust) and the pubky-testnet integration tests. It's a good option for self-contained tests or CI environments where you don't want to manage a separate database. + +Enable it in your crate: + +```toml +[dev-dependencies] +pubky-testnet = { version = "", features = ["docker-postgres"] } +``` + +### Per-test container + +Each `.with_docker_postgres()` call starts a separate PostgreSQL container. Simple but expensive for large test suites: + +```rust +use pubky_testnet::EphemeralTestnet; + +#[tokio::test] +async fn my_test() { + let testnet = EphemeralTestnet::builder() + .with_docker_postgres() + .build() + .await + .unwrap(); + + let homeserver = testnet.homeserver_app(); +} +``` + +### Shared container + +For many tests, share one Docker PostgreSQL instance with `DockerPostgres::shared()`. Each testnet still creates its own ephemeral database, so test data remains isolated: + +```rust +use pubky_testnet::docker_postgres::DockerPostgres; +use pubky_testnet::EphemeralTestnet; + +#[tokio::test] +async fn test_one() { + let pg = DockerPostgres::shared().await; + let testnet = EphemeralTestnet::builder() + .postgres(pg.connection_string().unwrap()) + .build() + .await + .unwrap(); + + let homeserver = testnet.homeserver_app(); +} +``` + +## End-to-End Tests + +The [`e2e`](../e2e) crate contains tests that cover cross-crate workflows using `pubky-testnet`. Run them with: + +```bash +TEST_PUBKY_CONNECTION_STRING='postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true' \ + cargo test -p e2e +``` + +## Common Commands + +Run the homeserver tests against external PostgreSQL: + +```bash +TEST_PUBKY_CONNECTION_STRING='postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true' \ + cargo test -p pubky-homeserver --all-features +``` + +Run the testnet tests with Docker PostgreSQL: + +```bash +cargo test -p pubky-testnet --features docker-postgres +``` + +Run the full workspace test suite: + +```bash +TEST_PUBKY_CONNECTION_STRING='postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true' \ + cargo test --workspace --all-features +``` diff --git a/e2e/README.md b/e2e/README.md index 868b8f0a6..a9e932dae 100644 --- a/e2e/README.md +++ b/e2e/README.md @@ -14,4 +14,4 @@ the individual components and not all of Pubky. ### E2E Testing E2E tests cover multiple workspace members. Test full workflows. -It's recommended to use `pubky-testnet` which provides a convinient way to run Pubky components. \ No newline at end of file +It's recommended to use `pubky-testnet` which provides a convenient way to run Pubky components. diff --git a/pubky-homeserver/README.md b/pubky-homeserver/README.md index 56c06a470..4e08f6443 100644 --- a/pubky-homeserver/README.md +++ b/pubky-homeserver/README.md @@ -1,126 +1,81 @@ # Pubky Homeserver -Pubky homeserver that acts as user's agent on the Internet, providing data availability and more. +A homeserver for Pubky. Stores and serves user data via HTTP APIs with public-key authentication. -## Usage +For standalone deployment, see the [install guide](../docs/INSTALL.md). -### Library +## Development -Use the Homeserver as a library in other crates/binaries or for testing purposes. -The `HomeserverApp` is all bells and wistles included. +Run the homeserver directly from the source tree: -```rust -use anyhow::Result; -use pubky_homeserver::HomeserverApp; +```bash +cargo run -p pubky-homeserver -- --data-dir ~/.pubky +``` -#[tokio::main] -async fn main() -> anyhow::Result<()> { - let app = HomeserverApp::run_with_data_dir_path(PathBuf::from("~/.pubky")).await?; - println!( - "Homeserver HTTP listening on {}", - server.core().icann_http_url() - ); - println!( - "Homeserver Pubky TLS listening on {} and {}", - server.core().pubky_tls_dns_url(), - server.core().pubky_tls_ip_url() - ); - println!( - "Admin server listening on http://{}", - server.admin().listen_socket() - ); - tokio::signal::ctrl_c().await?; - - println!("Shutting down Homeserver"); - Ok(()) -} +See [config.sample.toml](config.sample.toml) for all configuration options. + +## Library Usage + +Use the homeserver as a library in other crates or for testing. + +```toml +[dependencies] +pubky-homeserver = "0.x" # replace with the latest version ``` -Run the app with a temporary directory and your custom config. This is a good way to test the server. +`HomeserverApp` starts the full server stack (client server, admin server, metrics server, DHT republishers): ```rust -use anyhow::Result; -use pubky_homeserver::{HomeserverApp, DataDirMock}; +use pubky_homeserver::HomeserverApp; +use std::path::PathBuf; #[tokio::main] async fn main() -> anyhow::Result<()> { - let mut config = ConfigToml::default(); // Use ConfigToml::test() for random ports. - // Set config values however you like - config.admin.admin_password = "alternative_password".to_string(); - // Creates a temporary directory that gets cleaned up - // as soon as the app is dropped. - let mock_dir = DataDirMock::new(config, None).unwrap(); - let app = HomeserverApp::run_with_data_dir_mock(mock_dir).await.unwrap(); + let app = HomeserverApp::start_with_persistent_data_dir_path( + PathBuf::from("~/.pubky") + ).await?; + + println!("Homeserver HTTP: {}", app.icann_http_url()); + println!("Homeserver Pubky TLS: {}", app.pubky_url()); + + if let Some(admin) = app.admin_server() { + println!("Admin server: http://{}", admin.listen_socket()); + } + + tokio::signal::ctrl_c().await?; + Ok(()) } ``` -Run the `HomeserverCore` only without the admin server. +For testing, use `MockDataDir` to create a temporary directory that is cleaned up on drop. Enable the `testing` feature: -```rust -use anyhow::Result; -use pubky_homeserver::HomeserverCore; +```toml +[dev-dependencies] +pubky-homeserver = { version = "0.x", features = ["testing"] } +``` -#[tokio::main] -async fn main() -> anyhow::Result<()> { - let mut core = HomeserverCore::from_data_dir_path(PathBuf::from("~/.pubky")).await?; - core.listen().await?; - println!( - "Homeserver HTTP listening on {}", - core().icann_http_url() - ); - println!( - "Homeserver Pubky TLS listening on {} and {}", - core().pubky_tls_dns_url(), - core().pubky_tls_ip_url() - ); -} +```rust,ignore +use pubky_homeserver::{HomeserverApp, MockDataDir, ConfigToml}; + +let config = ConfigToml::default_test_config(); +let mock_dir = MockDataDir::new(config, None).unwrap(); +let app = HomeserverApp::start_with_mock_data_dir(mock_dir).await.unwrap(); ``` ### Binary -Use `cargo run -- --data-dir=~/.pubky`. +See [Install and Run Pubky Homeserver](../docs/INSTALL.md) for full setup instructions. + +```bash +pubky-homeserver --data-dir ~/.pubky +``` ## Caching and Proxies Tenant-private responses must never be stored by shared caches. `/priv/...` data responses and `/events-stream` use `Cache-Control: no-store` and vary on `pubky-host`, `Authorization`, and `Cookie`; `/pub/...` file validators keep -their existing tenant-aware caching behavior. +their existing tenant-aware caching behavior. Note: CORS preflight `OPTIONS` is -handled upstream by the CORS layer and carries no private body. - -## Signup Token - -If homeserver is set to require signup tokens, you can create a new signup token using the admin endpoint: - -```rust,ignore -let response = pubky_client - .get(&format!("http://127.0.0.1:6288/generate_signup_token")) - .header("X-Admin-Password", "admin") // Use your admin password. This is testnet default pwd. - .send() - .await - .unwrap(); -let signup_token = response.text().await.unwrap(); -``` - -via CLI with `curl` - -```bash -curl -X GET "http://127.0.0.1:6288/generate_signup_token" \ - -H "X-Admin-Password: admin" - # Use your admin password. This is testnet default pwd. -``` - -or from JS - -```js -const url = "http://127.0.0.1:6288/generate_signup_token"; -const response = await client.fetch(url, { - method: "GET", - headers: { - "X-Admin-Password": "admin", // use your admin password, defaults to testnet password. - }, -}); -const signupToken = await response.text(); -``` +handled upstream by the CORS layer and carries no private body. \ No newline at end of file diff --git a/pubky-testnet/README.md b/pubky-testnet/README.md index 5d18f0728..8f305d50c 100644 --- a/pubky-testnet/README.md +++ b/pubky-testnet/README.md @@ -9,36 +9,20 @@ Two testnet types are provided: | [`EphemeralTestnet`] | Random | In-memory | Automated tests (`#[tokio::test]`) - parallel-safe, no port conflicts | | [`StaticTestnet`] | Fixed, well-known | In-memory or persistent | Interactive / CLI use - browser tests, mobile apps, manual debugging | -## Table of Contents - -- [Prerequisites](#prerequisites) -- [EphemeralTestnet (Automated Tests)](#ephemeraltestnet-automated-tests) - - [Writing Tests](#writing-tests) - - [Docker PostgreSQL](#docker-postgresql) - - [Sharing Docker Postgres Across Tests](#sharing-docker-postgres-across-tests) - - [Custom Configuration](#custom-configuration) -- [StaticTestnet (CLI / Interactive)](#statictestnet-cli--interactive) - - [Fixed Ports](#fixed-ports) - - [In-Memory Mode](#in-memory-mode) - - [Persistent Mode](#persistent-mode) - - [Custom Homeserver Config](#custom-homeserver-config) -- [Troubleshooting](#troubleshooting) - ## Prerequisites All testnet modes require a PostgreSQL database. You can either: - **Use Docker Postgres** (recommended for tests) — enable the `docker-postgres` feature, no external setup needed. -- **Run your own Postgres** — set the `TEST_PUBKY_CONNECTION_STRING` environment variable. +- **Run your own Postgres** - set the `TEST_PUBKY_CONNECTION_STRING` environment variable. ```bash # Example: start a local Postgres container docker run --name pubky-postgres \ -e POSTGRES_USER=postgres \ -e POSTGRES_PASSWORD=postgres \ - -e POSTGRES_DB=pubky_homeserver \ -p 127.0.0.1:5432:5432 \ - -d postgres:18-alpine + -d postgres:18 ``` The `TEST_PUBKY_CONNECTION_STRING` environment variable is used by both testnet types to configure the database connection. @@ -75,7 +59,7 @@ For testing without a separate Postgres installation, enable the `docker-postgre ```toml [dev-dependencies] -pubky-testnet = { version = "0.9", features = ["docker-postgres"] } +pubky-testnet = { version = "", features = ["docker-postgres"] } ``` ```rust,no_run @@ -139,6 +123,37 @@ async fn test_two() { Each testnet still gets its own ephemeral database within the shared PostgreSQL instance, so tests remain isolated. +### External PostgreSQL + +If you prefer to use an external Postgres instance, set the `TEST_PUBKY_CONNECTION_STRING` environment variable: + +```bash +TEST_PUBKY_CONNECTION_STRING='postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true' \ + cargo test -p my-crate +``` + +Or pass the connection string programmatically: + +```rust,no_run +use pubky_testnet::{EphemeralTestnet, pubky_homeserver::ConnectionString}; + +#[tokio::test] +#[pubky_testnet::test] +async fn my_test() { + let connection_string = ConnectionString::new( + "postgres://postgres:postgres@localhost:5432/postgres?pubky-test=true" + ).unwrap(); + + let testnet = EphemeralTestnet::builder() + .postgres(connection_string) + .build() + .await + .unwrap(); +} +``` + +The `?pubky-test=true` parameter tells the homeserver to create an ephemeral `pubky_test_*` database. The `#[pubky_testnet::test]` macro ensures the database is cleaned up after the test completes or panics. + ### Custom Configuration ```rust,no_run @@ -216,20 +231,4 @@ Seed a custom config on first run (errors if `config.toml` already exists in the ```bash TEST_PUBKY_CONNECTION_STRING='postgres://postgres:postgres@localhost:5432/postgres' \ cargo run -p pubky-testnet -- --homeserver-config my-config.toml persist ./my-testnet-data -``` - -## Troubleshooting - -### Docker not running - -The `docker-postgres` feature requires Docker. If you see `"Is Docker running?"` errors, ensure the Docker daemon is started and your user has permission to access it (e.g., is in the `docker` group). - -### Docker Hub rate limits - -The Postgres image is pulled from Docker Hub. Anonymous pulls are limited to 100 per 6 hours. If you hit this, either `docker login` or pre-pull the image: - -```bash -docker pull postgres -``` - -Once cached locally, subsequent test runs won't pull again. +``` \ No newline at end of file