A Datasette <-> Litestream plugin.
The plugin requires a recent alpha version of Datasette 1.0:
pip install 'datasette>=1.0a20'
Then install this plugin in the same environment as Datasette:
datasette install datasette-litestream
To replicate my_database.db to S3, use the following configuration in your metadata.yaml file:
databases:
my_database:
plugins:
datasette-litestream:
replicas:
- url: s3://my-bucket/my_databaseThen make sure you export LITESTREAM_ACCESS_KEY_ID and LITESTREAM_SECRET_ACCESS_KEY with your S3 credentials (or AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY), then run with:
datasette my_database.db -m metadata.yaml
If you have multiple attached databases in Datasette and want to replicate all of them, the top-level all-replicate key can be used.
plugins:
datasette-litestream:
all-replicate:
- s3://my-bucket/$DB_NAMEWhen all-replicate is used, a new replica URL is generated for each attached database. In this case, if you had a database named parking_tickets and another named city_budget, then datasette-litestream will replicate them to s3://my-bucket/parking_tickets and s3://my-bucket/city_budget.
This is done with "variables" that datasette-litestream replaces in the all-replicate URL. The supported variables are:
$DB_NAME: The name of the Datasette database to replicate.$DB_DIRECTORY: The full parent directory that the SQLite database resides.$PWD: The current working directory of the Datasette process.
Some configuration in the metadata.yaml will be used to auto-generate the litestream.yml file under the hood. You can use this to customize the Litestream replication process.
The following are valid keys that are allowed when specifying top-level plugin configuration:
all-replicate: A template replica URL used to replicate all attached Datasette databases, see above for details.metrics-addr: Defines theaddr:Litestream option, which will expose a Prometheus endpoint at the given URL. Use with caution on public Datasette instances! When defined, the metrics info will appear on thedatasette-litestreamstatus page.access-key-id: An alternate way to provide a S3 access key (though theLITESTREAM_ACCESS_KEY_IDenvironment variable is preferred).secret-access-key: An alternate way to provide a S3 secret key (though theLITESTREAM_SECRET_ACCESS_KEYenvironment variable is preferred).session-token: Optional AWS session token for temporary credentials (e.g., when using AWS STS).credentials-file: Path to a JSON file containing credentials (see Dynamic Credentials below).credentials-command: A CLI command to execute that returns JSON credentials (see Dynamic Credentials below).credentials-refresh-interval: How often (in seconds) to check for credential changes. Required when usingcredentials-fileorcredentials-command.
None of these keys are required.
Example:
plugins:
datasette-litestream:
all-replicate:
- XXX
- YYY
metrics-addr: :5001
access-key-id: $YOUR_KEY
secret-access-key: $YOUR_SECRETFor environments where credentials rotate or are fetched dynamically (e.g., from a secrets manager), you can configure datasette-litestream to read credentials from a file or execute a command, and periodically check for changes.
Important: You cannot specify both credentials-file and credentials-command - use one or the other.
Create a JSON file with your credentials:
{
"access-key-id": "AKIAIOSFODNN7EXAMPLE",
"secret-access-key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"session-token": "optional-session-token-for-temporary-credentials"
}The session-token field is optional and only needed when using temporary AWS credentials (e.g., from AWS STS).
Then configure the plugin to read from this file:
plugins:
datasette-litestream:
credentials-file: /path/to/credentials.json
credentials-refresh-interval: 300 # Check every 5 minutesYou can also execute a CLI command that outputs JSON credentials. This is useful for integrating with secrets managers or credential vending services:
plugins:
datasette-litestream:
credentials-command: ./fetch_creds.sh --bucket my-bucket
credentials-refresh-interval: 300 # Check every 5 minutesThe command should output JSON to stdout in the same format:
{
"access-key-id": "AKIAIOSFODNN7EXAMPLE",
"secret-access-key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"session-token": "optional-session-token"
}The session-token field is optional.
- On startup, credentials are loaded from the file or command
- Every
credentials-refresh-intervalseconds, the file is re-read or the command is re-executed - If the credentials have changed,
datasette-litestreamwill:- Stop the current litestream process
- Update the configuration with new credentials
- Start a new litestream process
- If loading credentials fails during a refresh check, the Datasette process will exit with an error
The following options are allowed on database-level plugin configuration.
replicasmonitor-intervalcheckpoint-intervalmin-checkpoint-page-countmax-checkpoint-page-count
databases:
my_database:
plugins:
datasette-litestream:
replicas:
- s3://...
- file://...
monitor-interval: XXX
checkpoint-interval: XXX
min-checkpoint-page-count: XXX
max-checkpoint-page-count: XXXSee Litestream Database settings for more information.
To set up this plugin locally, first checkout the code. Then run the tests using uv:
cd datasette-litestream
uv run pytestTo run Datasette with the plugin installed:
uv run datasette -c config.yaml