Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions deploy/RESTORE.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ flash backups among them), one object per file as
`boards/owner-reports/sha256/<ab>/<sha256>`, never overwritten or deleted;
see step 3d.

**Backed up once, by hand:** the source recordings of the flight on
`/low-latency`, as `boards/flights/<name>/` with a `SHA256SUMS`; the streams
are rebuilt from them (`tools/flight-ab/README.md`).

**Not backed up, by decision:** the wall images (snapshots purge at 2 days and
cameras re-upload continuously), the firmware cache (`/srv/www/shared/firmware`,
one version of each image, rebuilt on the next request), `/srv/github-releases`
Expand Down Expand Up @@ -311,11 +315,13 @@ Only needed on a rebuilt host:
Wall is empty until cameras re-upload, so an empty directory owned by uid
1000 is a complete restore.
- `/srv/www/shared/media` — the recorded flight the A/B player on
`/low-latency` streams. **Not** in the backup: about 2.5 GB, rebuilt from the
two source recordings by `tools/flight-ab/run.sh` and uploaded as its README
says. The sources are on this host too (`/srv/www/flight-sources/`, not in
S3), so a rebuilt host needs them from wherever else they were kept. Until then the page shows its poster and the player says it cannot
play.
`/low-latency` streams. **Not** in the backup, by design: about 2.6 GB that
`tools/flight-ab/run.sh` rebuilds from the two source recordings, which
**are** backed up, once, at `s3://openipc-org-backup/boards/flights/<name>/`
with a `SHA256SUMS`. The recipe, commands and expected results are in
`tools/flight-ab/README.md` ("Rebuilding the published tree"). Until it is
rebuilt the page shows its poster and the player says the flight did not
load.
- **analytics**, via `deploy/install-analytics.sh` (#181). It installs
GoatCounter, its account and its systemd unit, and creates the site on first
run from `ANALYTICS_EMAIL` and `ANALYTICS_PASSWORD`. The SQLite database is
Expand Down
64 changes: 55 additions & 9 deletions tools/flight-ab/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,63 @@ the poster to `frontend/apps/site/src/assets/pages/flight-<name>.jpg`, and point
`frontend/apps/site/src/data/flight.ts` at the directory. The page's numbers
come from the same run as the streams, so they cannot disagree.

`/srv/www/shared/media/` is in no backup. It is rebuilt from the two source
recordings with this script, so the sources are kept on the host, outside every
served path, at `/srv/www/flight-sources/<name>/` with a `SHA256SUMS`:
`/srv/www/shared/media/` is in no backup, by design: it is 2.6 GB that this
script rebuilds from two source recordings, and the source recordings are
what is backed up.

| flight | file | sha256 |
|---|---|---|
| mabur-2026-10 | `onboard.mp4` (898 MB) | `eb6c961f1fec08997adf511563b97ffdd7c8bcf7b787715e053d25bd571a7f61` |
| mabur-2026-10 | `record-0015.mp4` (353 MB) | `31392ad36cc7abdb2ffae015c3ab988bb034cec752a0a6a367335f9ff094fa37` |
## The source recordings

They are not in the S3 backup: the backup's IAM user may write only its own
prefixes, and refused `media-sources/`.
Kept once, in the S3 backup bucket, beside a `SHA256SUMS`. Under `boards/`
because that is the prefix the backup's IAM user may write and outside the
daily/weekly/monthly expiry (`deploy/RESTORE.md`); the user cannot delete,
so treat what is there as permanent. A copy may also be on the host at
`/srv/www/flight-sources/<name>/`, outside every served path.

| flight | object | size | sha256 |
|---|---|---|---|
| mabur-2026-10 | `s3://openipc-org-backup/boards/flights/mabur-2026-10/onboard.mp4` | 898,285,071 | `eb6c961f1fec08997adf511563b97ffdd7c8bcf7b787715e053d25bd571a7f61` |
| mabur-2026-10 | `s3://openipc-org-backup/boards/flights/mabur-2026-10/record-0015.mp4` | 352,513,886 | `31392ad36cc7abdb2ffae015c3ab988bb034cec752a0a6a367335f9ff094fa37` |

`onboard.mp4` is the drone's recording, `record-0015.mp4` the ground station's,
both from gilankpam, who gave them for publishing. (`boards/flights/probe`, six
bytes, is a write test left behind because the user cannot delete it.)

## Rebuilding the published tree for mabur-2026-10

What `/low-latency` streams is `/media/flights/mabur-2026-10/v2/`
(`frontend/apps/site/src/data/flight.ts`). To make it again -- after a host
loss, or to change the ladder:

```bash
# 1. Fetch and check the sources (backup credentials: deploy/RESTORE.md, step 0).
mkdir -p tmp/flight-src && cd tmp/flight-src
for f in onboard.mp4 record-0015.mp4 SHA256SUMS; do
aws s3 cp "s3://openipc-org-backup/boards/flights/mabur-2026-10/$f" .
done
sha256sum -c SHA256SUMS && cd ../..

# 2. Align, cut, encode, package (about half an hour on 32 cores).
tools/flight-ab/run.sh --onboard tmp/flight-src/onboard.mp4 \
--gs tmp/flight-src/record-0015.mp4 --poster-at 120.71 \
--out tmp/flight-ab/publish/flights/mabur-2026-10/v2

# 3. Publish.
rsync -a --chmod=D0755,F0644 -e 'ssh -p 35242' \
tmp/flight-ab/publish/flights/ root@openipc.org:/srv/www/shared/media/flights/
```

The run must report the offset as 1.0043 s and the common window as
0.7046 s..360.7273 s, and its `stats.json` must equal
`frontend/apps/site/src/data/flight-mabur-2026-10.json`; the poster at 120.71 s
is `frontend/apps/site/src/assets/pages/flight-mabur-2026-10.jpg`. Those two
are already in the repository, so a rebuild for a restored host needs nothing
committed. Rebuild into `v2` there -- the address the page names. A changed
ladder, on the other hand, is a new `v<N>/` and a change to `flight.ts`, because
`/media/` is cached as immutable for a year.

The page trims its own window: the drone sits still on the ground for the
first seven seconds, so `flight.ts` starts the player eight seconds in. That is
not in the streams.

## Traps

Expand Down
Loading