Skip to content

Commit 58c6a6c

Browse files
frahlgclaude
andauthored
test(site): run siterig/short for every PR on the control path (#1906)
* test(site): run siterig/short for every PR on the control path The runner only ran the short set for PRs touching control, loadpoint, OCPP, telemetry or the rig. Core behaviour also lives in the control tick wiring, the driver host, plans, config limits, cars, sitepower, the bundled drivers and the solver bundle. Watch those too, pin the list in a test table, and fix the README: the runner's clone is ~/siterig/ftw, add a Linux user-service variant, explain the port range, and record the 27 October timing for the full set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(siterig): watch every package the control path imports units.LimitChargeSteps limits charger steps for loadpoint, OCPP and dispatch, yet a PR that changed only go/internal/units got no siterig/short. Watch units and the other internal packages the control packages import (gridcost, forecasting, sunpos, events, state, mdnsresolve). TestWatchedPathsCoverImports runs go list -deps on the control packages and fails when one of them imports an unwatched internal package. The README now says what the list covers and that a change to the API or UI alone does not run the rig. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 09ebd42 commit 58c6a6c

5 files changed

Lines changed: 166 additions & 17 deletions

File tree

‎go/test/site/runner/README.md‎

Lines changed: 69 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,30 @@ the result as a GitHub commit status on the exact commit it tested:
66
| Status | When | What runs |
77
|---|---|---|
88
| `siterig/full` | each new commit on `master` | every scenario |
9-
| `siterig/short` | the head of each open pull request from this repository that touches `go/internal/{control,loadpoint,ocpp,telemetry}` or `go/test/site` | the short set |
9+
| `siterig/short` | the head of each open pull request from this repository that touches a watched path (below) | the short set |
1010
| `siterig/random` | once a night on `master` | that night's random seeds |
1111

12+
The watched paths are the control packages, every internal package they
13+
import, and the rig itself:
14+
15+
- `go/cmd/ftw` (control tick wiring)
16+
- `go/internal/{control,loadpoint,ocpp,telemetry}`
17+
- `go/internal/{drivers,mdnsresolve}` (driver host, command deadlines)
18+
- `go/internal/{mpc,optimizercontract}` (plans)
19+
- `go/internal/{gridcost,forecasting,sunpos}` (plan inputs)
20+
- `go/internal/{config,configreload}` (limits)
21+
- `go/internal/units` (charge steps)
22+
- `go/internal/{cars,sitepower,events,state}`
23+
- `drivers/` and `optimizer/native/bundle/` (the bundled drivers and solver)
24+
- `go/test/site`
25+
26+
`go/cmd/ftw` imports every package, the API and UI among them; only its own
27+
files are watched. A change to the API or UI alone does not run the rig.
28+
29+
`watchedPaths` in `runner.go` is the list; `TestTouchesWatched` pins it.
30+
`TestWatchedPathsCoverImports` fails when a control package starts to
31+
import an internal package that is not on the list.
32+
1233
A commit that already has the status is not run again. To run one again,
1334
post a new `pending` status or remove the result and wait a day. No GitHub
1435
Actions workflow runs the rig; `native-release.yml` only reads
@@ -25,43 +46,76 @@ rig's timing, and the rig then fails the run as "host paused". Not the home
2546
energy box: the rig needs the CPU, and the box is production.
2647

2748
The rig uses 100 ports per lane from `-port-base` (default 18700) on
28-
127.0.0.1. Leave them free.
49+
127.0.0.1: lane k uses `port-base + 100k` up to 99 above it, so three lanes
50+
take 18700 to 18999. Leave them free. `make siterig` uses the same default,
51+
so on the runner's host run it with `FTW_SITE_PORT_BASE=19700`. A second
52+
runner on one host needs its own `-port-base` and `-work`.
2953

3054
Pull requests from forks are skipped unless `-allow-forks` is set: the rig
3155
builds and runs the pull request's code on this machine.
3256

3357
## Install
3458

59+
The runner keeps its clone in `~/siterig/ftw` (`-work` plus `ftw`) and
60+
clones it there on first start if it is missing. Build the binary from the
61+
same clone:
62+
3563
```bash
3664
gh auth login # an account that may write commit statuses on srcfl/ftw
3765
mkdir -p ~/siterig/bin
38-
git clone https://github.com/srcfl/ftw ~/siterig/src
39-
cd ~/siterig/src/go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner
66+
git clone https://github.com/srcfl/ftw ~/siterig/ftw
67+
cd ~/siterig/ftw/go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner
4068
~/siterig/bin/siterig-runner -once # one pass in the foreground
4169
```
4270

71+
To rebuild after a change to this directory:
72+
73+
```bash
74+
cd ~/siterig/ftw && git pull --quiet origin master
75+
cd go && go build -o ~/siterig/bin/siterig-runner ./test/site/runner
76+
```
77+
78+
then restart the service. The scenarios themselves always come from the
79+
commit under test.
80+
4381
macOS (launchd):
4482

4583
```bash
46-
sed "s/USER/$USER/g" ~/siterig/src/go/test/site/runner/energy.ftw.siterig-runner.plist \
84+
sed "s/USER/$USER/g" ~/siterig/ftw/go/test/site/runner/energy.ftw.siterig-runner.plist \
4785
> ~/Library/LaunchAgents/energy.ftw.siterig-runner.plist
4886
launchctl load ~/Library/LaunchAgents/energy.ftw.siterig-runner.plist
4987
```
5088

51-
Linux (systemd): edit `siterig-runner.service`, copy it to
52-
`/etc/systemd/system/` and run `sudo systemctl enable --now siterig-runner`.
89+
The log is `~/siterig/runner.log`.
90+
91+
Linux, as a system service: replace `USER` in `siterig-runner.service`,
92+
copy it to `/etc/systemd/system/` and run
93+
`sudo systemctl enable --now siterig-runner`. The log is
94+
`journalctl -u siterig-runner`.
95+
96+
Linux, as a user service. Root is needed once, for
97+
`loginctl enable-linger`, which keeps the service running after you log out
98+
and starts it at boot:
99+
100+
```bash
101+
mkdir -p ~/.config/systemd/user
102+
sed -e '/^User=/d' -e "s/USER/$USER/g" -e 's/multi-user.target/default.target/' \
103+
~/siterig/ftw/go/test/site/runner/siterig-runner.service \
104+
> ~/.config/systemd/user/siterig-runner.service
105+
systemctl --user daemon-reload
106+
systemctl --user enable --now siterig-runner
107+
sudo loginctl enable-linger "$USER"
108+
```
53109

54-
The log is `~/siterig/runner.log` (launchd) or `journalctl -u
55-
siterig-runner` (systemd). Rebuild the binary after a change to this
56-
directory; the scenarios themselves always come from the commit under test.
110+
The log is `journalctl --user -u siterig-runner`. Check `HOME` and `PATH`
111+
in the unit if your home is not `/home/$USER` or Go lives elsewhere.
57112

58113
## How long it takes
59114

60-
On an M5 MacBook (10 cores), 22 October 2026: the 23 scenarios of that
61-
morning took 72 minutes in three lanes. With the one-phase charger cases the
62-
set adds up to about 270 minutes, so three lanes need about 90 and four
63-
about 70; `s12-20` alone takes up to 56, so more than four do not help.
64-
Each lane is one `ftw` and one `simulator run`, both mostly idle.
115+
On an M5 MacBook (10 cores), 27 October 2026: the full set took about
116+
2 h 20 min in three lanes. `s12-20` alone took up to 56 minutes on
117+
22 October, so more than four lanes do not help. Each lane is one `ftw` and
118+
one `simulator run`, both mostly idle.
65119

66120
## Flags
67121

‎go/test/site/runner/main.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
// siterig-runner runs FTW's site test (go/test/site) on a machine of its own
22
// and publishes the result as a GitHub commit status on the exact commit:
33
// siterig/full on each new master commit, siterig/short on the head of each
4-
// open pull request that touches control, loadpoint, OCPP or telemetry code,
4+
// open pull request that touches code on the control path (watchedPaths),
55
// and siterig/random for a night's random seeds. native-release.yml reads
66
// siterig/full before it publishes a beta. See README.md.
77
package main

‎go/test/site/runner/runner.go‎

Lines changed: 19 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,12 +15,30 @@ const (
1515
)
1616

1717
// watchedPaths are the parts of the tree whose changes get the short set on
18-
// a pull request.
18+
// a pull request: the control packages, every internal package they import,
19+
// and the rig itself. TestWatchedPathsCoverImports keeps the imports in.
1920
var watchedPaths = []string{
21+
"go/cmd/ftw/", // control tick wiring
2022
"go/internal/control/", "go/internal/loadpoint/", "go/internal/ocpp/", "go/internal/telemetry/",
23+
"go/internal/drivers/", "go/internal/mdnsresolve/", // driver host, command deadlines
24+
"go/internal/mpc/", "go/internal/optimizercontract/", // plans
25+
"go/internal/gridcost/", "go/internal/forecasting/", "go/internal/sunpos/", // plan inputs
26+
"go/internal/config/", "go/internal/configreload/", // limits
27+
"go/internal/units/", // charge steps
28+
"go/internal/cars/", "go/internal/sitepower/",
29+
"go/internal/events/", "go/internal/state/",
30+
"drivers/", "optimizer/native/bundle/", // bundled drivers and solver
2131
"go/test/site/",
2232
}
2333

34+
// controlPackages are the watched packages whose imports must be watched
35+
// too. go/cmd/ftw is left out: it imports every package, the API and UI
36+
// among them.
37+
var controlPackages = []string{
38+
"control", "loadpoint", "ocpp", "telemetry", "drivers", "mpc",
39+
"optimizercontract", "config", "configreload", "cars", "sitepower",
40+
}
41+
2442
// job is one run of the rig on one commit.
2543
type job struct {
2644
Context string // which status it publishes

‎go/test/site/runner/runner_test.go‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package main
22

33
import (
4+
"os/exec"
45
"strings"
56
"testing"
67
"time"
@@ -75,6 +76,81 @@ func TestPlanRunsWhatIsDue(t *testing.T) {
7576
}
7677
}
7778

79+
func TestTouchesWatched(t *testing.T) {
80+
for _, tc := range []struct {
81+
file string
82+
want bool
83+
}{
84+
{"go/cmd/ftw/main.go", true},
85+
{"go/internal/control/dispatch.go", true},
86+
{"go/internal/loadpoint/loadpoint.go", true},
87+
{"go/internal/ocpp/handlers.go", true},
88+
{"go/internal/telemetry/store.go", true},
89+
{"go/internal/drivers/lua.go", true},
90+
{"go/internal/mpc/mpc.go", true},
91+
{"go/internal/optimizercontract/contract.go", true},
92+
{"go/internal/config/config.go", true},
93+
{"go/internal/configreload/apply.go", true},
94+
{"go/internal/cars/cars.go", true},
95+
{"go/internal/sitepower/sitepower.go", true},
96+
{"go/internal/units/charge_steps.go", true},
97+
{"go/internal/gridcost/gridcost.go", true},
98+
{"go/internal/state/store.go", true},
99+
{"go/internal/events/bus.go", true},
100+
{"drivers/sungrow.lua", true},
101+
{"drivers/BUNDLED_SOURCE.json", true},
102+
{"optimizer/native/bundle/manifest.json", true},
103+
{"go/test/site/rig_test.go", true},
104+
{"go/test/site/runner/runner.go", true},
105+
{"docs/architecture.md", false},
106+
{"web/components/theme.css", false},
107+
{"go/internal/api/api.go", false},
108+
{"go/internal/driverrepo/repo.go", false}, // shares a prefix with drivers/, not a path
109+
{"go/internal/configx/x.go", false}, // shares a prefix with config/
110+
{"optimizer/native/src/main.rs", false}, // only the bundle ships
111+
{"go/cmd/ftwctl/main.go", false}, // shares a prefix with cmd/ftw/
112+
{"web/drivers/list.js", false}, // drivers/ only at the root
113+
{".changeset/a.md", false},
114+
} {
115+
if got := touchesWatched([]string{tc.file}); got != tc.want {
116+
t.Errorf("%s: %v, want %v", tc.file, got, tc.want)
117+
}
118+
}
119+
if touchesWatched(nil) {
120+
t.Error("no files: watched")
121+
}
122+
if !touchesWatched([]string{"README.md", "go/internal/mpc/plan.go"}) {
123+
t.Error("one watched file among others: not watched")
124+
}
125+
}
126+
127+
// TestWatchedPathsCoverImports fails when a control package starts to
128+
// import an internal package that is not watched, so a change there would
129+
// skip the rig.
130+
func TestWatchedPathsCoverImports(t *testing.T) {
131+
const module = "github.com/srcfl/ftw/go/internal/"
132+
args := []string{"list", "-deps"}
133+
for _, p := range controlPackages {
134+
if !touchesWatched([]string{"go/internal/" + p + "/x.go"}) {
135+
t.Errorf("control package %s is not watched", p)
136+
}
137+
args = append(args, module+p)
138+
}
139+
out, err := exec.Command("go", args...).Output()
140+
if err != nil {
141+
t.Fatalf("go list: %v", err)
142+
}
143+
for _, dep := range strings.Fields(string(out)) {
144+
rest, ok := strings.CutPrefix(dep, module)
145+
if !ok {
146+
continue
147+
}
148+
if !touchesWatched([]string{"go/internal/" + rest + "/x.go"}) {
149+
t.Errorf("%s imports %s, which is not watched", strings.Join(controlPackages, ", "), rest)
150+
}
151+
}
152+
}
153+
78154
func TestDescribe(t *testing.T) {
79155
state, desc := describe([]byte(`[{"name":"s1-25","verdict":"pass"},{"name":"s2-16","verdict":"known"},{"name":"s9-25","verdict":"fail"}]`), 24*time.Minute)
80156
if state != "failure" || desc != "1 pass, 1 known, 1 fail of 3 in 24m0s: s9-25" {

‎go/test/site/runner/siterig-runner.service‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# systemd unit for siterig-runner (README.md). Replace USER and the paths,
22
# then: sudo cp siterig-runner.service /etc/systemd/system/ &&
33
# sudo systemctl enable --now siterig-runner
4+
# README.md also shows how to run it as a user service.
45
[Unit]
56
Description=FTW site test runner (siterig/short, siterig/full commit statuses)
67
After=network-online.target

0 commit comments

Comments
 (0)