test(interaction-harness): a busy proof judges each predicted move

internal/interaction-harness/README.md

@@ -321,11 +321,15 @@ decodes each frame once. A proof reads frames up to
 decode only those. Pass either as `control`: the component passes when it moved on every frame the control
 moved and its longest stall is no longer than the control's. Both are read from
 the same screencast frames, so a compositor animation misses none. A window in
-which the control moved on fewer than four frames fails, since the machine
-presented too little to judge; the failure names the cast's frames in the
-window and its longest gap. When no frame came for longer than
-`FRAME_STARVATION_MS` there too, the machine starved the run, and
-`busyWindowProof` throws `FrameStarvationError` instead of returning a verdict.
+which the control moved on fewer than four frames is under the floor. Under the
+floor, a run with no component failure on the frames shown is too little to
+judge: `busyWindowProof` throws `FrameStarvationError` instead of returning a
+verdict. The error names the control's moves, the cast's frames in the window
+and its longest gap. A component that missed a frame the control moved on, or
+stalled longer than it, failed on frames the compositor presented, so the
+verdict keeps that red. A run the machine starved, with no frame for longer
+than `FRAME_STARVATION_MS`, throws either way. `maxMedianFrameMs` judges the
+cast's pace, not the component, so under the floor it keeps no red.
 A run of frames with no new position counts as
 moved when the change that ends it is a single cast step: the component moves
 under one step a frame there, as on an easing tail, which the cast cannot show
@@ -353,6 +357,33 @@ the cast's step explains is then too little to judge: `busyWindowProof` throws
 there and the cast's step. A verdict that fails anyway comes back as it is, so
 a frozen or main-thread component stays a red on the component.
 
+A window that caught an animation (`blockMainThread` with `during`) predicts
+each frame instead, when two things hold. The proof passes both controls,
+`ahead` included, so every frame has a time of its own. The engine is
+Chromium. A geometry replay also needs the harness marker on the caught
+element. Once the loop ends, the
+page replays the element's geometry animations, paused, on a hidden copy of
+its box. The copy is aligned to the element, and its box is read every 2 ms.
+A fade or a pseudo-element replays one numeric channel. The tracked value at
+rest and at the curve's end map that channel to a position. The proof dates
+each frame on the controls' clock, from the controls' starts that the window
+keeps. A frame predicted to move a full cast step since the last new position must
+show one, so a hold ends once its moves add up to a step.
+A frame predicted to move less is too little to judge, and so is a frame with
+no control reading of its own. Each such frame is named in the proof's
+`unjudged` list and in its summary. It counts neither as moved nor as missed,
+and `judgeEveryFrame` throws for none. When too few judged frames are left,
+the proof throws `FrameStarvationError`. Where a replay predicts, one the
+proof cannot read throws `BusyPredictionError`: an ancestor moves too, or a
+run composites onto another. A replay whose travel or first judged position
+differs from the cast's by more than two steps or a tenth of the travel falls
+back by name. So does a geometry window without the marker, a window without
+`ahead` or in WebKit, and a window whose run left the element before the
+replay. WebKit reads the element's box at the loop's end before its
+animation's current time, so the replay cannot be aligned there. A window that falls back is judged exactly as without a
+replay, and its notes say why. The proof gives its `judged` part in the
+cast's stamps, so a spec selects frames by it as before.
+
 `controlClock(control)` reads the compositor's own time in each frame off the
 control's position, for a proof that judges motion against time, not against
 the screencast's stamps. A loaded machine stamps frames late, and
@@ -409,8 +440,13 @@ belongs to its run. A run whose control moved on fewer frames than
 and the pool leaves it out, so it never counts as a pass. A run whose block
 began too late for its motion is left out the same way: its `runs` entry
 is null, and `late` holds its `BusyWindowTooLateError`. The pool reads each
-run without the starvation throw. The pool fails with
-fewer than `minJudgedRuns` judged runs (half of all the runs, rounded up). In each judged run,
+run without the starvation throw. With fewer than `minJudgedRuns` judged runs
+(half of all the runs, rounded up), or a control that moved on fewer than four
+frames over them, the pool is under the floor. Under the floor, a pool with no
+component failure on the frames shown is too little to judge, and it throws
+`FrameStarvationError`. A pool left short of judged runs by a run that began
+too late fails instead, and so does a pool whose judged runs show a component
+red. In each judged run,
 the component must move on every frame its control moved on, so a freeze fails
 its run.
 

internal/interaction-harness/src/busy.control.test.ts

@@ -313,14 +313,67 @@ describe('busyWindowProof against a control', () => {
     );
   });
 
-  it('fails a window in which the control itself presented too little to judge', () => {
+  it('names a window in which the control itself presented too little to judge as starvation, though every frame came', () => {
     const still = asControl(
       moving().map(f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame),
     );
-    const proof = busyWindowProof(moving(), BLOCK, {control: still});
+    const run = () => busyWindowProof(moving(), BLOCK, {control: still});
+    expect(run).toThrow(FrameStarvationError);
     // It names how starved the cast was: here every frame came, 60 Hz.
+    expect(run).toThrow(
+      /busyWindowProof: the page drew \d+ frames while a 700 ms block held the main thread, and the control moved on 0 of them, where the proof needs 4\. No frame came for 1[67]\.\d ms: the compositor presented too little in this window to judge/,
+    );
+  });
+
+  it("names a window under the control's floor as starvation when no frame came for less than 50 ms: a loaded machine's cast, its component on every frame the control moved on", () => {
+    // A 152 ms block: the cast delivered 8 frames, none for 44 ms. The control
+    // moved on 3 of them, as on a loaded runner.
+    const block = {from: 1000, to: 1152};
+    const times = [1008, 1028, 1048, 1092, 1104, 1116, 1128, 1146];
+    const moves = new Set([1048, 1092, 1128]);
+    let y = 0;
+    const loaded = [
+      ...moving().filter(f => f.t < block.from),
+      ...times.map(t => ({t, box: null}) as TrackedFrame),
+      ...moving().filter(f => f.t > block.to),
+    ].map(f => {
+      if (f.t < block.from || f.t > block.to || moves.has(f.t)) {
+        y += 3;
+      }
+      return {...f, box: {x: 0, y, width: 10, height: 10}} as TrackedFrame;
+    });
+    const run = () =>
+      busyWindowProof(loaded, block, {control: asControl(loaded)});
+    expect(run).toThrow(FrameStarvationError);
+    expect(run).toThrow(
+      /busyWindowProof: the page drew 8 frames while a 152 ms block held the main thread, and the control moved on 3 of them, where the proof needs 4\. No frame came for 44\.0 ms: the compositor presented too little in this window to judge/,
+    );
+  });
+
+  it('still fails a main-thread component under the floor: it misses the frames the control moved on, and the cast kept pace', () => {
+    const block = {from: 1000, to: 1090};
+    const frozen = moving().map(f =>
+      f.t > block.from && f.t < block.to + 20
+        ? ({...f, box: {...f.box!, y: 3 * at(block.from)}} as TrackedFrame)
+        : f,
+    );
+    const proof = busyWindowProof(frozen, block, {
+      control: asControl(moving()),
+    });
     expect(proof.verdict.failures.join('\n')).toMatch(
-      /the control moved on 0 frames in the judged part, under 4: the compositor presented too little in this window to judge \(the cast delivered \d+ frames in the \d+ ms block, none for 1[67] ms\)/,
+      /moved on 0 of the 3 frames the control moved/,
+    );
+  });
+
+  it("names a window under the floor as starvation when only the cast's median interval fails: the median is the cast's pace, not the component", () => {
+    const run = () =>
+      busyWindowProof(moving(), short(1000), {
+        control: asControl(moving()),
+        maxMedianFrameMs: 10,
+      });
+    expect(run).toThrow(FrameStarvationError);
+    expect(run).toThrow(
+      /the control moved on 3 of them, where the proof needs 4/,
     );
   });
 
@@ -448,15 +501,29 @@ function starvedCast(): TrackedFrame[] {
   );
 }
 
+/** The numbers a `FrameStarvationError` from `run` names. */
+function starvationNumbers(run: () => unknown) {
+  try {
+    run();
+  } catch (error) {
+    if (error instanceof FrameStarvationError) {
+      return error.numbers;
+    }
+    throw error;
+  }
+  throw new Error('no FrameStarvationError');
+}
+
 /** A 90 ms block from `from`: its judged part holds three 60 Hz frames. */
 const short = (from: number) => ({from, to: from + 90});
 
 describe('pooledBusyProof', () => {
   it('judges a block too short for one run over pooled runs: one run cannot decide, three runs of a component that moves with the control pass', () => {
     const control = asControl(moving());
-    const single = busyWindowProof(moving(), short(1000), {control});
-    expect(single.verdict.failures.join('\n')).toMatch(
-      /the control moved on 3 frames in the judged part, under 4/,
+    const single = () => busyWindowProof(moving(), short(1000), {control});
+    expect(single).toThrow(FrameStarvationError);
+    expect(single).toThrow(
+      /the control moved on 3 of them, where the proof needs 4/,
     );
     const runs = [1000, 1300, 1600].map(from => ({
       tracked: moving(),
@@ -623,20 +690,22 @@ describe('pooledBusyProof', () => {
     }
   });
 
-  it('judges the control over the pool too: runs in which it presented too little fail', () => {
+  it('names a pool whose control presented too little in every run as starvation', () => {
     const still = asControl(
       moving().map(f => ({...f, box: {...f.box!, y: 0}}) as TrackedFrame),
     );
-    const pooled = pooledBusyProof(
-      [1000, 1300].map(from => ({
-        tracked: moving(),
-        control: still,
-        window: short(from),
-      })),
+    const run = () =>
+      pooledBusyProof(
+        [1000, 1300].map(from => ({
+          tracked: moving(),
+          control: still,
+          window: short(from),
+        })),
+      );
+    expect(run).toThrow(FrameStarvationError);
+    expect(run).toThrow(
+      /pooledBusyProof: .*0 of 2 runs judged, under 1, and the control moved on 0 frames over them: the compositor presented too little in the others to judge/,
     );
-    expect(pooled.verdict.failures).toEqual([
-      '0 of 2 runs judged, under 1: the compositor presented too little in the others to judge',
-    ]);
   });
 
   /** `frames` held at the position they had at `from`, from `from` to `to`. */
@@ -665,20 +734,84 @@ describe('pooledBusyProof', () => {
     );
   });
 
-  it('fails a pool in which every run but one was too little to judge, even when that run holds enough frames', () => {
+  it('names a pool in which every run but one was too little to judge as starvation, even when that run holds enough frames', () => {
     const windows = [1000, 1300, 1600].map(from => ({from, to: from + 200}));
     const starved = held(held(moving(), 1300, 1500), 1600, 1800);
+    const pool = () =>
+      pooledBusyProof(
+        windows.map(window => ({
+          tracked: moving(),
+          control: asControl(starved),
+          window,
+        })),
+      );
+    expect(pool).toThrow(FrameStarvationError);
+    expect(pool).toThrow(
+      /pooledBusyProof: the page drew \d+ frames while 3 blocks held the main thread, and 1 of them had enough control frames to judge, where the proof needs 2\. .*1 of 3 runs judged, under 2, and the control moved on \d+ frames over them: the compositor presented too little in the others to judge/,
+    );
+  });
+
+  it('names a pool of four runs with one judged as starvation: its component moved on every frame its control moved on', () => {
+    const windows = [1000, 1250, 1500, 1750].map(short);
+    const control = asControl(
+      held(held(held(moving(), 1250, 1340), 1500, 1590), 1750, 1840),
+    );
+    const pool = () =>
+      pooledBusyProof(
+        windows.map(window => ({tracked: moving(), control, window})),
+      );
+    expect(pool).toThrow(FrameStarvationError);
+    expect(pool).toThrow(/1 of 4 runs judged, under 2/);
+    // Short of judged runs, the error counts every block's frames.
+    const numbers = starvationNumbers(pool);
+    expect(numbers.frames).toBe(20);
+    expect(numbers.longestGapMs).toBeCloseTo(FRAME, 1);
+  });
+
+  it('names the longest gap of every block in a pool short of judged runs, where the unjudged blocks drew no frame', () => {
+    const windows = [1000, 1250, 1500, 1750].map(short);
+    const control = asControl(
+      held(held(held(moving(), 1250, 1340), 1500, 1590), 1750, 1840),
+    );
+    const tracked = moving().filter(
+      f => !windows.slice(1).some(w => f.t > w.from && f.t < w.to),
+    );
+    const numbers = starvationNumbers(() =>
+      pooledBusyProof(windows.map(window => ({tracked, control, window}))),
+    );
+    expect(numbers.frames).toBe(5);
+    expect(numbers.longestGapMs).toBeCloseTo(90, 5);
+  });
+
+  it("names a pool short of judged runs as starvation when only the runs' median intervals fail", () => {
+    const windows = [1000, 1250, 1500, 1750].map(short);
+    const control = asControl(
+      held(held(held(moving(), 1250, 1340), 1500, 1590), 1750, 1840),
+    );
+    const pool = () =>
+      pooledBusyProof(
+        windows.map(window => ({tracked: moving(), control, window})),
+        {maxMedianFrameMs: 10},
+      );
+    expect(pool).toThrow(FrameStarvationError);
+    expect(pool).toThrow(/1 of 4 runs judged, under 2/);
+  });
+
+  it('still fails a pool short of judged runs when its judged run shows a component red', () => {
+    const windows = [1000, 1250, 1500, 1750].map(short);
+    const control = asControl(
+      held(held(held(moving(), 1250, 1340), 1500, 1590), 1750, 1840),
+    );
     const pooled = pooledBusyProof(
       windows.map(window => ({
-        tracked: moving(),
-        control: asControl(starved),
+        tracked: held(moving(), 1000, 1090),
+        control,
         window,
       })),
     );
-    expect(pooled.judged).toEqual([true, false, false]);
-    expect(pooled.control.moved).toBeGreaterThanOrEqual(4);
+    expect(pooled.judged).toEqual([true, false, false, false]);
     expect(pooled.verdict.failures.join('\n'), pooled.summary).toMatch(
-      /1 of 3 runs judged, under 2: the compositor presented too little in the others to judge/,
+      /run 1: moved on 0 of the 3 frames its control moved/,
     );
   });