Skip to content

test(interaction-harness): a busy proof judges each predicted move - #10812

Merged
cixzhang merged 3 commits into
meta:mainfrom
vjeux:test/busy-proof-predicted-move
Nov 9, 2026
Merged

cixzhang merged 3 commits into
meta:mainfrom
vjeux:test/busy-proof-predicted-move

Conversation

@vjeux

@vjeux vjeux commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Maintainer impact

The busy double-tap zoom proof fails Chromium CI now and then. Under the CI profile, the current rule fails 10 of 240 recorded casts.

Both come from one cause. The rule guesses each frame's move from the frames around it. A cast that stamps frames late, or a spring's slow start, defeats that guess.

Intended invariant

spec:AST-094 FR5: motion keeps advancing at display rate through a main-thread block. The busy proof judges that claim frame by frame. A frame the cast cannot judge is named, and is never a red on the component or a pass.

Change

  • A window that caught an animation can predict each frame's move. After the busy loop, the page replays the element's own runs on a hidden copy of its box.
  • Prediction runs in Chromium, for a marked element, when the proof runs the control ahead.
  • Each such frame is judged by its prediction. A frame predicted to move a full cast step must show a new position. A frame predicted to move less is named in unjudged.
  • Frames are dated on the controls' clock when a prediction runs. blockMainThread now returns the controls' starts for that.
  • Every other window falls back by name. The cast-step rule judges it exactly as without a replay, and its notes say why. This covers no marker, no control ahead, WebKit, and a replay that disagrees with the cast.
  • judgeEveryFrame keeps its meaning there. In WebKit the zoom spec still refuses an excused frame by name. With a prediction, no frame is excused, so nothing throws.
  • Why the control ahead is required: under the CI profile, Chromium also stamps frames in bursts. Dating a frame without its own reading from a neighbour gave a false 2 px move and a false red on a recorded cast.
  • WebKit cannot predict: its element box at the loop's end lags the animation's current time, so the replay cannot be aligned.
  • The zoom spec starts the control ahead and passes the controls' starts.
  • Builds on test(interaction-harness): too little to judge is named starvation #10788 and fix(interaction-harness): WebKit busy proofs date frames on the controls #10772, both merged.

Evidence

  • Failure before the change: the current rule fails 10 of 240 recorded CI-profile zoom casts.
  • Success after the change: the same 240 casts all pass, each sub-step frame named.
  • Product/runtime behavior verified unchanged: test code only.
  • CI impact: every busy proof keeps its verdict, double-judged on the same frames.

Scope

  • No intended public API, product behavior, visual direction, or policy change.
  • Product changes discovered during the work were removed or split.
  • Public text and artifacts contain no internal Meta context.

Testing

Merge conditions

Line numbers and results are at 99a1628.

  1. The predicted move composes both curves, not one channel. The page replays every geometry run of the photo together: the translate (900 ms) and the scale (979 ms). It plays them on a hidden copy and reads it every 2 ms (busy.ts:1074, :1110, :1185). One channel alone misses: a single curve predicted 1.76 px where the edge moved about 0.85 px. A frame is named only when its composed move since the last new position is under one step (busy.ts:2768–2800, :2792). Every other frame the control moved on must show a new position.
  2. minJudgedFrames is unchanged and still required (busy.ts:2871, :2886). Too few judged frames throw FrameStarvationError (busy.ts:1925).
  3. The frozen and main-thread mutants still fail in both engines at CI's profile (LightboxZoom.interaction.browser.spec.ts:1264–1265). At 99a1628, both mutant tests passed 3 of 3 times in each engine, in the CI-profile container with one worker. So the proof failed every slowed zoom. WebKit falls back by rule (busy.ts:2004) and keeps the judgeEveryFrame throw (LightboxZoom.interaction.browser.spec.ts:1153, :1156).
  4. Zoom casts: the 200 recorded CI-profile casts cited before fail 0 under this rule, where the current rule fails 8. With 40 more, it is 0 of 240 against 10.

The CI profile is the Playwright container with --cpus=4, CI=1, LP_NUM_THREADS=4, one worker and eight busy loops. d5ac99dba2, 4bcd673, a670d34 and e4f6b93 are this change at earlier heads.

  • Zoom proof, Chromium, CI profile:
    • the 240 recorded casts judged again under this head's rule: 0 fail; the current rule fails 10 of them. Some read a full-step frame as a miss, others excuse a sub-step frame;
    • live, at d5ac99dba2, whose zoom path is this head's: 40 of 40, with 26 to 41 control frames judged and 54 named;
    • live at 4bcd673, a670d34, 381d381 and ffae3a0: 40 of 40 each.
  • Every busy proof, double-judged at 4bcd673, one worker. Each call is judged by both rules on the same frames.
    • Chromium, 21 files twice, 56 calls: 0 verdict changes; 986 tests passed, 0 failed.
    • Only the zoom proof, which runs the control ahead, judges other frames than today: 80 instead of 86, with 5 named and none unread.
    • Every other window falls back by name: 18 without the control ahead, 6 without the marker.
    • WebKit, CI profile, 11 calls: every window with a replay falls back by name. One verdict changes: a zoom run starves by name, with 3 control frames where the proof needs 4. The harness review: "The dating with the controls' true starts places the frame at stamp +187.6 ms at content time −1 ms, before the block, so it leaves the judged part. It showed the page before the block, not block evidence. Every in-block frame is still judged."
    • That WebKit run: 399 passed, 15 failed. In WebKit every proof falls back by name. So apart from that zoom run, each failure is judged as without this change.
  • busy.predicted.test.ts, 27 cases:
    • two recorded CI-profile casts: the current rule reds one and excuses the other, and the prediction passes both with the tail named;
    • in WebKit under judgeEveryFrame, the second cast and a synthetic slow start throw BusyFrameUnjudgedError; in Chromium they pass, the frames named;
    • a hold whose predicted moves add up to a step fails, although each frame alone moves less;
    • a hold that begins on a frame with no control reading fails too;
    • a replay that moves a quarter less, or sits a quarter of its travel off, falls back by name;
    • the fallback equals the proof without a replay, in Chromium and in WebKit, except its note;
    • WebKit, a proof without the control ahead, and a proof without a control all fall back by name. So does a replay the proof cannot read there;
    • a fade predicts from one channel. A fade that holds bends its fit, falls back by name, and equals the proof without the replay;
    • synthetic curves: a held frame, a frozen part and a half-rate component fail;
    • too few judged frames starve by name, and the error names the motion, not the machine. A control under the floor starves by name too, with or without a replay;
    • where a replay predicts, one it cannot read throws BusyPredictionError.
  • busy.unjudged.test.ts (test(interaction-harness): a busy proof names a frame it cannot judge #10765's 5 cases) is unchanged and passes.
  • Harness unit suite at this head: 278 passed (15 files).
  • Zoom proof, its slowed mutants and the Switch colour proofs, Chromium: 18 passed.
  • tsc (harness and core): 0 errors. ESLint: 0. Prettier clean. pnpm review:pre --base bd57e0a91b --head 99a1628b5f: 1685 authored lines, merges cleanly, 0 findings.

A busy proof whose control moved on fewer than minJudgedFrames frames
threw FrameStarvationError only when the cast also showed no frame for
longer than FRAME_STARVATION_MS. On a loaded runner the gap can stay
under that. The same shortfall then came back as verdict failures, and
the spec failed as if the component had stalled.

Under the floor, a run with no component failure on the frames shown
is now too little to judge. busyWindowProof throws FrameStarvationError
with the control's moves, the floor, the cast's frames and its longest
gap, counting every frame the cast drew. A component that missed a
frame the control moved on, or stalled longer than it, still fails the
verdict. The cast's median interval judges no component, so it keeps no
red there. pooledBusyProof does the same for too few judged runs or
control frames. A pool left short of judged runs by a run that began
too late keeps its failure. Every change turns a verdict failure into
named starvation.
@vercel

vercel Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
astryx Ready Ready Preview Nov 9, 2026 6:11pm UTC

Request Review

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 9, 2026
@astracat-bot

astracat-bot Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

AI review status for this pull request.

Review status Updated
🟢 Reviewed (for maintainers only) Nov 9, 2026, 6:23 PM UTC

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

PR Analysis Report

Current run: The current CI run concluded skipped. Current analysis is unavailable.


Generated by PR Enrichment workflow | View current CI run

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: replaying the caught animation is a sound way to stop the zoom proof's false reds, but two harness defects need fixing first.

  • Frames predicted to move under one cast step also count as component edges (busy.ts:2758-2805), so a component frozen for 300 ms where the replay predicts 0.4 px a frame passes with 18 frames named while the cast-step rule fails it on the same frames; measure the predicted move from the last frame that showed a new position, so an accumulated full step is judged.
  • predicts (busy.ts:1968-1971) ignores animation.unpredictable, so a window whose replay could not be read throws BusyPredictionError in WebKit, without the ahead control and with no control at all, although the README says those windows fall back by name; count unpredictable in predicts and cover the WebKit and no-ahead cases in the test for an unpredictable window.

Both cases reproduce with the synthetic curve and controls of busy.predicted.test.ts. Not blocking: the fade and pseudo-element (progress) path has no test or caller, and the README and the predictionOf JSDoc disagree with the code on when prediction needs the marker and when it throws.

[Automated review]

@cixzhang

cixzhang commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

/approve-spec 0339061

@cixzhang

cixzhang commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

/approve-spec bbc656c

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested: both findings from the last review are fixed and now covered by tests, but the fix for the sub-step hold opens a similar gap.

  • A hold that begins on a frame with no control reading is never judged (busy.ts:2775-2785, then counted as a component edge at busy.ts:2820): on the synthetic curve and controls of busy.predicted.test.ts, hiding both controls in the frame that first shows a held position and holding for 300 ms passes with 18 frames named clock, while the cast-step rule fails the same frames, so decide undated from frame i and the frame before it as before, measure the move from the run's first frame that has its own reading, and add that case to the tests.

Not blocking: the starvation for too many named frames counts judged frames as frames drawn and blames a loaded machine (busy.ts:2871-2875), and the README and the busyWindowProof JSDoc still make the harness marker a condition that the fade path does not check.

[Automated review]

A busy proof whose window caught an animation now predicts each frame's
move, in Chromium, when the proof runs the control ahead. After the busy
loop, the page replays the element's own runs on a hidden copy of its
box. A frame the control moved on must show a new position when its
predicted move is a full cast step. A frame predicted to move less is
named as too little to judge. With a prediction, frames are dated on
the controls' clock, from the starts the window now keeps.

Any other window falls back by name to the cast-step rule, exactly as
without a replay: no marker, no control ahead, WebKit, or a replay
that disagrees with the cast. judgeEveryFrame keeps refusing an excused
frame there. The zoom spec starts the control ahead.
@vjeux
vjeux force-pushed the test/busy-proof-predicted-move branch from bbc656c to 9956ef8 Compare November 9, 2026 16:27
@cixzhang

cixzhang commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

/approve-spec 9956ef8

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending CI: CI has not run on this head yet. It runs once the pull request is marked ready for review and a maintainer adds the ci:run label.

The hold that begins on a frame with no control reading and the starvation message from the last review are fixed, each with a test.

Not blocking:

  • Miss: a hold whose judged frames each follow a frame with no control reading still passes (busy.ts:2790-2791); deciding undated from series[i] and series[anchor] closes it and keeps every existing case and both recorded casts unchanged.
  • The README's fallback paragraph, the controlPageTimes JSDoc and the zoom spec's verdict comment lag the code: a fade with no end anchor is judged on the controls' clock with no note, the median lag is undocumented, and named frames now leave the "every frame the control moves" count.

[Automated review]

}
const move = Math.abs(predictedAt(t) - predictedAt(series[anchor].t));
const undated =
unread.has(series[i].frame) || unread.has(series[i - 1].frame);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not blocking. The move is now measured from anchor, but undated still looks at series[i - 1]. So when every other frame of a hold has no reading from either control, each judged frame is named clock and the hold is never judged: on the synthetic curve of busy.predicted.test.ts (3 px a frame), a 300 ms hold with both controls hidden on every other frame passes with 8 frames named, while the cast-step rule fails it ("moved on 33 of the 41 frames", "longest stall 300.0 ms"). unread.has(series[i].frame) || unread.has(series[anchor].frame) judges it ("moved on 33 of the 40 frames"), and the 27 cases and both recorded casts keep their results.

@cixzhang

cixzhang commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

/approve-spec 99a1628

@astracat-bot astracat-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved pending CI: CI has not run on this head yet. It runs once the pull request is marked ready for review and a maintainer adds the ci:run label.

The merge of main leaves this pull request's own changes as they were, and the earlier fixes stay covered by tests.

Not blocking:

  • A hold whose judged frames each follow a frame with no control reading still passes (busy.ts:2790-2791); deciding undated from series[i] and series[anchor] closes it and keeps all 27 cases passing.
  • The README's fallback paragraph, the controlPageTimes JSDoc and the zoom spec's verdict comment lag the code: a fade with no end anchor is judged on the controls' clock with no note, the median lag is undocumented, and named frames now leave the "every frame the control moves" count.
  • Miss: the readBusyWindow and BlockedWindow.controlStarts JSDoc (busy.ts:1941-1946, busy.ts:684) do not say that a predicting Chromium window is dated on the controls' clock, or that the window's starts are read only then.

[Automated review]

@cixzhang cixzhang added the ci:run A maintainer lets this outside contributor's pull request run CI (AST-030 FR17) label Nov 9, 2026
@vjeux
vjeux marked this pull request as ready for review November 9, 2026 20:08
@cixzhang
cixzhang merged commit 86fcdd3 into meta:main Nov 9, 2026
122 checks passed
@vjeux
vjeux deleted the test/busy-proof-predicted-move branch November 9, 2026 20:31

This branch was successfully deployed

1 active deployment
Preview — 99a1628b Deployed Nov 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:run A maintainer lets this outside contributor's pull request run CI (AST-030 FR17) CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants