fix(test): repair the e2e suites - #2640
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
✅ Deploy Preview for vjs10-site ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for vjs-registry canceled.
|
@videojs/cdn
@videojs/core
@videojs/element
@videojs/html
@videojs/media
@videojs/react
@videojs/spf
@videojs/store
@videojs/utils
@videojs/cloudflare-video
@videojs/dash-video
@videojs/hlsjs-video
@videojs/mux-audio
@videojs/mux-video
@videojs/native-hls-video
@videojs/shaka-video
@videojs/spotify-audio
@videojs/tiktok-video
@videojs/twitch-video
@videojs/vimeo-video
@videojs/wistia-video
@videojs/youtube-video
@videojs/google-cast
@videojs/mux-data
commit: |
Contributor
📦 Bundle Size Report🎨 @videojs/html — 30 small size changes
Presets (7)
Media (18)
Players (5)
Skins (29)
UI Components (62)
⚛️ @videojs/react — 2 small size changes
Presets (7)
Media (22)
Extensions (2)
Players (5)
Skins (18)
UI Components (39)
🧩 @videojs/core — no changesEntries (76)
🏷️ @videojs/element — no changesEntries (2)
📦 @videojs/store — no changesEntries (3)
🔧 @videojs/utils — no changesEntries (13)
📦 @videojs/cdn — no changes📦 @videojs/cloudflare-video — no changes📦 @videojs/dash-video — no changes📦 @videojs/google-cast — no changes📦 @videojs/hlsjs-video — no changes📦 @videojs/media — no changesEntries (3)
📦 @videojs/mux — no changes📦 @videojs/mux-audio — no changesEntries (2)
📦 @videojs/mux-data — no changes📦 @videojs/mux-video — no changesEntries (2)
📦 @videojs/native-hls-video — no changes📦 @videojs/shaka-video — no changes📦 @videojs/spf — no changesEntries (7)
📦 @videojs/spotify-audio — no changes📦 @videojs/tiktok-video — no changes📦 @videojs/twitch-video — no changes📦 @videojs/vimeo-video — no changes📦 @videojs/wistia-video — no changesEntries (2)
📦 @videojs/youtube-video — no changesℹ️ How to interpretEach entry is independently bundled, minified, and brotli-compressed. Initial size includes its static import graph; lazy dynamic chunks are reported separately. Entries are not additive because their dependency graphs overlap. Preset rows represent realistic combined bundles. Changes of 300 B or less across initial, lazy, and total size are collapsed, not discarded. Run |
Contributor
|
E2E triage: inconclusive. WebKit has a stale/incorrect portrait-thumbnail assertion; Chromium has a repeatable captions snapshot mismatch that needs a local rendering check before classifying it as intentional or a regression. Tracking: #2642. |
mihar-22
changed the title
Contributor
|
E2E triage: real regression. The Chromium sandbox failures consistently hit a Vite overlay because generated registry modules import unresolved bare |
Contributor
|
E2E triage: expected change. The persistent minimal-video fullscreen failure is a stale absolute thumbnail-to-time gap floor (6–7px measured vs. 8px expected); CSS/Tailwind parity and the CSS visual baseline pass. Two other failures passed on retry. Tracking issue: #2644. |
mihar-22
added
the
record:snapshots
mihar-22
added
record:snapshots
Contributor
Contributor
|
E2E triage: expected change — the 16 new player-layout visual cases lack committed Chromium baselines; this run’s snapshot artifact generated them. Reused and updated #2644 with the run evidence, including the three persistent minimal fullscreen spacing assertions and two retry-passing parity flakes. |
mihar-22
removed
the
record:snapshots
Contributor
The CDN pages import @videojs/cdn since #2598, but the e2e package never declared it, so Vite could not resolve the imports and every CDN test failed. The player prepare task also builds the CDN bundles so a local run does not depend on a separate command.
Since #2574 the store clears waiting as soon as currentTime advances past the stall, so a stub on a playing video lost its waiting state before the indicator delay elapsed. Freeze currentTime at the stall point alongside paused and readyState.
The volume indicator shakes at a boundary level and prints the level it lands on, so the CSS and Tailwind panels only agreed when both started at the same volume. Set a mid-range volume in every panel before the ArrowUp shortcut.
Drop the sandbox width control test from skin parity: it exercised the shell, the sandbox suite already covers it, and it broke when #2586 moved the control into a closed options panel. Move the consumer-override test to the player suite, where the packaged CSS skin is what consumers restyle and WebKit also runs. Document which suite owns what in each Playwright config.
E2E only ran on pushes to main or labeled pull requests, so three regressions merged unnoticed. Pull requests touching the e2e app, sandbox, any package, the lockfile, or the workflow now run the player and skin parity suites; the test:e2e label still forces a run.
WebKit exposes the AirPlay button, whose accessible name also contains "Play", so the substring match resolved to two buttons.
The base `video` rule sizes the media in the React skin, but the HTML skin slots the page's video through its shadow root, where that rule cannot reach it. #2550 removed the legacy `::slotted(video)` rule with the legacy skins, so the video kept its intrinsic size and overflowed the container, which also pushed native captions out of view. Size the slotted video from the container styles for the Shadow DOM variant.
WebKit does not apply `@scope` rules to an element that hosts a shadow root, and the slider thumbnail is the one such element in the HTML skin, so its max height never applies there. The portrait thumbnail e2e records that as a fixme on WebKit instead of failing the run.
…in paths WebKit doubles the wall time of the player job, so pull requests run Chromium and pushes to main cover both browsers. Skin parity compiles the authored skins, so it only runs for pull requests that touch the skins, the compiler, the framework packages, the sandbox, or the parity suite.
WebKit never matches an `@scope` rule whose subject hosts a shadow root, so the slider thumbnail lost its max height and clipping in Safari. Style rules can now declare `shadowHost`, and the renderer keeps those rules outside the scope block with the scope root as an ancestor, the same way it already keeps slotted rules. The thumbnail image rule opts in, and the portrait thumbnail e2e runs on WebKit again.
…w gaps The thumbnail keeps a placeholder box until its storyboard image arrives, so on a slow runner the preview gaps were read against the placeholder and came up short of the pinned minimum.
A build job compiles the packages, CDN bundles, generated skins, and sandbox setup once per run and saves this run's Vite Task cache; every browser job restores it instead of rebuilding. The sandbox suite moves to its own job so it runs beside the player suite rather than after it. Pull requests retry a failing test once, since the second retry mostly re-records the same flake.
…s first Skin parity captures the CSS panel in the page as the reference for the Tailwind panel instead of also holding it to a stored PNG; the player suite owns the baselines over time. The contracts wait for the store's volume, for fonts, and for running transitions before reading, and round centered insets to whole pixels, since those were the sub-pixel and mid-transition differences behind the recurring flakes.
The e2e app gains minimal-skin pages for both frameworks and media types, and a visual spec holds each packaged skin to a baseline at the widths where its layout changes.
…ntainer Baselines only match when recorded in the same image the E2E workflow runs in, so this records the requested spec with --update-snapshots there and uploads the snapshot directories as an artifact.
A manually dispatched workflow only exists to the API once it is on the default branch, so the recording job lives in the E2E workflow instead and runs when a pull request carries the `record:snapshots` label.
…gaps The seek preview scales in when hovered, and a rect read mid-transition shifts the gap between the time label and the thumbnail by a few pixels.
Chromium and WebKit baselines for every packaged skin at its layout widths, recorded by the labeled snapshot job so they match what the E2E workflow compares against.
The gaps are measured against the time label's box, which follows the fallback font's metrics until Inter has loaded, so CI read them a pixel or two short. The indicator contract also records the text each indicator shows, so a width difference names the level behind it.
Inter loads one weight at a time on first use, so labels that appear after an interaction, the keyboard indicators and the fullscreen seek preview, were measured in the fallback face in CI. The contracts settle fonts after those labels appear, the contrast contract waits out its color transition, and the fullscreen gap pins follow the settled measurement, which reads two pixels smaller than the fallback did.
mihar-22
force-pushed
the
fix/e2e-suite-failures
branch
from
September 7, 2026 09:23
4ecb547 to
87b396a
Compare
mihar-22
marked this pull request as ready for review
September 7, 2026 09:41
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 87b396a. Configure here.
sampotts
approved these changes
Sep 7, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The E2E workflow has been red on every push to main since August 27. Three regressions merged without running the browser suites, and two tests were flaky. This repairs the suites, tightens what the skin-parity suite owns, fixes the regressions the repaired suites then surfaced, and makes the browser suites cheaper to run on pull requests.
Changes
@videojs/cdn, so the CDN pages resolve again, and its prepare task builds the CDN bundles.currentTimeat the stall, matching how the store clearswaitingsince fix(core): stop waiting from latching when readyState never recovers #2574.Unblocking the CDN pages let the rest of the serial visual spec run again for the first time since Sept 3, which surfaced regressions on main:
::slotted(video)rule and the basevideorule cannot cross the shadow boundary, so the page's video kept its intrinsic size, overflowed the container, and pushed native captions out of view. The container styles now size it for the Shadow DOM variant. The existing captions baseline passes unchanged with this fix.@scoperules on shadow hosts. The slider thumbnail is the only such element in the HTML skin, so its max height and clipping never applied in Safari. VJSC style rules can now declareshadowHost, and the renderer emits those rules outside the scope block with the scope root as an ancestor, the same way it already handles slotted rules. The portrait thumbnail test runs on WebKit again.cnhelper when shadcn's remoteutilsregistry item changed toexport { cn } from "cn". fix(packages): adopt cn for class name merging #2641 on main adopts thecnpackage, which resolves it; this branch is rebased on top of it.Suite organization, second pass:
record:snapshotsand the E2E workflow records the player visual baselines in its container and uploads them as an artifact, since baselines only match when recorded in that image.CI cost:
Testing
Local runs on macOS, with the PNG comparisons left to CI since the baselines are recorded there.
pnpm typecheck, lint on changed filesThe E2E workflow on this PR runs without the
test:e2elabel, which exercises the path gating. The run on the final pre-rebase commit was green across the build, player, sandbox, skin parity, and registry jobs, with one known flaky parity test passing on retry. Pushes to main will also run WebKit.Note
Medium Risk
Touches VJSC CSS emission and shipped skin layout (shadow DOM / WebKit), plus CI path gating that could skip suites on PRs if patterns are wrong; test-only and workflow changes otherwise limit runtime risk.
Overview
Repairs long-failing browser E2E and reshapes CI so PRs only run suites when relevant paths change (or
test:e2e/record:snapshotslabels apply). A shared build job compiles packages, CDN, skins, and sandbox once and saves a Vite task cache that player, sandbox, skin-parity, and snapshot jobs replay; PR player runs Chromium only (WebKit stays on push to main), sandbox is its own job, and PR retries drop from 2 to 1.Player suite gains minimal-skin generated pages, layout visual baselines for default/minimal × HTML/React × video/audio, and a consumer CSS override test moved from skin-parity. CDN pages work again via
@videojs/cdnandprepare:playerdepending onbuild:cdn. The buffering indicator test pinscurrentTimeat the stall sowaitingstays observable.Skin-parity drops stored PNG baselines and compares Tailwind to an in-page CSS capture instead; contracts add
presetVolume, font/animation settling, and inset rounding to cut Inter-related flakes.Product/CSS: HTML skins get a shadow-dom container variant so slotted video fills the player; VJSC adds
shadowHostso rules on shadow-root hosts (e.g. thumbnail image) emit outside@scopewith:where(scope)ancestors—fixing WebKit skipping scoped rules on shadow hosts.Reviewed by Cursor Bugbot for commit 7169f56. Bugbot is set up for automated code reviews on this repo. Configure here.