Skip to content

feat(mwpw-199088): add Phase A beacon telemetry and QA introspection hooks - #531

Open
sanrai wants to merge 2 commits into
mainfrom
canary-beacon-poc
Open

feat(mwpw-199088): add Phase A beacon telemetry and QA introspection hooks#531
sanrai wants to merge 2 commits into
mainfrom
canary-beacon-poc

Conversation

@sanrai

@sanrai sanrai commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Phase A canary observability infrastructure for CaaS.

Two new helper modules:

  • beacon.js: navigator.sendBeacon-based telemetry to LANA with consent gating, URL-param overrides for QA, and a read-only DOM assertions runner.
  • qa-hooks.js: window.caas introspection API for headless agents. Pure DOM read; no mutations.

Wires both into app.jsx (boot) and Container.jsx (cards fetch lifecycle). All call sites are try/catch wrapped so telemetry failures can never throw into the React tree.

Phase A cohort is hardcoded to 'stable' for all users; the UUID-hash 1% split lands in Phase B without changing the public API of this module. No UX changes in this PR.

URL-param overrides for QA:
?caas_cohort=canary|stable force cohort (sets forcedCohort:true)
?caas_consent=1 bypass consent on non-dev hosts
?caas_debug=1 log every beacon payload to console

Phase A canary observability infrastructure for CaaS.

Two new helper modules:
- beacon.js: navigator.sendBeacon-based telemetry to LANA with consent
  gating, URL-param overrides for QA, and a read-only DOM assertions runner.
- qa-hooks.js: window.caas introspection API for headless agents.
  Pure DOM read; no mutations.

Wires both into app.jsx (boot) and Container.jsx (cards fetch lifecycle).
All call sites are try/catch wrapped so telemetry failures can never throw
into the React tree.

Phase A cohort is hardcoded to 'stable' for all users; the UUID-hash 1%
split lands in Phase B without changing the public API of this module.
No UX changes in this PR.

URL-param overrides for QA:
  ?caas_cohort=canary|stable  force cohort (sets forcedCohort:true)
  ?caas_consent=1             bypass consent on non-dev hosts
  ?caas_debug=1               log every beacon payload to console
Resolves Container.jsx conflict in the no-cards-returned branch. Both
main's removeCollectionFromPage() call (added in PR #465 for the
events-origin case) and our beaconTargetMissing() telemetry now coexist.

The order in the resolved block:
  1. logLana("no cards return by query...")
  2. beaconTargetMissing({ reason: 'no_cards_returned', ... })
  3. if (originSelection === 'events' && box.current) removeCollectionFromPage()
  4. return

Telemetry fires before the DOM mutation so the beacon captures the
state at the moment the empty-collection condition was detected.

All other main changes (72 files: workflows, docs, card components,
CHANGELOG, etc.) pulled in unchanged. No conflicts in any other file.
@github-actions

Copy link
Copy Markdown

🤖 Agent QA review — interactive + visual diff (advisory, non-blocking)

Drove the PR build on the live business.adobe.com collection (filtered, searched, paginated, inspected cards), guided by a PR-vs-stable visual diff (0.00% of pixels changed) and the PR code diff. Verdict: PASS.

QA Report — PR #531: feat(mwpw-199088): add Phase A beacon telemetry and QA introspection hooks

URL Tested

https://business.adobe.com/resources/main.html

Diff Analysis

The pixel diff shows 0.00% of pixels changed — no visual regressions. This matches the PR description: "No UX changes in this PR."


Functional Tests

1. window.caas Introspection API (qa-hooks.js)

PASSwindow.caas is initialized correctly:

  • window.caas.version = "0.55.1"
  • window.caas.dump = function ✓
  • window.caas.waitForReady = function (returns a Promise) ✓
  • window.__caasReady = true

2. window.caas.dump() State

PASSdump() returns a well-structured snapshot:

  • cardCount: 9 (9 cards rendered)
  • cards[] array with id, style, title, description, ctaText, ctaHref, badges for each card
  • search object: { present: true, value: "", placeholder: "Search Here", ariaLabel: "Search Here" }
  • filters object: { panel: "left", groups: [Products (27 items), Content type (N items), Industry] } — all items with selected: false defaults
  • sort, pagination keys present
  • url, ts, ready metadata present ✓

3. caas:ready / markCaasReady Signal

PASSwindow.__caasReady = true confirms the markCaasReady() call from Container.jsx fired successfully after cards were fetched and processed.

4. waitForReady() Promise

PASS — Returns a resolved Promise (since __caasReady is already true by the time of the call).

5. Page Rendering

PASS — The page renders correctly:

  • Filter panel visible with "Refine Your Results", Search box, Products / Content Type / Industry filter groups
  • 9 cards rendered with images, titles, descriptions, and CTAs
  • Result count shows "3690 results"
  • Sort control present ("Date: Newest To Oldest")
  • No layout issues, overlapping, misalignment, or missing UI elements observed

6. Console Errors

PASS — No JavaScript console errors recorded.

7. Beacon Module (beacon.js)

PASS (structural) — The beacon infrastructure loaded without errors. The beaconCardsRendered, beaconPageLoad, scheduleAssertions call sites are all wrapped in try/catch per the PR, so telemetry failures cannot propagate to the React tree. Cohort is hardcoded to 'stable' for all users (Phase A), consistent with PR description.

8. URL-Param QA Overrides (code review)

The PR documents three QA overrides:

  • ?caas_cohort=canary|stable — force cohort
  • ?caas_consent=1 — bypass consent on non-dev hosts
  • ?caas_debug=1 — log every beacon payload to console

These are wired into beacon.js and not user-visible, consistent with the "no UX changes" claim.


Accessibility

Not fully audited within turn budget, but no obvious a11y regressions visible from screenshots. The filter panel, search input, and card grid structure appear unchanged from stable.


Summary

  • No visual regressions (0.00% pixel diff confirmed)
  • No JavaScript errors on boot or card load
  • window.caas API fully functional: version, dump(), waitForReady() all work as specified
  • Cards rendered correctly with all expected data
  • Telemetry infrastructure is non-intrusive: all call sites are try/catch wrapped, no side effects on UX
  • Phase A cohort = 'stable' for all users as documented — no split traffic risk

No bugs found. The PR delivers exactly what it describes: invisible telemetry/observability infrastructure with zero UX impact.

PR / stable / diff screenshots + console + axe artifacts in the workflow run.

@github-actions

Copy link
Copy Markdown

AI Code Review

Critical Issues

1. beaconPageLoad() fires before React mounts (broken timing)

File: app.jsx, line ~64

initReact(document);

// Fire the page_load beacon after React is mounted.
try {
    beaconPageLoad();
} catch (e) { /* swallow */ }

initReact calls ReactDOM.render which is asynchronous — the component tree is not mounted when beaconPageLoad() fires. The comment says "after React is mounted" but this is incorrect. This will consistently capture wrong timing data and may reference DOM state that doesn't exist yet.


2. start variable used in beaconFetchFail / beaconCardsRendered but never shown defined in diff scope

File: Container.jsx, lines ~1078, ~1235

responseTimeMs: Date.now() - start,
fetchDurationMs: Date.now() - start,

If start is not defined in the enclosing fetch scope before the request fires, this will throw ReferenceError (or produce NaN). The diff doesn't show where start is initialized — if it's missing, this is a runtime crash inside the fetch chain that could break card rendering despite the outer try/catch, since the inner .then() callbacks are not wrapped.


3. beaconTargetMissing and beaconFetchFail called outside try/catch in fetch chain

File: Container.jsx, lines ~1071, ~1075

beaconTargetMissing({ reason: 'empty_collection', ... });
/* istanbul ignore next */
return Promise.reject(...);
beaconFetchFail({ url, ... });
return Promise.reject(...);

These calls are not wrapped in try/catch. If beacon throws, it will propagate into the Promise chain and could alter error handling behavior or swallow the original rejection. The beaconCardsRendered call later is wrapped — this inconsistency is a bug.


4. Beacon module truncated — getUrlParam function is incomplete

File: beacon.js, line ~399

function getUrlParam(name) {
    try {
        return new URLSearchParams(window.location.search).get(name);
    } catch (e) {
        ret   // <-- truncated: "return null" cut off

The file is cut off mid-function. This means the module will have a syntax error and fail to parse entirely, breaking the entire app boot since app.jsx imports from it at the top level. The try/catch blocks in app.jsx won't help — a module-level syntax error throws before execution reaches them.


5. markCaasReady called on every successful fetch, not just the final one

File: Container.jsx, lines ~1244-1252

If the collection does multiple fetches (partial + full load as the comment acknowledges), markCaasReady fires multiple times. If qa-hooks exposes caas:ready as a one-time signal (typical for "ready" events), subsequent firings could confuse headless agents into re-running assertions against an intermediate state. The scheduleAssertions debounce handles the assertions case but markCaasReady itself is not debounced.

@sanrai sanrai closed this Aug 5, 2026
@sanrai sanrai reopened this Aug 5, 2026
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown

🧪 Feature QA review — injected feature test (advisory, non-blocking)

Last updated Aug 4, 2026, 11:42 PM PT · commit ea219dd · pull_request.

Overall: SKIPPED

Not an injectable feature -- skipped.

This PR only adds telemetry beacons and a QA introspection API (window.caas hooks) with no config- or card-data-driven rendering changes to exercise via CONFIG/COLLECTION DATA overrides.

This PR's change isn't driven by config/collection data the harness can force, so a feature test wouldn't be meaningful. (The visual/smoke review still applies.)

Screenshot in the workflow run.

Review history (1 run)
  • Aug 4, 2026, 11:42 PM PT · SKIPPED · ea219dd · pull_request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants