fix: resolve nested window offsets for widget elements in dump ui - #55
Conversation
XCTest reports frames relative to the screen, except for elements hosted in a nested window (e.g. a WidgetKit extension), where frames reset to be relative to that window's own origin. This made home screen widget elements (like a Calendar widget's date label) report coordinates near 0,0 instead of their real screen position. windowContextID marks the boundary between windows: when it changes between a nonzero parent and a different nonzero child, rebase that subtree's frames onto the screen-absolute position the parent already resolved, and carry the same offset down through every descendant still inside that window.
WalkthroughThe change adds 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@DeviceKitTests/XCTest/AXElement.swift`:
- Around line 241-246: Remove the trailing comma after the "Height" entry in the
resolvedFrame AXFrame literal, leaving all other frame values unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c296f0ee-02f7-4f8c-a2b0-5cedc53f82ce
📒 Files selected for processing (2)
DeviceKitTests/JSONRPC/Handlers/DumpUI.swiftDeviceKitTests/XCTest/AXElement.swift
| let resolvedFrame: AXFrame = [ | ||
| "X": (frame["X"] ?? 0) + offsetX, | ||
| "Y": (frame["Y"] ?? 0) + offsetY, | ||
| "Width": frame["Width"] ?? 0, | ||
| "Height": frame["Height"] ?? 0, | ||
| ] |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Remove the trailing comma flagged by SwiftLint.
SwiftLint's trailing_comma rule flags line 245. Remove the trailing comma after the last element of the resolvedFrame literal.
🧹 Proposed fix
let resolvedFrame: AXFrame = [
"X": (frame["X"] ?? 0) + offsetX,
"Y": (frame["Y"] ?? 0) + offsetY,
"Width": frame["Width"] ?? 0,
- "Height": frame["Height"] ?? 0,
+ "Height": frame["Height"] ?? 0
]📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| let resolvedFrame: AXFrame = [ | |
| "X": (frame["X"] ?? 0) + offsetX, | |
| "Y": (frame["Y"] ?? 0) + offsetY, | |
| "Width": frame["Width"] ?? 0, | |
| "Height": frame["Height"] ?? 0, | |
| ] | |
| let resolvedFrame: AXFrame = [ | |
| "X": (frame["X"] ?? 0) + offsetX, | |
| "Y": (frame["Y"] ?? 0) + offsetY, | |
| "Width": frame["Width"] ?? 0, | |
| "Height": frame["Height"] ?? 0 | |
| ] |
🧰 Tools
🪛 SwiftLint (0.65.0)
[Warning] 245-245: Collection literals should not have trailing commas
(trailing_comma)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@DeviceKitTests/XCTest/AXElement.swift` around lines 241 - 246, Remove the
trailing comma after the "Height" entry in the resolvedFrame AXFrame literal,
leaving all other frame values unchanged.
Source: Linters/SAST tools
1 participant
Summary
device.dump.uireported wrong screen coordinates for elements hosted inside a nested window (e.g. a home screen widget's WidgetKit extension) — frames came back relative to that window's own origin instead of the screen, since XCTest's snapshot frame is only screen-absolute up to the point wherewindowContextIDchanges.AXElement.resolvingNestedWindowOffsets(), which walks the tree and, on a nonzero-to-different-nonzerowindowContextIDtransition, rebases that subtree's frames onto the screen-absolute position the parent already resolved — carrying the same offset through every descendant still inside that window.DumpUI.swift'selementHierarchy.Root cause
Confirmed via
rawformat dump of a home screen with a Calendar widget: the widget's date label reportedx=17.67, y=27.99(relative to the widget's own WidgetKit extension window), while the widget itself sits atx=204, y=67on screen. Also confirmed this is not something WebDriverAgent/Appium handles correctly either — same bug reproduces there.Test plan
x=221.67, y=95.0, matching the hand-computed expected screen position (204+17.67,67+28).windowContextIDnever changes).Related: mobile-next/mobilewright#238