Skip to content

Strings tab, a picture for every step, inputText by selector, faster iOS calls - #257

Merged
handstandsam merged 1 commit into
mainfrom
upstream-2026.09.30
Oct 1, 2026
Merged

handstandsam merged 1 commit into
mainfrom
upstream-2026.09.30

Conversation

@handstandsam

@handstandsam handstandsam commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Syncs the latest upstream changes through 2026.10.01, bringing in:

  • Strings tab in the report: every screen's visible strings are recorded as it's captured, with each string cut out of its screenshot.
  • A picture for every step: a step with no screenshot gets a frame from the session video. Thumbnails play the step's recording, and the Timeline can seek anywhere and open any captured event.
  • inputText by selector: it can tap its field by selector, including by the field's label, and now types into password fields on web pages inside Android apps.
  • Koog is the only agent: the legacy TrailblazeRunner and MULTI_AGENT_V3 agents are removed.
  • Decision engine (experimental, off by default): it can make the agent's routine moves without an LLM call, using any POST /v1/systemone server. It also says which element a tap hit.
  • Faster iOS: AXe tool calls share screen reads within a call, and iOS snapshots no longer repeat the same tap point. assertVisible no longer crashes on cells with a disabled child.
  • iOS reliability: stopping the daemon no longer leaves XCTest runners behind to kill the next one, and a dead runner now says so and is rebuilt on reconnect instead of reporting "No device connected". Home-screen icons and other elements under zero-size wrappers now show in the element list.
  • Tools per platform: a session now only sees the tools that run on its platform, for in-process scripted tools too. Scripted tool results now include the screen after the action.
  • Android: the runner recovers after an emulator reboot. Session video is timed by when the device drew each frame.
  • Web: checks fail fast on a page that has stopped responding instead of waiting it out. Screenshots no longer stall on pages with many links, and several CLI rough edges are fixed. WebM recordings now play in Chrome.
  • CLI: a run that times out is cancelled on the daemon, so a retry doesn't run beside it. Quiet CLI commands no longer silence the daemon's other logs.
  • Errors: ask, blaze and verify fail when the LLM call fails.
  • README checks: example README commands that pass undeclared CLI flags are now caught.
  • Security: patched vulnerable MCP SDK, logging, metrics, jsoup and fast-uri dependencies.
  • Cleanup: removed unreferenced code and the report's Map tab.

@handstandsam
handstandsam merged commit d97b97c into main Oct 1, 2026
12 checks passed
@handstandsam
handstandsam deleted the upstream-2026.09.30 branch October 1, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant