Skip to content

ci: capture Windows JVM hang diagnostics - #1786

Closed
TyceHerrman wants to merge 10 commits into
oraios:mainfrom
TyceHerrman:agent/jdtls-hang-diagnostics
Closed

ci: capture Windows JVM hang diagnostics#1786
TyceHerrman wants to merge 10 commits into
oraios:mainfrom
TyceHerrman:agent/jdtls-hang-diagnostics

Conversation

@TyceHerrman

@TyceHerrman TyceHerrman commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add a per-test watchdog for the Windows JVM pytest batch
  • capture all Python threads, the pytest process tree, JDTLS JVM thread dumps, and active JDTLS workspace logs
  • fall back from jcmd Thread.print -l to jstack -l when JVM attachment fails
  • capture after 10 minutes and terminate after 12 minutes in normal runs
  • repeat the cross-language sequence surrounding the failed Java startup on this diagnostic branch
  • shorten branch-only capture/termination to 5/7 minutes, preserve each attempt log, and continue past fast assertion failures until a hang is captured or all repetitions finish
  • run 20 cold JDTLS workspaces after a 50-warm-start run completed without a hang
  • print any Python/JVM dump into the Actions job log before artifact upload

Why

The Windows JVM job in run 30496140622 stalled in a Java test until the 60-minute job timeout. The existing log does not show whether Python was waiting for IntelliCode or ServiceReady, or whether JDTLS was blocked inside Maven/m2e initialization.

The first 50-repetition run did not hang. Attempt 26 failed quickly because Java hover information omitted the expected Javadoc. Its artifact correctly contained only pytest.log; the watchdog had not fired.

A second 50-repetition run also completed without a hang. Eight attempts failed with the same missing-Javadoc result. A third run cleared only the isolated JDTLS workspace between attempts; all 20 repetitions passed without a hang or missing-Javadoc failure.

This draft PR remains diagnostic instrumentation. It is intended to capture the evidence needed to distinguish a Serena/SolidLSP wait or event-ordering problem from an upstream JDTLS/m2e deadlock.

Validation

  • uv run pytest test/test_ci_hang_diagnostics.py -q
  • uv run poe lint
  • uv run poe type-check
  • forced-hang smoke test confirming the artifact completes before pytest-timeout exits the process
  • workflow YAML parsed successfully; actionlint reported no diagnostics on the changed lines

@TyceHerrman

Copy link
Copy Markdown
Contributor Author

Closing this diagnostic draft as superseded by the focused production fix in #1790.

The branch completed its intended investigation:

  • the deliberate Windows canary captured Python threads, the process tree, a live JDTLS jcmd Thread.print -l, phase/status history, bounded LSP metadata, and the Eclipse workspace log
  • that canary showed healthy JDTLS initialization before the deliberate Python-side pause
  • the original reference-test boundary then passed in 3 full-batch attempts, 25 fresh-workspace attempts, and 25 reused-workspace attempts
  • the three full-batch failures occurred later in the unrelated safe-delete test after repeated mutable-suite execution produced pupackage test_repo;; no watchdog capture was produced

Because none of the 53 target executions produced a natural stall, there is no JVM/Python dump supporting a JDTLS, m2e, or Buildship issue. No upstream issue is being filed.

#1790 retains the durable Serena behavior: bounded IntelliCode and ServiceReady waits, exact phase and last-status reporting, and JDTLS termination on timeout. The cache-correctness change remains separate in #1788. The investigation summary and upstream filing boundary are recorded in #1789.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant