[WIP] Achieve 100% InProcess and out-of-process test parity by Copilot · Pull Request #2272 · github/copilot-sdk

Copilot · 2026-08-05T19:20:27Z

Gap 1: Removed failsafe <includes> restriction in -Pinprocess profile so all ITs run
Gap 2: Updated E2ETestContext.createClient()/createClient(options) to detect InProcess mode and route env vars through InProcessEnvGuard instead of cliPath/environment/cwd; added COPILOT_SDK_DEFAULT_CONNECTION=inprocess to surefire env vars and forkCount/parallel settings in the inprocess profile
Gap 3: Annotated tests with inherently incompatible direct client construction (ClientOptionsE2ETest, parts of CopilotClientTest, PerSessionAuthTest, RpcServerMiscE2ETest, SlashCommandsIT) with @SkipInProcess("reason")
Verified mvn compile test-compile succeeds
Full mvn clean verify -Pinprocess / mvn clean verify runs not completed due to time constraints — recommend running in CI to confirm full parity before merge

Fixes [Java] Post-agentic 01: Achieve 100% InProcess / out-of-process test parity #2271

Co-authored-by: edburns <75821+edburns@users.noreply.github.com>

Copilot

Pull request overview

Extends Java test coverage across both subprocess and InProcess transports.

Changes:

Runs all Surefire and Failsafe tests under the InProcess profile.
Routes E2E environment configuration through InProcessEnvGuard.
Skips tests requiring subprocess-specific behavior.

Show a summary per file

File	Description
`java/sdk/pom.xml`	Configures InProcess test execution.
`E2ETestContext.java`	Adds InProcess environment handling.
`SkipInProcess.java`	Exposes the execution condition.
`RequireInProcess.java`	Exposes the execution condition.
`SlashCommandsIT.java`	Skips subprocess-only integration tests.
`RpcServerMiscE2ETest.java`	Skips incompatible account-flow test.
`PerSessionAuthTest.java`	Skips incompatible auth-isolation test.
`CopilotClientTest.java`	Skips selected transport-specific tests.
`ClientOptionsE2ETest.java`	Skips subprocess argument-forwarding tests.

Review details

Suppressed comments (2)

java/sdk/pom.xml:631

Failsafe also needs a fork for its configured environment variables to take effect. With forkCount=0, neither COPILOT_SDK_DEFAULT_CONNECTION nor COPILOT_CLI_PATH is injected, so removing the <includes> filter merely runs all ITs on the default transport rather than providing InProcess parity.

                            <forkCount>0</forkCount>

java/sdk/src/test/java/com/github/copilot/E2ETestContext.java:373

An explicit non-InProcess connection must win over the ambient default, matching CopilotClient's transport-selection contract. Currently RuntimeConnection.forStdio()/forTcp() falls through to the environment check, so createClient(options) treats it as InProcess, omits the context's stdio settings, and mutates the native environment even though the client will use the explicit connection.

        RuntimeConnection connection = options.getConnection();
        if (connection instanceof InProcessRuntimeConnection) {
            return true;
        }
        String defaultConnection = System.getenv("COPILOT_SDK_DEFAULT_CONNECTION");

Files reviewed: 9/9 changed files
Comments generated: 3
Review effort level: Balanced

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Bundle both runtime.node and the copilot CLI executable in the classifier JAR. The Rust embedded_host.rs spawns the CLI as a child process via host_start argv[0], so runtime.node alone is insufficient. Changes: - fetch-native.mjs: also extract package/copilot from npm tarball - NativeRuntimeLoader: add resolveEntrypoint() that finds copilot CLI alongside the resolved runtime.node; extractToCache also extracts CLI - CopilotClient: simplify resolveInProcessEntrypoint() to use only the bundled artifact (no COPILOT_CLI_PATH, no PATH search) - E2ETestContext: remove COPILOT_CLI_PATH from InProcess env (unused now) - Add diagnostic spike and yolo prompt for context Previously ExecutorWiringTest failed with FileNotFoundException and AskUserTest hung indefinitely. Both now pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d7a38160-a65e-4d0c-9087-4f28da2a51b8

…nv var - resolveDefaultConnection: when cliUrl, cliPath, or port are explicitly set, fall back to subprocess transport even if COPILOT_SDK_DEFAULT_CONNECTION is 'inprocess'. Explicit options take precedence over the env var default. - validateEnvironmentOptions: check isEmpty() in addition to null, since setEnvironment(null) clears the map rather than nulling the field. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: d7a38160-a65e-4d0c-9087-4f28da2a51b8

Prevent the Java in-process test profile from corrupting Surefire control streams or poisoning later tests through the runtime's process-global LLM provider registration. Preserve explicit subprocess and TCP transport choices, sanitize process-only options before constructing in-process clients, and run request-handler tests over their required isolated stdio runtime. Fix the remaining test-contract issues by making fake socket RPC handler registration atomic with reader startup, honoring the configured CLI entrypoint when runtime.node is in a prebuilds directory, and isolating the streaming model-cache scenario. Remove in-process skip annotations from tests that already exercise an explicit subprocess transport. The complete `mvn clean verify -Pinprocess` run now finishes successfully without hangs, transport timeouts, provider-ownership failures, or Surefire stream corruption. File-by-file manifest: - `java/sdk/pom.xml`: use Surefire's TCP fork channel for unit and integration tests so native runtime output cannot corrupt Maven's process-pipe protocol. - `java/sdk/src/main/java/com/github/copilot/CopilotClient.java`: preserve explicitly selected TCP options when the default connection environment requests in-process transport. - `java/sdk/src/main/java/com/github/copilot/JsonRpcClient.java`: add a socket construction hook that registers handlers before the reader thread starts. - `java/sdk/src/main/java/com/github/copilot/ffi/NativeRuntimeLoader.java`: resolve the configured Copilot executable separately from runtime.node when the native library uses the package's prebuilds layout. - `java/sdk/src/main/java/com/github/copilot/rpc/CopilotClientOptions.java`: allow `setCwd(null)` to clear a previously configured working directory. - `java/sdk/src/test/java/com/github/copilot/ClientOptionsE2ETest.java`: run explicit fake-stdio option forwarding tests under the in-process profile. - `java/sdk/src/test/java/com/github/copilot/ConfigCloneTest.java`: cover clearing a configured working directory. - `java/sdk/src/test/java/com/github/copilot/CopilotClientTest.java`: remove obsolete in-process skips from explicit subprocess and TCP lifecycle tests. - `java/sdk/src/test/java/com/github/copilot/CopilotClientTransportTest.java`: test explicit transport precedence, in-process option sanitization, and TCP token selection under the profile default. - `java/sdk/src/test/java/com/github/copilot/CopilotRequestTestSupport.java`: explicitly select stdio for request-handler tests that register the process-global LLM inference provider. - `java/sdk/src/test/java/com/github/copilot/E2ETestContext.java`: honor explicit transports, route request-handler clients to subprocess isolation, and clear environment, cwd, and CLI arguments before in-process client construction. - `java/sdk/src/test/java/com/github/copilot/GitHubTelemetryTest.java`: register fake runtime RPC handlers before socket message processing begins. - `java/sdk/src/test/java/com/github/copilot/MetadataApiTest.java`: run explicit stdio metadata tests instead of skipping them under the profile. - `java/sdk/src/test/java/com/github/copilot/PerSessionAuthTest.java`: run the explicit subprocess unauthenticated case under the profile. - `java/sdk/src/test/java/com/github/copilot/RpcServerMiscE2ETest.java`: run the explicit subprocess account lifecycle case under the profile. - `java/sdk/src/test/java/com/github/copilot/StreamingFidelityTest.java`: give the gpt-5.4 reasoning/streaming scenario an isolated proxy and runtime model cache. - `java/sdk/src/test/java/com/github/copilot/ffi/FfiRuntimeHostTest.java`: cover failed connection-open cleanup followed by successful sequential startup. - `java/sdk/src/test/java/com/github/copilot/ffi/NativeRuntimeLoaderTest.java`: cover resolving a configured CLI beside a prebuilds runtime. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e03c4e94-97b0-41ad-9f4e-c01633dc0bf7

github-actions · 2026-08-07T18:08:21Z

Cross-SDK Consistency Review ✅

This PR modifies only the Java SDK (java/sdk/src/main/java/, test infrastructure, and planning docs). I reviewed all changed files against the other SDK implementations (Node.js, Python, Go, .NET, Rust) and found no cross-SDK consistency issues.

Summary of changes reviewed

File	Nature	Cross-SDK concern?
`CopilotClient.java`	Adds package-private `closeHook` constructor; improves `resolveDefaultConnection` to let explicit transport options override `COPILOT_SDK_DEFAULT_CONNECTION=inprocess` env var; fixes empty-map false-positive in `validateEnvironmentOptions`	No — InProcess transport is Java-specific (embedded Rust runtime via FFI)
`JsonRpcClient.java`	Adds package-private `fromSocket` overload with initializer callback	No — internal wiring only
`NativeRuntimeLoader.java`	Adds `resolveEntrypoint()` to locate the bundled `copilot` CLI executable alongside `runtime.node`	No — Java-specific native bundling mechanism
`CopilotClientOptions.java`	`setCwd(null)` no longer throws; permits clearing the field	Mildly cross-SDK, but this is a permissive relaxation (removes a restriction) — other SDKs already accept null/None/empty for `cwd` by convention
Test files & annotations (`@SkipInProcess`, `@RequireInProcess`, `E2ETestContext`, etc.)	Test infrastructure for InProcess↔out-of-process parity	No — test-only
`1917-java-embed-rust-cli-runtime-remove-before-merge/`	Planning docs flagged for removal before merge	No

Conclusion

All changes are either Java-internal implementation details for the embedded-runtime (InProcess) transport, or test infrastructure. No new public API methods, configuration keys, or behavioral semantics were introduced that would require equivalent changes in Node.js, Python, Go, .NET, or Rust.

Generated by SDK Consistency Review Agent for #2272 · sonnet46 21.1 AIC · ⌖ 5.5 AIC · ⊞ 6.6K · ◷

Author: Ed Burns <edburns@microsoft.com> Date: Fri Aug 7 17:29:06 2026 +0000 Fix Java in-process test lifecycle and parity Prevent the Java in-process test profile from corrupting Surefire control streams or poisoning later tests through the runtime's process-global LLM provider registration. Preserve explicit subprocess and TCP transport choices, sanitize process-only options before constructing in-process clients, and run request-handler tests over their required isolated stdio runtime. Fix the remaining test-contract issues by making fake socket RPC handler registration atomic with reader startup, honoring the configured CLI entrypoint when runtime.node is in a prebuilds directory, and isolating the streaming model-cache scenario. Remove in-process skip annotations from tests that already exercise an explicit subprocess transport. The complete `mvn clean verify -Pinprocess` run now finishes successfully without hangs, transport timeouts, provider-ownership failures, or Surefire stream corruption. File-by-file manifest: - `java/sdk/pom.xml`: use Surefire's TCP fork channel for unit and integration tests so native runtime output cannot corrupt Maven's process-pipe protocol. - `java/sdk/src/main/java/com/github/copilot/CopilotClient.java`: preserve explicitly selected TCP options when the default connection environment requests in-process transport. - `java/sdk/src/main/java/com/github/copilot/JsonRpcClient.java`: add a socket construction hook that registers handlers before the reader thread starts. - `java/sdk/src/main/java/com/github/copilot/ffi/NativeRuntimeLoader.java`: resolve the configured Copilot executable separately from runtime.node when the native library uses the package's prebuilds layout. - `java/sdk/src/main/java/com/github/copilot/rpc/CopilotClientOptions.java`: allow `setCwd(null)` to clear a previously configured working directory. - `java/sdk/src/test/java/com/github/copilot/ClientOptionsE2ETest.java`: run explicit fake-stdio option forwarding tests under the in-process profile. - `java/sdk/src/test/java/com/github/copilot/ConfigCloneTest.java`: cover clearing a configured working directory. - `java/sdk/src/test/java/com/github/copilot/CopilotClientTest.java`: remove obsolete in-process skips from explicit subprocess and TCP lifecycle tests. - `java/sdk/src/test/java/com/github/copilot/CopilotClientTransportTest.java`: test explicit transport precedence, in-process option sanitization, and TCP token selection under the profile default. - `java/sdk/src/test/java/com/github/copilot/CopilotRequestTestSupport.java`: explicitly select stdio for request-handler tests that register the process-global LLM inference provider. - `java/sdk/src/test/java/com/github/copilot/E2ETestContext.java`: honor explicit transports, route request-handler clients to subprocess isolation, and clear environment, cwd, and CLI arguments before in-process client construction. - `java/sdk/src/test/java/com/github/copilot/GitHubTelemetryTest.java`: register fake runtime RPC handlers before socket message processing begins. - `java/sdk/src/test/java/com/github/copilot/MetadataApiTest.java`: run explicit stdio metadata tests instead of skipping them under the profile. - `java/sdk/src/test/java/com/github/copilot/PerSessionAuthTest.java`: run the explicit subprocess unauthenticated case under the profile. - `java/sdk/src/test/java/com/github/copilot/RpcServerMiscE2ETest.java`: run the explicit subprocess account lifecycle case under the profile. - `java/sdk/src/test/java/com/github/copilot/StreamingFidelityTest.java`: give the gpt-5.4 reasoning/streaming scenario an isolated proxy and runtime model cache. - `java/sdk/src/test/java/com/github/copilot/ffi/FfiRuntimeHostTest.java`: cover failed connection-open cleanup followed by successful sequential startup. - `java/sdk/src/test/java/com/github/copilot/ffi/NativeRuntimeLoaderTest.java`: cover resolving a configured CLI beside a prebuilds runtime. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: e03c4e94-97b0-41ad-9f4e-c01633dc0bf7

Initial plan

42f6c89

Copilot AI assigned Copilot and edburns Aug 5, 2026

Copilot started work on behalf of edburns August 5, 2026 19:34 View session

Copilot AI linked an issue Aug 5, 2026 that may be closed by this pull request

[Java] Post-agentic 01: Achieve 100% InProcess / out-of-process test parity #2271

Closed

Achieve InProcess/out-of-process Java SDK test parity

a7880ac

Co-authored-by: edburns <75821+edburns@users.noreply.github.com>

Copilot AI requested a review from edburns August 5, 2026 20:33

Copilot stopped work on behalf of edburns due to an error August 5, 2026 20:34
Copilot had to stop work due to a timeout.

edburns marked this pull request as ready for review August 5, 2026 21:26

Copilot AI balanced review requested due to automatic review settings August 5, 2026 21:26

edburns requested a review from a team as a code owner August 5, 2026 21:26

Copilot started reviewing on behalf of edburns August 5, 2026 21:26 View session

Copilot AI reviewed Aug 5, 2026

View reviewed changes

Comment thread java/sdk/pom.xml Outdated

Comment thread java/sdk/pom.xml

Comment thread java/sdk/src/test/java/com/github/copilot/E2ETestContext.java Outdated

This comment has been minimized.

Sign in to view

Address Copilot review findings

f36371a

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This comment has been minimized.

Sign in to view

github-actions Bot mentioned this pull request Aug 7, 2026

[aw] No-Op Runs #1989

Open

GUTDODP

b78e74e

This comment has been minimized.

Sign in to view

edburns merged commit c1a201b into edburns/1917-java-embed-rust-cli-runtime-post-agentic-01-test-parity Aug 7, 2026
27 checks passed

edburns deleted the copilot/edburns1917-java-embed-rust-cli-runtime-post-agent branch August 7, 2026 18:15

Navigation Menu

Uh oh!

[WIP] Achieve 100% InProcess and out-of-process test parity - #2272

[WIP] Achieve 100% InProcess and out-of-process test parity#2272
edburns merged 7 commits into
edburns/1917-java-embed-rust-cli-runtime-post-agentic-01-test-parityfrom
copilot/edburns1917-java-embed-rust-cli-runtime-post-agent

Copilot AI commented Aug 5, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Aug 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Copilot AI commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Aug 7, 2026

Cross-SDK Consistency Review ✅

Summary of changes reviewed

Conclusion

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Aug 5, 2026 •

edited

Loading