[Java] Post-agentic 01: Achieve 100% InProcess / out-of-process test parity #2271

Overview

Achieve 100% test parity between the InProcess (FFI) transport and the existing out-of-process (stdio/TCP) transport. Every Surefire unit test and Failsafe integration test that runs under the default transport must also run under the InProcess transport, or be explicitly annotated with @SkipInProcess with a documented reason.

Branch: edburns/1917-java-embed-rust-cli-runtime-post-agentic-01-test-parity on upstream

Plan and supporting resources

On the edburns/1917-java-embed-rust-cli-runtime-post-agentic-01-test-parity branch, the directory 1917-java-embed-rust-cli-runtime-remove-before-merge contains the plan (1917-embed-cli-runtime-ignorance-reduction-plan.md) and supporting resources (spikes, prototypes, diagrams).

Read the entire plan before working.

Relevant plan sections to carefully re-read

• Section 3.11 — E2E testing with InProcess transport — Resolution: Read the full evidence in 1917-java-embed-rust-cli-runtime-remove-before-merge/spike-3-11-replay-proxy-and-in-process/. Key answers:
  1. Replay proxy works with InProcess — the replay proxy intercepts HTTP calls to COPILOT_API_URL. The runtime reads COPILOT_API_URL from the native process environment block.
  2. Use real runtime.node binary — from the copilot-native module.
  3. No mock native library for E2E — only unit tests use mocks.
  4. Reuse existing YAML snapshots — HTTP traffic is identical regardless of transport.
  5. Run full E2E suite under both transports — subprocess (existing job A) and InProcess (new job B with -Pinprocess Maven profile).
• InProcessEnvGuard requirement — calls SetEnvironmentVariableW (Windows) or setenv() (Linux/macOS) via JNA to mutate process environment before host_start. Restores on close(). Already implemented in com.github.copilot.ffi.InProcessEnvGuard.
• Concurrency must be 1 — InProcessEnvGuard mutates process-global state. Use surefire.forkCount=1 or JUnit 5 @ResourceLock.
• @SkipInProcess annotation — Already implemented in com.github.copilot.e2e.SkipInProcess. JUnit 5 condition annotation for tests incompatible with InProcess transport (e.g., per-client environment variables — see issue [Tracking] In-process (FFI) items to be cleaned up #1934).
• Section 3.12 — CI/CD workflow changes — Resolution: java-sdk-inprocess job in java-sdk-tests.yml already exists.
• Hard scope invariant: Linux-x64 only. No other platform E2E tests in this phase.

Current state

The following infrastructure already exists on the branch:

• InProcessEnvGuard in com.github.copilot.ffi — sets/restores native env vars via JNA.
• @SkipInProcess annotation in com.github.copilot.e2e — JUnit 5 condition for skipping incompatible tests.
• @RequireInProcess annotation in com.github.copilot.e2e — JUnit 5 condition for InProcess-only tests.
• InProcessTransportIT — single Failsafe smoke test that passes under -Pinprocess.
• -Pinprocess Maven profile — activates InProcess transport for Failsafe.
• java-sdk-inprocess CI job — runs mvn clean verify -Pinprocess.

Identified gaps

Gap 1: Failsafe IT parity

The -Pinprocess profile failsafe configuration only includes **/InProcessTransportIT.java. The other 6 ITs are excluded:

• ErgonomicToolDefinitionIT
• GitHubTelemetryForwardingIT
• InternalExecutorProviderIT
• LowLevelToolDefinitionIT
• SlashCommandsIT
• SystemMessageSectionsIT

Fix: Remove the <includes> restriction from the inprocess profile failsafe config so all ITs run. Annotate any IT that is inherently incompatible with @SkipInProcess (with reason).

Gap 2: Surefire E2E test parity

Many surefire tests use E2ETestContext and exercise real CLI interactions. These never run under InProcess because:

1. The CI step no longer sets COPILOT_SDK_DEFAULT_CONNECTION=inprocess (we removed it to fix 195 surefire failures caused by env var leaking into tests that set cliPath/environment/cwd).
2. E2ETestContext.createClient() unconditionally sets cliPath, environment, and cwd on CopilotClientOptions — all three are rejected by CopilotClient.validateEnvironmentOptions() for InProcess connections.

Fix: Update E2ETestContext.createClient() to detect InProcess mode and:

• Skip setting cliPath, environment, and cwd on options.
• Instead, use InProcessEnvGuard to set the required env vars in the native process environment.
• Re-add COPILOT_SDK_DEFAULT_CONNECTION=inprocess to surefire <environmentVariables> in the inprocess profile (not the CI step env).

Gap 3: Tests with direct client creation

Some tests create CopilotClient directly (not via E2ETestContext) with stdio-specific options (cliPath, cliArgs, cwd). These are inherently incompatible with InProcess transport.

Fix: Annotate each with @SkipInProcess and a reason. Examples:

• ClientOptionsE2ETest — tests CLI argument forwarding.
• Tests in CopilotClientTest that set cliArgs or test CLI startup behavior.

Deliverables

Files to modify

1. java/sdk/pom.xml — Update the -Pinprocess profile:

   • Remove <includes> restriction from failsafe so all ITs run.
   • Add COPILOT_SDK_DEFAULT_CONNECTION=inprocess to surefire <environmentVariables>.
   • Ensure forkCount=1 and parallel=none for both surefire and failsafe.

2. java/sdk/src/test/java/com/github/copilot/E2ETestContext.java — Update createClient() methods to:

   • Detect InProcess mode (via explicit connection or COPILOT_SDK_DEFAULT_CONNECTION env var).
   • Skip incompatible options (cliPath, environment, cwd).
   • Use InProcessEnvGuard for native env setup.

3. Various test classes — Add @SkipInProcess("reason") to tests that are inherently incompatible with InProcess transport.

No files to create

All required infrastructure (InProcessEnvGuard, @SkipInProcess, @RequireInProcess) already exists.

Gating tests and criteria

1. Full parity verified: mvn clean verify -Pinprocess from java/ runs all surefire and failsafe tests under InProcess transport (except those annotated @SkipInProcess).
2. Same test count: The number of tests run + skipped under -Pinprocess equals the number run + skipped under the default profile.
3. Standard tests unaffected: mvn clean verify (without -Pinprocess) passes with no regressions.
4. CI passes: Both java-sdk and java-sdk-inprocess CI jobs pass.
5. Spotless compliance: mvn spotless:check passes.
6. Every @SkipInProcess has a reason documenting why the test is incompatible.

Out of scope

• Testing on any platform other than linux-x64.
• Adding new test scenarios — this task is about running existing tests under both transports.
• Changes to production code in src/main/java (except if CopilotClient.validateEnvironmentOptions needs adjustment).

Parent issue: #2268