Skip to content
Open
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
2d688cc
docs(testing): ship XML documentation with the Repl.Testing package
carldebilly Sep 10, 2026
bfee74c
feat(defaults): reach the SIGTERM claim in-process
carldebilly Sep 10, 2026
fc66770
feat(defaults): make the registration platform declarable, without in…
carldebilly Sep 10, 2026
b5b8a34
feat(testing): add a deterministic process-signal harness
carldebilly Sep 10, 2026
b49d0cd
feat(testing): add a real-process signal probe
carldebilly Sep 11, 2026
cb847ba
test: stop Given_ProcessSignalExitCodePolicy running in parallel
carldebilly Sep 11, 2026
1fffb6e
docs: document process-signal testing and where its limits are
carldebilly Sep 11, 2026
e0edd9f
fix(testing): close what the local review panel found
carldebilly Sep 11, 2026
79a0f31
fix(testing): address the PR review
carldebilly Sep 12, 2026
e5c8f2f
fix(testing): close the second review wave
carldebilly Sep 14, 2026
4b02768
fix(testing): close the third review wave
carldebilly Sep 14, 2026
8ba874f
fix(testing): close the local review panel's findings
carldebilly Sep 14, 2026
690e539
fix(testing): pair registrations by run, not by order
carldebilly Sep 14, 2026
e740b62
fix(testing): refuse a delivery an interactive session would take
carldebilly Sep 14, 2026
2cadcf8
fix(testing): declare the platform in the interactive-ownership guard
carldebilly Sep 14, 2026
d02df18
fix(testing): bound the exit drain and decide refusal inside the disp…
carldebilly Sep 15, 2026
509e69f
fix(testing): drain on end-of-stream and refuse a timeout that cannot…
carldebilly Sep 15, 2026
00b67c6
fix(testing): watch the process, not the pipe, when waiting for exit
carldebilly Sep 15, 2026
ab996ac
fix(testing): one drain deadline, and one truth per promise
carldebilly Sep 15, 2026
4a557b3
fix(testing): refuse a timeout that cannot bound the wait it is given to
carldebilly Sep 15, 2026
ddbdf6b
fix(testing): close the boundary-value holes across the public options
carldebilly Sep 15, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@
- `Repl.Mcp`
- MCP (Model Context Protocol) integration: `UseMcpServer()`, `BuildMcpServerOptions()`, tool/resource/prompt mapping, MCP Apps UI resources, client roots, transport factory.
- `Repl.Testing`
- In-memory multi-session testing toolkit (`ReplTestHost`, `ReplSessionHandle`, typed execution results/events).
- In-memory multi-session testing toolkit (`ReplTestHost`, `ReplSessionHandle`, typed execution results/events),
plus process-signal testing: `ReplProcessSignalHarness` drives the lifecycle in memory for any
declared platform, and `ReplProcessProbe` spawns an application for what only a real process shows.
See [Testing toolkit](testing-toolkit.md#process-signals).
- `Repl.Tests`
- Unit tests for pure logic and contracts.
- `Repl.IntegrationTests`
Expand All @@ -45,6 +48,10 @@ reverse is not guaranteed: a release's `Repl.Defaults` may call APIs that only e
release's `Repl.Core`. Upgrading one and pinning the other is unsupported. The `Repl` meta-package
takes both at matched versions, which is the reason to prefer it.

`Repl.Testing` belongs to that same set. It drives the signal coordinators and the session sink through
internals both assemblies grant it, so it is version-matched to them too — a newer `Repl.Testing`
against an older `Repl.Core` or `Repl.Defaults` is unsupported for the same reason.

## Quality gates

- Strict build rules from `src/Directory.Build.props`:
Expand Down
4 changes: 4 additions & 0 deletions docs/best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -363,6 +363,10 @@ That keeps status/progress/problem events out of the main Spectre surface and av

Use `UseCliProfile()` (or an explicit `ProcessSignalHandlingMode.Automatic`) for a standalone CLI where Repl is the process owner. An unprofiled `ReplApp.Create()` remains caller-owned. Use `UseEmbeddedConsoleProfile()` or explicitly set `ProcessSignalHandlingMode.None` when an ASP.NET Core host, worker service, test runner, or another command framework already owns console cancellation and shutdown. Feed that host's cancellation token into `RunAsync` instead of installing competing handlers. External `IServiceProvider`, `IHost`, and `IReplHost` overloads always remain caller-owned and diagnose an explicit `Automatic` request instead of applying it.

Whichever ownership you pick, assert it rather than assume it: `ReplProcessSignalHarness` covers the
signal lifecycle in memory and `ReplProcessProbe` covers what only a spawned process can show. See
[Testing toolkit](testing-toolkit.md#process-signals).

Supplying a `ReplRunOptions` instance for an unrelated setting preserves the profile default because `ProcessSignalHandling` is nullable:

```csharp
Expand Down
6 changes: 6 additions & 0 deletions docs/configuration-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,12 @@ A record passed to `app.RunAsync(...)` to control runtime behavior. Separate fro

### Process signal handling

Everything in this section is testable without spawning a process: `Repl.Testing`'s
`ReplProcessSignalHarness` drives the lifecycle in memory and can declare the platform whose decisions
apply, so a Windows wiring decision is assertable from Linux and the other way round. See
[Testing toolkit](testing-toolkit.md#process-signals), which is also where the boundary is written
down — what an in-memory test can prove, and what needs a real process.

`ProcessSignalHandling` applies only to standalone `Run`/`RunAsync` overloads that use the app's internally configured services. Overloads that receive an external `IServiceProvider`, `IHost`, or `IReplHost` do not install the standalone process-signal bridge; the external owner remains responsible for translating shutdown into the caller-owned cancellation token. Passing an explicit `Automatic` value to one of those overloads writes a diagnostic to the active error channel and ignores the value. If such a run enters Repl's interactive loop, that loop still retains its own console command-cancellation policy.

The mode that actually applies to a run is resolved in this order:
Expand Down
155 changes: 155 additions & 0 deletions docs/testing-toolkit.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,161 @@ await using var host = ReplTestHost.Create(
- `NormalizeAnsi`: strips ANSI escape sequences from `OutputText` when `true`.
- `RunOptionsFactory`: provides base `ReplRunOptions` for each session.

## Process Signals

Signal handling is not a session concern, so it has its own entry points rather than living on
`ReplTestHost`. There are two, and they prove different things:

- `ReplProcessSignalHarness` drives the whole lifecycle **in memory**. Deterministic, fast, runs
everywhere, and covers every decision the framework makes.
- `ReplProcessProbe` **spawns your application** and sends it real signals. Slower and platform-bound,
and the only way to show that a process actually terminates.

Reach for the harness first. Use the probe for the handful of guarantees the harness cannot honestly
make.

### Deterministic, in memory

```csharp
await using var harness = ReplProcessSignalHarness.Create(CreateApp);
var run = await harness.StartRunAsync("work");

var delivery = harness.SendSignal(ReplProcessSignal.Interrupt);
var result = await run.Completion;

delivery.Should().Be(ReplSignalDelivery.CancellationRequested);
result.OutcomeKind.Should().Be(ReplExecutionOutcomeKind.Interrupted);
result.ExitCode.Should().Be(130);
```

`SendSignal` is synchronous on purpose: a signal callback owes the operating system a suppression
decision before it returns, so the framework decides synchronously and so does this. Await
`run.Completion` to see what the decision did.

Runs execute as **standalone invocations**, not hosted sessions — the same classification a
process-owning `Main` gets. Commands gated to the CLI channel are therefore present, and handlers see
`IReplIoContext.IsHostedSession == false`.

`RunTimeout` is measured against the clock rather than against the run agreeing to stop, so a command
that never observes its cancellation token still fails the test instead of hanging the suite. Such a
run cannot be killed: the harness abandons it, and disposal says so, because it still holds a place in
the process-wide signal epoch.

**`StartRunAsync` guarantees the signal will reach the run, not that the command is running.** The
signal scope is installed around the whole run, before its arguments are parsed, so the command body
Comment thread
carldebilly marked this conversation as resolved.
Outdated
has usually not started when the call returns. If you are asserting on what the command did — that its
cleanup ran, say — have the command say when it is running:

```csharp
var started = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
app.Map("work", async (CancellationToken ct) =>
{
started.TrySetResult();
try { await Task.Delay(Timeout.Infinite, ct); return "unreachable"; }
finally { /* cleanup you want to assert on */ }
});
// ...
var run = await harness.StartRunAsync("work");
await started.Task;
harness.SendSignal(ReplProcessSignal.Interrupt);
```

Start a second run to test a **late joiner** — a run that begins while a signal is already claimed and
inherits that epoch. Starting is serialised; the runs are not. Note that an epoch resets once its last
run drains, so the earlier run has to still hold its scope for the window to exist.

### Declaring a platform

By default the harness applies the decisions of the platform you are on. Declare another one and they
become assertable from anywhere:

```csharp
await using var harness = ReplProcessSignalHarness.Create(
CreateApp,
options => options.Platform = ReplPlatformProfile.Windows);
var run = await harness.StartRunAsync("work");

// Ctrl+Break is a signal on Windows and nothing anywhere else — this passes on Linux too.
harness.SendSignal(ReplProcessSignal.Break).Should().Be(ReplSignalDelivery.CancellationRequested);
```

Profiles: `Current`, `Windows`, `Unix` (Linux and macOS decide identically here, so they share one),
`Android`, `Browser`, `IOS`, `TvOS`. The last four have no signal bridge, so a run under them degrades
to caller-owned handling and says so once.

A declared platform changes **decisions only**: the harness never installs an operating-system signal
registration on that platform's behalf. That matters more than it sounds — .NET accepts a `SIGTERM`
registration on Windows too, so nothing but this rule would stop a declared-Unix test from installing
a live handler in your test process.

One consequence to know: a profile with no signal bridge (`Browser`, `Android`, `IOS`, `TvOS`)
registers no console cancel-key handler either, so a real Ctrl+C aimed at your runner during such a run
takes its normal course rather than being claimed by it.

Profiles are the only way to build one — the flags are readable so you can assert on them, not
settable, so combinations no device has cannot be constructed by mistake.

### One harness at a time

Signal handling is process-global, and a harness owns it for its lifetime — really owns it, not by
convention. While it holds ownership, a second harness is refused, and so is any other run that would
install its own signal handling: such a run joins the harness's isolated epoch, gets cancelled by its
synthetic signals, and releases its readiness wait. Failing that run loudly is the point. Configure
your framework so it does not happen:

| Framework | Parallel by default? | What to add |
| --- | --- | --- |
| MSTest | yes | `[DoNotParallelize]` on the test class |
| xUnit | yes, across classes | a shared `[Collection("...")]`, or `[assembly: CollectionBehavior(DisableTestParallelization = true)]` |
| NUnit | no | `[NonParallelizable]`, only if you opted into parallelism |

Sharding across separate processes needs none of this: the state is per-process. And the package
references no test framework, so it cannot apply any of these for you.

### What only a real process can prove

```csharp
await using var probe = ReplProcessProbe.Start("./my-app", ["wait", markerPath]);
await probe.WaitForOutputAsync("READY");
await probe.SendSignalAsync(ReplProcessSignal.Terminate);

(await probe.WaitForExitAsync()).Should().Be(143);
```

Output is drained continuously, so a chatty child never blocks; every wait reports what was captured
when it fails; and disposal kills the process tree so a failed assertion cannot leak a running
process.
Comment thread
carldebilly marked this conversation as resolved.
Outdated

**Signals are delivered on Unix only.** Sending one to another process on Windows needs a console
control event and console attachment rather than a signal, which is deliberately out of scope until
the Windows lifecycle work lands. `SendSignalAsync` throws `PlatformNotSupportedException` there;
spawning, waiting on output and exit codes all still work, so a cross-platform suite shares everything
but the delivery. `ReplProcessSignal.Break` is refused on every platform — it is a Windows console
event, and the nearest Unix signal is one the framework deliberately leaves unclaimed.

One trap worth knowing: `probe.Output` holds what the child flushed. To observe what happened during a
shutdown the process might not survive, have the application append to a file and read that.

### What each half proves

| | Harness | Probe |
| --- | --- | --- |
| The decision about each signal | yes | — |
| Diagnostics the framework wrote | yes, on `harness.DiagnosticText` | in the child's output |
| The exit code the run resolves to | yes | yes, as the shell sees it |
| Cleanup was given time to run | yes | yes |
| Every platform's wiring decisions | yes, from any host | only the host's own |
| **The process actually terminated** | **no** | **yes** |

A second signal makes the harness report `WouldTerminateProcess`. That is the framework's decision and
nothing more: nothing dies in-process, so the run keeps unwinding and your code after the call keeps
executing. Read it as "Repl chose not to intervene again", never as "it stopped".

Signal diagnostics are written from whichever context delivers the signal — an operating-system
callback thread with no session, in production — so they belong to the delivery rather than to a run.
`harness.DiagnosticText` holds them; `ReplSignalRunResult.DiagnosticText` holds what the run itself
wrote.

## Notes

- The harness is in-memory and transport-agnostic (no loopback network stack required).
Expand Down
6 changes: 4 additions & 2 deletions eng/ci/process-signal-stress.sh
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,11 @@ run_stress \
"FullyQualifiedName~Given_ProcessSignalCancellationScope" \
25

# Given_ProcessSignalHarness does not contain the substring "Given_ProcessSignals" — there is no "s"
# after "Signal" — so it needs its own clause rather than riding along on the one below.
run_stress \
"process-signal integration stress" \
"$integration_iterations" \
src/Repl.IntegrationTests/Repl.IntegrationTests.csproj \
"FullyQualifiedName~Given_ProcessSignals" \
8
"FullyQualifiedName~Given_ProcessSignals|FullyQualifiedName~Given_ProcessSignalHarness|FullyQualifiedName~Given_ProcessProbe" \
39
Loading
Loading