feat: add ANSI parsing support on Windows (VT input hybrid)

[eitsupi]

Fix #737
Fix #962

Summary

• Keep the existing Win32 input path as the default on Windows. Raw mode preserves ENABLE_VIRTUAL_TERMINAL_INPUT instead of enabling it implicitly.
• Enable VT input only for the lifetime of EnableBracketedPaste, allowing modern Windows terminals to deliver bracketed paste without changing keyboard semantics for every raw-mode user.
• Preserve ANSI-only bracketed paste in environments such as Git Bash and mintty, where TERM indicates ANSI support but no Win32 CONIN$ console is available.
• Share the ANSI parser between Unix and Windows by moving it from sys/unix/parse to sys/parse.
• Keep WindowsEventSource construction side-effect free and select the VT or VK path from the actual console mode for each input batch.
• Bridge Windows mouse capture through both WinAPI mode flags and the VT ?1003;1006 representation, so mouse input continues across VT transport changes.
• Fall back to the existing ReadConsoleInput VK handling when VT input is inactive or unavailable.

Scope on Windows

The source reads the console mode once per batch. Character-bearing key-down records use the shared ANSI parser only while VT input is active; other records continue through the Win32 path when the host supplies them.

Area	Bracketed paste disabled	Bracketed paste enabled / VT input active
Keyboard	Existing Win32 translation, preserving Press and Release; a held key may be exposed as additional Press records	Character and ANSI-sequence presses use the shared parser. Host-supplied releases remain supported, but Windows Terminal/ConPTY may expose a Press-only VT transport
Bracketed paste (Event::Paste)	Not enabled	Decoded by the shared ANSI parser
Alt+numpad	Existing Win32 handling	Preserves host-supplied releases and suppresses only the adjacent duplicate character record observed on conhost
Mouse	Win32 MOUSE_EVENT path	SGR VT reports through the shared parser; WinAPI capture remains enabled for conhost and legacy fallback
Focus and resize	Existing INPUT_RECORD handling	Existing records remain supported when supplied by the host
UTF-16 surrogate pairs	Existing VK handling	Separate surrogate buffers for the VT and VK paths

Keyboard input is still expected to be read in raw mode. Reading events without entering raw mode remains valid for resize, focus, and mouse events and does not change the console mode as a side effect.

Explicitly out of scope: the kitty keyboard protocol. supports_keyboard_enhancement() still returns Ok(false) on Windows and PushKeyboardEnhancementFlags still reports ErrorKind::Unsupported. Sharing the parser removes one decoding obstacle, but negotiating that protocol on Windows is separate work.

Console-mode lifecycle

• WindowsEventSource::new() opens CONIN$ but does not modify its mode.
• enable_raw_mode() captures the first-touch console mode, changes only the Windows raw-mode bits, and preserves the current VT-input bit.
• disable_raw_mode() restores the cooked-mode bits while likewise preserving the current VT-input bit.
• On ANSI-capable Windows terminals with a Win32 console, EnableBracketedPaste flushes prior output, saves the current VT-input bit in a bracketed-paste-specific state, enables VT input, writes CSI ? 2004 h, and flushes again before returning.
• DisableBracketedPaste writes CSI ? 2004 l, flushes it, and only then restores the VT-input bit observed immediately before the paired enable. It preserves every other current console-mode bit and consumes the saved state only after a successful restore.
• An unpaired disable leaves the console mode unchanged. If the paired enable found no CONIN$ console, both commands still emit their ANSI sequences and safely skip the WinAPI mode side effects.
• Missing-console errors are recognized narrowly; access-denied and genuine console-mode failures still abort before an unsafe state transition.
• The paired bracketed-paste commands are synchronization boundaries on Windows and are not reference-counted; independent nested ownership is not supported.
• The event source does not cache VT support for its lifetime. Console and ConsoleMode share the same CONIN$ handle, and the real VT and raw-mode state is read once per batch.

The ordering is deliberate: VT input must be active before enabling the terminal protocol, while the disable sequence must be flushed before restoring the input transport. Failures short-circuit later steps so the parser transport is not disabled before the terminal has accepted CSI ? 2004 l.

Command execution on Windows

The hidden Command implementation hooks distinguish:

• the historical WinAPI fallback used when ANSI is unsupported;
• WinAPI side effects that run before ANSI;
• WinAPI side effects that run before ANSI and force a final flush;
• WinAPI side effects that run after ANSI has been flushed.

This preserves the existing legacy fallback while supporting the coordinated mouse and bracketed-paste operations. Hybrid commands are rejected by the internal fmt::Write formatting path because it cannot provide the required I/O flush ordering.

Mouse transport

Windows mouse capture now keeps the WinAPI flags and, when ANSI is supported, also emits ?1003;1006h/l.

• With VT input off, terminals continue to deliver Win32 MOUSE_EVENT records.
• With VT input on for bracketed paste, Windows Terminal/ConPTY can deliver SGR mouse reports through the shared parser.
• The two representations preserve conhost/legacy fallback and avoid the mouse regression described by Mouse input differences when using ENABLE_VIRTUAL_TERMINAL_INPUT microsoft/terminal#15296. The VT representation matches the bridge used by [Conpty] Pass through request for mouse mode to the Terminal microsoft/terminal#9970.

Key releases and Alt+numpad

When a host supplies character-bearing key-up records during VT input, releases are normalized through the same control-character and uppercase rules as the shared ANSI parser. This keeps the public Press and Release identities aligned.

Some hosts do not supply those records. In particular, Windows Terminal/ConPTY was observed to expose ordinary characters, function keys, modified characters, and Alt-code input as Press-only while VT input is active. This PR does not synthesize missing releases because doing so would duplicate real releases on other hosts and would invent hold, repeat, and ordering semantics.

For the conhost Alt+numpad trace, suppression is limited to the exact adjacent sequence: an Alt-bearing VK_PACKET release followed immediately by the same character as a key-down record with no virtual-key code. Mismatches, non-key records, and VT mode changes clear the candidate.

Input batching and ordering

• Available records are fetched with read_console_input() and only the records actually returned are processed. This removes the additional per-record blocking race introduced by repeatedly calling read_single_input_event(). The pre-existing race where another console reader drains the entire queue between readiness and ReadConsoleInputW remains.
• Incomplete ANSI sequences survive input-batch boundaries.
• The parser records the queue position at which an incomplete sequence began. A later flush() or completion inserts the parsed event at that position, preserving its order relative to mouse, focus, resize, and VK-translated events.
• ANSI responses with no Windows consumer, such as cursor-position and keyboard-enhancement reports, are discarded at the Windows source boundary instead of remaining permanently in the shared reader queue.

Esc and Alt sequence boundary

With VT input enabled for bracketed paste, Windows delivers keyboard input as the same byte stream used by Unix terminals. This carries the same inherent ambiguity between a standalone Esc key and the prefix of an Alt or CSI sequence.

The parser uses the currently available input batch: a quickly followed key can combine with Esc as an Alt-prefixed sequence, while a prefix split at a queue-empty boundary can be emitted as a standalone Esc. This PR intentionally keeps that Unix-compatible behavior rather than adding a Windows-only timing threshold.

Once the complete bracketed-paste opener (ESC [ 200 ~) has been recognized, temporary queue drains do not split the paste payload. A split at the opening Esc remains subject to the same boundary.

Diagnostic example

windows-vt-input provides two phases in one run:

1. Raw mode with bracketed paste disabled, for ordinary/F-key/modified-key Press and Release behavior plus Win32 mouse input.
2. Bracketed paste enabled, for multiline Event::Paste, SGR mouse, physical-numpad Alt codes, and the documented Press-only host behavior.

It drains input already queued when the event source is constructed (typically the command-launch Enter release) before Phase 1 statistics begin. Its key accounting distinguishes additional Press records emitted while a key is held from genuinely unmatched identities, while retaining signed detection for release-first, extra-release, modifier-mismatch, and Press-only cases. It also reports narrowly matched Alt-code duplicate candidates, mouse activity without flooding moved events, mode transitions, partial-setup cleanup, and restoration of VT, mouse, window, and extended console-input flags. When VT input was already active before startup, Phase 1 advances on the transition-key press because ConPTY may not provide its release; otherwise it observes the press and release before enabling paste.

Follow-up work

An unterminated bracketed paste can keep the shared parser waiting indefinitely and grow its buffer without a bound. This behavior already exists on Unix and is now reachable on Windows. Choosing a buffer limit and recovery behavior affects both platforms and is deferred to a separate parser-hardening change.

Tested

• cargo fmt --all -- --check
• cargo test --locked --all-targets --all-features -- --test-threads 1: 153 library tests passed, 0 failed, 7 ignored, plus integration targets and 4 diagnostic-accounting example tests
• cargo clippy --locked --all-targets --all-features -- -D warnings
• cargo check --locked --all-targets --no-default-features (one pre-existing file_descriptor::read dead-code warning)
• cargo test --locked --all-targets --no-default-features: 52 passed, 0 failed, 1 ignored, plus integration and example targets
• The lifecycle-scoped implementation at 245d5c6 was manually validated on both Windows Terminal and modern conhost: raw mode preserved VT input off; Phase 1 retained Win32 Press/Release and mouse behavior; Phase 2 delivered multiline Event::Paste, SGR click/drag/wheel input, and one Alt+numpad character Press without a duplicate; cleanup restored the original VT, mouse, window, and extended input bits.
• The latest 94ed715 changes only diagnostic startup-event and held-key accounting based on those traces. Its four example tests cover the balance invariants, and a modern-conhost run confirmed that the startup Enter release is separated, additional held-key Press records do not inflate imbalance, and a Press-only VT identity still remains visible as one mismatch.

[soerennielsen]

I verified this with Forge code -> reedline -> crossterm.
Patched cargo.toml with:

// Patches to enable Windows VT input and bracketed paste support
// See: #870
[patch.crates-io]
crossterm = { git = "https://github.com/eitsupi/crossterm", branch = "feature/windows-vt-input" }

It works, THANKS!

I just hope the maintainers will merge and release a new version soon 🙏

[eitsupi]

@easyinplay Thanks for your detailed review! I'll take a look.

[eitsupi]

@easyinplay Thank you for the detailed review. I have pushed a set of changes addressing the findings at the current HEAD.

Findings 1–4, 6, and 7 are fixed in code. Finding 5 remains an intentional Unix-compatible Esc/Alt ambiguity and is now documented.
The batch-read regression is reduced to the pre-existing master race.
The unterminated bracketed-paste issue is deferred to a separate cross-platform parser follow-up.

I also added a Windows diagnostic example for the mode lifecycle, key-event kinds, and bracketed paste.
Could you please re-review the current HEAD?

[easyinplay]

@eitsupi Thanks for working through all of these. Six findings addressed in one push, finding 5 documented as an intentional trade rather than quietly kept, and a first-party diagnostic example on top. That last one is the part that keeps paying off: it gives the next person a way to check the mode lifecycle and key kinds without building their own probe. This PR has been waiting since February for reasons that have nothing to do with the work in it.

Re-reviewed 11e4912. All six findings you addressed are fixed, and I confirmed each one against the code rather than taking the commit messages for it. Finding 5 reads fine as an intentional, now-documented trade.

Four new things below. One of them will fail CI and is a one-line fix, so it leads. Two of the others are consequences of the finding 1 fix rather than pre-existing problems.

I also have a correction to make to my own finding 1. The behavior I reported was real, but the mechanism I gave for the Alt-code half of it was wrong, and the raw records I captured for this pass show why.

---

cargo clippy -- -D warnings fails on this HEAD

.github/workflows/ci.yml:65, verbatim:

cargo clippy --locked --all-targets --all-features -- -D warnings

error: this loop could be written as a `for` loop
    --> src/event/sys/parse.rs:1833:9
     |
1833 |         while let Some(event) = self.next() {
     |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `for event in self.by_ref()`
     |
     = note: `-D clippy::while-let-on-iterator` implied by `-D warnings`

error: could not compile `crossterm` (lib) due to 1 previous error

That is next_event(), from c12062b8. Clean on 3ca5429 with the same toolchain, so it came with this push. The suggested for event in self.by_ref() builds on Windows and takes the command back to clean.

Leading with it because it is one line and the rest of this can wait for a normal round.

---

Character key-up now works, and two things follow from how it works

Finding 1 is fixed. source/windows.rs:103's else arm now takes every record the VT arm doesn't, and the comment says exactly what it's for. Measured in conhost on this HEAD:

Key Char('a') NONE Press
Key Char('a') NONE Release     +778 us

against 4 Press / 0 Release on the previous HEAD. Arrow and function keys are unchanged, as expected. Alt+numpad produces a character again.

Both of the following are new behavior introduced by that fix, not pre-existing. I don't think either is a reason to hold the PR, but the second one is a visible correctness bug and neither is mentioned in the description.

Raw INPUT_RECORDs below come from a read-only dump taken through the same CONIN$ handle crossterm uses, after crossterm's own enable_raw_mode() has run. Nothing is injected and the dump changes no mode. conhost, mode=0x000003f0, VT_INPUT=true, Windows 11 26200. cks is control_key_state in hex.

2. Press and Release disagree for Ctrl+Shift+letter

Physically typed Ctrl+Shift+A:

down=true  vk=0x11 scan=0x1D u_char=0x0000 cks=0x28   ctrl down
down=true  vk=0x10 scan=0x2A u_char=0x0000 cks=0x38   shift down
down=true  vk=0x00 scan=0x00 u_char=0x0001 cks=0x00   VT byte, cks cleared
down=false vk=0x41 scan=0x1E u_char=0x0001 cks=0x38   key-up, verbatim
down=false vk=0x11 scan=0x1D u_char=0x0000 cks=0x30
down=false vk=0x10 scan=0x2A u_char=0x0000 cks=0x20

delivered as

Key Char('a') CONTROL         Press
Key Char('A') SHIFT | CONTROL Release

The key code and the modifiers both differ between the two halves of one keypress. The cause is structural rather than a slip in the routing: the VT byte is 0x01, which has no way to encode shift, and conhost clears control_key_state on the record it rewrites. The key-up keeps cks=0x38 (NUMLOCK | SHIFT | LEFT_CTRL) because VT translation passes key-ups through untouched. So the VT arm can only ever report CONTROL here while the VK arm reports SHIFT | CONTROL, and this will hold for every Ctrl+Shift+letter.

An application pairing press to release by (code, modifiers) gets a Release matching no Press, and the Press stays outstanding. That is the same stuck-key symptom finding 1 was about, reached a different way, so it seems worth deciding deliberately rather than inheriting.

This also closes the gap I was explicit about last time. I wrote then that I had not watched a physically typed character key-up under VT and that the conjunction was inference. It isn't any more; the trace above is a real keystroke.

3. Alt+numpad delivers the character twice

Alt+numpad 0233:

down=true  vk=0x00 scan=0x00 u_char=0x001B cks=0x00   ESC, VT byte
down=true  vk=0x00 scan=0x00 u_char=0x00E9 cks=0x00   e-acute, VT byte
down=false vk=0xE7 scan=0x00 u_char=0x00E9 cks=0x22   key-up, vk=0xE7 (VK_PACKET)
down=true  vk=0x00 scan=0x00 u_char=0x00E9 cks=0x00   e-acute, VT byte again
down=false vk=0x12 scan=0x38 u_char=0x0000 cks=0x20   alt up, u_char is zero

delivered as

Key Char('é') ALT  Press
Key Char('é') ALT  Release
Key Char('é') NONE Press

Three events, and two of them are Press. A text field receiving this inserts the character twice.

Tracing them: the first two records are ESC followed by the character as VT bytes, which the shared parser combines into Alt+é. The VK_PACKET key-up is what the new else arm now routes to handle_key_event, giving the Release with ALT from cks=0x22 (NUMLOCK | LEFT_ALT_PRESSED). The third record is the character arriving as a VT byte a second time, with no modifiers.

The doc comment on examples/windows-vt-input.rs gives the expected observation as "Alt+numpad 0233 producing a character event", singular, so I don't think this is intended.

Correction to my finding 1

My finding 1 claimed that with VT input active, Alt+numpad 0233 loses the character because is_alt_code at sys/windows/parse.rs:202 becomes unreachable. The loss was real on the previous HEAD, but that rationale was wrong, and the last record above is why: the Alt key-up carries u_char=0x0000 under VT input. is_alt_code requires u_char != 0, so it cannot fire under VT regardless of how records are routed. It isn't being bypassed; it has nothing to act on.

What actually happens is that conhost supplies the character itself as VT bytes. On the previous HEAD those still arrived, but the discarded key-up meant nothing completed, and on this HEAD they arrive alongside a third delivery from the VK arm. I based the earlier claim on injected records rather than a typed Alt sequence, and injection doesn't reproduce what conhost does with a real Alt code.

The consequence for the fix is that the two arms can now both produce a character for the same keystroke, and only the record's own fields distinguish "this character already went out as a VT byte" from "this character exists only on the VK record".

---

4. An unterminated bracketed paste now withholds non-key events too

The ordering fix in a6599bad is right, and test_pending_sequence_at_queue_front_blocks_later_event covers the case it was written for. That case always resolves, because a lone ESC parses to KeyCode::Esc once flush() passes more=false.

The bracketed-paste case can't resolve. parse_csi_bracketed_paste returns Ok(None) regardless of input_available — deliberately, so a queue drain can't split a payload, which I noted approvingly last time — and flush_impl preserves the buffer on Ok(None). With buffer_event_position == 0 the new guard in Iterator::next then holds back everything queued behind it, and nothing can lift it.

Same reproducer against both trees:

let mut p = Parser::default();
p.advance(b"\x1b[200~hello", false);
p.push_event(InternalEvent::Event(Event::Resize(80, 24)));
p.push_event(mouse(7));
p.flush();
let drained: Vec<_> = std::iter::from_fn(|| p.next()).collect();

where mouse(7) is any InternalEvent::Event(Event::Mouse(..)).

	drained
3ca5429	[Resize(80, 24), Mouse(col 7)]
11e4912	[] (buffer.len() == 11, buffer_event_position == 0)

You already have the unterminated paste itself on the deferred list and I'm not asking to move it. The change is in what it costs when it happens: previously the application stopped receiving keys, and now it stops receiving Event::Resize as well, so it can't even redraw at the right size while wedged. If the eventual parser-hardening change caps the buffer, this goes away with it. It seemed worth recording next to that item rather than discovering it later.

---

5. Three symbols are dead on Windows now

A non-test Windows build:

cargo build --locked --lib --no-default-features --features windows,events

warning: function `parse_event_with_raw_mode` is never used
  --> src\event\sys\parse.rs:40:15
warning: methods `advance` and `flush` are never used
    --> src\event\sys\parse.rs:1773:19

Clean on 3ca5429. Checking the three:

• Parser::advance (:1773) has no allow at all. Live on Unix (source/unix/tty.rs:150, source/unix/mio.rs:99), dead on Windows, which uses advance_with_raw_mode.
• Parser::flush (:1853) has #[cfg_attr(unix, allow(dead_code))]. It has no non-test caller on either platform; Windows uses flush_with_raw_mode.
• parse_event_with_raw_mode (:40) has #[cfg_attr(not(windows), allow(dead_code))] but is only called from tests, so Windows warns too.

Nothing fails on this: the warnings do show up in the Windows test job, which builds --all-targets and so compiles the non-test lib, but -D warnings only runs in the ubuntu clippy job. It reaches downstream Windows builds, though, which is why I'd rather mention it than not.

This is partly my fault for how I phrased the cfg_attr note last time. push_event is correct as written.

---

A concern I raised and then withdrew

I thought the new buffer_event_position guard in Iterator::next might stall Unix, since Parser is shared and the Unix sources have no flush() call to lift it. It can't. The Unix sources only ever call advance and next, never push_event, so internal_events is populated exclusively by insert_buffered_event, which runs only when the buffer empties. A non-empty buffer therefore implies nothing was queued after the sequence began, and the guard can only ever refuse a pop that would have returned None. cargo test --locked --lib --all-features on Linux: 146 passed, 0 failed, 7 ignored.

Recording it because I nearly filed it.

---

Method

Two trees at 3ca5429 and 11e4912 from the same clone, same toolchain, same machine, so every before/after above differs only in HEAD. Windows 11 26200, rustc 1.96.0; the Linux runs are the same source in WSL. Three programs behind it: the JSONL event logger from my earlier comment, the read-only raw record dump quoted above, and the two-tree parser reproducer. Happy to post any of them.

Not covered

Same list as last time, none of it exercised on this HEAD: legacy console (ForceV2=0), the disable_mouse_capture() path, large pastes, a CRLF clipboard, WaveTerm. The re-check of bracketed paste across the terminal set is also not redone here, since nothing in this push touches the paste path I measured before. Say which of these would help and I'll run them.

@joshka For what it's worth, this looks close to done to me. Everything I raised in the adversarial pass has been dealt with, and what's left is one CI line plus two behaviors that fall out of the key-up fix and want a deliberate decision rather than more digging. I hope the review was some use in getting it to a state you can take another look at.

[larsch]

I tested this PR through xi-agent on native Windows 11 (10.0.26100) in Windows Terminal. Bracketed paste works correctly, but enabling VT input causes a mouse-input regression: Crossterm no longer emits mouse events, so click/drag selection and wheel scrolling stop working.

The issue appears to be the interaction described in microsoft/terminal#15296. With ENABLE_VIRTUAL_TERMINAL_INPUT active, Windows Terminal/ConPTY delivers mouse input as VT sequences rather than Win32 MOUSE_EVENT records. Crossterm's Windows EnableMouseCapture deliberately takes only the Win32 path (is_ansi_code_supported() == false), so it sets ENABLE_MOUSE_INPUT but does not emit the VT mouse-tracking sequences.

I confirmed a local workaround: keep the existing Win32 EnableMouseCapture, and on Windows additionally write:

CSI ? 1000 h
CSI ? 1002 h
CSI ? 1003 h
CSI ? 1015 h
CSI ? 1006 h

with the corresponding disable sequences in reverse order during cleanup:

CSI ? 1006 l
CSI ? 1015 l
CSI ? 1003 l
CSI ? 1002 l
CSI ? 1000 l

With dual capture enabled, all of the following work in Windows Terminal:

• native Event::Paste for multiline bracketed paste;
• wheel scrolling;
• click/drag mouse selection;
• normal cleanup on exit.

I did not observe duplicated mouse events. The Win32 capture should remain enabled for conhost, which may continue delivering MOUSE_EVENT records.

Reproduction:

1. Enable raw mode.
2. Execute Crossterm EnableMouseCapture and EnableBracketedPaste.
3. Read events using this PR in Windows Terminal.
4. Bracketed paste works, but moving/clicking/scrolling produces no usable Event::Mouse.
5. Explicitly enable the VT mouse modes above; mouse events resume.

The review currently states that preventing ANSI execution for EnableMouseCapture means the terminal will not send SGR mouse reports and thus avoids duplicate delivery. That appears not to account for Windows Terminal switching away from Win32 mouse records when VT input is enabled. Supporting both capture mechanisms on Windows, or otherwise coordinating mouse capture with the new VT-input mode, seems necessary.

[eitsupi]

Thank you both!
I'll take a look into these.

[eitsupi]

@easyinplay @larsch Thank you for your feedback.
Based on it, I’ve revised the Windows input implementation and updated the PR description with the design rationale and validation results.

The latest version has been validated on both Windows Terminal and the modern Windows Console Host, including raw-mode key releases, bracketed paste, mouse input, Alt+numpad input, and console-mode restoration.

Could you please take another look when you have a chance?

[larsch]

Tested the latest PR HEAD (94ed715) through xi-agent on native Windows 11.

I removed xi-agent’s local VT mouse workaround and verified the upstream implementation directly in both Windows Terminal and modern Console Host (conhost). In both environments:

• bracketed paste still produces native paste events correctly;
• mouse-wheel scrolling works;
• click-and-drag text selection works;

This addresses the mouse regression I previously reported. I did not observe missing or duplicated mouse events. Thanks for incorporating the feedback.