Repository navigation
Support multi-threading in hypervisors - #16
Conversation
Guest threads now run on several host threads at once on a hypervisor backend. A kernel lock covers every host thread that touches emulator, OS layer or backend state, and the backend releases it only for the time a processor spends in the guest. Scheduler state that used to sit on the emulator moved to a per host thread worker, so each worker holds its own current thread, context flag and register scratch. A dispatched thread is hidden from every scan and given back on every route out of a slice. Idle workers wait on the kernel lock and a sweep thread pulses one at the idle cadence, because a timed wake has no producer. `--no-smp` runs the single thread scheduler unchanged and `--cores=<n>` sets how many threads run together, both forwarded to a guest process the emulator launches. The backend interface gains ProcessorLimit, UseRunLock, StopThread and StopAllProcessors. The WHP and KVM bindings give each guest thread its own virtual processor, so a thread switch moves no register state, and a thread that cannot get one waits. Per processor state moved out of the binding and into the processor object, including the register cache, the stepped completion, the stop request and the GDT, TSS and interrupt stack. A slice deadline is named by generation so a late cancel cannot end the slice that replaced it. A rebuild keeps a mapping whose pages still map as installed instead of replacing it, because a processor delivering an exception through the gap cannot recover. KVM offsets the counter of a later virtual processor onto the first one, and RDTSC is answered from the real counter when the backend has one. NtGetContextThread, NtSetContextThread and NtTerminateThread park the target thread before they read or change it, and the context calls answer STATUS_UNSUCCESSFUL when it never reaches a safe point. NtSuspendThread and NtQueueApcThread end the target's slice instead of the calling one. A thread that has ended releases its TEB, its initial stack and its initial context, and its thread object stays only while a handle refers to it. NtLockFile retries a conflicting lock through a slice rather than sleeping on the host thread. The 64 bit fields of KUSER_SHARED_DATA are written with the high half first, so a reader that retries while High1Time differs from High2Time reads a whole value, and the hypervisor shared page stores its offset before the sequence that publishes it. The module list poll reads the PEB_LDR_DATA list head instead of watching writes to the page it shares with the loader lock. On Linux the syscall helper takes the current thread id from the emulator and keeps CpuidEnabled on the thread rather than on the helper.
|
@claude review |
|
Claude finished @AdvDebug's task in 7m 55s —— View job Review: Support multi-threading in hypervisorsTodo list
This is a large, well-thought-out change (the kernel-lock discipline, per-processor state split, and generation-tagged slice deadlines are all sound design choices), but I found a few concrete correctness/concurrency issues that are worth fixing before merge, plus a couple of lower-confidence items worth double-checking. High confidence
Worth double-checking (lower confidence)
Minor
What looks solidPer-processor register cache/GDT/TSS/interrupt-stack isolation, the deadline-generation pairing in |
|
For the rest of the issues they are tested locally and didn't trigger anything, so that should be okay. |
1 participant
Guest threads now run on several host threads at once on a hypervisor backend. A kernel lock covers every host thread that touches emulator, OS layer or backend state, and the backend releases it only for the time a processor spends in the guest. Scheduler state that used to sit on the emulator moved to a per host thread worker, so each worker holds its own current thread, context flag and register scratch. A dispatched thread is hidden from every scan and given back on every route out of a slice. Idle workers wait on the kernel lock and a sweep thread pulses one at the idle cadence, because a timed wake has no producer.
--no-smpruns the single thread scheduler unchanged and--cores=<n>sets how many threads run together, both forwarded to a guest process the emulator launches.The backend interface gains ProcessorLimit, UseRunLock, StopThread and StopAllProcessors. The WHP and KVM bindings give each guest thread its own virtual processor, so a thread switch moves no register state, and a thread that cannot get one waits. Per processor state moved out of the binding and into the processor object, including the register cache, the stepped completion, the stop request and the GDT, TSS and interrupt stack. A slice deadline is named by generation so a late cancel cannot end the slice that replaced it. A rebuild keeps a mapping whose pages still map as installed instead of replacing it, because a processor delivering an exception through the gap cannot recover. KVM offsets the counter of a later virtual processor onto the first one, and RDTSC is answered from the real counter when the backend has one.
NtGetContextThread, NtSetContextThread and NtTerminateThread park the target thread before they read or change it, and the context calls answer STATUS_UNSUCCESSFUL when it never reaches a safe point. NtSuspendThread and NtQueueApcThread end the target's slice instead of the calling one. A thread that has ended releases its TEB, its initial stack and its initial context, and its thread object stays only while a handle refers to it. NtLockFile retries a conflicting lock through a slice rather than sleeping on the host thread.
The 64 bit fields of KUSER_SHARED_DATA are written with the high half first, so a reader that retries while High1Time differs from High2Time reads a whole value, and the hypervisor shared page stores its offset before the sequence that publishes it. The module list poll reads the PEB_LDR_DATA list head instead of watching writes to the page it shares with the loader lock.
On Linux the syscall helper takes the current thread id from the emulator and keeps CpuidEnabled on the thread rather than on the helper.