Skip to content

machine: snapshot master FPU at prepare time for cross-process forks #85

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fwsGonzo merged 2 commits into from
Jul 6, 2026
Merged

machine: snapshot master FPU at prepare time for cross-process forks #85

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
973ac11
machine: snapshot master FPU at prepare time for cross-process forks
perbu Jul 5, 2026
bb38a40
arm64: snapshot FPU state for prepared forks
perbu Jul 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions lib/tinykvm/machine.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,10 +168,14 @@ Machine::Machine(const Machine& other, const MachineOptions& options)
m_fds->reset_to(*other.m_fds);
}

/* Copy register state from the master machine */
/* Copy register state from the master machine. FPU state comes from the
master's prepare-time snapshot rather than a live KVM_GET_FPU: when this
fork is constructed from a master inherited over fork(), the master's
vCPU fd is not ioctl-able from this process and KVM_GET_FPU fails with
-EIO. See Machine::prepared_fpu_registers(). */
auto& m_regs = other.registers();
this->set_registers(m_regs);
this->set_fpu_registers(other.fpu_registers());
this->set_fpu_registers(other.prepared_fpu_registers());
}

__attribute__ ((cold))
Expand Down Expand Up @@ -288,10 +292,11 @@ bool Machine::reset_to(const Machine& other, const MachineOptions& options)
#endif

if (options.reset_copy_all_registers) {
/* Copy register state from the master machine */
/* Copy register state from the master machine (FPU from the master's
prepare-time snapshot; see the fork constructor). */
auto& m_regs = other.registers();
this->set_registers(m_regs);
this->set_fpu_registers(other.fpu_registers());
this->set_fpu_registers(other.prepared_fpu_registers());
}
if (options.reset_enter_usermode) {
/* Enforce usermode (default). This will crash guests
Expand Down
14 changes: 14 additions & 0 deletions lib/tinykvm/machine.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,18 @@ struct Machine
void set_registers(const tinykvm_regs&);
tinykvm_fpuregs fpu_registers() const;
void set_fpu_registers(const tinykvm_fpuregs&);
/* FPU state snapshotted by prepare_copy_on_write(). Fork construction and
fork reset read this instead of issuing KVM_GET_FPU against the master's
vCPU, so both remain usable from a process that cannot ioctl the master's
fds -- e.g. a child that inherited the master over fork() and is building
its own VM from the master's copy-on-write memory (KVM vCPU ioctls require
the caller's mm to be the VM-creating mm). Requires a prepared master: the
snapshot is only valid because the master is frozen after prepare and is
never run again, so it equals a live KVM_GET_FPU at fork time. */
const tinykvm_fpuregs& prepared_fpu_registers() const noexcept {
assert(m_prepped && "prepared_fpu_registers() requires prepare_copy_on_write()");
return m_prepared_fpu_regs;
}
const kvm_sregs& get_special_registers() const;
void set_special_registers(const kvm_sregs&);
std::pair<__u64, __u64> get_fsgs() const;
Expand Down Expand Up @@ -391,6 +403,8 @@ struct Machine
bool m_verbose_mmap_syscalls = false;
bool m_verbose_thread_syscalls = false;
void* m_userdata = nullptr;
/* See prepared_fpu_registers(). */
tinykvm_fpuregs m_prepared_fpu_regs {};

std::string_view m_binary;

Expand Down
17 changes: 17 additions & 0 deletions lib/tinykvm/vcpu.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,16 @@
extern "C" int close(int);
extern "C" void tinykvm_timer_signal_handler(int);
#define TINYKVM_USE_SYNCED_SREGS 1
/* Constructing or resetting a fork from a master inherited over fork() requires
that reading the master's registers never ioctls the master's vCPU fd, which
the child process cannot use. With synced sregs, get_special_registers()
reads the mmap'd kvm_run page instead of KVM_GET_SREGS; FPU is served from
the prepare-time snapshot (Machine::prepared_fpu_registers()). If this is
ever disabled, get_special_registers() reverts to KVM_GET_SREGS and
cross-process fork construction breaks with -EIO. */
#if !TINYKVM_USE_SYNCED_SREGS
#error "TINYKVM_USE_SYNCED_SREGS must be enabled for cross-process fork construction"
#endif

@fwsGonzo fwsGonzo Jul 6, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this break ARM? I thought I saw that ARM didn't support this?

@perbu perbu Jul 6, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. this is true.

checked on native ARM64. The existing ARM test suite passed, but I found one ARM-specific gap: PR 85 only snapshotted prepared FPU state in the non-ARM vcpu.cpp; ARM’s prepare_copy_on_write() was still leaving the prepared FP/SIMD snapshot zeroed. I pushed bb38a40, which snapshots it on ARM too and adds a regression test for fork construction and reset_to(). /tests/run_unit_tests.sh passes on ARM64.

sloppy.


#ifndef SYS_gettid
#error "SYS_gettid unavailable on this system"
Expand Down Expand Up @@ -421,6 +431,13 @@ void Machine::prepare_copy_on_write(size_t max_work_mem,
{
this->m_prepped = true;

/* Snapshot FPU state to userspace while this master's vCPU fd is still
ours to ioctl. Fork construction and fork reset consume the snapshot via
prepared_fpu_registers(), which keeps both usable from a process that
inherited this master over fork(). The master is frozen hereafter, so the
snapshot stays equal to a live KVM_GET_FPU. */
this->m_prepared_fpu_regs = this->fpu_registers();

/* Make each writable page read-only, causing page fault.
any page after the @shared_memory_boundary is untouched,
effectively turning it into a shared memory area for all. */
Expand Down
Loading