Media Engine LLE: add homebrew-focused ME support - #21554
Conversation
|
This is very cool, thanks for doing this! I'm targeting getting this in after 1.20.4, which is kind of a smaller bugfix/UI release. Will take a little while to review as well. There are some CI errors but they should be fairly easily resolved, I think. Let me know if you have any issues. |
|
By the way, it's great that you're relying on the IR JIT infra. The old non-IR JITs are going away in the future, likely for 1.21. |
hrydgard
left a comment
There was a problem hiding this comment.
First batch of comments, don't have time for more today unfortunately :)
| #if PPSSPP_PLATFORM(MAC) && PPSSPP_ARCH(ARM64) | ||
| // On Apple Silicon with MAP_JIT, use pthread_jit_write_protect_np | ||
| // instead of mprotect for W^X toggling. | ||
| if (PlatformIsWXExclusive()) { | ||
| if (memProtFlags & MEM_PROT_WRITE) { | ||
| pthread_jit_write_protect_np(false); // Allow writing | ||
| } else if (memProtFlags & MEM_PROT_EXEC) { | ||
| pthread_jit_write_protect_np(true); // Allow execution | ||
| sys_icache_invalidate((void *)ptr, size); | ||
| } | ||
| return true; | ||
| } | ||
| #endif |
There was a problem hiding this comment.
Why are these changes made as part of this commit?
If these improve some kind of compatibility or avoids some deprecation, we should move them to a separate PR and merge separately first.
| }; | ||
|
|
||
|
|
||
| const HLEModule moduleList[] = |
There was a problem hiding this comment.
Don't use this moduleList thing. Instead, make new Register_...() functions, and importantly, call them at the end of RegisterAllModules.
Otherwise syscall module numbering gets messed up which breaks save state compatibility.
| {0XD13BDE95, &WrapI_V<sceKernelCheckThreadStack>, "sceKernelCheckThreadStack", 'i', "" ,HLE_KERNEL_SYSCALL }, | ||
| {0X1839852A, &WrapU_UUU<sceKernelMemcpy>, "sceKernelMemcpy", 'x', "xxx" ,HLE_KERNEL_SYSCALL }, | ||
| {0XFA835CDE, &WrapI_I<sceKernelGetTlsAddr>, "sceKernelGetTlsAddr", 'i', "i" ,HLE_KERNEL_SYSCALL }, | ||
| {0XF987B1F0, &WrapU_U<sceKernelReleaseIntrHandler>, "sceKernelReleaseIntrHandler", 'x', "x" ,HLE_KERNEL_SYSCALL }, |
There was a problem hiding this comment.
Add new functions at the end of the list.
| using namespace Arm64Gen; | ||
| using namespace Arm64IRJitConstants; | ||
|
|
||
| static bool IsMeSensitiveHwPage(u32 address) { |
There was a problem hiding this comment.
Maybe we can get a comment here about what those ranges are?
And I feel this function might belong elsewhere (ME MemMap.h or something) but can be moved later.
|
|
||
| case IROp::Load8: | ||
| mips->r[inst->dest] = Memory::ReadUnchecked_U8(mips->r[inst->src1] + inst->constant); | ||
| mips->r[inst->dest] = Memory::Read_U8(mips->r[inst->src1] + inst->constant); |
There was a problem hiding this comment.
These are intentionally unchecked for speed. The IR interpreter is what allows us to run on iOS at all, and we really need all the speed we can get.
There was a problem hiding this comment.
After analyzing this, I agree the unchecked path should stay for the common case. The issue is that ME homebrew code (and some SC homebrew code) accesses HW registers (0xBC100044, 0xBC800xxx, etc.) directly via loads/stores that go through the IR interpreter. In JIT mode, the fault handler transparently handles these by catching the page fault on unmapped HW addresses. The IR interpreter doesn't have an equivalent safety net — ReadUnchecked_U32(0xBC100044) would access base + 0x3C100044 which is unmapped.
My plan is to revert the global Read_U32 change and instead:
- Add
IROp::LoadHw32andIROp::StoreHw32— checked variants that go throughRead_U32/Write_U32for proper HW register dispatch. - Generalize
ApplyMeMemoryValidationso it also runs for main CPU IR blocks when ME is enabled. The pass already identifies constant HW addresses viaIsMeSensitiveHwPage()— it would rewriteLoad32 → LoadHw32andStore32 → StoreHw32for those.
This keeps ReadUnchecked for all normal RAM/VRAM accesses (zero overhead), and only HW register accesses go through the checked path. The limitation is that non-constant (dynamic) HW addresses won't be caught by the pass, but those are extremely rare in practice — PSP homebrews always use lui + offset patterns for HW registers.
What do you think?
4fa11c9 to
c17fd94
Compare
|
Your new first commit says "This is needed for proper JIT operation on recent macOS versions.". But it's not, it's working just fine without that change. Can certainly merge it anyway, but seems weird to me to make that assertion. Anyhow, I'll get back to reviewing tomorrow. |
Add bare-metal Media Engine (ME) LLE support for PSP homebrew. The ME is a secondary MIPS core used for parallel computation. Features: - ME boot/enable via 0xBC10004C register write - SC<->ME soft interrupts via 0xBC100044 - HW mutex (0xBC100048) for synchronization - ME exception handler + ERET instruction support - ME IR compilation with ApplyMeMemoryValidation pass - ME native ARM64 JIT with real dispatcher - DMACplus (3 channels, LLI chaining) - VME CSC (YCbCr->RGB) hardware emulation - sceSysreg_driver and sceMeCore_driver HLE - Per-core cycle accounting and clock scaling - ImGui debugger windows for ME state
Add MIPSDebugInterface for the Media Engine (debugMe) alongside the existing debugr4k for the main CPU. The CPU menu now has a 'Media Engine' section with separate windows: - ME Debugger (disassembly view) - ME GPR regs - ME FPR regs These can be opened alongside the main CPU windows simultaneously. DrawGPRs/DrawFPRs/DrawVFPU accept optional title/openFlag parameters. ImDisasmWindow::Draw accepts optional title/openFlag for reuse.
J_CC(CC_Z, bail, true) passed a FixupBranch as the second arg, but the x64 emitter only accepts (CCFlags, bool) or (CCFlags, const u8*, bool). Use a second FixupBranch bail2 and resolve both at the bail target.
Add a 'Media Engine Core' dropdown in Settings > Developer Tools with options: Disabled / Interpreter / IR Interpreter / Native JIT (recommended). The Dynarec/JIT option (value 1) is hidden since it's not a valid ME backend. Native JIT is hidden on unsupported architectures. Default changed from IR Interpreter to Native JIT.
Move sceKernelReleaseIntrHandler, sceKernelRegisterIntrHandler, and sceKernelEnableIntr entries to the end of the InterruptManagerForKernel table to preserve existing syscall numbering and save state compatibility.
Add detailed comments explaining the ME-sensitive HW register pages: System Controller, ME interrupt regs, ME/SC communication, VME (CSC), and DMACplus. Reference from x64 copy to ARM64 for full documentation.
c17fd94 to
53dc7bd
Compare
You were right, those MAP_JIT / pthread_jit_write_protect_np changes don't belong in this PR. After testing, I confirmed that all my tests pass across all three backends (interpreter, IR, native JIT) without them. The changes were originally introduced by AI that incorrectly diagnosed a W^X issue while fixing a different problem. I've removed them from the PR entirely. Apologies for the noise. |
Move sceSysEventForKernel, sceSysreg_driver, and sceMeCore_driver out of moduleList[] and into proper Register_*() functions called at the end of RegisterAllModules(), following PPSSPP convention of one file per module: - Core/HLE/sceSysEvent.cpp/.h (sceSysEventForKernel) - Core/HLE/sceSysreg.cpp/.h (sceSysreg_driver) - Core/HLE/sceMeCore.cpp/.h (sceMeCore_driver) This preserves syscall numbering for savestate compatibility.
Move the ME hardware page check function from three separate static copies (ARM64 backend, x64 backend, IRPassSimplify) into a single implementation in MemMapFunctions.cpp with declaration in MemMap.h. All three consumers (NeedsGenericMeHwAccess in both JIT backends and ApplyMeMemoryValidation in the IR pass) now use Memory::IsMeSensitiveHwPage().
53dc7bd to
90955c7
Compare
|
Sorry, I ran out of time last week and this week is pretty busy too, I'll get some reviewing done though. It's a complex PR. |
| // Provide a minimal PspSysEventHandler for ME startup code. | ||
| // It uses a fixed kernel address and the name "SceMeRpc". | ||
| static u32 sceKernelReferSysEventHandler() { | ||
| const u32 handlerAddr = 0x88000100; |
There was a problem hiding this comment.
I think we should coordinate the choice of this address with the other things writing handlers at the start of kernel memory. This will lead to trouble eventually otherwise.
| static u32 sceSysregMeResetEnable371() { return 0; } | ||
| static u32 sceSysregMeBusClockEnable371() { return 0; } | ||
| static u32 sceSysregMeResetDisable371() { Core_EnableME(); return 0; } | ||
| static u32 sceSysregVmeResetEnable371() { return 0; } | ||
| static u32 sceSysregAvcResetEnable371() { return 0; } | ||
| static u32 sceSysregMeBusClockDisable371() { return 0; } |
There was a problem hiding this comment.
Let's have all of these do this:
return hleLogDebug(Log::ME, 0);
so we can trace them.
Same with sceKernelRegisterSysEventHandler and friends.
| loadStaticRegisters_ = nullptr; | ||
| } | ||
|
|
||
| restoreRoundingMode_ = AlignCode16(); |
There was a problem hiding this comment.
There's a lot of stuff duplicated here from the main dispatcher generator. Can we extract parts into utility functions, or maybe even better, have one single shared dispatcher generator for each architecture, taking an enum about which type of dispatcher to generate? enum class CoreType { Main, ME }; or something like that.
hrydgard
left a comment
There was a problem hiding this comment.
Have you tested this with save states? Two scenarios:
- Save and load state successfully within a homebrew app that uses this
- Save state before this change in any game, load the same state after this change.
| struct FastCacheEntry { | ||
| u32 pc; | ||
| u32 pad; // Align nativeEntry to 8 bytes within 16-byte entry | ||
| const u8 *nativeEntry; |
There was a problem hiding this comment.
Hm, I wonder if this should be an offset instead of a direct pointer, would make the structure smaller.
Actually, it seems to me that this type of "fast cache" could lead to a lot of thrashing if two commonly used functions share the same 4096-alignment, so to speak.
Might as well make the cache the size of the ME RAM, avoiding collisions entirely? It's not that big. Such a cache could also get rid of the "pc" value in each entry since there are no ambiguities.
| // ERET (0x42000018) and HALT (0x70000000) change control flow or stop | ||
| // execution entirely. They must terminate the IR block so the caller | ||
| // can react (e.g., redirect PC or stop the ME). | ||
| if (op.encoding == 0x42000018 || op.encoding == 0x70000000) { |
There was a problem hiding this comment.
Instead of directly handling them here, can't it be done in the opcode handlers? Would be nicer than bypassing the dispatch like this.
Or is there a reason this can't be done?
|
|
||
| case IROp::Load8: | ||
| mips->r[inst->dest] = Memory::ReadUnchecked_U8(mips->r[inst->src1] + inst->constant); | ||
| mips->r[inst->dest] = Memory::Read_U8(mips->r[inst->src1] + inst->constant); |
| compilerEnabled_ = false; | ||
| #endif | ||
| while (mips->downcount >= 0) { | ||
| if (!Memory::IsValid4AlignedAddress(mips->pc)) { |
|
|
||
| // Media Engine state and scheduling. | ||
|
|
||
| static bool meEnabled = false; |
There was a problem hiding this comment.
I want to move more towards putting major chunks of state in structs. Even if it's just g_MeState or something. A lot of individual values will be tricky to "lift" to heap allocations later (I have long term plans of making it possible to run two PSP instances within one process).
| #if PPSSPP_ARCH(ARM64) | ||
| uint32_t val = (uint32_t)context->CTX_REG(info.Rt); | ||
| if (info.size == 2) { | ||
| Memory::Write_U32(val, guestAddress); |
|
Are you interested in addressing my comments so we can get this in? Otherwise I might take this over some time in the future, but probably not soon. |
|
I will but maybe not soon but I'll try |
|
Perhaps the ME execution should be moved out from |
This adds homebrew-focused Media Engine LLE support to PPSSPP.
The goal of this branch is to make ME bare-metal homebrews usable without disturbing the main CPU JIT or the existing execution model. The implementation keeps the ME isolated in its own state and scheduler path, adds the missing ME-side hardware behavior needed by real homebrew, and wires it into interpreter, IR, and native execution paths.
Main pieces included in this PR:
A few implementation details were important for keeping this upstreamable:
About autotests / CI:
PPSSPP CI currently runs
python test.py -g --graphics=software, which only covers thetests_goodlist intest.py. The existingpspautotests/tests/me/metest is a legacy interactive sample rather than a self-terminating stdout autotest, and it is also explicitly ignored today. Because of that, this PR does not add ME coverage to the standard pspautotests CI path yet. I plan to handle that as a separate pspautotests/update-submodule follow-up once there is a proper ME autotest suitable for the headless harness.Testing performed on this branch:
me-core=0,me-core=2, andme-core=3:me-soft-interruptme-minimal-handlerme-spinlockpsp-me-bench-customme-dmacplus-drawing-transferme-feed-graphics-engineAll of the above passed on the ARM64 host used for development.
Note on x64:
The initial x64 native ME backend is included in this branch and compiles cleanly, but I have not yet runtime-validated it on a real x64 host. The ARM64 path and the interpreter/IR paths have been validated with the test matrix above.