Make julialauncher config reads lock-free - #1533
Conversation
Launching julia could previously stall on the configuration lock whenever another process held the exclusive lock (e.g. a `remove`/`gc` deleting a large installation). The launcher only ever used its shared lock to get a consistent snapshot of juliaup.json — and since all config writes now replace the file atomically (temp file + rename), that snapshot consistency no longer needs a lock at all. - Add `load_config_db_lockfree` and switch julialauncher to it, so launching Julia can never block on the configuration lock. - Create the initial juliaup.json atomically (temp file + rename) instead of writing it in place, closing the one window where a lock-free reader could observe an empty/partial config file. - Treat a zero-length config file (leftover from an interrupted initial setup of an older juliaup) like a missing one instead of failing to parse it. - Document the lock-free read path and the "config writes must be atomic" invariant in devdocs/locking_and_update_flows.md. - Add unit tests, including a regression test that a lock-free read succeeds while the exclusive lock is held. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Hooray! You may need to use |
tempfile's persist() renames via MoveFileExW, which fails while any process has the destination file open. With lock-free launcher reads a writer's commit can now overlap a reader holding juliaup.json open, so a commit could transiently fail on Windows. Route all config-file replacements through std::fs::rename instead, which on Windows renames with FILE_RENAME_FLAG_POSIX_SEMANTICS | FILE_RENAME_FLAG_REPLACE_IF_EXISTS (falling back to MoveFileExW only on filesystems without support), so the replacement succeeds while readers still hold the old file open. On Unix this is the same rename(2) as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@topolarity look good now? |
|
Yes, I think that should resolve it! Only other thought is that despite what the OP says w.r.t. #400, it does seem like the new behavior makes that window somewhat worse since you will no longer be blocked from loading an install during the time that its files are being deleted But that may be handled best in a follow-up tackling that specific issue? |
|
Yeah. I do think we want to close #400 but the issue with vscode poking through the juliaup api to binaries directly makes it harder to solve. Need @davidanthoff's input there. We can merge this for now at least. |
3 participants
Claude:
Follow-up to #1517. That PR shrank how long the exclusive configuration lock is held; this one removes the launcher from the lock contention picture entirely: launching
juliacan no longer block on the configuration lock, ever.Why this is safe
The launcher only ever took a shared lock, inside
load_config_db, and released it before resolving the binary path and exec'ing Julia. The only thing that lock bought was a guarantee of not reading a half-writtenjuliaup.json. Since all config writes go through an atomic temp-file + rename (save_config_db), a lock-free reader always sees either the old or the new file in full — no lock needed.One hole had to be closed to make that invariant total: the initial config file was created by opening with
create(true)and writing JSON in place, so a lock-free reader racing first-time setup could observe an empty or partial file. The initial config is now also created via temp-file + rename (create_initial_config_file), so an incompletejuliaup.jsoncan never be observed on disk.Changes
load_config_db_lockfree;julialauncheruses it for both config reads (startup and post-auto-install reload). The lockedload_config_dbis unchanged for alljuliaupcommands.load_mut_config_dbcreates the initial config atomically instead of writing in place (it still holds the exclusive lock while doing so).devdocs/locking_and_update_flows.mddocuments the lock-free read path and adds the design rule that alljuliaup.jsonwrites must be atomic replacements, since that invariant is now load-bearing.What this does and doesn't fix
With this change the launcher cannot be stalled even by the remaining long exclusive-lock holders (
remove/gcdeleting a large install under the lock). It does not change the situation in #400 (an in-use installation being deleted) — but it also doesn't meaningfully widen that race: the lock was already released before exec, so the unprotected window between config read and launch existed before and is unchanged. #400 remains the complementary fix, planned separately along with moving directory deletion out from under the exclusive lock (rename-to-trash).🤖 Generated with Claude Code