Skip to content

Reject n_jobs=0 in MultiprocessingBackend before the nesting checks - #1838

Open
ckarnell wants to merge 1 commit into
joblib:mainfrom
ckarnell:fix-njobs-zero-daemon
Open

Reject n_jobs=0 in MultiprocessingBackend before the nesting checks#1838
ckarnell wants to merge 1 commit into
joblib:mainfrom
ckarnell:fix-njobs-zero-daemon

Conversation

@ckarnell

Copy link
Copy Markdown

SequentialBackend, PoolManagerMixin and LokyBackend all reject n_jobs=0 as their first act. MultiprocessingBackend gets there last, after the daemon and nesting checks, so in a daemonic process the invalid value is swallowed and reported as a nesting problem instead.

>>> # inside a daemonic process, e.g. a worker
>>> with parallel_config(backend='multiprocessing'):
...     joblib.effective_n_jobs(0)
UserWarning: Multiprocessing-backed parallel loops cannot be nested, setting n_jobs=1
1

The other three raise there. ValueError: n_jobs == 0 in Parallel has no meaning. effective_n_jobs is public and in __all__, so this is the documented error going missing on one backend.

test_invalid_njobs already asserts the behaviour for every backend and passes today, because it goes through Parallel. When the backend returns 1, configure raises FallbackToBackend(SequentialBackend(...)), and the sequential backend then raises for its own reasons. The error a caller sees is right, and it's arriving from a different backend than the one they asked for.

Three lines. Nothing moves. The check is added at the top instead of reordering what's there, so the daemon and nesting branches behave exactly as before for every valid n_jobs.

Test is test_invalid_njobs_in_daemon_process, parametrized over the four backends and going through the public effective_n_jobs instead of Parallel, since Parallel recovers and hides it. Reverting _parallel_backends.py fails it on multiprocessing alone with DID NOT RAISE, and the other three pass either way. test_parallel.py is 575 passed, 24 skipped, 3 xpassed, against 574 and the one failure on revert.

@ckarnell
ckarnell force-pushed the fix-njobs-zero-daemon branch from 134ea20 to 5b4f57b Compare August 19, 2026 13:29
@codecov

codecov Bot commented Aug 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.07%. Comparing base (804f472) to head (47c17de).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1838   +/-   ##
=======================================
  Coverage   94.06%   94.07%           
=======================================
  Files          46       46           
  Lines        8070     8082   +12     
=======================================
+ Hits         7591     7603   +12     
  Misses        479      479           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

MultiprocessingBackend validates n_jobs last, after the daemon and nesting
checks, so in a daemonic process effective_n_jobs(0) returns 1 with a nesting
warning instead of raising, unlike the other three backends. Add the n_jobs==0
check at the top (additive, reorders nothing). Adds
test_invalid_njobs_in_daemon_process, going through the public effective_n_jobs
because Parallel recovers and hides the defect.
@ckarnell
ckarnell force-pushed the fix-njobs-zero-daemon branch from 5b4f57b to 47c17de Compare August 19, 2026 14:31
@Nanored4498

Copy link
Copy Markdown
Contributor

LGTM. As some users might currently use the effective_n_jobs(0) with 'multiprocessing', I prefer waiting for another reviewer opinion before merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants