fix(docker): don't abort Docker startup when .env is missing

[YZJF]

Related to #2416 — but see the review discussion: the exact error in that issue is a
different failure mode and I could not reproduce it, so this PR no longer claims to close it.

Why

.env and frontend/.env are gitignored, so a fresh clone has neither. On main, both
documented entry points then abort on Windows with a confusing "cannot find the file
specified" from Compose, before any service starts:

from a fresh clone (no .env)	main	this branch
docker compose -f docker/docker-compose-dev.yaml up --build (repo root)	env file ... not found	ok
make docker-start	env file ... not found	ok

Making the env_file entries optional uses Compose's long-form path/required
syntax, which needs Compose >= 2.24. This PR declares that floor, fails early from
the wrapper with an actionable message, and documents DEER_FLOW_ROOT for direct callers.

What changed

• make docker-start copies .env / frontend/.env from examples when missing.
• Dev env_file entries are required: false (Compose >= 2.24).
• Wrapper checks Compose version (probes docker compose, then docker-compose) and
  exits with an upgrade hint when too old.
• logs / stop / restart only run a read-only preflight (compose file + version +
  DEER_FLOW_ROOT); they never create env files.
• Direct Compose docs now set DEER_FLOW_ROOT (bash + PowerShell). No ${DEER_FLOW_ROOT:-$PWD}
  fallback: PowerShell/cmd do not export PWD, so a default would silently break on Windows.
• Compose is invoked after cd docker/ with relative filenames.
• Prerequisites updated in CONTRIBUTING.md, README.md, and README_zh.md.

Surface area

• Frontend UI — page / component / setting / interaction under frontend/
• Backend API — endpoint / SSE event / request-response shape under backend/app
• Agents / LangGraph — agent node, graph wiring, langgraph.json, or prompt change
• Sandbox — docker/ or sandboxed execution
• Skills — change under skills/
• Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json
• Default behavior change — missing .env is tolerated by the dev compose file;
  Compose < 2.24 is rejected by make docker-start with an upgrade message;
  logs/stop/restart no longer create env files
• Docs / tests / CI only — no runtime behavior change

Bug fix verification

• Fresh-clone state (.env deleted): config / up --build --dry-run succeed here; fail on main.
• Compose v2.23.3 against this file: services.gateway.env_file.0 must be a string (documented in the compose header).
• Wrapper with stubbed 2.23.3: actionable upgrade message, exit 1.
• Documented direct command with DEER_FLOW_ROOT set: 0 warnings; host paths render to the checkout.
• logs/stop/restart on a clean tree: no .env created; still succeed without .env.example.
• Unit tests: version boundaries, hyphenated-binary fallback, env-file create/preserve, read-only commands.

Open points for maintainers

• Prod compose (docker/docker-compose.yaml) still uses short-form env_file — left out of scope on purpose.
• Direct callers cannot get the wrapper's version check (YAML cannot run code); they rely on docs + the compose-file header quoting the old-client error.
• Version undetectable (neither binary answers): warn and proceed, so environments without Docker are not hard-blocked.

AI assistance

Tool(s) used: Cursor

How you used it: AI helped reproduce the Windows Docker / Compose failures, draft the
script and compose changes, run red/green verification (including a real Compose v2.23.3
client), and write this PR text. I reviewed every line.

• I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

[CLAassistant]

All committers have signed the CLA.

[willem-bd]

Two blocking concerns:

1. [P1] Existing backend tests fail. The new prepare_compose_env preflight exits before the mocked Compose command in test_compose_commands_set_deer_flow_root_before_compose for both logs --gateway and restart. I reproduced this locally as 2 failures / 6 passes, and backend-unit-tests is now failing in CI. Please update the fixture to provide the compose/example files, or separate the DEER_FLOW_ROOT setup from environment preparation.

2. [P2] The reported reproduction is unchanged. Issue #2416 invokes docker-compose -f docker/docker-compose-dev.yaml up --build directly, which bypasses scripts/docker.sh; therefore neither the new env-file creation nor the wrapper cd behavior applies. The base compose filename was already relative and normal wrapper invocations already ran after cd "$DOCKER_DIR"; only the AIO overlay path changed. Please either address the direct invocation or explicitly document that the supported resolution is make docker-start and adjust the issue claim accordingly.

[YZJF]

Thanks for the careful review — both concerns are addressed in 5b4cbc7.

[P1] Backend tests

Reproduced on the previous head: test_docker_sandbox_mode_detection.py +
test_compose_default_bind_host.py gave 2 failed / 11 passed, matching your CI run.

I took the fixture route. _seed_compose_env() now writes docker-compose-dev.yaml,
.env.example and frontend/.env.example into the temp root before the script is sourced,
so prepare_compose_env reaches the mocked COMPOSE_CMD. The same selection is now
30 passed / 0 failed.

I kept the preflight where it is rather than splitting it from the DEER_FLOW_ROOT setup,
since it has to run for every Compose subcommand and not just start. Happy to split it
instead if you'd prefer the fixture stay untouched.

[P2] Direct invocation, and the issue claim

You're right, and it goes further than that: I can't reproduce #2416 at all. The reported
message

open C:\Users\Thin\deer-flow\docker\docker-compose-dev.yaml: Le fichier spécifique est introuvable.

is the "compose file itself is absent" error — docker compose -f docker/does-not-exist.yaml config
produces exactly that shape. A missing env_file produces a different one:

env file C:\...\deer-flow\.env not found: CreateFile ...: The system cannot find the file specified.

So I've dropped the Fixes #2416 claim and retitled the PR. That reporter's case needs more
information from them.

What this PR does fix is adjacent and reproducible every time. .env is gitignored, so a
fresh clone has none, and on main both entry points abort on Windows before anything
useful happens:

from a fresh clone (no .env)	main	this branch
docker compose -f docker/docker-compose-dev.yaml up --build (repo root)	env file ... not found	ok
make docker-start	env file ... not found	ok

Marking the dev env_file entries required: false fixes the direct invocation without a
prior .env; scripts/docker.sh still creates the files, so the wrapper path is unchanged.
Verified with config and up --build --dry-run, via both docker compose and
docker-compose, with and without .env present — 12 DEER_FLOW_* keys still render when
it exists.

Two things I'd like your call on:

• The long-form env_file syntax requires Compose >= 2.24. The repo doesn't document a
  version floor today and uses no other modern Compose syntax, so this would introduce one.
• docker/docker-compose.yaml has the same exposure for make up. I left it out to keep
  this PR scoped — say the word and I'll include it.

If you'd rather not take the compose change at all, I'm glad to reduce this to the fixture
fix plus a docs note that make docker-start is the supported entry point.

[willem-bd]

The previous test failures and the missing-env direct-invocation case are improved, and the targeted suites plus current CI are green. Three compatibility issues remain: the undocumented Compose 2.24 minimum, the direct command rendering incorrect host paths without DEER_FLOW_ROOT, and non-start commands creating configuration files.

[willem-bd]

[P1] Please declare or avoid the new Compose 2.24 minimum. The long-form env_file required field is not understood by older Compose v2 clients, so configurations that previously ran make docker-start now fail while parsing this file, before the wrapper-created env files can help. The prerequisites currently list Docker Desktop or Engine without a Compose version floor, and the wrapper has no version check. Either preserve the previous syntax/support or document Compose >= 2.24 and fail early with an actionable version check, including for direct callers.

[willem-bd]

[P2] This direct command also needs to set DEER_FLOW_ROOT to the absolute checkout path. Running the documented command with it unset emits warnings and renders DEER_FLOW_HOST_BASE_DIR as /backend/.deer-flow and THREADS_HOST_PATH as /backend/.deer-flow/threads. Those are host-side paths in AIO/provisioner mode, so the command gets past the missing env file but remains misconfigured. Please document the platform-appropriate setting or make Compose derive it reliably.

[willem-bd]

[P2] Please keep logs, stop, and restart read-only with respect to configuration. prepare_compose_env calls ensure_from_example for both env files, so these commands now mutate a fresh checkout and can even refuse to stop running containers if an example file is unavailable. Since the Compose entries are optional after this change, split the shared preflight into compose-file validation plus DEER_FLOW_ROOT setup, and run env-file creation only from start.

[YZJF]

Thanks for the second review — all three compatibility points are addressed in 0bedc01 (on top of 5b4cbc7).

Summary of this round

Concern	What we did
[P1] Undocumented Compose 2.24 minimum	Declared the floor + early actionable check in the wrapper
[P2] Direct command without DEER_FLOW_ROOT misconfigures host paths	Documented platform-specific setting; no silent $PWD fallback
[P2] logs/stop/restart created env files	Split preflight; env-file creation only from start

---

[P1] Compose >= 2.24

Chose declare + fail early, not “preserve the old syntax” — keeping the short-form env_file would bring back the Windows missing-.env abort this PR is fixing, and YAML cannot branch on client version.

Where it is declared

• CONTRIBUTING.md prerequisites
• README.md / README_zh.md Docker sections
• docker/docker-compose-dev.yaml header (first block), including the exact old-client error:
  validating docker-compose-dev.yaml: services.gateway.env_file.0 must be a string
  (reproduced with docker/compose-bin:v2.23.3 against this file; the same client parses upstream main fine)

Wrapper check (require_compose_version in scripts/docker.sh)

• Runs inside compose_preflight, which start calls before creating any config files
• Probes docker compose version --short, then falls back to docker-compose version --short (hyphenated binary)
• Too old → exit 1 with upgrade URL
• Neither binary answers → warn and proceed (so no-Docker CI / shells are not hard-blocked)

Direct callers

Compose YAML cannot run a version check. Direct docker compose / docker-compose users get the raw parser error above; the file header and CONTRIBUTING call that out explicitly. Happy to tighten further if you want a different policy (e.g. hard-fail when version is undetectable).

---

[P2] DEER_FLOW_ROOT for the documented direct command

Chose document the platform-appropriate setting over “make Compose derive it.”

We tried the obvious ${DEER_FLOW_ROOT:-$PWD} default. $PWD is an environment variable in POSIX shells, but not in PowerShell or cmd (verified: after clearing a inherited PWD, PowerShell has no Env:PWD). A default that only works on macOS/Linux would silently reintroduce the wrong host paths on Windows — the platform this variable exists to get right. Wrong-but-plausible paths (/backend/.deer-flow) mount empty dirs instead of failing, so we require an explicit value for direct callers; make docker-start still sets it automatically.

Docs now

# macOS / Linux
DEER_FLOW_ROOT="$PWD" docker compose -f docker/docker-compose-dev.yaml up --build

# Windows PowerShell
$env:DEER_FLOW_ROOT = (Get-Location).Path
docker compose -f docker/docker-compose-dev.yaml up --build

Verification of that documented command

	DEER_FLOW_ROOT warnings	rendered DEER_FLOW_HOST_BASE_DIR
unset	3	/backend/.deer-flow
set as documented	0	<checkout>/backend/.deer-flow

The “why no $PWD fallback” rationale is also in a comment above THREADS_HOST_PATH in the compose file so a later edit does not reintroduce it by accident.

---

[P2] Keep logs / stop / restart read-only

Split the old prepare_compose_env as requested:

• compose_preflight → require_compose_file + require_compose_version + ensure_deer_flow_root (read-only; used by start/logs/stop/restart)
• ensure_env_files → only called from start

Live check on a fresh tree (examples present, no .env yet)

command	creates .env?
logs --gateway / stop / restart	no
start (ensure_env_files)	yes

Also verified: with no .env.example at all, logs/stop/restart still complete (they no longer inherit ensure_from_example's exit 1).

---

Tests added / extended

• Version floor: 2.23.3 / 2.5.0 reject; 2.24.0 / desktop / 3.x pass; empty → warn+proceed
• Hyphenated fallback: plugin missing + docker-compose 2.24.0 passes; 2.23.3 rejects
• ensure_env_files creates from examples; does not overwrite existing .env
• logs/stop/restart set DEER_FLOW_ROOT, create no env files, run without examples
• Existing test_dev_compose_env_files_are_optional still locks the long-form syntax

Local targeted run: 47 passed (plus one unrelated local AV false-positive deleting skillscan/orchestrator.py mid-import — not present in CI).

---

Intentionally out of scope / open calls

1. docker/docker-compose.yaml (prod / make up) still has short-form required env_file. Same missing-.env exposure. Left out to keep this PR scoped — say the word and I will mirror the optional syntax + docs there.
2. Direct callers still have no programmatic version gate (inherent). Docs + header quote the old error.
3. Undetectable Compose version warns rather than hard-fails. Prefer hard-fail? Easy to flip.
4. start always prints Using DEER_FLOW_ROOT=... (previously only when the script set a default). Small UX change; intended for visibility.

Happy to trim further (e.g. docs-only + fixture-only) if you would rather not take the Compose 2.24 floor at all — but that would mean dropping required: false and leaving the fresh-clone direct-invocation failure on Windows.

[willem-bd]

Thanks for addressing the previous review rounds. The earlier concerns are resolved, but one compatibility issue remains on 0bedc015:

[P2] The docker-compose fallback is only used for version detection. _compose_version_short() accepts a valid standalone docker-compose installation when the docker compose plugin is unavailable, but COMPOSE_CMD remains hardcoded to docker compose. As a result, the standalone 2.24 binary passes require_compose_version(), then start, logs, stop, and restart immediately invoke the unavailable plugin and fail.

I reproduced this by making docker compose version unavailable and returning 2.24.0 from docker-compose version: preflight succeeds, but the next $COMPOSE_CMD operation calls docker compose and exits. Please select and retain the Compose executable for both the version check and subsequent operations (preferably as an array), and extend the fallback test through an actual wrapper operation rather than stopping at require_compose_version().

Focused validation otherwise passed: 32 targeted tests, Compose config rendering with Compose 5.1.4, bash -n, and git diff --check.

[YZJF]

Thanks — good catch. Fixed in abf0c6b.

What was wrong

_compose_version_short / the probe could accept a standalone docker-compose 2.24+, but COMPOSE_CMD stayed hardcoded to docker compose. Preflight passed; the next $COMPOSE_CMD call then hit the missing plugin.

There was a second footgun in the first attempt at fixing this: assigning COMPOSE_BIN inside raw="$(...)" runs in a subshell, so the selection was discarded. Selection now happens in the current shell via _probe_compose.

What changed

• COMPOSE_BIN array holds the chosen executable (docker compose or docker-compose)
• _refresh_compose_cmd rebuilds COMPOSE_CMD from that array after a successful probe
• start / logs / stop / restart therefore use the same binary the version check accepted
• Fallback test no longer stops at require_compose_version: with the plugin stubbed out and docker-compose returning 2.24.0, it runs stop and asserts the hyphenated binary received down

Verification

scenario	result
plugin missing + docker-compose 2.24 → stop	COMPOSE_CMD=docker-compose ..., down invoked on that binary
same, then DooD overlay append	docker-compose ... -f docker-compose-dev.yaml -f docker-compose.dood.yaml
both available	prefers docker compose
hyphenated 2.23.3	still rejected at version check

One residual note (not changed here): execution still goes through the COMPOSE_CMD string for compatibility with existing test overrides (COMPOSE_CMD=capture_compose); the selected executable itself is kept in the COMPOSE_BIN array as suggested. Happy to switch call sites to "${COMPOSE_BIN[@]}" ... in a follow-up if you prefer that shape.