fix(authz): recheck policy before sandbox reuse

backend/packages/harness/deerflow/agents/middlewares/AGENTS.md

@@ -1,4 +1,4 @@
 ### Middleware Chain
 
 Persisted delegation verdicts are untrusted durable context; ledger rendering revalidates them and ignores malformed values.
 
@@ -61,7 +61,7 @@
    forgeries). Consumers pop it; the publisher and the consumer share only that
    contract module.
 10. **SandboxAuditMiddleware** - Audits sandboxed shell/file operations for security logging before tool execution. Command classification is **defense-in-depth and audit, not a security boundary** — the sandbox itself is the isolation boundary. Command substitution is judged by *position*, not by the presence of `$(`: a substitution in **command position** (`$(curl url)`, `` `curl url` ``, the word after a `|`/`&&`/`;`, or any `eval`/`source` argument) executes fetched or interpreted content and is blocked, while **value position** (`x=$(curl url)`, `echo $(curl url)`, an argument, a `for` word list) only captures output and passes (#4611). `_HIGH_RISK_COMMAND_POSITION_PATTERNS` is therefore matched anchored against each split sub-command, never against the whole compound string, and `_split_compound_command(split_pipes=True)` supplies those sub-commands; rules that span a pipe (`| sh`, `base64 -d | ...`) still rely on `_classify_command`'s whole-command Pass 1. `_COMMAND_POSITION_PREFIX` extends the anchor over leading variable assignments and exec wrappers (`FOO=1 $(curl url)`, `env`/`command`/`builtin`/`exec`/`nohup`/`time`/`sudo`/`doas`), which are still command position; its assignment branch requires whitespace before the substitution, which is exactly what keeps `x=$(curl url)` in value position. Two execution contexts are deliberately **position-blind** and matched against the whole command in Pass 1, because they execute what they receive wherever they appear (including as an argument to something else, e.g. `xargs sh -c "$(curl url)"`): an `eval`/`source` argument, and an interpreter's **code-string flag** — `-c` (shells, `python`), `-e` (`perl`/`ruby`/`node`), `-p` (`perl`/`node`), `-r` (`php`) — plus the here-string (`<<<`) that reaches the same place through stdin. All three substitution spellings (`$(cmd`, `<(cmd`, `` `cmd ``) share one `_RISKY_SUBSTITUTION` opener so a rule cannot cover one spelling and miss another. An unquoted newline splits like `;`, because it separates statements the same way: leaving it joined let `echo hi\n$(curl url)` evade the anchored rules that its `;` spelling triggers. A heredoc body is data rather than statements, so `_split_compound_command` records headers (`<<EOF`, `<<-EOF`, `<<'EOF'`) and consumes their bodies verbatim at the newline that starts them — otherwise a body line beginning with `$(curl url)` would be promoted to a command position the shell never creates. Two things that look like headers must not open one, or a body that never terminates swallows every following statement: `<<<` is a here-string (both a lookahead and a lookbehind are needed, or the trailing `<<` of `<<< "text"` reads as a heredoc with delimiter `text`), and a `<<` inside `$(( ... ))` / `(( ... ))` is a bit shift, so arithmetic depth is tracked alongside the quote flags. That is a heuristic, not shell parsing: it exists only to avoid manufacturing command positions *and* to avoid destroying real ones. An unterminated body consumes the rest of the string; an unclosed `((` only disables heredoc detection, so newlines keep splitting and the failure direction stays towards seeing more command positions rather than fewer. Known, deliberate gaps: process substitution outside `eval`/`source` (`. <(curl u)`) is not detected — closing it would require real shell parsing, which is out of scope for this layer. Two-step forms (`x=$(curl u); eval "$x"`) are inherent rather than incidental: any rule that allows output capture allows the first statement, and connecting it to the later `eval` needs dataflow analysis, not pattern matching. There is currently no config gate: the middleware is appended unconditionally in `_build_runtime_middlewares`, so it applies to both the lead agent and subagents.
-11. **ReadBeforeWriteMiddleware** - *(optional, if `read_before_write.enabled`, default on)* Outermost write gate (issue #3857): `read_file` stamps a content hash onto its ToolMessage; `write_file` (append/overwrite-existing) and `str_replace` are blocked unless the newest mark for that path matches the file's current hash. Sits outside ToolProgressMiddleware and ToolErrorHandlingMiddleware so a blocked write returns immediately without consuming a ToolProgress slot. Blocked results call `normalize_tool_result` directly to stamp `deerflow_tool_meta` (`recoverable_by_model=True`) before returning, keeping the result well-formed for any outer consumer. Marks live on messages, so summarization dropping the read result invalidates the gate automatically; writes never refresh marks, forcing a re-read between consecutive edits. Gate check + tool execution are serialized per (thread, path) so same-turn parallel writes cannot reuse one stale mark; on sandboxes whose `read_file` reports failures as `"Error: ..."` strings instead of raising (AIO/E2B), uninspectable targets fail open (creation proceeds, no mark stamped)
+11. **ReadBeforeWriteMiddleware** - *(optional, if `read_before_write.enabled`, default on)* Outermost write gate (issue #3857): `read_file` stamps a content hash onto its ToolMessage; `write_file` (append/overwrite-existing) and `str_replace` are blocked unless the newest mark for that path matches the file's current hash. Sits outside ToolProgressMiddleware and ToolErrorHandlingMiddleware so a blocked write returns immediately without consuming a ToolProgress slot. Blocked results call `normalize_tool_result` directly to stamp `deerflow_tool_meta` (`recoverable_by_model=True`) before returning, keeping the result well-formed for any outer consumer. Marks live on messages, so summarization dropping the read result invalidates the gate automatically; writes never refresh marks, forcing a re-read between consecutive edits. Gate check + tool execution are serialized per (thread, path) so same-turn parallel writes cannot reuse one stale mark; on sandboxes whose `read_file` reports failures as `"Error: ..."` strings instead of raising (AIO/E2B), uninspectable targets fail open (creation proceeds, no mark stamped). The middleware also owns the sandbox authorization scope for these composed calls: pre-write inspection, the tool body, and post-read hashing share one sync/async provider decision, while `SandboxAuthorizationError` bypasses the generic inspection fail-open paths and becomes an error ToolMessage.
 12. **ToolProgressMiddleware** - *(optional, if `tool_progress.enabled`)* State-machine-based stagnation guard (RFC #3177). Outer wrapper around ToolErrorHandlingMiddleware so its `wrap_tool_call` receives results already stamped with `deerflow_tool_meta`. Tracks per-(thread, tool) consecutive "no-new-info" calls across three error categories: (a) `recoverable_by_model=True` (no_results, not_found, permission, Jaccard-duplicate success): ACTIVE → WARNED (terminal — hint re-injected on each subsequent problem); (b) `recoverable_by_model=False, action≠stop` (rate_limited, transient): ACTIVE → WARNED → BLOCKED after `warn_escalation_count` more problems; (c) `recoverable_by_model=False, action=stop` (auth, config, internal): immediately BLOCKED on first occurrence. **Division of labor with LoopDetectionMiddleware:** ToolProgressMiddleware is a result-quality guard — fires after tool execution and blocks specific tools that stop producing new information; LoopDetectionMiddleware is a call-pattern guard — fires after the model responds and hard-stops the whole turn when the model repeatedly issues identical tool_calls. Both can inject HumanMessage hints in the same model call without conflict; neither reads the other's internal state.
 13. **ToolReceiptMiddleware + ToolErrorHandlingMiddleware** - `ToolReceiptMiddleware` is *(optional, if `verification.receipts_enabled`, default on)*. It is the **outermost `wrap_tool_call` layer** — registered ahead of entries 9-12 — because Guardrail/SandboxAudit/ReadBeforeWrite/ToolProgress can short-circuit a call with their own ToolMessage (and SandboxAudit rebuilds medium-risk results); an inner receipt layer would silently gap the ledger on those results (ordering constraints in `deerflow.extensions.ordering`). Normal results still carry the `deerflow_tool_meta` status ToolErrorHandlingMiddleware stamps on the inner return path; short-circuit messages self-stamp meta or fall back to `message.status`. It stamps deterministic provenance (tool name, status, args/output hashes, byte count, timestamp) onto direct `ToolMessage` results and every matching `ToolMessage` carried in `Command.update.messages`, including delegated `task`, `present_file`, `view_image`, and `tool_search` results; before model calls it derives a hidden receipt ledger (display ids r1..rN) from message state, and when the 2,000-character budget is exceeded the newest receipts are retained in chronological order with their original ids plus an older-receipts omission marker. Rendering returns both the text and its retained receipt subset; every response that received a ledger carries only that exact server-owned subset, never omitted receipts. Snapshot validation accepts a strictly consecutive positive original-id range (for example `r24`–`r30`) rather than requiring `r1`, so subagent terminal citation verification resolves ids against evidence present in the citing turn even when later summarization drops and renumbers tool messages. Model-generated citation IDs are digit-bounded before integer conversion; oversized IDs are ignored as malformed input rather than raising through task write-back. Citation parsing deduplicates exact `(id, anchor)` pairs, not IDs alone, so repeated identical references stay compact while every distinct anchor claim is verified. Gateway strips delegated receipts/verdicts from external messages. `ToolErrorHandlingMiddleware` receives `AppConfig`, converts tool exceptions into error `ToolMessage`s so the run can continue instead of aborting, stamps every result with `deerflow_tool_meta` (status / error_type / recoverable_by_model / recommended_next_action / source) via `tool_result_meta.normalize_tool_result`, stamps structured metadata for task exception wrappers, and stamps skill-read metadata for downstream durable-context capture. Task tool result text is generated from the same status/result/error inputs as the structured metadata so callers do not hand-write a second protocol string.
 

backend/packages/harness/deerflow/agents/middlewares/read_before_write_middleware.py

@@ -39,8 +39,13 @@
 from langgraph.prebuilt.tool_node import ToolCallRequest
 from langgraph.types import Command
 
-from deerflow.agents.middlewares.tool_result_meta import normalize_tool_result
-from deerflow.sandbox.tools import read_current_file_content
+from deerflow.agents.middlewares.tool_result_meta import normalize_tool_result, stamp_exception_meta
+from deerflow.sandbox.exceptions import SandboxAuthorizationError
+from deerflow.sandbox.tools import (
+    read_current_file_content,
+    sandbox_authorization_scope,
+    sandbox_authorization_scope_async,
+)
 
 logger = logging.getLogger(__name__)
 
@@ -105,21 +110,29 @@ def wrap_tool_call(
             path = self._requested_path(request)
             if path is None:
                 return handler(request)
-            with self._lock_for(request, path):
-                blocked = self._check_write_gate(request)
-                if blocked is not None:
-                    # Stamp deerflow_tool_meta so ToolProgressMiddleware can classify
-                    # the blocked write even though it bypasses ToolErrorHandlingMiddleware.
-                    return normalize_tool_result(blocked)
-                return handler(request)
+            try:
+                with sandbox_authorization_scope(request.runtime):
+                    with self._lock_for(request, path):
+                        blocked = self._check_write_gate(request)
+                        if blocked is not None:
+                            # Stamp deerflow_tool_meta so ToolProgressMiddleware can classify
+                            # the blocked write even though it bypasses ToolErrorHandlingMiddleware.
+                            return normalize_tool_result(blocked)
+                        return handler(request)
+            except SandboxAuthorizationError as exc:
+                return self._authorization_error_result(request, exc)
         if name in _READ_TOOLS:
             path = self._requested_path(request)
             if path is None:
                 return handler(request)
-            with self._lock_for(request, path):
-                result = handler(request)
-                self._attach_read_mark(request, result)
-                return result
+            try:
+                with sandbox_authorization_scope(request.runtime):
+                    with self._lock_for(request, path):
+                        result = handler(request)
+                        self._attach_read_mark(request, result)
+                        return result
+            except SandboxAuthorizationError as exc:
+                return self._authorization_error_result(request, exc)
         return handler(request)
 
     @override
@@ -133,32 +146,54 @@ async def awrap_tool_call(
             path = self._requested_path(request)
             if path is None:
                 return await handler(request)
-            # threading.Lock may be released from a different thread than the
-            # acquiring one, so acquiring in a worker thread and releasing on
-            # the event-loop thread is safe.
-            lock = self._lock_for(request, path)
-            await asyncio.to_thread(lock.acquire)
             try:
-                blocked = await asyncio.to_thread(self._check_write_gate, request)
-                if blocked is not None:
-                    return normalize_tool_result(blocked)
-                return await handler(request)
-            finally:
-                lock.release()
+                async with sandbox_authorization_scope_async(request.runtime):
+                    # threading.Lock may be released from a different thread than the
+                    # acquiring one, so acquiring in a worker thread and releasing on
+                    # the event-loop thread is safe.
+                    lock = self._lock_for(request, path)
+                    await asyncio.to_thread(lock.acquire)
+                    try:
+                        blocked = await asyncio.to_thread(self._check_write_gate, request)
+                        if blocked is not None:
+                            return normalize_tool_result(blocked)
+                        return await handler(request)
+                    finally:
+                        lock.release()
+            except SandboxAuthorizationError as exc:
+                return self._authorization_error_result(request, exc)
         if name in _READ_TOOLS:
             path = self._requested_path(request)
             if path is None:
                 return await handler(request)
-            lock = self._lock_for(request, path)
-            await asyncio.to_thread(lock.acquire)
             try:
-                result = await handler(request)
-                await asyncio.to_thread(self._attach_read_mark, request, result)
-                return result
-            finally:
-                lock.release()
+                async with sandbox_authorization_scope_async(request.runtime):
+                    lock = self._lock_for(request, path)
+                    await asyncio.to_thread(lock.acquire)
+                    try:
+                        result = await handler(request)
+                        await asyncio.to_thread(self._attach_read_mark, request, result)
+                        return result
+                    finally:
+                        lock.release()
+            except SandboxAuthorizationError as exc:
+                return self._authorization_error_result(request, exc)
         return await handler(request)
 
+    @staticmethod
+    def _authorization_error_result(request: ToolCallRequest, exc: SandboxAuthorizationError) -> ToolMessage:
+        """Return the normal tool-level denial instead of failing open or the run."""
+        tool_name = str(request.tool_call.get("name") or "unknown_tool")
+        tool_call_id = str(request.tool_call.get("id") or "missing-tool-call-id")
+        detail = str(exc).strip() or exc.__class__.__name__
+        message = ToolMessage(
+            content=f"Error: {detail}",
+            tool_call_id=tool_call_id,
+            name=tool_name,
+            status="error",
+        )
+        return stamp_exception_meta(message, f"{exc.__class__.__name__}: {detail}")
+
     # -- locking ---------------------------------------------------------
 
     def _lock_for(self, request: ToolCallRequest, path: str) -> threading.Lock:
@@ -193,6 +228,8 @@ def _check_write_gate(self, request: ToolCallRequest) -> ToolMessage | None:
         except FileNotFoundError:
             # write_file creates the file; str_replace surfaces its own error.
             return None
+        except SandboxAuthorizationError:
+            raise
         except Exception:
             logger.warning("read-before-write gate could not inspect %r; allowing the write (fail-open)", path, exc_info=True)
             return None
@@ -246,6 +283,8 @@ def _attach_read_mark(self, request: ToolCallRequest, result: ToolMessage | Comm
             return
         try:
             content = self._content_reader(request.runtime, path)
+        except SandboxAuthorizationError:
+            raise
         except Exception:
             logger.debug("read-before-write mark skipped for %r: file not hashable", path, exc_info=True)
             return