Skip to content

feat(cli): add runtime.log_file YAML default for --log-file #473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nskun (nskun) wants to merge 4 commits into
base: main
Choose a base branch
from
Open

feat(cli): add runtime.log_file YAML default for --log-file #473

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4b98de5
feat(cli): allow runtime.log_file in workflow YAML to set the default…
nskun Aug 21, 2026
8142eb1
docs(cli): document runtime.log_file YAML setting
nskun Aug 21, 2026
78e328e
docs: document runtime.log_file across guides
nskun Aug 21, 2026
df91d12
style(cli): format log file warning
nskun Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion docs/cli-reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ conductor run <workflow.yaml> [OPTIONS]
| `--provider PROVIDER` | `-p` | Override provider (copilot, claude, claude-agent-sdk, hermes) |
| `--dry-run` | | Show execution plan without running |
| `--skip-gates` | | Auto-select first option at human gates |
| `--log-file <auto\|PATH>` | `-l` | Write full debug output to a file |
| `--log-file <auto\|PATH>` | `-l` | Write full debug output to a file. Overrides `workflow.runtime.log_file` when supplied. |
| `--web` | | Start a real-time web dashboard |
| `--web-bg` | | Run in background, print dashboard URL, exit |
| `--web-port PORT` | | Port for web dashboard (0 = auto-select) |
Expand All @@ -75,6 +75,19 @@ conductor run <workflow.yaml> [OPTIONS]
> [root-level options](#root-level-options), which must appear *before* the
> `run` subcommand: `conductor --quiet run workflow.yaml`.

### File Logging

Debug file logging can be configured with `workflow.runtime.log_file` or the
`--log-file` / `-l` option. Both forms accept `auto`, which creates a
timestamped log in the OS temporary directory's `conductor/` subdirectory, or
an explicit file path. Relative paths are resolved from the Conductor process's
current working directory.

The YAML setting applies to both `conductor run` and `conductor resume`. An
explicit CLI option takes precedence over `workflow.runtime.log_file`. If
neither is set, debug file logging is disabled. See
[Runtime Configuration](configuration.md#debug-file-logging) for the YAML form.

### Examples

#### Basic Execution
Expand Down
19 changes: 19 additions & 0 deletions docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ workflow:
default_model: gpt-5.2
temperature: 0.7
max_tokens: 4096
log_file: auto # Optional: "auto" or a file path; disabled when omitted
default_reasoning_effort: medium # low | medium | high | xhigh | max (optional)
default_context_tier: default # default | long_context (optional, Copilot only)
# Provider-specific settings...
Expand Down Expand Up @@ -390,6 +391,24 @@ workflow:
- `0.4 - 0.7`: Balanced (general Q&A, analysis)
- `0.8 - 1.0`: Creative (brainstorming, content generation)

### Debug File Logging

Use `runtime.log_file` to enable debug file logging by default for both
`conductor run` and `conductor resume`:

```yaml
workflow:
runtime:
log_file: auto # Or ./logs/run.log
```

`auto` creates a timestamped `.log` file in the `conductor/` subdirectory of
the OS temporary directory. Relative paths are resolved from the Conductor
process's current working directory. An explicit CLI `--log-file` value takes
precedence; if both settings are omitted, file logging remains disabled. See
[Workflow Syntax](workflow-syntax.md) and the
[CLI reference](cli-reference.md#file-logging).

## Claude-Specific Configuration

### Max Tokens
Expand Down
11 changes: 11 additions & 0 deletions docs/workflow-syntax.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,7 @@ workflow:
default_model: gpt-5.2
temperature: 0.7
max_tokens: 4096
log_file: auto # Optional: "auto" or file path (default: no file logging)
default_reasoning_effort: medium # Optional: low | medium | high | xhigh | max
# Workflow-wide default for reasoning /
# extended-thinking effort. Inherited by
Expand All @@ -80,6 +81,16 @@ workflow:
# parent directory of the workflow YAML file.
```

**Runtime file logging** enables debug file logging using the same destination
forms as CLI `--log-file`.
Set `log_file: auto` to generate a timestamped file in the OS temporary
directory's `conductor/` subdirectory, or specify a path such as
`./logs/run.log`. Relative paths are resolved from the Conductor process's
current working directory. The setting applies to both `conductor run` and
`conductor resume`. An explicitly supplied CLI `--log-file` takes precedence;
if neither value is set, file logging remains disabled. See the
[CLI reference](cli-reference.md#conductor-run) for the command-line option.

**Workflow metadata** is included verbatim in the `workflow_started` event and lets downstream consumers (dashboards, queue runners, observability tools) adapt without parsing the YAML. CLI `--metadata key=value` flags merge on top of YAML metadata (CLI wins on conflicts).

**Instructions files** are loaded once and prepended to every agent's rendered prompt. They are inherited by sub-workflows and persisted in checkpoints so resume continues to use the same instructions. Use the YAML `instructions:` list for workflow-pinned context, or pass `--workspace-instructions` on the CLI to auto-discover `AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`, and `.github/instructions/**/*.instructions.md` (recursive; only files marked `applyTo: "**"` in YAML frontmatter are loaded — see the [Workspace Instructions section in the CLI reference](cli-reference.md#workspace-instructions) for full details) by walking from CWD up to the git root.
Expand Down
1 change: 1 addition & 0 deletions plugins/conductor/skills/conductor/references/authoring.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ workflow:
runtime:
provider: copilot # copilot (default), claude, claude-agent-sdk, hermes (experimental), or openai-agents
default_model: gpt-5.2 # Default model for agents
log_file: auto # Full debug log: "auto" or a file path (optional; CLI --log-file wins)
temperature: 0.7 # 0.0-1.0 (optional)
max_tokens: 4096 # Max output tokens per response (optional)
timeout: 600 # Per-request timeout in seconds (optional)
Expand Down
20 changes: 17 additions & 3 deletions plugins/conductor/skills/conductor/references/execution.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ conductor run <workflow.yaml> [OPTIONS]
| `--web-bg` | Run in background, print dashboard URL, exit |
| `--web-port PORT` | Port for web dashboard (0 = auto) |
| `--no-interactive` | Disable Esc-to-interrupt capability |
| `--log-file`, `-l PATH` | Write full debug output to file (`auto` for auto-generated) |
| `--log-file`, `-l <auto\|PATH>` | Write full debug output to file; overrides `runtime.log_file` when supplied |
| `--workspace-instructions` | Auto-discover `AGENTS.md`, `CLAUDE.md`, `.github/copilot-instructions.md`, and `.github/instructions/**/*.instructions.md` (only files marked `applyTo: "**"`) and prepend them to every agent prompt |
| `--instructions PATH` | Path to a specific instruction file to prepend (repeatable) |

Expand Down Expand Up @@ -198,7 +198,7 @@ conductor resume --from <checkpoint.json> [OPTIONS]
| `--provider`, `-p PROVIDER` | Override provider for the resumed run |
| `--metadata`, `-m KEY=VALUE` | Workflow metadata, merged on top of YAML metadata (repeatable) |
| `--skip-gates` | Auto-select first option at human gates |
| `--log-file`, `-l PATH` | Write debug output to file |
| `--log-file`, `-l <auto\|PATH>` | Write debug output to file; overrides `runtime.log_file` when supplied |
| `--no-interactive` | Disable Esc-to-interrupt |
| `--web` | Start real-time web dashboard for the resumed run |
| `--web-port PORT` | Port for the dashboard (0 = auto) |
Expand Down Expand Up @@ -414,7 +414,21 @@ conductor run workflow.yaml --log-file auto
conductor -s run workflow.yaml --log-file debug.log
```

Capture full debug output to a file. Combine with `--silent` for quiet terminal with full logging. Auto mode generates files in `$TMPDIR/conductor/`.
Set a workflow default so the flag does not need to be repeated:

```yaml
workflow:
runtime:
log_file: auto # Or ./logs/run.log
```

Both the YAML field and CLI option accept `auto`, which creates a timestamped
file in the OS temporary directory's `conductor/` subdirectory, or an explicit
file path. Relative paths are resolved from the Conductor process's current
working directory. The YAML setting applies to both `run` and `resume`; an
explicit CLI `--log-file` value takes precedence. If neither is set, debug file
logging is disabled. Combine with `--silent` for a quiet terminal with file
logging enabled.

### Dry Run

Expand Down
2 changes: 2 additions & 0 deletions plugins/conductor/skills/conductor/references/yaml-schema.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@ workflow:
provider: string | object # "copilot" (default), "claude", "claude-agent-sdk", "hermes", or "openai-agents"
# — or a ProviderSettings object (see below)
default_model: string # Default model for all agents
log_file: string # Full debug log path, or "auto" for a generated temp file (optional)
# CLI --log-file overrides this value; omitted means no file logging
temperature: float # 0.0-1.0, controls randomness (optional, copilot/claude/hermes)
max_tokens: integer # Max OUTPUT tokens per response, 1-200000 (optional, copilot/claude/hermes)
timeout: float # Per-request timeout in seconds (optional, default: 600, copilot/claude only)
Expand Down
54 changes: 36 additions & 18 deletions src/conductor/cli/run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,22 @@ def init_file_logging(log_path: Path) -> None:
)


def _try_init_file_logging(log_path: Path | None) -> bool:
"""Initialize file logging, returning whether initialization succeeded."""
if log_path is None:
return False

try:
init_file_logging(log_path)
except OSError as e:
_verbose_console.print(
styled("[bold yellow]Warning:[/bold yellow] Cannot open log file {}: {}", log_path, e)
)
return False

return True


def close_file_logging() -> None:
"""Close file logging and clean up resources."""
global _file_console, _file_handle
Expand Down Expand Up @@ -1959,15 +1975,7 @@ async def run_workflow_async(
start_time = time.time()

# Initialize file logging if requested
if log_file is not None:
try:
init_file_logging(log_file)
except OSError as e:
_verbose_console.print(
styled(
"[bold yellow]Warning:[/bold yellow] Cannot open log file {}: {}", log_file, e
)
)
_try_init_file_logging(log_file)

# Always create event emitter and JSONL log subscriber
emitter = WorkflowEventEmitter()
Expand Down Expand Up @@ -1995,6 +2003,15 @@ async def run_workflow_async(
config = load_config(workflow_path)
verbose_log_timing("Configuration loaded", time.time() - load_start)

if log_file is None:
configured = config.workflow.runtime.log_file
if configured is not None:
if configured.lower() == "auto":
log_file = generate_log_path(workflow_path.stem)
else:
log_file = Path(configured)
_try_init_file_logging(log_file)
Comment on lines +2008 to +2013

@jrob5756 Jason Robert (jrob5756) Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four things land in this block.

generate_log_path() is outside the guard. Its mkdir(parents=True, exist_ok=True) still raises FileExistsError when $TMPDIR/conductor is a file, and PermissionError or ENOSPC on a constrained runner. A logging setting shouldn't be able to abort the run, and it's inconsistent with the explicit-path branch one line below, which warns and continues.

Path(configured) doesn't expand ~. Since init_file_logging creates parents, log_file: ~/logs/run.log quietly writes into a directory named ~ under the CWD and reports success. The shell saves the CLI flag from this; YAML doesn't.

A NUL byte in the value raises ValueError, which except OSError doesn't catch, so it escapes as an unhandled traceback.

And the bool from _try_init_file_logging is dropped here, so log_file keeps naming a file that was never opened. Nothing reads it wrongly today, but only because the _file_console is not None check at line 2313 happens to cover for it.

Suggested change
if configured is not None:
if configured.lower() == "auto":
log_file = generate_log_path(workflow_path.stem)
else:
log_file = Path(configured)
_try_init_file_logging(log_file)
if configured is not None:
try:
if configured.strip().lower() == "auto":
log_file = generate_log_path(workflow_path.stem)
else:
log_file = Path(configured).expanduser()
except (OSError, ValueError) as e:
_verbose_console.print(
styled(
"[bold yellow]Warning:[/bold yellow] Cannot prepare log file {}: {}",
configured,
e,
)
)
log_file = None
if log_file is not None and not _try_init_file_logging(log_file):
log_file = None


# Merge CLI metadata on top of YAML-declared metadata
if metadata:
config.workflow.metadata.update(metadata)
Expand Down Expand Up @@ -2592,15 +2609,7 @@ async def resume_workflow_async(
start_time = time.time()

# Initialize file logging if requested
if log_file is not None:
try:
init_file_logging(log_file)
except OSError as e:
_verbose_console.print(
styled(
"[bold yellow]Warning:[/bold yellow] Cannot open log file {}: {}", log_file, e
)
)
_try_init_file_logging(log_file)

# Always create event emitter and JSONL log subscriber (parity with run)
emitter = WorkflowEventEmitter()
Expand Down Expand Up @@ -2659,6 +2668,15 @@ async def resume_workflow_async(
# (workflow_root) and to seed the synthetic replay fallback.
config = load_config(resolved_workflow_path)

if log_file is None:
configured = config.workflow.runtime.log_file
if configured is not None:
if configured.lower() == "auto":
log_file = generate_log_path(resolved_workflow_path.stem)
else:
log_file = Path(configured)
_try_init_file_logging(log_file)
Comment on lines +2674 to +2678

@jrob5756 Jason Robert (jrob5756) Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same four issues as the run side. Given these two blocks are identical apart from the path variable, and AGENTS.md names drift between run_workflow_async and resume_workflow_async as a hazard, I'd lift this into one helper rather than fix it twice.

A shared resolve_log_file(value, stem) next to generate_log_path would also absorb the two copies in cli/app.py (:663 and :1157). Those have already drifted: app.py:1158 falls back to a "resume" stem where this line uses the checkpoint-derived one. Four copies of the auto vocabulary means the expanduser() and strip() fixes have to be applied four times to hold.

One more thing specific to resume. Because the YAML value is read after the checkpoint block, this log misses Loading checkpoint, the workflow-hash-changed warning, and Resuming from agent, all of which --log-file captures. Two spellings of the same setting produce different logs, and the docs present them as equivalent.


# Merge CLI metadata on top of YAML-declared metadata (parity with run)
if metadata:
config.workflow.metadata.update(metadata)
Expand Down
3 changes: 3 additions & 0 deletions src/conductor/config/schema.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3332,6 +3332,9 @@ def _coerce_provider(cls, value: Any) -> Any:
default_model: str | None = None
"""Default model for agents that don't specify one."""

log_file: str | None = None
"""Default log file path, or ``auto`` to generate one automatically."""
Comment on lines +3335 to +3336

@jrob5756 Jason Robert (jrob5756) Aug 22, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of invariants the type could carry instead of the call site.

An empty or whitespace-only value clears the is not None guard downstream and becomes Path("."), which fails as IsADirectoryError and prints Cannot open log file .: with an errno naming ''. Neither half points back at this field. SkillDiscoveryConfig.validate_exclude a few lines below rejects blank entries for the same reason, and session_key at line 1532 already uses the constraint below.

Stripping also fixes " auto ", which currently becomes a file named ␣auto␣ rather than matching the sentinel.

The docstring is worth two additions as well: auto is matched case-insensitively, and only the top-level workflow's value is read. A sub-workflow setting this gets no log, no warning and no error, which is surprising when working_dir, skills and default_model all apply per sub-workflow. runtime.checkpoint has the same root-only scope and says so.

Suggested change
log_file: str | None = None
"""Default log file path, or ``auto`` to generate one automatically."""
log_file: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1)] | None = None
"""Default log file path, or ``auto`` (case-insensitive) to generate one automatically.
Honoured only for the top-level workflow; ignored in a ``type: workflow`` sub-workflow.
"""


mcp_servers: dict[str, MCPServerDef] = Field(default_factory=dict)
"""MCP server configurations keyed by server name."""

Expand Down
Loading
Loading