Skip to content

Commit d14d6c9

Browse files
fix(sandbox): harden OCI workspace handling
Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
1 parent 6e40ae1 commit d14d6c9

13 files changed

Lines changed: 319 additions & 73 deletions

File tree

.agents/skills/openshell-cli/SKILL.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -223,8 +223,8 @@ openshell sandbox ssh-config my-sandbox >> ~/.ssh/config
223223
# Upload local files to the sandbox working directory
224224
openshell sandbox upload my-sandbox ./src
225225

226-
# Download files from sandbox
227-
openshell sandbox download my-sandbox /sandbox/output ./local-output
226+
# Download files from sandbox (replace /workspace with the path from `pwd -P`)
227+
openshell sandbox download my-sandbox /workspace/output ./local-output
228228
```
229229

230230
Uploads honor `.gitignore` by default. Add `--no-git-ignore` only when ignored files are intentionally in scope.

architecture/compute-runtimes.md

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -147,13 +147,15 @@ primary GID. It does not rewrite the account files.
147147

148148
Docker and Podman use an absolute OCI working directory as the workspace. An
149149
empty or root (`/`) declaration falls back to `/sandbox`, which OpenShell
150-
creates inside the container when needed. Before direct or SSH children start,
151-
the supervisor transfers ownership of only the workspace directory to the
152-
completed UID/GID. Image-provided contents and nested mounts retain their
153-
ownership. The resolved workspace is also the child cwd and `HOME`, the
154-
automatic writable policy path, and Podman's managed workspace-volume mount
155-
target. Kubernetes/OpenShift keep their `/sandbox` PVC and `fsGroup` behavior,
156-
and VM keeps its `/sandbox` guest initialization path.
150+
creates inside the container when needed. Protected system trees and
151+
OpenShell-reserved paths cannot become workspaces. The container runtime starts
152+
the supervisor from `/`; before direct or SSH children start, the supervisor
153+
validates the resolved path and transfers ownership of only the workspace
154+
directory to the completed UID/GID. Image-provided contents and nested mounts
155+
retain their ownership. The resolved workspace is also the child cwd and
156+
`HOME`, the automatic writable policy path, and Podman's managed
157+
workspace-volume mount target. Kubernetes/OpenShift keep their `/sandbox` PVC
158+
and `fsGroup` behavior, and VM keeps its `/sandbox` guest initialization path.
157159

158160
Sandbox creation fails before the workload becomes ready when a required image
159161
identity is absent, malformed, unknown, ambiguous, or resolves to UID/GID 0.

crates/openshell-cli/src/ssh.rs

Lines changed: 22 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1115,7 +1115,15 @@ async fn ssh_run_capture_stdout(session: &SshSessionConfig, command: &str) -> Re
11151115
output.status
11161116
));
11171117
}
1118-
Ok(String::from_utf8_lossy(&output.stdout).trim().to_string())
1118+
decode_ssh_probe_stdout(output.stdout)
1119+
}
1120+
1121+
fn decode_ssh_probe_stdout(stdout: Vec<u8>) -> Result<String> {
1122+
let stdout = String::from_utf8(stdout)
1123+
.map_err(|error| miette::miette!("ssh probe returned non-UTF-8 output: {error}"))?;
1124+
let stdout = stdout.strip_suffix('\n').unwrap_or(&stdout);
1125+
let stdout = stdout.strip_suffix('\r').unwrap_or(stdout);
1126+
Ok(stdout.to_string())
11191127
}
11201128

11211129
fn validate_discovered_workspace_root(root: &str) -> Result<String> {
@@ -2092,6 +2100,19 @@ mod tests {
20922100
}
20932101
}
20942102

2103+
#[test]
2104+
fn ssh_probe_output_only_removes_the_protocol_line_ending() {
2105+
assert_eq!(
2106+
decode_ssh_probe_stdout(b"/workspace/project \n".to_vec()).unwrap(),
2107+
"/workspace/project "
2108+
);
2109+
assert_eq!(
2110+
decode_ssh_probe_stdout(b"/workspace/project\r\n".to_vec()).unwrap(),
2111+
"/workspace/project"
2112+
);
2113+
assert!(decode_ssh_probe_stdout(vec![0xff]).is_err());
2114+
}
2115+
20952116
#[test]
20962117
fn build_single_file_tar_cmd_inserts_double_dash_before_basename() {
20972118
// Without `--`, a basename such as `--checkpoint-action=...` would be

crates/openshell-core/src/driver_mounts.rs

Lines changed: 105 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,46 @@ const RESERVED_MOUNT_TARGETS: &[&str] = &[
3232
"/run/netns",
3333
];
3434

35+
/// Container filesystem trees that image metadata must never turn into an
36+
/// agent-owned writable workspace.
37+
const PROTECTED_WORKSPACE_TREES: &[&str] = &[
38+
"/bin",
39+
"/boot",
40+
"/dev",
41+
"/etc",
42+
"/lib",
43+
"/lib64",
44+
"/proc",
45+
"/root",
46+
"/run",
47+
"/sbin",
48+
"/sys",
49+
"/usr/bin",
50+
"/usr/lib",
51+
"/usr/lib64",
52+
"/usr/local/bin",
53+
"/usr/local/lib",
54+
"/usr/local/lib64",
55+
"/usr/local/sbin",
56+
"/usr/local/share",
57+
"/usr/sbin",
58+
"/usr/share",
59+
"/var/log",
60+
];
61+
62+
/// Broad container roots that are unsafe as workspaces themselves, while
63+
/// application-specific descendants remain valid.
64+
const PROTECTED_WORKSPACE_ROOTS: &[&str] = &[
65+
"/home",
66+
"/mnt",
67+
"/opt",
68+
"/srv",
69+
"/usr",
70+
"/usr/local",
71+
"/var",
72+
"/var/lib",
73+
];
74+
3575
/// Compatibility workspace used when an OCI image has no usable working
3676
/// directory and by drivers whose workspace remains fixed.
3777
pub const DEFAULT_WORKSPACE_ROOT: &str = "/sandbox";
@@ -139,8 +179,11 @@ pub fn resolve_oci_workspace_root(working_dir: &str) -> Result<String, String> {
139179
if working_dir.is_empty() || working_dir == "/" {
140180
return Ok(DEFAULT_WORKSPACE_ROOT.to_string());
141181
}
142-
if working_dir.as_bytes().contains(&0) {
143-
return Err("OCI WorkingDir must not contain NUL bytes".to_string());
182+
if working_dir != working_dir.trim() {
183+
return Err("OCI WorkingDir must not contain surrounding whitespace".to_string());
184+
}
185+
if working_dir.chars().any(char::is_control) {
186+
return Err("OCI WorkingDir must not contain control characters".to_string());
144187
}
145188
if !working_dir.starts_with('/') {
146189
return Err(format!(
@@ -159,13 +202,32 @@ pub fn resolve_oci_workspace_root(working_dir: &str) -> Result<String, String> {
159202
));
160203
}
161204

162-
Ok(working_dir.trim_end_matches('/').to_string())
205+
let workspace_root = working_dir.trim_end_matches('/').to_string();
206+
let workspace_path = Path::new(&workspace_root);
207+
if PROTECTED_WORKSPACE_ROOTS
208+
.iter()
209+
.any(|protected| workspace_path == Path::new(protected))
210+
|| PROTECTED_WORKSPACE_TREES
211+
.iter()
212+
.any(|protected| path_is_or_under(workspace_path, Path::new(protected)))
213+
|| RESERVED_MOUNT_TARGETS.iter().any(|reserved| {
214+
let reserved = Path::new(reserved);
215+
path_is_or_under(workspace_path, reserved) || path_is_or_under(reserved, workspace_path)
216+
})
217+
{
218+
return Err(format!(
219+
"OCI WorkingDir '{working_dir}' conflicts with a protected container path"
220+
));
221+
}
222+
223+
Ok(workspace_root)
163224
}
164225

165-
/// Reject a user-supplied mount that would replace the resolved workspace
166-
/// root. Mounts below the workspace remain valid.
226+
/// Reject a user-supplied mount that would replace or contain the resolved
227+
/// workspace root. Mounts below the workspace remain valid.
167228
pub fn validate_workspace_mount_target(target: &str, workspace_root: &str) -> Result<(), String> {
168-
if normalize_mount_target(target) == workspace_root {
229+
let normalized_target = normalize_mount_target(target);
230+
if path_is_or_under(Path::new(workspace_root), Path::new(&normalized_target)) {
169231
return Err(format!(
170232
"mount target '{target}' is reserved for the OpenShell workspace"
171233
));
@@ -202,6 +264,8 @@ mod tests {
202264
validate_workspace_mount_target("/sandbox/", "/sandbox").unwrap_err();
203265
validate_workspace_mount_target("/workspace/", "/sandbox").unwrap();
204266
validate_workspace_mount_target("/workspace/cache", "/workspace").unwrap();
267+
validate_workspace_mount_target("/workspace", "/workspace/project").unwrap_err();
268+
validate_workspace_mount_target("/workspace-other", "/workspace/project").unwrap();
205269
}
206270

207271
#[test]
@@ -227,6 +291,8 @@ mod tests {
227291
"/workspace/./project",
228292
"/workspace//project",
229293
"/workspace\0project",
294+
"/workspace ",
295+
"/workspace\nproject",
230296
] {
231297
assert!(
232298
resolve_oci_workspace_root(invalid).is_err(),
@@ -235,6 +301,39 @@ mod tests {
235301
}
236302
}
237303

304+
#[test]
305+
fn oci_workspace_root_rejects_protected_container_paths() {
306+
for invalid in [
307+
"/etc",
308+
"/etc/project",
309+
"/usr",
310+
"/usr/bin/project",
311+
"/var",
312+
"/var/log/project",
313+
"/opt",
314+
"/opt/openshell/project",
315+
] {
316+
assert!(
317+
resolve_oci_workspace_root(invalid).is_err(),
318+
"expected protected workspace '{invalid}' to be rejected"
319+
);
320+
}
321+
322+
for valid in [
323+
"/app",
324+
"/home/app",
325+
"/opt/app",
326+
"/usr/src/app",
327+
"/var/lib/app",
328+
] {
329+
assert_eq!(
330+
resolve_oci_workspace_root(valid).unwrap(),
331+
valid,
332+
"expected application workspace '{valid}' to remain valid"
333+
);
334+
}
335+
}
336+
238337
#[test]
239338
fn container_target_rejects_reserved_openshell_tls_legacy_path() {
240339
let err = validate_container_mount_target("/etc/openshell-tls/client").unwrap_err();

crates/openshell-driver-docker/README.md

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,9 @@ root (`/`) declaration falls back to `/sandbox`, which OpenShell creates when
3333
necessary. The supervisor transfers ownership of only the workspace directory
3434
to the completed identity before direct or SSH children start. Existing
3535
contents and nested bind or volume mounts retain their ownership. The workspace
36-
is the child cwd and `HOME`; an invalid, symlinked, or unpreparable workspace
37-
fails sandbox startup.
36+
is the child cwd and `HOME`. Protected system trees and OpenShell-reserved
37+
paths cannot become workspaces. The supervisor starts from `/`, then reports an
38+
invalid, symlinked, or unpreparable workspace as a readiness failure.
3839

3940
Docker containers join an OpenShell-managed bridge network. The driver injects
4041
`host.openshell.internal` and `host.docker.internal` so supervisors have stable
@@ -86,9 +87,10 @@ optional `selinux_label` of `shared` (applies `:z`) or `private` (applies
8687
`subpath`. User-supplied bind and volume mounts are read-only by default; set
8788
`read_only: false` to make them writable. Mount `source`, `target`, and
8889
`subpath` values must not contain surrounding whitespace. Mount targets must be
89-
absolute container paths and must not replace the resolved workspace root.
90-
Nested workspace mounts remain valid. Mounts also must not overlap OpenShell
91-
supervisor files, `/etc/openshell`, `/etc/openshell-tls`, or `/run/netns`.
90+
absolute container paths and must not replace or contain the resolved workspace
91+
root. Nested workspace mounts remain valid. Mounts also must not overlap
92+
OpenShell supervisor files, `/etc/openshell`, `/etc/openshell-tls`, or
93+
`/run/netns`.
9294

9395
Example named-volume usage:
9496

crates/openshell-driver-docker/src/lib.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2436,6 +2436,9 @@ fn build_container_create_body_for_image(
24362436
Ok(ContainerCreateBody {
24372437
image: Some(image.id.clone()),
24382438
user: Some("0".to_string()),
2439+
// The image workspace may need to be created or rejected by the
2440+
// supervisor, so do not let the OCI runtime chdir there first.
2441+
working_dir: Some("/".to_string()),
24392442
env: Some(build_environment_for_oci_user(sandbox, config, &image.user)),
24402443
entrypoint: Some(vec![SUPERVISOR_MOUNT_PATH.to_string()]),
24412444
// Clear the image CMD so Docker does not append inherited args to the

crates/openshell-driver-docker/src/tests.rs

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,6 +584,7 @@ fn container_creation_uses_inspected_immutable_image() {
584584

585585
assert_eq!(body.image.as_deref(), Some("sha256:immutable"));
586586
assert_eq!(body.user.as_deref(), Some("0"));
587+
assert_eq!(body.working_dir.as_deref(), Some("/"));
587588
assert_eq!(
588589
body.cmd.as_deref(),
589590
Some(&["--workdir".to_string(), "/workspace/project".to_string()][..])
@@ -614,6 +615,26 @@ fn container_creation_rejects_invalid_oci_working_dir() {
614615
assert!(err.message().contains("must be an absolute container path"));
615616
}
616617

618+
#[test]
619+
fn container_creation_rejects_protected_oci_working_dir() {
620+
let metadata = DockerImageMetadata {
621+
id: "sha256:immutable".to_string(),
622+
user: "1234:1235".to_string(),
623+
working_dir: "/etc".to_string(),
624+
};
625+
let err = build_container_create_body_for_image(
626+
&test_sandbox(),
627+
&runtime_config(),
628+
&DockerSandboxDriverConfig::default(),
629+
None,
630+
&metadata,
631+
)
632+
.unwrap_err();
633+
634+
assert_eq!(err.code(), tonic::Code::FailedPrecondition);
635+
assert!(err.message().contains("protected container path"));
636+
}
637+
617638
#[test]
618639
fn container_creation_reserves_resolved_workspace_root_but_allows_nested_mounts() {
619640
let metadata = DockerImageMetadata {
@@ -638,6 +659,27 @@ fn container_creation_reserves_resolved_workspace_root_but_allows_nested_mounts(
638659
.contains("reserved for the OpenShell workspace")
639660
);
640661

662+
let ancestor_mount: DockerSandboxDriverConfig = serde_json::from_value(serde_json::json!({
663+
"mounts": [{"type": "tmpfs", "target": "/workspace"}]
664+
}))
665+
.unwrap();
666+
let nested_metadata = DockerImageMetadata {
667+
working_dir: "/workspace/project".to_string(),
668+
..metadata.clone()
669+
};
670+
let err = build_container_create_body_for_image(
671+
&test_sandbox(),
672+
&runtime_config(),
673+
&ancestor_mount,
674+
None,
675+
&nested_metadata,
676+
)
677+
.unwrap_err();
678+
assert!(
679+
err.message()
680+
.contains("reserved for the OpenShell workspace")
681+
);
682+
641683
let nested_mount: DockerSandboxDriverConfig = serde_json::from_value(serde_json::json!({
642684
"mounts": [{"type": "tmpfs", "target": "/workspace/cache"}]
643685
}))

crates/openshell-driver-podman/README.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,10 @@ necessary. The supervisor transfers ownership of only the workspace directory
2323
to the completed identity before direct or SSH children start. Existing
2424
contents and nested bind or volume mounts retain their ownership. The workspace
2525
is the child cwd and `HOME`, and the Podman-managed workspace volume is mounted
26-
there so normal volume copy-up preserves image content. An invalid, symlinked,
27-
or unpreparable workspace fails sandbox startup.
26+
there so normal volume copy-up preserves image content. Protected system trees
27+
and OpenShell-reserved paths cannot become workspaces. The supervisor starts
28+
from `/`, then reports an invalid, symlinked, or unpreparable workspace as a
29+
readiness failure.
2830

2931
For a rootless networking deep dive, see [NETWORKING.md](NETWORKING.md).
3032

@@ -97,8 +99,8 @@ read-only by default; set `read_only: false` to make them writable. Podman
9799
image and volume mounts do not support `subpath` in OpenShell driver config.
98100
Mount `source` and `target` values must not contain surrounding whitespace.
99101
Mount targets must be absolute container paths and must not replace the
100-
resolved workspace root. Nested workspace mounts remain valid. Mounts also
101-
must not overlap OpenShell supervisor files, `/etc/openshell`,
102+
resolved workspace root or any of its parents. Nested workspace mounts remain
103+
valid. Mounts also must not overlap OpenShell supervisor files, `/etc/openshell`,
102104
`/etc/openshell-tls`, or `/run/netns`.
103105

104106
Example named-volume usage:

0 commit comments

Comments
 (0)