Repository navigation
fix(lib): concat list entries broke on relative paths and quotes in paths - #673
Conversation
…aths
concatenate_video_files writes an ffmpeg concat demuxer list file into
the system temp directory, with entries written verbatim as
"file '<path>'". Two failure modes:
1. Relative input paths: the concat demuxer resolves relative entries
against the directory of the LIST FILE (the temp folder), not the
caller's working directory, so every relative path became a dangling
reference and av.open failed with FileNotFoundError on a file that
exists.
2. Quotes in paths: a single quote in any component ("John's slides")
terminated the quoted entry early and made it unparseable.
Fix both in one place: entries are written as absolute (resolved)
paths, with single quotes escaped shell-style ('\''), which is the
quoting ffmpeg's av_get_token() understands.
Adds regression tests for both cases (relative CWD-dependent input,
and a path containing a single quote), asserting a decodable output.
The helper moved to module level to keep ruff's complexity limit happy.
Not up to standards ⛔🔴 Issues
|
| Category | Results |
|---|---|
| Security | 2 high |
| CodeStyle | 1 minor |
🟢 Metrics 0 duplication
Metric Results Duplication 0
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #673 +/- ##
=======================================
Coverage 79.10% 79.10%
=======================================
Files 23 23
Lines 2178 2178
=======================================
Hits 1723 1723
Misses 455 455 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Addresses review feedback on jeertmans#673. I tested the suggested shlex.quote form empirically and it unfortunately can't be used here: shlex quotes for POSIX shells (' -> '"'"'), while ffmpeg's concat demuxer parses entries with av_get_token, which only understands backslash escapes and '\'' -style quoting — a shlex-quoted path containing a single quote fails to parse (details in the PR thread). Kept the av_get_token-compatible escaping, and switched str(resolve()) to resolve().as_posix() from the suggestion.
|
Thanks for the review! I applied the CHANGELOG suggestion and switched to On # path: /tmp/x/John's slides/video.mp4
f"file '{quote(path)}'\n" # -> parse fails, FileNotFoundError
f"file {quote(path)}\n" # -> parse fails too
"file '" + path.replace("'", "'\\''") + "'\n" # -> works(The doubled quoting in the first form also breaks plain paths only when they contain quotes, so the failure is silent until a user has a So I kept the |
|
Thanks for checking, see my latest #673 (comment) |
|
Thanks @Agi-Asi! A typo in one of the comments has been reported by the pre-commit codspell check, could you fix it? |
|
Note on CI: the remaining |
|
pre-commit.ci run |
|
Thanks for your contribution @Agi-Asi! |
…aths (jeertmans#673) * fix(lib): concat list entries broke on relative paths and quotes in paths concatenate_video_files writes an ffmpeg concat demuxer list file into the system temp directory, with entries written verbatim as "file '<path>'". Two failure modes: 1. Relative input paths: the concat demuxer resolves relative entries against the directory of the LIST FILE (the temp folder), not the caller's working directory, so every relative path became a dangling reference and av.open failed with FileNotFoundError on a file that exists. 2. Quotes in paths: a single quote in any component ("John's slides") terminated the quoted entry early and made it unparseable. Fix both in one place: entries are written as absolute (resolved) paths, with single quotes escaped shell-style ('\''), which is the quoting ffmpeg's av_get_token() understands. Adds regression tests for both cases (relative CWD-dependent input, and a path containing a single quote), asserting a decodable output. The helper moved to module level to keep ruff's complexity limit happy. * chore(changelog): add entry for jeertmans#673 * review: use as_posix() and apply changelog suggestion Addresses review feedback on jeertmans#673. I tested the suggested shlex.quote form empirically and it unfortunately can't be used here: shlex quotes for POSIX shells (' -> '"'"'), while ffmpeg's concat demuxer parses entries with av_get_token, which only understands backslash escapes and '\'' -style quoting — a shlex-quoted path containing a single quote fails to parse (details in the PR thread). Kept the av_get_token-compatible escaping, and switched str(resolve()) to resolve().as_posix() from the suggestion. * review: inline escaping into concatenate_video_files * chore: fix codespell finding (unparseable -> unparsable) * Apply suggestion from @jeertmans --------- Co-authored-by: Agi-Asi <206806952+Agi-Asi@users.noreply.github.com> Co-authored-by: Jérome Eertmans <jeertmans@icloud.com>
Description
concatenate_video_fileswrites an ffmpeg concat demuxer list file into the system temp directory, with entries written verbatim asfile '<path>'. Two failure modes fall out of that:Relative input paths silently break. The concat demuxer resolves relative entries against the directory of the list file (the temp folder), not the caller's working directory — so every relative segment path became a dangling reference and
av.openfailed withFileNotFoundErroron files that exist. Reproduced locally by callingconcatenate_video_files([Path('tests/data/video.mp4'), ...])from the repo root.Quotes in paths make entries unparseable. A single quote anywhere in a component (
John's slides/…) terminates the quoted entry early. Any user whose project lives under such a directory gets a confusing parse/FileNotFoundErrorfailure from deep inside PyAV.Fix, in one place (
_concat_list_entry): entries are written as absolute (resolved) paths, with single quotes escaped shell-style ('\'') — the quoting ffmpeg'sav_get_token()understands.Today's internal callers happen to always pass absolute paths, which is why the pipeline works — but
concatenate_video_filesis part of the importable API surface, and both failure modes produce errors that look exactly like the "file vanished mid-render" class of reports (e.g. theErrno 22 Invalid argumentfamily in #540 is being debugged around this same function). Hardening the list-file writing removes two entire failure modes from the suspect list.Testing
monkeypatch.chdir), and a path containing a single quote — both assert a decodable output. Both fail onmainand pass with this change.pytest tests/test_utils.py— 4 passed.ruff check/ruff format --checkclean (the helper moved to module level to stay under the complexity limit).