fix: prevent out-of-bounds read in UNICHAR::UTF8ToUTF32 - #4609
fix: prevent out-of-bounds read in UNICHAR::UTF8ToUTF32#4609SomSamantray wants to merge 4 commits into
Conversation
Adds UnicharTest.TruncatedUtf8 covering 2/3/4-byte truncated prefixes, a mid-string truncated prefix, and an illegal leading continuation byte. The test triggers the out-of-bounds read in UNICHAR::UTF8ToUTF32 on unfixed code (issue tesseract-ocr#4495).
utf8_step() reports the full multibyte width from the leading byte alone, so a truncated prefix (e.g. "\xE8\0") made the iterator read one byte past the end of the string. Clamp the per-character step to the remaining bytes and return an empty vector for truncated input. Fixes tesseract-ocr#4495.
Reword the comment to describe the rejection behavior accurately (reject a truncated trailing sequence) rather than "clamping", per code review feedback.
Up to standards ✅🟢 Issues
|
|
Please mention if you used an AI assistant in the commit messages. See Git history for examples how to do this. |
Already addressed in commit |
There was a problem hiding this comment.
Pull request overview
Prevents out-of-bounds reads when converting truncated UTF-8 sequences.
Changes:
- Rejects multibyte sequences exceeding the remaining input length.
- Adds regression tests for truncated and illegal UTF-8 prefixes.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/ccutil/unichar.cpp |
Adds bounds validation before decoding. |
unittest/unichar_test.cc |
Adds regression coverage for malformed UTF-8. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
UNICHAR::UTF8ToUTF32no longer reads past the end of a string that ends with a truncated multibyte UTF-8 prefix. Previously, a string like"\xE8\0"made the iterator advance past the NUL terminator and read one byte out of bounds (a stack/global buffer overflow depending on where the input lives). Such input now returns an empty vector, matching the documented "empty on invalid UTF-8" contract.The fix clamps the per-character step to the number of bytes remaining before the end of the string, rejecting a truncated trailing sequence before any dereference.
Validation
UnicharTest.TruncatedUtf8covering 2/3/4-byte truncated prefixes, a mid-string truncated prefix, and an illegal leading continuation byte.utf8_step) and passes after the fix.unichar_testpasses 3/3 in both ASan and non-ASan builds; relatednormstrngs_test(17 pass) andunicharset_test(4 pass) are unaffected under ASan.Fixes #4495