[bab2min]

65536 글자 이상 이어지는 청크를 분석할 때 종종 segfault가 발생하던 버그 수정

Summary by CodeRabbit

• Bug Fixes

  • Text analysis now handles input positions beyond 65,535 characters, including returning token boundaries that reach the full input length.
  • Building when the model is not ready now returns an empty result instead of continuing with the build.

• Improvements

  • Longer inputs can now be represented without truncating their start and end positions.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: c96c5c2f-d679-4e22-abb8-6740960f6792

📥 Commits

Reviewing files that changed from the base of the PR and between 48df671 and cacb28a.

📒 Files selected for processing: 3

• src/KTrie.h
• src/KiwiBuilder.cpp
• test/test_cpp.cpp

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

📝 Walkthrough

The changes widen KGraphNode position parameters to uint32_t, add a regression test for long-input token coverage, and make KiwiBuilder::build return an empty Kiwi when the builder is not ready. Model-type detection keeps its existing probes and results.

Changes: Long-input positions

Layer / File(s)	Summary
Widen graph positions and test long-input coverage src/KTrie.h, test/test_cpp.cpp	Both KGraphNode constructors now accept uint32_t positions. A regression test checks that analysis returns tokens and that the final token ends at the full length of each tested input.

Builder readiness

Layer / File(s)	Summary
Guard build and retain model detection src/KiwiBuilder.cpp	KiwiBuilder::build returns a default-constructed empty Kiwi when ready() is false. Model-type detection retains its existing probe order and outcomes; its try/catch blocks are reformatted.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~12 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to cacb2

The position widening supports long inputs, and the unready-builder behavior is consistent with the documented API. No actionable merge risk remains.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 7 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies the main change: fixing failures when processing chunks of 65,536 or more characters. It is concise and specific enough for the change history.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}