Skip to content

feat(doc-index): section-granularity inference, hashing, and caching fixes #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tkcoding wants to merge 4 commits into
base: main
Choose a base branch
from
+1,860 −3
Open

feat(doc-index): section-granularity inference, hashing, and caching fixes #109

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d5e2be7
feat(toc,doc-index): add JIT-retrieval readiness checks and cached do…
Aug 28, 2026
512975d
feat(doc-index): infer real section granularity, hash sections for st…
Aug 28, 2026
66b9cc2
fix(doc-index): resolve CodeRabbit review findings on PR #109
Aug 31, 2026
9cf51f4
fix(doc-index): resolve second CodeRabbit review round on PR #109
Aug 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -569,6 +569,10 @@ coverage.xml
**/.cache/decisions.jsonl.1
**/.cache/decisions.jsonl.lock

# doc-index cache (doc_index.py) — per-project structural index for JIT
# retrieval, read-once-per-file. Rebuilds automatically on content change.
**/.cache/doc-index/

# Superpowers brainstorming/planning specs (local-only)
docs/superpowers/

Expand Down
33 changes: 33 additions & 0 deletions architecture/features/traceability-validation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
- [List ID Kinds](#list-id-kinds)
- [Validate TOC](#validate-toc)
- [TOC Utilities](#toc-utilities)
- [Document Index](#document-index)
- [Markdown Parsing Utilities](#markdown-parsing-utilities)
- [Fixing Prompt Enrichment](#fixing-prompt-enrichment)
- [Headings Contract Validation](#headings-contract-validation)
Expand Down Expand Up @@ -416,6 +417,9 @@ Catches structural and traceability issues that AI agents miss or hallucinate
4. [x] - `p1` - Insert/update TOC using heading-based insertion (`## Table of Contents`) for kit file generator - `inst-toc-util-insert-heading`
5. [x] - `p1` - Process file: strip manual TOC, insert marker-based TOC, write if changed - `inst-toc-util-process-file`
6. [x] - `p1` - Validate TOC: check existence, anchor validity, completeness, staleness - `inst-toc-util-validate`
7. [x] - `p1` - Parse headings with line numbers, fence-aware (shared by doc-index and the JIT-retrieval readiness checks, which need section boundaries the plain heading list doesn't carry) - `inst-toc-util-parse-headings-lines`
8. [x] - `p1` - Collect JIT-retrieval readiness warnings for a document: gathers all four signals below over *every* heading level, independent of whatever level cap the caller configured for TOC-completeness checking - `inst-toc-jit-readiness-collect`
9. [x] - `p1` - Compute the four JIT-retrieval readiness signals -- duplicate heading titles, heading depth jumps, oversized sections (`--max-section-lines`), and a missing top-of-file description/frontmatter block -- all warning-only, never errors (see constructorfabric/studio#104) - `inst-toc-jit-readiness`

**Supporting**:
- [x] - `p1` - Imports, constants, fence tracking, GitHub anchor slug generation - `inst-toc-util-datamodel`
Expand All @@ -437,6 +441,35 @@ Catches structural and traceability issues that AI agents miss or hallucinate
- [x] - `p1` - Heading-based TOC new-insert branch: inject `## Table of Contents` before first heading when absent - `inst-toc-util-insert-heading-new`
- [x] - `p1` - TOC validate init: build heading list and expected TOC string before comparison checks - `inst-toc-util-validate-init`

### Document Index

- [x] `p1` - **ID**: `cpt-studio-algo-traceability-validation-doc-index`

**Input**: Markdown file path

**Output**: A structural index (every heading's line range, plus a coarser "one chunk per real section" grouping at an inferred heading level, each with a content hash and a summary slot) cached per file, keyed by a stat-based fingerprint

A cached, read-once-per-file structural index for Markdown JIT retrieval (see
constructorfabric/studio#104): parsing a file's headings/section boundaries
happens once, not once per query, until the file's content actually changes.
The cache-validity fingerprint is deliberately metadata-only (`mtime` + file
size via `Path.stat()`), never a content hash — the point of the cache is to
avoid reading the file at all on a hit, and a content hash would defeat that
by requiring the read it's meant to save.

1. [x] - `p1` - Build a fresh structural index: parse headings + line ranges from current content, compute the stat-based fingerprint - `inst-doc-index-build`
2. [x] - `p1` - Load a cached index for a file, validated against current stat metadata (no content read on a hit); returns `None` if missing, stale, or corrupt - `inst-doc-index-load`
3. [x] - `p1` - Persist an index to its cache location; no-ops silently outside a Studio-adapted project - `inst-doc-index-save`
4. [x] - `p1` - Return the cached index or build-and-cache a fresh one; reports cache hit/miss for benchmarking - `inst-doc-index-get-or-build`
5. [x] - `p1` - Attach a one-line, LLM-authored summary to a cached section by its `line_start`, for a future per-section-summary caller - `inst-doc-index-annotate`
6. [x] - `p1` - Infer which heading level represents one retrievable section: the most-recurring level wins over a level that appears only once (however shallow), since PDF-conversion heading levels don't reliably encode true nesting depth — a fixed level assumption silently produces a degenerate mega-section on such documents - `inst-doc-index-infer-level`
7. [x] - `p1` - Group headings at exactly the inferred level into retrieval sections (off-level headings stay inside whichever section they fall under, never split one apart); hash each section's own text for section-granularity staleness detection - `inst-doc-index-retrieval-sections`
8. [x] - `p1` - Diff the current file against its last cached build at section granularity: which retrieval sections are unchanged vs. changed, or whether the section count itself changed (a structural change, matched by position not heading text, since duplicate titles are real) - `inst-doc-index-diff-stale`

**Supporting**:
- [x] - `p1` - Stat-based cache-validity fingerprint (`mtime_ns` + size); resolved from the file's own path, never a content hash - `inst-doc-index-etag`
- [x] - `p1` - Resolve the cache file location within the Studio directory owning the indexed file, resolved from the file's own path (not the process's working directory) - `inst-doc-index-cache-path`

### Markdown Parsing Utilities

- [x] `p1` - **ID**: `cpt-studio-algo-traceability-validation-parsing-utils`
Expand Down
9 changes: 8 additions & 1 deletion skills/studio/scripts/studio/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,10 @@ def _cmd_eval(argv: List[str]) -> int:
from .commands.eval import cmd_eval
return cmd_eval(argv)

def _cmd_doc_index(argv: List[str]) -> int:
from .commands.doc_index import cmd_doc_index
return cmd_doc_index(argv)

# =============================================================================
# ADAPTER COMMAND
# =============================================================================
Expand Down Expand Up @@ -216,6 +220,7 @@ def _cmd_map(argv: List[str]) -> int:
"resolve-vars": "Resolve template variables to absolute paths",
"toc": "Generate/update Table of Contents",
"chunk-input": "Chunk oversized workflow input into line-bounded Markdown files",
"doc-index": "Build/reuse a cached heading index for a Markdown file (read once, not per query)",
"pdsl": "Validate PDSL prompt blocks",
"workspace-init": "Initialize multi-repo workspace",
"workspace-add": "Add a source to workspace config",
Expand All @@ -232,7 +237,7 @@ def _cmd_map(argv: List[str]) -> int:
("Validation", ["validate", "validate-kits", "validate-toc", "spec-coverage", "check-language"]),
("Search & Navigation", ["list-ids", "list-id-kinds", "get-content", "where-defined", "where-used"]),
("Kit Management", ["kit"]),
("Utility", ["toc", "chunk-input", "pdsl"]),
("Utility", ["toc", "chunk-input", "doc-index", "pdsl"]),
("Workspace", ["workspace-init", "workspace-add", "workspace-info", "workspace-sync"]),
("Delegation", ["delegate"]),
("Diagnostics", ["doctor"]),
Expand Down Expand Up @@ -263,6 +268,7 @@ def _cmd_map(argv: List[str]) -> int:
"validate-toc": "_cmd_validate_toc",
"spec-coverage": "_cmd_spec_coverage",
"chunk-input": "_cmd_chunk_input",
"doc-index": "_cmd_doc_index",
"workspace-init": "_cmd_workspace_init",
"workspace-add": "_cmd_workspace_add",
"workspace-info": "_cmd_workspace_info",
Expand Down Expand Up @@ -296,6 +302,7 @@ def _cmd_map(argv: List[str]) -> int:
_cmd_validate_toc,
_cmd_spec_coverage,
_cmd_chunk_input,
_cmd_doc_index,
_cmd_workspace_init,
_cmd_workspace_add,
_cmd_workspace_info,
Expand Down
64 changes: 64 additions & 0 deletions skills/studio/scripts/studio/commands/doc_index.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
"""Studio doc-index command — build/reuse a cached structural index for a
Markdown file, so heading-based JIT retrieval reads a file's structure once,
not once per query.

Thin CLI wrapper around ``studio.utils.doc_index``.

@cpt-flow:cpt-studio-flow-traceability-validation-validate:p1
"""

import argparse
from pathlib import Path
from typing import List

from ..utils.doc_index import get_or_build_doc_index
from ..utils.ui import ui


def cmd_doc_index(argv: List[str]) -> int:
"""Build (or reuse the cached) structural index for a Markdown file."""
p = argparse.ArgumentParser(
prog="cfs doc-index",
description=(
"Build or reuse a cached heading/section index for a Markdown file, "

@ainetx ainetx Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cfs doc-index --help never explains how section_level is inferred

Severity: Minor

Problem
cfs doc-index picks section_level via a frequency-based heuristic in infer_section_level(): the heading level used most often wins, levels appearing exactly once are excluded as candidates, and ties (or an all-singletons document) fall back to the shallowest level. This choice directly determines which lines get grouped into retrieval_sections in both JSON and human-readable output. None of this is mentioned anywhere a CLI user would see it.

How to reproduce
Run cfs doc-index --help (or read the argparse.ArgumentParser description in commands/doc_index.py). The description reads only: "Build or reuse a cached heading/section index for a Markdown file, so navigation reads the file's structure once, not once per query." No mention of section_level, how it's chosen, or that it can differ from the "obvious" chapter level in irregularly-leveled documents (the exact PDF-conversion scenario infer_section_level()'s own docstring cites as its motivating case).

Expected behavior
A user seeing "section_level": 5 (say) instead of the level they expected should have a way, via --help or the command's own output, to understand that the level was inferred from heading-frequency, not simply "H1/H2 is always the section level."

Actual behavior
--help gives no indication that section-level selection is heuristic at all.

Impact
Minor UX/discoverability gap — doesn't cause incorrect behavior, but makes an already-nontrivial, silently-applied heuristic harder to audit or trust from the CLI alone.

Suggested correction
Extend the argparse.ArgumentParser(description=...) in cmd_doc_index() (or add a short note to _human_doc_index()'s output) with one sentence, e.g.: "section_level is inferred from the most frequently repeated heading level (ties prefer the shallower level); a level used only once is never chosen."

How to verify
Run cfs doc-index --help after the fix and confirm the heuristic is described.

"so navigation reads the file's structure once, not once per query."
),
)
p.add_argument("file", help="Markdown file path")
p.add_argument(
"--rebuild",
action="store_true",
help="Force a fresh build even if a valid cached index exists",
)
args = p.parse_args(argv)

filepath = Path(args.file).resolve()
if not filepath.is_file():
ui.result(
{"file": str(filepath), "status": "ERROR", "message": "File not found"},
human_fn=lambda d: ui.error(f"{d['file']}: {d['message']}"),
)
return 2

index = get_or_build_doc_index(filepath, force_rebuild=args.rebuild)

output = {
"file": str(filepath),
"cache_hit": index["cache_hit"],
"total_lines": index["total_lines"],
"section_count": len(index["sections"]),
"sections": index["sections"],
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
ui.result(output, human_fn=_human_doc_index)
return 0


def _human_doc_index(data: dict) -> None:
ui.header("Doc Index")
hit = "cache hit — reused existing index" if data["cache_hit"] else "cache miss — built fresh index"
ui.substep(hit)
ui.substep(f"{data['section_count']} section(s), {data['total_lines']} total lines")
for s in data["sections"]:
summary = f" — {s['summary']}" if s.get("summary") else ""
ui.substep(f" H{s['level']} [{s['line_start']}-{s['line_end']}] {s['heading']}{summary}")
ui.blank()
9 changes: 8 additions & 1 deletion skills/studio/scripts/studio/commands/validate_toc.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
from pathlib import Path
from typing import List

from ..utils.toc import add_toc_max_level_argument, validate_toc
from ..utils.toc import DEFAULT_MAX_SECTION_LINES, add_toc_max_level_argument, validate_toc
from ..utils.ui import ui
# @cpt-end:cpt-studio-algo-traceability-validation-validate-toc:p1:inst-toc-imports

Expand All @@ -31,6 +31,12 @@ def cmd_validate_toc(argv: List[str]) -> int:
help="Markdown file path(s) to validate",
)
add_toc_max_level_argument(p)
p.add_argument(
"--max-section-lines",
type=int,
default=DEFAULT_MAX_SECTION_LINES,
help=f"Warn when a section exceeds this many lines (default: {DEFAULT_MAX_SECTION_LINES})",
)
p.add_argument(
"--verbose",
action="store_true",
Expand Down Expand Up @@ -63,6 +69,7 @@ def cmd_validate_toc(argv: List[str]) -> int:
content,
artifact_path=filepath,
max_heading_level=args.max_level,
max_section_lines=args.max_section_lines,

@ainetx ainetx Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmd_validate_toc aborts the entire multi-file validation batch on any unhandled per-file read error, discarding already-collected results

Severity: Minor-to-moderate

Problem
In skills/studio/scripts/studio/commands/validate_toc.py::cmd_validate_toc, the per-file loop only guards the "file not found" case (if not filepath.is_file()). The actual read, content = filepath.read_text(encoding="utf-8"), has no exception handling. Any other read failure on any file in the batch — permission denied, non-UTF-8/binary content, or a TOCTOU race — raises uncaught out of the loop, aborting the command before it ever reaches ui.result(...). This discards results already accumulated for every file validated earlier in the same invocation. Separately, FILE_READ_ERROR/FILE_LOAD_ERROR codes already exist in utils/error_codes.py but are unused here (error_codes isn't even imported in validate_toc.py).

How to reproduce

# files = [good.md, bad.md, good2.md]; bad.md has invalid UTF-8 bytes or revoked permissions
cmd_validate_toc([str(good), str(bad), str(good2)])
# raises uncaught after good.md was already validated and appended to `results` -- that
# result is lost, and good2.md is never reached

Expected behavior
A read failure on one file in a cfs validate-toc a.md b.md c.md batch should be recorded as a per-file ERROR result (using FILE_READ_ERROR/FILE_LOAD_ERROR), consistent with the existing "File not found" handling, and the batch should continue validating the remaining files.

Actual behavior
An unhandled exception propagates out of the loop, the command crashes with a raw traceback, and results already computed for earlier files are never emitted.

Impact
In CI or batch-validation contexts, a single bad file (encoding issue, permission glitch, or a file removed mid-run) silently destroys the whole run's output rather than surfacing that one file's problem alongside the rest.

Suggested correction
Wrap the filepath.read_text(...) call in a try/except (OSError, UnicodeDecodeError) inside the loop, append an ERROR entry to results (using error_codes.FILE_READ_ERROR/FILE_LOAD_ERROR) analogous to the existing "File not found" branch, increment total_errors, and continue to the next file.

How to verify
Add a regression test with a batch of 2+ files where one has invalid UTF-8; assert the command still emits results for the other files, marks the bad file as an ERROR with a stable error code, and does not raise.

)

errors = report.get("errors", [])
Expand Down
Loading
Loading