feat(agent): read chat attachments on demand

[lyingbug]

Description

Agent conversations expose uploaded attachments through compact manifests and read_file at attachment:// paths. Small ready files remain inline; larger files can be paged or searched on demand, including during history replay. Reading and image-observation reuse work independently of a sandbox.

• Read accepted parsed attachments beyond the sandbox's 8 MiB download limit while respecting per-result byte and output budgets. Unicode-safe next_offset / next_line_offset cursors cover oversized lines, expired excerpts, and staged copies. Reserve metadata space before pagination so registry truncation cannot skip content.
• Default image reads to one image and brief plus focus. Guidance reserves full for an explicit close-reading request and caps full reads at one image; explicit brief batches are capped at four. Shorten the shared VLM prompts and preserve partial failures and continuation.
• For non-vision chat models, persist successful image observations with the document. Cache identity includes the prepared-image hash, model ID, and normalized prompt/detail/focus. Equivalent reads reuse observations across turns and authorized forks. Failed or empty observations are retried on a later read, without a duplicate engine fallback in the same call. New descriptions have a 30-second deadline and bounded concurrency.
• Store observations in existing document metadata with optimistic updates that preserve parser metadata and concurrent writes. Bound the cache to 64 entries and 1 MiB per document; retention and deletion follow the document lifecycle. No database schema migration is needed for this cache.
• Reconcile originals and ready parsed text at turn start. A successful read immediately syncs newly-ready parsed text, requested images, and available observations into the optional sandbox. Observations record source, detail, focus, model, and time. Later staging preserves these copies; a read restores them after a sandbox rebuild. Projection failures leave durable reads usable.
• Native vision models receive image pixels directly and do not incur an extra VLM call to manufacture a cached description. Quick Answer includes standalone image uploads; Agent reads extracted PDF/Office figures explicitly. Image bytes are omitted from persisted file-tool results.
• Retain originals, parsed content, and extracted images while live conversations reference them, including forks after parent deletion. New objects avoid provider-expiring temporary buckets; indexed reference checks leave unreferenced uploads eligible for cleanup.

Architecture

application/attachment owns session-authorized reads, parse waits, image preparation, observation reuse, and optional projection coordination. read_file consumes interfaces.AttachmentReader and handles URI routing, pagination, and presentation. Repository and sandbox adapters provide persistence and copies; the Agent engine handles model transport and generic image fallback.

Split upload/lifecycle handling, parsing/OCR/chunking, and prompt assembly into separate service files. Document responsibilities and runtime flow in internal/application/attachment/README.md.

Validation

Passed:

• Relevant suites: go test ./internal/application/attachment ./internal/application/repository ./internal/agent/... ./internal/application/service ./internal/types ./internal/handler/session ./internal/models/chat ./internal/modelcontext ./internal/im ./internal/database
• Final affected-interface, tool, and service checks after the last changes.
• Focused attachment/cache tests with -race.
• go build ./..., go vet ./..., and diff whitespace checks.
• golangci-lint run --new-from-rev=ef54e835c907718cedf7316bfa93daf69a28b2ba ./... — 0 issues.

Regression coverage includes 9 MiB paging/search, lossless multilingual long-line continuation, output-budget reconstruction, bounded image reads, authorization before storage access, cross-reader cache reuse, detail/focus/model/image invalidation, failed-description retry, concurrent cache updates, tenant isolation, document deletion, same-turn parsed-file synchronization, and sandbox reconstruction without another VLM call. The original three pagination regressions failed against the previous PR head before their fixes.

Tests run with the local LOG_FORMAT override removed because it affects existing logging assertions. PostgreSQL/MySQL integration, frontend checks, the full repository test suite, and live-model conversations were not rerun for this update. Earlier PR validation covered retention and index migrations on PostgreSQL 17.

Compatibility notes

Existing originals and images in legacy temporary buckets are not migrated; provider lifecycle deletion can still make them unavailable. Uploads and reads continue to use the default file service. This change does not add tenant-specific storage routing.

Observation reuse requires the same model ID and normalized reading prompt; it does not infer semantic equivalence between different focus strings. Native-vision reasoning is not persisted as a separate attachment observation.