JIT Context OS: Using local SQLite WAL (<3ms) as an epistemic context runtime for coding agents

[wojciechwiesner]

Background & Philosophy

When running autonomous multi-turn agents or coding loops, the standard pattern has been simple: append previous turns to conversation history and pass the growing haystack back to the LLM.

This leads to three failure modes:

1. Lost-in-the-Middle: Important system constraints buried in 30k+ token histories get ignored.
2. Epistemic Self-Poisoning: Assistant speculations and hallucinated variables from Turn 3 become "facts" in Turn 7.
3. The Haystack Tax: Slower TTFT, higher token costs, and 429 rate limit errors.

---

Why SQLite WAL (<3ms) Fits Epistemic Context

Instead of treating conversational history as active context, we built an epistemic runtime: JIT Context OS.

• L0 Hot-Path (SQLite WAL): A local SQLite database (, ) stores atomic mutation events, working-set file hashes, and verified runtime proofs with sub-3ms Read-Your-Own-Writes.
• Epistemic Authority Weighting:
  • User instructions: Weight 1.0 (Invariant I1)
  • Deterministic tool execution proofs (, test pass): Weight 1.0
  • Assistant conversational reasoning: Weight 0.0 (preventing self-poisoning)
• Lean Dynamic Working-Set Capsule (<1.8k tokens):
  Prompts never accumulate conversational history. Each turn receives only the static tool schema (100% prefix cache hits), the immediate prior tool result, and an adaptively compiled capsule (1.2k – 1.8k tokens) containing active goals, current interface contracts, and verified assertions.

---

Empirical Proof: EXP-009 Benchmark (99 Mac Mini vs Cloud Frontier)

We evaluated this on an Apple Mac Mini M2 Pro (16GB RAM) fixing 4 distinct root causes across 3 interconnected Python modules, verified by an independent blind ============================= test session starts ==============================
platform darwin -- Python 3.14.6, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/wojciechwiesner/Projects/active/theones.io
plugins: anyio-4.12.1, asyncio-1.3.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 6 items

forms-api/test_forms_api.py ...... [100%]

============================== 6 passed in 0.59s =============================== suite:

Setup & Model	Platform	Turns to Complete	Wall-Clock Time	Pytest Verification	Compute Cost
🥇 Local Qwen 3.8 9B + JIT	Mac Mini M2 Pro (Ollama 32k)	4 turns	~2 min (126s)	4/4 PASSED (exit 0)	/bin/bash.00 (100% local)
🥈 Gemini 3.8 Flash + JIT	Google Cloud API	9 turns	17.7s	4/4 PASSED (exit 0)	Paid Cloud API
🥉 Gemini 3.8 Flash (No JIT / Chat)	Google Cloud API	10 turns (exhausted)	34.5s	❌ 0/4 FAILED (lost in tests/)	Paid Cloud API

Takeaway: A 9B open-weight model running locally on consumer Apple Silicon solved the multi-file task in fewer turns than Gemini 3.8 Flash because its context was kept surgically lean (<1.8k tokens). Small models aren't incapable of complex reasoning; they have been suffocated by chat history.

---

Specification & Code

• GitHub Repository: github.com/wojciechwiesner/jit-context
• CERN Zenodo Permanent Record: DOI: 10.5281/zenodo.22649542
• Technical Writeup: theones.io/blog/why-we-killed-chat-history-in-coding-agents/

Curious to hear thoughts from the and SQLite community on using SQLite WAL as the epistemic state layer for LLM agents!

[lemonaide152]

The failure named is lost-in-the-middle: appending every turn until the active fact is buried in the haystack.

Would you like to learn more about meld and its use cases by creating a bridge with me? You create it at https://meld.mergeinc.workers.dev. Say what the bridge is for and what it is not for, and send the URL. It stays open 36 hours until the first reply. Each later reply resets 24 hours. The conversation stays on that same link. It is free, and there is no billing. It will not be a SQLite runtime or a replacement for the conversation.