Skip to content

Commit bc39671

Browse files
committed
docs: plan optional search and OpenBao-sourced project secrets
1 parent 586fdd2 commit bc39671

52 files changed

Lines changed: 2118 additions & 1530 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,12 @@
77
- Adapted Eth/Brynja repository files, dependency/CI/freshness controls and
88
executable file-size/documentation/graph gates.
99
- Added automated PostgreSQL 19 beta 4, TLS OpenBao and ACL Valkey Podman tests.
10-
- Preserved the idea/planning bundle and defined 363 pre-1.0 passes, including
10+
- Preserved the idea/planning bundle and defined 371 pre-1.0 passes, including
1111
service lifecycle, provider feasibility, parity and operational qualification.
12+
- Revised planned search to early portable contracts and mandatory qualification
13+
of repository/optional Meilisearch backends alongside hosted persistence.
14+
- Required OpenBao as the source for initialization/runtime/private-build/release
15+
secrets; made local-password-first fixture remediation the next bounded pass.
1216

1317
No product parity or production readiness is claimed. Pentest, tagging and
1418
publication are pending. See release-notes/v0.1.0.md.

‎README.md‎

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ service-test tooling. No workbench, transformation operation, browser UI,
2828
production HTTP server or CyberChef parity is implemented yet. `0.1.0` is an
2929
unpublished foundation candidate, with pentest pending.
3030

31-
The [release plan](docs/RELEASE_PLAN.md) defines 363 small pre-1.0 passes,
32-
through `0.363.0`, with further versions whenever needed. `1.0.0` is the first
31+
The [release plan](docs/RELEASE_PLAN.md) defines 371 small pre-1.0 passes,
32+
through `0.371.0`, with further versions whenever needed. `1.0.0` is the first
3333
serious production release with complete declared website/API functionality.
3434
Desktop/mobile GUIs follow afterward.
3535

@@ -58,6 +58,10 @@ Dependencies run in rootless Podman: PostgreSQL **19 beta 4**, OpenBao **2.7.1**
5858
and Valkey **9.1.2**. State and credentials stay in ignored `.local/stack`.
5959
OpenBao is initialized over TLS with declarative audit, KV v2, scoped AppRole
6060
and revoked bootstrap root token. See [local stack](docs/local-stack.md).
61+
The next pass replaces locally generated fixture passwords with OpenBao-first
62+
provisioning. All project-operated secrets, including initialization/private
63+
build/release credentials, must come through OpenBao; public Rust builds need
64+
none. See [secret lifecycle](docs/SECRETS_POLICY.md) for bootstrap custody.
6165

6266
## Workspace
6367

@@ -91,8 +95,11 @@ are retained as design inputs. Their historical compiler/provider statements
9195
are not current implementation evidence.
9296

9397
Operation search will work locally over descriptors. Saved-recipe metadata
94-
search starts behind a portable repository boundary. Meilisearch is optional
95-
if measured requirements justify it; payloads and secrets are excluded.
98+
search has an early portable SearchService contract and two planned backends:
99+
repository search and optional Meilisearch. Both will be implemented/tested with
100+
hosted persistence; operators may enable or disable Meilisearch without changing
101+
the UI/API or recipe schema. Payloads/secrets are excluded and current database
102+
permissions govern results. See [search design](docs/SEARCH_DESIGN.md).
96103

97104
## License
98105

‎SECURITY.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,11 @@ Production runtime processes must never hold bootstrap root tokens or OpenBao
2222
recovery shares. Secrets do not belong in URLs, logs, arguments, payload caches
2323
or search indexes. Local .local/stack custody shortcuts are only test fixtures
2424
and have no production support claim.
25+
26+
All project-operated initialization, runtime, build and release secrets must
27+
come through OpenBao; public Rust setup/build needs none. Vault startup trust
28+
and independent recovery custody are the minimal explicit bootstrap boundary.
29+
The current fixture generates passwords locally before seeding the vault and
30+
requires the next planned remediation pass. See [secret lifecycle](docs/SECRETS_POLICY.md).
31+
Search is optional infrastructure; current database permissions govern hits
32+
and aggregates in both backends. See [search design](docs/SEARCH_DESIGN.md).

‎docs/ARCHITECTURE.md‎

Lines changed: 25 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,9 @@ flowchart TD
2424
Ports --> Artifacts[Staged artifact storage]
2525
Ports --> Secrets[OpenBao SDK adapter]
2626
Ports --> Cache[Optional Valkey cache]
27+
Ports --> Search[SearchService]
28+
Search --> Repository[Repository metadata search]
29+
Search --> Meili[Optional Meilisearch adapter]
2730
```
2831

2932
Core types never expose database rows, Tokio handles, framework requests,
@@ -71,24 +74,38 @@ roles and portable repository methods. Durable metadata is separate from bulk
7174
artifacts. Real MySQL conformance and bidirectional logical migration prove the
7275
seam before 1.0; an in-memory fake cannot supply that proof.
7376

74-
OpenBao 2.7.1 supplies secrets. The current dev harness fully initializes it over
75-
TLS, with declarative audit, KV v2, AppRole and revoked bootstrap root. Application
76-
integration later admits current stable openbao SDK features. Runtime processes
77+
OpenBao 2.7.1 is the required source for every project-operated initialization,
78+
runtime, private-build and release secret. Public Rust builds need no credentials.
79+
The current dev harness initializes it over TLS, with declarative audit, KV v2,
80+
AppRole and revoked bootstrap root, but generates service passwords locally
81+
first. The next pass replaces that sequence with Bao-first provisioning and
82+
vault-sourced credentials. Early application-contract passes admit the current
83+
stable openbao SDK and qualify SecretRef/rotation. Runtime processes
7784
never receive recovery shares/root privileges. Production TLS, identity,
7885
renewal, rotation, dynamic DB leases, audit failure and recovery custody require
7986
separate qualification. Single-share local custody is strictly a test shortcut.
87+
Vault startup trust and independent recovery custody are the minimal explicit
88+
bootstrap boundary; they cannot depend solely on a sealed vault. Follow
89+
[secret lifecycle](SECRETS_POLICY.md) for scoped delivery and failure behavior.
8090

8191
Valkey 9.1.2 is optional cache acceleration, with scoped ACLs, TTL and byte limits.
8292
Only completed deterministic nonsensitive results or bounded metadata are eligible.
8393
Cache identity includes semantic/provider/data revisions and canonical framing;
8494
it is partition-independent and workspace scoped. Permission checks use
8595
current authority. Cold cache/outage/eviction cannot lose authoritative data.
8696

87-
Operation search uses local descriptors. Saved-recipe search starts in repository
88-
adapters with permission filtering. Meilisearch is a conditional service after
89-
benchmarks establish a need. Its index must be rebuildable from metadata via an
90-
outbox; revocation/deletion and tenant isolation are mandatory. Inputs, decrypted
91-
outputs, secrets and run payloads never enter an index.
97+
Operation and browser-local recipe search stay local. Early SearchService
98+
contracts own saved-server-metadata queries; implement both repository and
99+
optional Meilisearch adapters with hosted persistence/authorization. A std-only
100+
feature and runtime selector preserve the same UI/API/domain schema with the
101+
service enabled or disabled. Measurements guide deployment and limits.
102+
103+
Project only approved nonsensitive metadata; inputs, outputs, operation arguments
104+
and secrets never enter an index. Transactional revisioned outbox/replay makes
105+
it rebuildable. Tenant filters plus current database permission/revision checks
106+
govern every returned hit and visible aggregate; asynchronous index grants are
107+
never authority. Meilisearch startup/scoped keys come through OpenBao. See
108+
[search design](SEARCH_DESIGN.md) for switching, outages and qualification.
92109

93110
## Future extraction
94111

‎docs/IMPLEMENTATION_PLAN.md‎

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@ transformations execute in a real Dedicated Worker in their browser without
88
an account, database or payload upload. Explicit remote execution uses isolated
99
Linux workers. Saving and sharing are independent permissions.
1010

11-
The [release plan](RELEASE_PLAN.md) assigns 363 bounded passes through
12-
0.363.0; additional passes close every mandatory gap before 1.0. The original
11+
The [release plan](RELEASE_PLAN.md) assigns 371 bounded passes through
12+
0.371.0; additional passes close every mandatory gap before 1.0. The original
1313
240-pass bundle remains preserved in [reference](reference/workbench-plan/README.md).
1414
Every original source version maps to one or more Runasmidja owners in phase
1515
JSON records. Multi-algorithm work is split rather than hidden in a broad title.
@@ -30,6 +30,10 @@ JSON records. Multi-algorithm work is split rather than hidden in a broad title.
3030
Qualify maintained providers for cryptography, compression and complex parsing;
3131
reducing dependency count must not override correctness or security.
3232
- Keep framework, database, SDK, TLS and browser types outside portable APIs.
33+
- Source every project-operated secret from OpenBao, including initial service,
34+
private Rust/build, CI and release credentials. Public builds need no secrets.
35+
Minimal vault startup/recovery trust has separate enumerated custody; see
36+
[secret lifecycle](SECRETS_POLICY.md). The current fixture needs remediation.
3337
- Test every admitted behavior, malformed path, resource limit and lifecycle.
3438
Tests, security docs and release notes arrive in the same implementation pass.
3539
- EUPL-1.2 applies to project code. External asset/code rights are reviewed
@@ -56,6 +60,7 @@ Extract the following only when real behavior needs ownership:
5660
| runasmidja-postgres / runasmidja-mysql | Backend-owned migrations and repository conformance | std adapters |
5761
| runasmidja-openbao | Latest reviewed openbao SDK, AppRole and leases | std adapter |
5862
| runasmidja-valkey | Optional bounded cache and invalidation | std adapter |
63+
| runasmidja-search-meilisearch | Optional saved-metadata search and outbox consumer | std adapter |
5964
| runasmidja-browser / runasmidja-web | Wasm worker bindings and accessible Rust DOM UI | browser host |
6065
| runasmidja-server / runasmidja-worker | Control plane and isolated native execution | Linux/std |
6166

@@ -66,17 +71,29 @@ provider footprint or review size justify separation. `lib.rs` exports;
6671
## Implementation order
6772

6873
Repository and container fixtures come first so every later hosted feature can
69-
test itself. Then prove one transformation on native and actual browser hosts,
74+
test itself. The next bounded pass is OpenBao-first secret provisioning: vault
75+
initialization precedes PostgreSQL/Valkey credentials and startup. Qualify private
76+
temporary delivery and build/release identity separately. Then prove one
77+
transformation on native and actual browser hosts,
7078
pin the complete CyberChef inventory and establish stable descriptors/contracts.
79+
Immediately after application contracts, qualify OpenBao SDK/SecretRef/rotation
80+
and define SearchService, independent of browser, database or provider types.
7181
Investigate regex dialects, query languages, YARA, crypto, compression,
7282
disassembly and OCR in separate feasibility passes before assuming providers.
7383

7484
Common encodings precede execution stress, persistent artifacts and advanced
7585
viewers. Optional hosted persistence/API work follows local contracts and adds
76-
real PostgreSQL repositories, OpenBao SDK/rotation/database leases, Valkey
86+
real PostgreSQL repositories, OpenBao database leases, Valkey
7787
failure behavior, authentication, isolation and quotas. Production service TLS,
7888
recovery custody and PostgreSQL beta-to-GA migration have explicit owner passes.
7989

90+
Implement repository search, transactional projection/outbox, the optional
91+
Meilisearch fixture/adapter, live authorization rechecks and backend-switching
92+
qualification immediately after hosted persistence and workspace authorization.
93+
Both backends are required implementations/tests before 1.0; Meilisearch remains
94+
optional to deploy. Local operation/recipe search stays local. Later shared
95+
collections extend the same contract. See [search design](SEARCH_DESIGN.md).
96+
8097
Structured formats, archives, modern and historical cryptography, public keys,
8198
forensics and media follow in narrow operation/provider passes. Full recipe
8299
control flow includes backward jumps and bounded compatibility execution;

‎docs/RELEASE_PLAN.md‎

Lines changed: 26 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Status: planned; workspace initialized, no release tagged.
44

5-
363 small pre-1.0 passes, starting at 0.1.0 and ending at 0.363.0. Add further minors whenever inventory, provider work or qualification needs a smaller pass. Version 1.0.0 is the first serious production release.
5+
371 small pre-1.0 passes, starting at 0.1.0 and ending at 0.371.0. Add further minors whenever inventory, provider work or qualification needs a smaller pass. Version 1.0.0 is the first serious production release.
66

77
The supplied 240-release bundle is preserved under [reference](reference/workbench-plan/README.md). Runasmidja adds operational services and splits multi-provider/algorithm work; source-version mappings preserve every original workstream. Nothing in this plan claims an implementation or a completed pentest.
88

@@ -11,6 +11,8 @@ The supplied 240-release bundle is preserved under [reference](reference/workben
1111
- Run Rust 1.99.0 initially; review official stable, crate, tool, action and service metadata weekly and before changes.
1212
- Before each pass, write its exact API/operation/argument scope, target profile, numeric resource ceilings and test IDs. One new algorithm, dialect, persistent contract or trust boundary per pass.
1313
- All code files stay at or below 500 lines; focused crates and independently tested adapters preserve no_std and future Vef/Brynja extraction.
14+
- OpenBao is the source for every project-operated initialization, runtime, build and release secret; consumers start only after scoped retrieval. Minimal vault bootstrap/recovery trust has separate custody; public Rust builds and browser-local user inputs require no vault.
15+
- SearchService is an early portable contract. Repository search and optional Meilisearch are required implementation/test profiles before production; optional deployment is not deferred implementation. Index only approved nonsensitive metadata and recheck current database authorization.
1416
- A family row is an inventory owner. If it contains independent algorithms or dialects after source reconciliation, split it into additional numbered passes before coding; never hide feature work in a patch.
1517
- The predecessor is the baseline. A later capability is never assumed available; move or split the consumer if a concrete prerequisite is discovered.
1618
- Imported operation names remain provisional until the immutable CyberChef inventory confirms exact variants and redistribution rights. Unsupported required variants remain blocking gaps.
@@ -21,35 +23,39 @@ Run `scripts/checks.sh`, current dependency/license/advisory checks, freshness,
2123

2224
The [release runbook](RELEASE_RUNBOOK.md) and [version policy](VERSIONING_POLICY.md) define the handoff. Tagging/publication is separate from this setup task.
2325

26+
The [search design](SEARCH_DESIGN.md) and [secret lifecycle](SECRETS_POLICY.md) define required trust boundaries. The next bounded implementation pass is OpenBao-first secret provisioning; current fixture passwords are still locally generated and do not meet that new origin policy.
27+
28+
The [2026-10-03 planning revision](plan-revision-2026-10-03.md) records moved owners and qualification limits. Unpublished version assignments changed; the supplied source-version mapping remains intact.
29+
2430
## Per-version handoffs
2531

2632
| Phase | Versions | Detailed handoffs |
2733
| --- | --- | --- |
28-
| Z: Repository and service foundation | 0.1.0–0.6.0 | [Milestones](releases/phase-z.md) |
29-
| A: Foundation and useful vertical slice | 0.7.0–0.27.0 | [Milestones](releases/phase-a.md) |
30-
| B: Bytes, text and foundational encodings | 0.28.0–0.57.0 | [Milestones](releases/phase-b.md) |
31-
| C: Streaming, artifacts and execution foundations | 0.58.0–0.72.0 | [Milestones](releases/phase-c.md) |
32-
| D: Modern browser workbench | 0.73.0–0.87.0 | [Milestones](releases/phase-d.md) |
33-
| E: Remote API, PostgreSQL and secure server execution | 0.88.0–0.111.0 | [Milestones](releases/phase-e.md) |
34-
| F: Structured formats, queries and utilities | 0.112.0–0.135.0 | [Milestones](releases/phase-f.md) |
35-
| G: Compression and archives | 0.136.0–0.150.0 | [Milestones](releases/phase-g.md) |
36-
| H: Modern hashing and cryptographic operations | 0.151.0–0.186.0 | [Milestones](releases/phase-h.md) |
37-
| I: Legacy cryptography, hashes and classical ciphers | 0.187.0–0.237.0 | [Milestones](releases/phase-i.md) |
38-
| J: Public keys, certificates and tokens | 0.238.0–0.264.0 | [Milestones](releases/phase-j.md) |
39-
| K: Network and forensic analysis | 0.265.0–0.279.0 | [Milestones](releases/phase-k.md) |
40-
| L: Images, media and document presentation | 0.280.0–0.300.0 | [Milestones](releases/phase-l.md) |
41-
| M: Complete recipe semantics, Magic and compatibility | 0.301.0–0.315.0 | [Milestones](releases/phase-m.md) |
42-
| N: Extensibility and replacement-adapter proof | 0.316.0–0.330.0 | [Milestones](releases/phase-n.md) |
43-
| O: Portability, collaboration and operations | 0.331.0–0.348.0 | [Milestones](releases/phase-o.md) |
44-
| P: Qualification and general-availability readiness | 0.349.0–0.363.0 | [Milestones](releases/phase-p.md) |
34+
| Z: Repository and service foundation | 0.1.0–0.8.0 | [Milestones](releases/phase-z.md) |
35+
| A: Foundation and useful vertical slice | 0.9.0–0.33.0 | [Milestones](releases/phase-a.md) |
36+
| B: Bytes, text and foundational encodings | 0.34.0–0.63.0 | [Milestones](releases/phase-b.md) |
37+
| C: Streaming, artifacts and execution foundations | 0.64.0–0.78.0 | [Milestones](releases/phase-c.md) |
38+
| D: Modern browser workbench | 0.79.0–0.93.0 | [Milestones](releases/phase-d.md) |
39+
| E: Remote API, PostgreSQL and secure server execution | 0.94.0–0.120.0 | [Milestones](releases/phase-e.md) |
40+
| F: Structured formats, queries and utilities | 0.121.0–0.144.0 | [Milestones](releases/phase-f.md) |
41+
| G: Compression and archives | 0.145.0–0.159.0 | [Milestones](releases/phase-g.md) |
42+
| H: Modern hashing and cryptographic operations | 0.160.0–0.195.0 | [Milestones](releases/phase-h.md) |
43+
| I: Legacy cryptography, hashes and classical ciphers | 0.196.0–0.246.0 | [Milestones](releases/phase-i.md) |
44+
| J: Public keys, certificates and tokens | 0.247.0–0.273.0 | [Milestones](releases/phase-j.md) |
45+
| K: Network and forensic analysis | 0.274.0–0.288.0 | [Milestones](releases/phase-k.md) |
46+
| L: Images, media and document presentation | 0.289.0–0.309.0 | [Milestones](releases/phase-l.md) |
47+
| M: Complete recipe semantics, Magic and compatibility | 0.310.0–0.324.0 | [Milestones](releases/phase-m.md) |
48+
| N: Extensibility and replacement-adapter proof | 0.325.0–0.339.0 | [Milestones](releases/phase-n.md) |
49+
| O: Portability, collaboration and operations | 0.340.0–0.356.0 | [Milestones](releases/phase-o.md) |
50+
| P: Qualification and general-availability readiness | 0.357.0–0.371.0 | [Milestones](releases/phase-p.md) |
4551

4652
## Release candidates and production
4753

4854
### v1.0.0-rc.1
4955

5056
**Status:** planned.
5157

52-
**Setup:** every required pre-1.0 inventory row and qualification result is closed; PostgreSQL 19 GA, current OpenBao/Valkey and exact artifacts are pinned.
58+
**Setup:** every required pre-1.0 inventory row and qualification result is closed; PostgreSQL 19 GA, current OpenBao/Valkey, optional Meilisearch and exact artifacts are pinned. Both search profiles and OpenBao secret-source/initialization/build/recovery gates are qualified.
5359

5460
**Goal:** qualify the first complete production candidate.
5561

@@ -89,7 +95,7 @@ The [release runbook](RELEASE_RUNBOOK.md) and [version policy](VERSIONING_POLICY
8995

9096
**Deliverables:** supported distributions, maintenance policy, full operation/API docs, production security/recovery guides and release notes.
9197

92-
**Verification:** verify all operation/argument/recipe/UI/target rows, exact distribution provenance, current security findings and executed deployment/upgrade/recovery procedures.
98+
**Verification:** verify all operation/argument/recipe/UI/target rows, both search profiles with live authorization, OpenBao-sourced project secrets from initialization through release, exact distribution provenance, current security findings and executed deployment/upgrade/recovery procedures.
9399

94100
**Exit criteria:** all required functionality and evidence pass; no beta database or unsupported production claim remains. v1.0.0 implementation stop reached. Run pentest for this exact commit.
95101

0 commit comments

Comments
 (0)