diff --git a/bpa/CHANGELOG.md b/bpa/CHANGELOG.md index b456f207d..20e8cff97 100644 --- a/bpa/CHANGELOG.md +++ b/bpa/CHANGELOG.md @@ -9,7 +9,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), ### Added - Both input doors settle recent duplicates at their gates, before the payload spools: the store keeps a small LRU of every id it commits (or refuses as a duplicate), and the gates probe it with a pure in-memory check — no storage round-trip, and the backends' trait surface is unchanged. A replay is DoS-shaped traffic, so a cache hit costs the sender nothing but its headers (and any declared payload peek): CLA ingress acks and drops the replay (reception still reported per RFC 9171 §5.6, counted under `received.duplicate`), and the raw originate door surfaces `DuplicateBundle` before consuming the caller's payload stream past any declared peek. The ADU doors carry no probe — their ids are freshly generated, so only `insert_metadata`'s refusal backstops them. The probe is never authoritative — copies racing in concurrently, ids past the cache's horizon, and a cold cache after restart are all still arbitrated by `insert_metadata`'s atomic refusal, exactly as before. - `ApplicationSink::send_streamed(destination, total_len, stream, lifetime, options)` — the streamed ADU originate, completing the pull-shaped `Receiver` seam for application origination. The declaration frames the bundle's wire form before the first segment is pulled (`bpv7`'s `Builder::build_stream` emits the prefix; the payload block's CRC is computed as the payload flows; a declared Originate payload peek holds the app's first bytes before the gate), and the stream is one-shot: an over- or under-delivering producer is rejected (`PayloadTooLarge`/`PayloadUnderrun`, nothing persisted), and a vanishing producer cancels the send (`StreamCancelled`). The trait method has a provided implementation — buffer and delegate to `send`, same error surface — so existing sinks (the gRPC bridge included) compile unchanged; the BPA's own sink streams end to end, and the whole-buffer `send` is pure sugar over the streamed door (a resident payload is a one-segment stream), so the two doors are wire-equivalent by construction. Neither door retries a duplicate bundle id: `CreationTimestamp::now` is process-monotonic, so `DuplicateBundle` from an application send can only mean a collision with a pre-restart bundle and is surfaced to the caller. -- `VerifyContext::payload_peek` and `ClassifyContext::payload_peek` lend a filter the payload's resident prefix. The input doors hold the declared payload peek: before an input chain runs, the header pass at CLA ingress and at the raw originate door pulls segments until the payload's first min(P, payload length) bytes have arrived (P being the largest peek declared at that hook), feeding each through the payload's tail; a hook with no peek declared pulls nothing extra. The bytes may precede the payload's CRC and BIB checks, which the drain settles before the bundle commits or its route executes. A BCB-covered payload, or a fragment past the payload's start, has no peek, and the header pass holds nothing for it. +- `VerifyContext::payload_peek` and `ClassifyContext::payload_peek` lend a filter the payload's resident prefix. The input doors hold the declared payload peek: before an input chain runs, the header pass at CLA ingress and at the raw originate door pulls segments until the payload's first min(P, payload length) bytes have arrived (P being the largest peek declared at that hook), feeding each through the payload's tail; a hook with no peek declared pulls nothing extra. The bytes may precede the payload's CRC and BIB checks, which the drain settles before the bundle commits or its route executes. A BCB-covered payload, or a fragment past the payload's start, has no peek, and the header pass holds nothing for it. A Deliver Verifier declares a peek through `FilterPack::deliver_verifier_with_peek`; the delivery door holds the payload's first min(P, payload length) bytes, read from the store, before the Deliver Verifiers run. - Annotation slots — embedder-private, typed per-bundle metadata in the classification group (see [Annotation slots](docs/filter_subsystem_design.md#annotation-slots--embedder-private-metadata)): a filter pair declares a slot with the `slot!` macro, a `static` `filter::slots::Slot` named at rest by its declaring path, so slot names cannot collide and nothing is registered; Rust visibility on the static gates every read and write. Classifiers write through `MetadataDelta::set`; filters read via `BundleMetadata::slot`, or borrow without copying via `slot_str`/`slot_bytes` (`filter::slots::Blob` is the owned byte-string value type, since bare byte containers deliberately encode with CBOR array semantics). Values are canonically-encoded CBOR at rest, persisted with the bundle as written: filters are trusted, so the BPA checks nothing a Classifier writes. - Classifier-selected routing inputs — the first routing consumer of classification (design: [docs/routing_table_redesign.md](docs/routing_table_redesign.md)): Classifiers emit the per-bundle `{table, key}` lookup inputs through the new per-field `MetadataDelta::route_table`/`route_key` options (`Some` writes under last-writer-wins across the sequential chain, `None` expresses no opinion and preserves the stored value — one classifier may set the table and a later one the key; the table is the new opaque `routing::TableId`, whose one public value is `TableId::DEFAULT` until tables can be named), persisted in the classification group as the new `RoutingKey` sub-record (serde-elided when unset) and read back via the new `BundleMetadata::route_key`/`route_table` accessors. Every RIB lookup — the ingress gate's decision of record and every re-dispatch alike — resolves its inputs from the bundle's record: `Rib::find` walks `route_key.unwrap_or(destination)` in `route_table.unwrap_or(default)`, and reads no policy. The key replaces the destination for the entire walk, deciding node-level disposition — producers own the segment-routing skip-self discipline — and naming any table other than the default (the only table until the tables tranche) is a strict no-match: the bundle Waits for its topology, never falls through; an explicit default in either half is indistinguishable from unset. A bundle the lookup cannot place parks under the same inputs: a local key waits for that service to register, and any other key, or a table other than the default, waits for a route. The `RoutingKey`'s influence ends at route selection: it picks the ECMP group, never the member (see the flow-label entry under Changed). Until the policy-epoch machinery lands ([docs/storage_consistency_updates.md](docs/storage_consistency_updates.md)), the persisted inputs refresh only by re-classification. - `BundleStatus::same_queue` — queue-identity comparison (peer/service/queue, ignoring per-bundle payload fields), used by the storage channels to recover queue membership without matching the whole assignment record. @@ -21,6 +21,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - `Bytes` implements `stream::Receiver` — a whole in-memory buffer is a one-`Final`-segment stream, zero-copy — and `stream::buffer_stream` assembles a segment stream into a contiguous buffer, enforcing `total_len` exactness and 32-bit addressability via the new `stream::BufferError` (which converts into `cla::Error`/`services::Error` through the new `PayloadUnaddressable` variants). ### Changed +- The delivery door streams: `deliver_bundle` opens the stored bundle through `Store::load_stream` and holds it only until its headers are resident. The scheduled §5.1.1 removals and the Deliver Rewriters edit one editor over the record's block index, and the Deliver Verifiers read a snapshot of its staged edits, so no delivery materialises the bundle for its filters. A raw `Service` receives the rebuild's chunks as segments when a stage edited, and otherwise the stored bundle, the payload streaming on from the store either way, so it may receive several segments where it received one `Final`. An `Application` receives a plaintext payload as it streams from the store, its stored data and nothing after it, announced at the stored data length; a BCB-covered payload is still decrypted whole, from the stored bundle the door pulls, and announced at its decrypted length, since a ciphertext that carries its AES-GCM tag (RFC 9173 §4.4, this node's own form) is 16 bytes longer than its plaintext. The stages run for their verdict either way: they touch neither the payload nor its protection. The record keeps the stored block index throughout. +- The egress door streams: `forward_bundle` opens the stored bundle through the new `Store::load_stream` seam and holds it only until its headers are resident, and the attempt's one rebuild reaches `Cla::forward` as segments — the outer array's head, each chunk of the rebuild (an edited block's new bytes, a kept block's resident bytes zero-copy), then the stored payload — with `total_len` from the rebuilt block index. Nothing on the egress path flattens the bundle, and a CLA receives several segments where it received one `Final`. `Store::load_stream`'s interim body yields the whole stored buffer as one segment, so an Egress Rewriter still reads the payload; once a backend streams its load, a payload not yet loaded reads as `NotResident`. +- The per-hop writes and the scheduled §5.1.1 removals at the output doors edit over the record's block index, the parse ingress verified, instead of re-parsing the stored bytes without keys. A key-holding forwarder therefore updates a Previous Node, Hop Count or Bundle Age block that a BIB it decrypted at ingress provably does not cover, stripping the block from its BCB and writing it in plaintext, where it used to refuse the write (a Previous Node or Bundle Age refusal parked the bundle; a refused Hop Count travelled unchanged). A block an encrypted BIB may or does cover is still refused. +- Ingress refuses a bundle in which an encrypted BIB provably covers a Previous Node or Bundle Age block, deleting it with the reason "Conflicting security operation" (RFC 9172 §7.1), where the bundle previously parked at every egress attempt until it expired. A keyed pass that decrypts the BIB proves the combination, or, without the key, every BCB-covered block other than a BIB being one of those two; anything short of proof passes as before. - The originate doors are restructured onto the streamed pipeline, with a validation split and one shared admission stage (and renamed to match: `Dispatcher::originate` / `Dispatcher::originate_raw`, crate-internal). The ADU door (`originate`) performs no parse or validation — the `Builder` emits valid bundles by construction — while the raw door (`originate_raw`) is the security boundary and now runs the same strict machinery as CLA ingress: `parse_headers` keyed header verification on the stream (retiring the door's one-shot `parse_validate_with_provider` and its interim `concat_stream` buffering — service bytes are no longer accumulated before parsing), the source-endpoint check, the lifetime/hop admission gate, and — new to raw sends — the config-gated RFC 9171 validity checks. Both doors merge at the admission stage: the gate decisions (the Originate filter chain on the resident prefix with the header pass's decoded BCB sets, then the route lookup **at the door as the decision of record**) settle before the payload spools, then the record is written once at `Dispatching`, and the routing decision executes as a fresh CLA arrival's does (decided again if the table changed during the drain; the admin-endpoint arm queues for the processing pool, since no origination is a fragment), so originated bundles otherwise no longer transit the dispatch queue (`DispatchPending` now belongs to the re-dispatch paths). Every rejection surfaces as a `services::Error` to the originator and never a status report: parse/verification failures return the error itself (`InvalidBundle`), gate and filter and route drops return `Dropped(reason)` — a reasonless (silent) `Drop` route surfaces as `Dropped(None)` where the CLA gate stays silent — and a route or chain rejection returns with nothing spooled, never awaiting the payload. The declared-wire-size bound now applies to both doors at the shared stage (the ADU door's first size bound), and originate rejections that carry a reason code — an invalid raw bundle, the admission gates, the filter chain, a route Drop, a payload verdict — are counted under the new `bpa.bundle.originated.dropped` counter with a `reason` label (size, length and cancellation refusals and duplicates are not, as at CLA ingress). - **BREAKING:** the raw originate door rejects fragments: a service-built bundle carrying fragment info surfaces the new `services::Error::FragmentedBundle`, before the admission gates and with nothing spooled or persisted. Fragmentation is a forwarding-time action (RFC 9171 §5.8) — a source never emits fragments — and admitting one would let a service fabricate pieces of an ADU it never sent whole. The ADU doors are unaffected: the `Builder` cannot produce a fragment by construction. - The streamed ingress drain runs behind the store's streaming write seam, strictly after the gate decisions. `Store::save_stream(&mut dyn Receiver)` is the seam at its target signature (`docs/streaming_pipeline_design.md` §3): one stream carries the whole bundle — a validating decorator (`ValidatingReceiver`) yields the resident head as its first segment, then validates the tail as it flows — and the interim body spools via `concat_stream` and commits through the backends' whole-buffer `save`; the storage tranche swaps the body and the backend trait with no caller changing. The gate decisions — Ingress chain and route lookup — settle before the drain begins, so a gate drop rejects with nothing spooled and nothing persisted, and a save failing post-stream validation is discarded (the streaming contract's stage-then-discard ordering). The size cap is enforced against the *declared* wire size the moment the headers parse — an over-cap bundle refuses before a single payload byte drains, and a producer exceeding its declaration trips the framing checks rather than a byte-count. The recovery paths take the same door: reassembly and restart-orphan re-admission hand their bytes as the bundle stream and the drain saves an admitted bundle fresh — there is no caller-stored special case, and each caller deletes its superseded copy after the pipeline settles. @@ -32,14 +36,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - RFC 9171 §5.6 Step 4 conformance: an unprocessable extension block flagged `report_on_failure` triggers its "Block unsupported" reception status report on the block flag alone — previously the report was also gated on the bundle-level receipt-report flag, so a block's demand on a non-reporting bundle was silently dropped. The demand survives the deletion a block's `delete_bundle_on_failure` orders: the report, then the deletion. A status report addressed to a null report-to endpoint is suppressed (reachable only via the block-flag trigger; the §4.2.4 admin-record/anonymous flag combinations are rejected at parse). Report construction is gated up front: with reporting disabled or a null report-to, no report is encoded and the `bpa.status_report.sent` counters no longer tick for reports that were never sent. - **BREAKING:** the IPN legacy 2-element re-encode is available as a built-in of the per-hop rewrite stage, configured with the new `BpaBuilder::ipn_legacy_peers(Vec)` (empty and inert by default). This replaces the `hardy-ipn-legacy-filter` crate, which is retired: nothing populates the `BundleMetadata::next_hop` field it read. A bundle carrying a BPSec operation that covers the primary block, as its target or through its scope (RFC 9173's default scope includes it), cannot be re-encoded without breaking that operation, so at a legacy next hop it is dropped with `UnexpectedSecurityOperation`; the editor's refusal of the primary edit, judged after the per-hop writes, decides it. - **BREAKING:** the Hop Count block is no longer liveness-critical, since its increment is a SHOULD (RFC 9171 §4.4.3). A bundle whose Hop Count this node cannot decrypt is accepted at ingress and at origination rather than rejected: the node enforces no hop limit on it and forwards the block unchanged, so a loop through nodes that all lack the key is bounded by the bundle's lifetime alone (any key holder on the path still enforces the limit). One whose ciphertext fails authentication is scheduled for removal with its BCB (RFC 9172 §5.1.1) rather than rejecting the bundle; the bundle then carries no Hop Count, so no later node enforces a hop limit and its lifetime alone bounds a loop on the rest of its path. An unclocked bundle's undecryptable Bundle Age stays fatal. +- Origination is not a hop: the originating node no longer increments the Hop Count, so a locally originated bundle leaves with the count it was built with, and the next node makes the first increment. +- A delivered payload that fails to decrypt, at an Application or at the administrative endpoint, is dropped with `FailedSecurityOperation`, or `UnknownSecurityOperation` when its BCB names a security context the node does not recognise, where both were `BlockUnintelligible`. Without a key, the bundle still parks until keys may arrive. The payload is extracted over the record's block index with the attempt's keys, so a delivery consults the key provider at most once, and not at all for a plaintext payload with no Deliver filters or scheduled removals. A delivered payload the stored bytes do not hold, or a payload BCB that does not decode, is a BPA bug or storage corruption, and fail-stops the process. - A forwarder increments the Hop Count only where it can safely update the block: its increment is a SHOULD (RFC 9171 §4.4.3), so under an encrypted BIB that may cover it the block travels unchanged with its security operations, where the bundle previously parked until expiry. - Origination runs a pre-store admission gate: an expired or hop-exhausted origination fails synchronously with `services::Error::Dropped(Some(LifetimeExpired`/`HopLimitExceeded))` instead of being stored and later reaped with a deletion report, and an Originate-filter drop returns its reason (`Dropped(Some(reason))`, previously `Dropped(None)`). - **BREAKING:** `BundleStatus::ForwardPending` carries the resolved adjacency in a new `next_hop: Eid` field — the routing decision rides the queue-assignment record, so the egress channels' at-least-once storage recovery re-delivers it intact — and the transient `BundleMetadata::next_hop` field is removed. Storage backends must persist the new field (SQLite reuses `status_param3`; PostgreSQL migration `0007_forward_pending_next_hop`) and their queue polls must match on queue identity rather than the whole status (see `same_queue` above); ownership swaps keep full-equality semantics. - **BREAKING:** `cla::Sink::dispatch` returns the new `#[must_use]` `cla::Acceptance` verdict, and `Err` no longer carries one. `Accepted` means the CLA may acknowledge the transfer — internal disposal of an invalid-but-complete or duplicate bundle is still acceptance, because refusing invalid content would invite the previous node's custody/retransmission machinery to resend identical bytes forever, so the first BPA to detect a content problem accepts and terminates the bundle. `Refused` means the CLA must not acknowledge (a stream that ends without its `Final`, or a bundle over `max_bundle_size`) and its peer retains responsibility. `Err` is reserved for sink faults (`Disconnected`, internal failures) — a CLA's own unregistration mid-stream now surfaces as `Err(Disconnected)` rather than a per-bundle outcome — and `StreamCancelled`/`PayloadTooLarge` are no longer returned by `dispatch` (both remain in use on the egress-door buffering paths). The bundle stream itself carries no verdict: the BPA may stop reading at any point (it may already have accepted — e.g. a duplicate identified from the headers), so a producer that sees `stream::SendError` stops pushing and takes the verdict from `dispatch`'s return value alone. - **BREAKING:** the size-carrying door errors are `u64`: `cla::Error::PayloadTooLarge`/`PayloadUnderrun` and `services::Error::PayloadTooLarge`/`PayloadUnderrun` carry `{ size: u64, ... }` (were `usize`), in the wire-size domain. The originate doors report declared wire sizes through them unchanged, so a declaration beyond a 32-bit target's address space surfaces as `PayloadTooLarge` with its true value (every cap is clamped to the address space); `PayloadUnaddressable` is `stream::buffer_stream`'s verdict for a whole-buffer consumer handed a stream it cannot address. The size bound has one seat: `bundle::parse::parse_headers` takes the cap as `u64` and refuses both the accumulated header bytes and the declared whole-bundle size (`HeaderFailure::TooLarge { size: u64, max: u64 }`) before any payload drains or keyed verification is spent; the ingress door's separate declared-size check is gone. The in-memory accumulator errors (`stream::ConcatError`/`BufferError`) deliberately keep `usize`: they measure buffered bytes, not wire declarations, and widen at the door conversions. - **BREAKING:** registration negotiates the bundle size cap, carried as `Option` end-to-end — the cap is admission policy, and `None` means no policy. A CLA's set-once declarations travel in the new `cla::ClaInit` struct — address type, lane count, and declared receive limit, all `Option`al, with `ClaInit::default()` declaring nothing — passed to `BpaBuilder::cla` and `BpaRegistration::register_cla` and snapshotted on the BPA-resident registration. The `Cla::address_type()` trait getter is removed along with the declaration parameters it accompanied: a getter implies a value the BPA may resample at any time, and these values are fixed at registration by construction (`Tcpclv4` exposes its declarations via the new `cla_init()` accessor). `cla::Cla::on_register` receives the *effective* cap — the minimum of the caps declared by either side, or `None` when neither declared one — for the CLA to propagate into its own transfer advertising, so an over-cap transfer is never offered: rejection is deterministic, and a peer that kept offering the same bundle would loop forever. `BpaBuilder::max_bundle_size` takes `NonZeroU64` (was `NonZeroUsize`); a cap beyond the target's addressable bound (`isize::MAX`) is clamped to it at construction, so the advertised and enforced values always agree. An unset cap still takes the private 64 MiB default while ingress accumulates in memory; the default becomes no-policy when the streaming pipeline moves enforcement to the u64 domain (see the storage leg in [refactor_plan.md](docs/refactor_plan.md)). -- **BREAKING:** filter registration is construction-frozen packs of three new filter kinds, replacing the runtime filter registry (the full design is [docs/filter_subsystem_design.md](docs/filter_subsystem_design.md)). Embedders implement `filter::{Verifier, Classifier, Rewriter}` — synchronous, trusted code receiving its kind's context (`filter::{VerifyContext, ClassifyContext, RewriteContext}`), which lends the wire bundle as a `hardy_bpv7::Bundle` (primary block and block headers) and a `hardy_bpv7::reader::Reader` over its block bodies — a `bpsec::DecryptingReader` over the node's keys, memoised so a covered block is decrypted at most once for the filters sharing it, reading each body as plaintext or BCB-decrypted (the payload's among them when resident) or as the `Availability` state that says why not; only a Rewriter's edits are scoped, to extension blocks — with the BPA-local record state (provenance, extension-field cache, annotation slots) through a separate getter, a `RewriteContext` also carrying its `filter::Boundary` (Egress, with the resolved next hop, or Deliver) and owning the `ExtensionEditor` — Verifiers and Classifiers returning a `filter::Verdict` (which derives `Clone`, `Copy`, `PartialEq`, `Eq` and `Hash`), and Rewriters returning nothing — collect them into a `filter::pack::FilterPack` (per-hook registration methods; the input-hook `_with_peek` variants declare a payload-peek prefix the input doors hold before their chains run), and hand the pack to the new `BpaBuilder::add_filters` before `build()`. The kind traits are implemented for `Box` and `Arc`, so a filter built from configuration registers as a boxed trait object and one instance can serve several hooks. Chain order is call order, within a pack and across `add_filters` calls; `build()` freezes the chains, which the dispatcher runs inline at the four hooks (Ingress, Originate, Egress, Deliver). Verifiers register at Ingress, Originate and Deliver, Classifiers at the input hooks, and Rewriters at Egress and Deliver, so nothing at Egress drops a bundle. Removed: `Bpa::register_filter`/`unregister_filter`, `BpaBuilder::filter`, the `Filter` and `Hook` enums, `ReadFilter`/`WriteFilter`, `ReadResult`/`WriteResult`, `Mutation`, `ExecResult`, and the name/`after` dependency graph with its `filter::Error` variants — filters no longer have names (a pack-prefixed diagnostic label replaces them) or a runtime lifecycle. Migrating an existing filter: a `ReadFilter` becomes a `Verifier`, except at Egress, which has none (per-adjacency policy belongs to routing, content policy to an Ingress Verifier); a `WriteFilter` that wrote `WritableMetadata` becomes an input-hook `Classifier` returning a `MetadataDelta` (a `flow_label` becomes an annotation slot); a `WriteFilter` that rewrote the bundle becomes an Egress or Deliver `Rewriter` editing extension blocks through bpv7's `ExtensionEditor` (whole-bundle replacement is gone: the primary, payload, and BIB/BCB blocks are out of scope); `ReadResult::Drop`/`WriteResult::Drop` become a Verifier's or Classifier's `Verdict::Drop`, with the same reason semantics (a Rewriter cannot drop); the `Hook` argument becomes the pack's per-hook registration method; and `async fn filter` becomes a synchronous `verify`/`classify`/`rewrite`. -- **BREAKING:** bundle mutation by filters is scoped to extension blocks: a `Rewriter` (Egress and Deliver hooks) edits through bpv7's `extension_editor::ExtensionEditor` (`insert`/`replace`/`remove`), which refuses the primary block, the payload, BIB/BCB, any block under (possibly unprovable) BPSec coverage, and any edit a receiver would reject; a refusal is the Rewriter's no-match path, and a block it inserted is a valid target for its own later `replace` or `remove`. Edits are in-memory and per transmission/delivery attempt — the stored bundle is never rewritten, so the `WriteResult` byte-return path and the post-rewrite re-validation re-parse are gone with it. A `Rewriter` execution failure — an invalid edit, or an edit whose materialised bytes do not re-parse — fail-stops the process naming the failing link's pack-prefixed label: like a storage fault, a rewrite that was meant to work and has not leaves further bundle processing undefined, so there is no error a caller could react to appropriately. An Egress or Deliver chain, or the per-hop rewrite, whose bytes fail to decode fail-stops the same way: they were validated at ingress, so the failure is a BPA bug or storage corruption. At Egress the Rewriters run before the per-hop writes (Previous Node, Hop Count, Bundle Age, and the legacy re-encode), which supersede a Rewriter's edits to the blocks they write. +- **BREAKING:** filter registration is construction-frozen packs of three new filter kinds, replacing the runtime filter registry (the full design is [docs/filter_subsystem_design.md](docs/filter_subsystem_design.md)). Embedders implement `filter::{Verifier, Classifier, Rewriter}` — synchronous, trusted code receiving its kind's context (`filter::{VerifyContext, ClassifyContext, RewriteContext}`), which lends the wire bundle as a `hardy_bpv7::Bundle` (primary block and block headers) and a `hardy_bpv7::reader::Reader` over its block bodies — a `bpsec::DecryptingReader` over the node's keys, memoised so a covered block is decrypted at most once for the filters sharing it, reading each body as plaintext or BCB-decrypted (the payload's among them when resident) or as the `Availability` state that says why not; only a Rewriter's edits are scoped, to extension blocks — with the BPA-local record state (provenance, extension-field cache, annotation slots) through a separate getter, a `RewriteContext` (`RewriteContext<'_, '_>`) also carrying its `filter::Boundary` (Egress, with the resolved next hop, or Deliver) and lending an `ExtensionEditor` handle on the engine's editor — Verifiers and Classifiers returning a `filter::Verdict` (which derives `Clone`, `Copy`, `PartialEq`, `Eq` and `Hash`), and Rewriters returning nothing — collect them into a `filter::pack::FilterPack` (per-hook registration methods; the input-hook `_with_peek` variants declare a payload-peek prefix the input doors hold before their chains run), and hand the pack to the new `BpaBuilder::add_filters` before `build()`. The kind traits are implemented for `Box` and `Arc`, so a filter built from configuration registers as a boxed trait object and one instance can serve several hooks. Chain order is call order, within a pack and across `add_filters` calls; `build()` freezes the chains, which the dispatcher runs inline at the four hooks (Ingress, Originate, Egress, Deliver). Verifiers register at Ingress, Originate and Deliver, Classifiers at the input hooks, and Rewriters at Egress and Deliver, so nothing at Egress drops a bundle. Removed: `Bpa::register_filter`/`unregister_filter`, `BpaBuilder::filter`, the `Filter` and `Hook` enums, `ReadFilter`/`WriteFilter`, `ReadResult`/`WriteResult`, `Mutation`, `ExecResult`, and the name/`after` dependency graph with its `filter::Error` variants — filters no longer have names (a pack-prefixed diagnostic label replaces them) or a runtime lifecycle. Migrating an existing filter: a `ReadFilter` becomes a `Verifier`, except at Egress, which has none (per-adjacency policy belongs to routing, content policy to an Ingress Verifier); a `WriteFilter` that wrote `WritableMetadata` becomes an input-hook `Classifier` returning a `MetadataDelta` (a `flow_label` becomes an annotation slot); a `WriteFilter` that rewrote the bundle becomes an Egress or Deliver `Rewriter` editing extension blocks through bpv7's `ExtensionEditor` (whole-bundle replacement is gone: the primary, payload, and BIB/BCB blocks are out of scope); `ReadResult::Drop`/`WriteResult::Drop` become a Verifier's or Classifier's `Verdict::Drop`, with the same reason semantics (a Rewriter cannot drop); the `Hook` argument becomes the pack's per-hook registration method; and `async fn filter` becomes a synchronous `verify`/`classify`/`rewrite`. +- **BREAKING:** bundle mutation by filters is scoped to extension blocks: a `Rewriter` (Egress and Deliver hooks) edits through bpv7's `extension_editor::ExtensionEditor` (`insert`/`replace`/`remove`), which refuses the primary block, the payload, BIB/BCB, any block under (possibly unprovable) BPSec coverage, and any edit a receiver would reject; a refusal is the Rewriter's no-match path, and a block it inserted is a valid target for its own later `replace` or `remove`. Edits are in-memory and per transmission/delivery attempt — the stored bundle is never rewritten, so the `WriteResult` byte-return path and the post-rewrite re-validation re-parse are gone with it. The Rewriters of an output hook edit the attempt's one editor, between the scheduled §5.1.1 removals and (at Egress) the per-hop writes, and each reads a snapshot of the edits staged before it, through the context's `bundle()` (a staged block's extents index nothing) and `reader()`; the output door rebuilds the bundle once. Every edit is checked when it is staged, so a failed rebuild — a BPA or bpv7 bug — fail-stops the process: like a storage fault, a rewrite that was meant to work and has not leaves further bundle processing undefined, so there is no error a caller could react to appropriately. An output chain works from the record's block index, the parse ingress verified, and decodes only the BCB operation sets it locates; a failure to decode them fail-stops the same way: they were validated at ingress, so the failure is a BPA bug or storage corruption. At Egress the Rewriters run before the per-hop writes (Previous Node, Hop Count, Bundle Age, and the legacy re-encode), which supersede a Rewriter's edits to the blocks they write. - **BREAKING:** the RFC 9171 validity checks are config-gated pipeline checks at the pre-drain ingress gate, controlled by the new `BpaBuilder::primary_block_integrity` and `bundle_age_required` setters (both strict by default): a policy rejection now happens before the payload is drained or anything is stored, reported like every gated drop (the combined reception + deletion status report, per the bundle's flags). A failing bundle is still accepted (the transfer is acknowledged) and never `Refused`, which stays reserved for truncated or over-cap transfers. `Rfc9171ValidityFilter`, `BundleValidityFilter`, the `filter::rfc9171` and `filter::validity` modules, and the `no-rfc9171-autoregister` cargo feature are removed; raw-bytes origination performs the equivalent inline lifetime/hop check. The checks screen arrivals only: a stored bundle is not re-checked (nor re-run through the Ingress chain) when the node restarts, so a node restarted with a tightened policy keeps, and delivers, bundles a relaxed configuration admitted — the removed filter re-ran for records recovered as `New`. - **BREAKING:** `WritableMetadata` and its `flow_label` field are removed with nothing left writing them: the classification group (annotation slots and the `route_table`/`route_key` routing inputs; `class` arrives with the policy tranche) is the new write path. The flow label as a policy input is gone with it: `FlowController::queue_for` takes no label (the controller owns per-peer queue assignment); a traffic-class parameter returns with the policy tranche (`policy_subsystem_redesign.md`). ECMP peer selection hashes the conversation `(source, destination)` — a settled boundary, not an interim: lookup state (the `RoutingKey`) selects the route, packet entropy selects the member within it, and policy-injected entropy is the flow-label input's seat. - An application send no longer loops rebuilding on a duplicate bundle id: `hardy-bpv7`'s `CreationTimestamp::now` is process-monotonic, so a builder-made id cannot collide with an id this process issued — the Builder ensures uniqueness rather than the caller checking for it. `services::Error::DuplicateBundle` from an application send is now surfaced to the caller instead of retried invisibly; it means a bundle with the same id is already in the store (in practice a pre-restart bundle after a backward clock step, made vanishingly unlikely by the nanosecond-seeded sequence floor). The monotonic `now()` equally covers the BPA's other minting sites (status reports, restart rebuilds), which previously had no collision handling at all. @@ -76,6 +82,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), - The `bpa.filter.error` metric: it counted an input chain whose resident prefix failed the chain's own decode, which cannot happen now that both input chains run on their door's header decode and decode nothing themselves. A dashboard keyed on it sees the series end. ### Fixed +- A forward that the CLA answers `StreamCancelled` re-enters dispatch at once, instead of parking in `Waiting` until the next routing or link event: a cancelled transfer is a failed acceptance the bundle is valid to retry. The re-dispatch re-claims the bundle with the conditional swap a deferred `Failed` outcome uses, so a concurrent resolution still wins. A CLA that cancels every attempt is offered the bundle again until it expires. Every other synchronous `forward` error still parks the bundle. - A bundle that arrives whole in a `Segment::Next`, or whose payload drains past its outer break before `Final`, commits only once the stream's `Final` arrives: a transfer the peer abandons after the bundle's last byte is refused (the CLA withholds its acknowledgement), where it was accepted. Bytes that follow a complete bundle are trailing data, dropped and reported like bytes past the outer break. A `Final` that ends the stream short of the bundle's end completes the transfer: the bundle is accepted and dropped, reported as an unintelligible block once its headers have parsed, however the transfer was segmented. A payload CRC mismatch is likewise reported however the bundle arrives, including one already resident when its headers parse, which failed without a report. - A peer removed while `add_peer` was still constructing it (a multi-await window) no longer strands RIB forward entries that nothing would ever clean up: peers are constructed complete and published once (no half-built peer is ever reachable), and the addition re-checks its address claim after construction, withdrawing the peer if a concurrent removal won. - The default (null) egress policy no longer panics when a CLA declares explicit egress lanes: the declaration is CLA-side shape — remote, once the v1 wire carries `lane_count` — and applying no policy imposes no lane constraint, so every bundle transmits with the next-free-lane directive and a multi-lane CLA still fans across its idle lanes. diff --git a/bpa/docs/TODO.md b/bpa/docs/TODO.md index 7c58305b1..c481e6a2b 100644 --- a/bpa/docs/TODO.md +++ b/bpa/docs/TODO.md @@ -128,6 +128,6 @@ Items the external reviews of the filter subsystem (2026-09-30, and its 2026-10- **No test observes the filter counters.** `bpa.filter.filtered` and `bpa.filter.modified` are incremented as documented, but the bpa tests install no metrics recorder, so nothing pins them. A `metrics_util::debugging::DebuggingRecorder` (a dev-dependency) around the engine's inline tests would assert each, by hook. -**No block-count cap.** The in-memory `Block` map costs ~774 B per wire block, so 262,146 tiny extension blocks in a 3.8 MB bundle take ~53× the wire size in heap before any filter runs, and the work, though linear, is synchronous (2.5 s measured, inside an async task with no yield point). Each editing Rewriter adds a full wire-form copy per attempt — a factor streamed egress shrinks to the header region (`streaming_pipeline_design.md` Phase B). A configurable maximum block count at the pre-drain gate, beside `max_bundle_size` and the RFC 9171 gates, would cap both; whether a cap conflicts with any conformance requirement is unchecked. +**No block-count cap.** The in-memory `Block` map costs ~774 B per wire block, so 262,146 tiny extension blocks in a 3.8 MB bundle take ~53× the wire size in heap before any filter runs, and the work, though linear, is synchronous (2.5 s measured, inside an async task with no yield point). Each editing Rewriter adds a snapshot of every block header per attempt — proportional to the block count, not the wire size: the output stages edit one editor over the resident bytes and rebuild once, and nothing on the egress path flattens the bundle. A configurable maximum block count at the pre-drain gate, beside `max_bundle_size` and the RFC 9171 gates, would cap both; whether a cap conflicts with any conformance requirement is unchecked. **The post-registration poll can re-attempt a fresh park.** `register_service` spawns `poll_service_waiting` asynchronously; landing after a bundle parks `WaitingForService`, it re-dispatches the bundle at once — one redundant attempt (failing the same way if the failure is deterministic), and a non-determinism tests must synchronise around (`bpa/tests/filter_dispositions.rs` waits on the park swap itself). The RIB's Waiting poller skips stale wakes since `Rib::request_poll`; the service poll has no equivalent. diff --git a/bpa/docs/component_test_plan.md b/bpa/docs/component_test_plan.md index 36aea6e7e..79b3f7bca 100644 --- a/bpa/docs/component_test_plan.md +++ b/bpa/docs/component_test_plan.md @@ -87,11 +87,13 @@ The unit testing strategy focuses on isolating complex logic from the async runt ### Suite E: Service Delivery Failure -*Objective: a delivery the service fails leaves the bundle in custody (`bpa/tests/pipeline.rs`).* +*Objective: a delivery the service fails leaves the bundle in custody, and a payload this node cannot decrypt as its security acceptor discards the bundle (`bpa/tests/pipeline.rs`).* | Test ID | Scenario | Procedure | Expected Result | | :--- | :--- | :--- | :--- | | **INT-BPA-05** | **`on_deliver` returns `Err`** | 1. Register an application whose `on_deliver` fails.
2. Originate a bundle to it from a second application.
3. Unregister the failing application and register a working one on the same service id. | The failed delivery parks the bundle `WaitingForService` rather than reporting it delivered and deleting it; the working receiver gets it re-delivered, payload intact. | +| **INT-BPA-16** | **Payload fails to decrypt** | 1. Register a key provider holding the payload's key.
2. Deliver, via a CLA, a deletion-report-requesting bundle whose encrypted payload was tampered with, addressed to a local application. | The bundle is dropped with one deletion report carrying `FailedSecurityOperation` (RFC 9172 §5.1.1); nothing is delivered. | +| **INT-BPA-17** | **Payload in an unknown security context** | 1. Deliver, via a CLA, a deletion-report-requesting bundle whose payload a BCB covers in a security context this node does not recognise, addressed to a local application. | The bundle is dropped with one deletion report carrying `UnknownSecurityOperation` (RFC 9172 §7.1); nothing is delivered. | ### Suite F: Deferred CLA Transfer Outcomes @@ -111,7 +113,7 @@ The unit testing strategy focuses on isolating complex logic from the async runt | Test ID | Scenario | Procedure | Expected Result | | :--- | :--- | :--- | :--- | -| **INT-BPA-12** | **Streamed origination and ingress** | 1. Originate through `ServiceSink::send` in several segments; drop the producer before `Final`; unregister the service while a send is parked; send a bundle whose source is not the service's endpoint; send a bundle whose Hop Count this node cannot decrypt.
2. Dispatch through `cla::Sink::dispatch` in several segments; unregister the CLA while a stream is parked; drop the producer before `Final`, including after a whole bundle in `Next` and after every byte of an oversized one. | Multi-segment origination and ingress match their whole-buffer forms (origination returns the built bundle's id and forwards it; ingress delivers locally). A producer dropped before `Final` cancels an origination (`StreamCancelled`, nothing enters custody) and is refused at ingress (`Acceptance::Refused`, nothing delivered), even once every byte of the bundle has arrived. Unregistration wakes a parked consumer with `StreamCancelled` at once. The spoofed source is rejected. An undecryptable Hop Count is accepted and forwarded unchanged. | +| **INT-BPA-12** | **Streamed origination and ingress** | 1. Originate through `ServiceSink::send` in several segments; drop the producer before `Final`; unregister the service while a send is parked; send a bundle whose source is not the service's endpoint; send a bundle carrying a Hop Count block, and one whose Hop Count this node cannot decrypt.
2. Dispatch through `cla::Sink::dispatch` in several segments; unregister the CLA while a stream is parked; drop the producer before `Final`, including after a whole bundle in `Next` and after every byte of an oversized one. | Multi-segment origination and ingress match their whole-buffer forms (origination returns the built bundle's id and forwards it; ingress delivers locally). A producer dropped before `Final` cancels an origination (`StreamCancelled`, nothing enters custody) and is refused at ingress (`Acceptance::Refused`, nothing delivered), even once every byte of the bundle has arrived. Unregistration wakes a parked consumer with `StreamCancelled` at once. The spoofed source is rejected. An originated bundle leaves with the Hop Count it was built with (origination is not a hop), and an undecryptable Hop Count is accepted and forwarded unchanged. | ## 5. Performance Benchmarks (REQ-13) diff --git a/bpa/docs/design.md b/bpa/docs/design.md index 9b56392a9..dc7179c33 100644 --- a/bpa/docs/design.md +++ b/bpa/docs/design.md @@ -208,6 +208,8 @@ Every `Accepted` resolves in exactly one of three ways: The deferred `Failed` retry loop is deliberately un-damped. An unreachable-but-routed peer cycles accept → probe → `Failed` → re-dispatch at the convergence layer's failure-discovery latency (for `tcpclv4`, a full dial cycle), bounded only by bundle lifetime. This is store-and-forward liveness for deployments without contact knowledge: with no route change and no inbound contact to wake `Waiting`, the loop is the only unilateral contact probe a plan-less static route has, and receiver-side deduplication absorbs any re-forwards it produces. Deployments with contact knowledge (contact plans, TVR) never run it — withdrawing the route parks the queue quietly. The churn is isolated per peer by the CLA's admission bound, and its pacing is a policy concern: retry damping is deferred to the policy subsystem with the rest of retry policy, and peer-health evidence feeding the RIB is routing-redesign work. The wrong-queue signal is unaffected: an address the CLA does not serve is still refused synchronously with `NoNeighbour` (whole-queue reset), and a removed peer's queues are swept event-driven at removal — dial exhaustion is the one case that moved, from link-scoped `NoNeighbour` to bundle-scoped `Failed`, because it is a probe result, not a queue-assignment error. +A synchronous failure — `forward` returning an error — is resolved by its kind. `StreamCancelled` reports a transfer cancelled before completion: a failed acceptance the bundle is valid to retry, so it re-enters Dispatch at once, exactly as a deferred `Failed` does, re-claimed out of `ForwardAckPending` by the same conditional swap. A CLA that cancels every attempt loops until the bundle expires. Any other error parks the bundle alone in `Waiting`, leaving the peer queue untouched: a synchronous failure can be deterministic and instantaneous, so it waits for the next routing or link event rather than re-running dispatch inline, and re-dispatches once if such an event landed while the transfer was in flight. + An outcome is honoured only if the named bundle is currently `ForwardAckPending` via a peer of the reporting CLA; anything else — already resolved, expired, another CLA's transfer — is logged and dropped. The check is enforced with a status-conditioned compare-and-swap on the persisted status (`MetadataStorage::swap_status`; the terminal `Completed` arm uses the conditional-tombstone form, `tombstone_if`, so a resolving bundle never transits a status another queue's poller could recover), so an outcome racing the peer sweep, the expiry reaper, or a duplicate of itself loses the swap and is dropped. Bundle lifetime is meant to bound retention even for a CLA that never resolves a transfer: a BPA-side guard timer times such a transfer out as outcome-unknown once its lifetime plus a grace period has passed, returning it to `Waiting`, where the dispatch expiry checkpoint drops it as a visible, attributable expiry. The guard timer is not yet built ([`TODO.md`](TODO.md)); until it is, a CLA that sits on a transfer holds it until its peer is removed or it unregisters, when the sweep resets it to `Waiting`. `ForwardAckPending { peer }` is persisted metadata status like any other, and is a holding state, not a queue (see [queue_architecture.md](queue_architecture.md)): bundles leave it only via the keyed outcome or a sweep (peer loss, or restart replay resetting it to `Waiting` exactly as `ForwardPending`, since registrations do not survive a restart) — the reaper defers it, because an accepted transfer cannot be recalled from the wire. The persisted status is the only state — outcome resolution is a metadata lookup by bundle ID. The retention cost is explicit: a bundle stays in the store from acceptance to outcome, bounded by the transfer duration and, for a CLA that never resolves it, by the life of the peer's registration — the price of honest reliability accounting, and how BP/LTP stacks already behave. So is a per-forward write cost: the claim persists `ForwardAckPending` before every offer, for every CLA — one extra metadata write per bundle even on the non-deferring path (a terminal `Sent` deletes it moments later) — unavoidable because an outcome can arrive before `forward` returns. diff --git a/bpa/docs/filter_subsystem_design.md b/bpa/docs/filter_subsystem_design.md index 02e01af47..a49c73c0c 100644 --- a/bpa/docs/filter_subsystem_design.md +++ b/bpa/docs/filter_subsystem_design.md @@ -106,7 +106,7 @@ Two findings carry the whole design: ## The filter kinds -Three kinds, defined in `bpa/src/filter/mod.rs` (`Verifier`, `Classifier`, `Rewriter`; rustdoc owns the signatures). Every invocation receives its kind's context — `VerifyContext`, `ClassifyContext` or `RewriteContext` — which lends the *wire* bundle through bpv7's own types — the `hardy_bpv7::Bundle` (primary block and structural block index) and a `hardy_bpv7::reader::Reader` over the block bodies — and the BPA-local record state (provenance, wire cache, classification, annotation slots) through a **separate getter** (ruling 2026-09-09): the immutable wire form and the record's mutable annotations are deliberately never one object, so the two cannot be confused. A `RewriteContext` also carries the Rewriter's `Boundary` and owns the `ExtensionEditor` its edits go through. The contexts are the extension point for filter inputs: a new input is a new getter, not a breaking change to every implementor, and the getters keep the reader's lifetime out of implementors' signatures. The reader is a `bpsec::DecryptingReader` over the resident source bytes, the decoded BCB OperationSets, and a `KeySource` resolved through the existing KeyProvider seam. A block body reads as an `Availability`: the plaintext or decrypted bytes when available, otherwise the state that says why not; a BCB-covered block the node holds no key for (`NoKey`) is the filter's no-match path, never an error it must handle, and `ReaderExt::extract` folds every unavailable state to `Ok(None)`. The reader memoises: one serves every filter that sees the same bytes (a whole input pass, the Verifiers closing the Deliver chain) and each Rewriter gets its own, so a covered block is decrypted at most once for the filters sharing it. Verifiers and Classifiers return the shared `Verdict` enum, so the drop path — and its status-report reason — is identical everywhere; a Rewriter returns nothing. +Three kinds, defined in `bpa/src/filter/mod.rs` (`Verifier`, `Classifier`, `Rewriter`; rustdoc owns the signatures). Every invocation receives its kind's context — `VerifyContext`, `ClassifyContext` or `RewriteContext` — which lends the *wire* bundle through bpv7's own types — the `hardy_bpv7::Bundle` (primary block and structural block index) and a `hardy_bpv7::reader::Reader` over the block bodies — and the BPA-local record state (provenance, wire cache, classification, annotation slots) through a **separate getter** (ruling 2026-09-09): the immutable wire form and the record's mutable annotations are deliberately never one object, so the two cannot be confused. A `RewriteContext` also carries the Rewriter's `Boundary` and owns the `ExtensionEditor` its edits go through. The contexts are the extension point for filter inputs: a new input is a new getter, not a breaking change to every implementor, and the getters keep the reader's lifetime out of implementors' signatures. The reader is a `bpsec::DecryptingReader` over the resident source bytes, the decoded BCB OperationSets, and a `KeySource` resolved through the existing KeyProvider seam. A block body reads as an `Availability`: the plaintext or decrypted bytes when available, otherwise the state that says why not; a BCB-covered block the node holds no key for (`NoKey`) is the filter's no-match path, never an error it must handle, and `ReaderExt::extract` folds every unavailable state to `Ok(None)`. The reader memoises: one serves every filter that sees the same bytes (a whole input pass, the Verifiers closing the Deliver chain) and the Rewriters of an output hook share one per snapshot of the attempt's staged edits, so a covered block is decrypted at most once for the filters sharing it. Verifiers and Classifiers return the shared `Verdict` enum, so the drop path — and its status-report reason — is identical everywhere; a Rewriter returns nothing. | Kind | May do | Mutates | Execution | Hooks | |---|---|---|---|---| @@ -120,7 +120,7 @@ Three kinds, defined in `bpa/src/filter/mod.rs` (`Verifier`, `Classifier`, `Rewr The two mutating kinds are duals by scope: the **Classifier** (inputs) writes *node-scoped* annotations — metadata this node's own downstream consumes; the **Rewriter** (egress) writes *network-scoped* annotations — extension blocks the next hops consume. A rule whose effect is local needs a delta; a rule whose effect must travel needs a block. -All three kinds are **streaming-immune** because none needs the payload to decide: at ingress, Verifiers and Classifiers need only the header blocks (plus any declared payload peek) and ride the pre-drain gate unchanged (`streaming_pipeline_design.md` §5.4); the Rewriter edits header blocks, which are resident at ClaSend where the per-hop rewrite already operates, while the payload streams past untouched. (A filter may still read a resident payload — a registered filter is trusted code — but nothing in the pipeline waits for one beyond the declared peek.) There is no late ingress pass — an early/late Ingress split would exist only to serve in-pipeline payload inspection, and with none there is nothing a late pass could see that the pre-drain pass cannot. +All three kinds are **streaming-immune** because none needs the payload to decide: at ingress, Verifiers and Classifiers need only the header blocks (plus any declared payload peek) and ride the pre-drain gate unchanged (`streaming_pipeline_design.md` §5.4); the Rewriter edits header blocks, which are resident at ClaSend where the per-hop rewrite already operates, while the payload streams past untouched. (A filter may still read a resident payload — a registered filter is trusted code — but nothing in the pipeline waits for one beyond the declared peek. At Egress the output door holds the stored bundle only until its headers are resident, so a payload still streaming from the store reads as `NotResident`; until a storage backend streams its load, the whole bundle is resident.) There is no late ingress pass — an early/late Ingress split would exist only to serve in-pipeline payload inspection, and with none there is nothing a late pass could see that the pre-drain pass cannot. The Classifier returns a *delta* rather than taking `&mut metadata` deliberately: the engine applies it, filters never touch `bundle.metadata` directly, the boundary stays clean for closed-source implementors, and delta application is idempotent — which the queue architecture's at-least-once semantics require of every processing block. A Classifier **sees the deltas applied by preceding links** of the same pass: the engine applies each delta before the next invocation. The reader/metadata split above is what makes this cheap and unambiguous — the reader lends only the wire form (invariant for the whole pass, one construction), while the `metadata` argument is re-borrowed per link over the advancing record state; a Classifier alters metadata through its delta and never the bundle itself. (An accumulate-then-apply variant — every link observing gate-time state, deltas merged after the pass — was considered and rejected 2026-09-09: per-link visibility is the designed behaviour.) @@ -154,14 +154,14 @@ The frozen chains run **synchronously, inline on the calling task** (`filter/eng - Filter invocations are synchronous by trait signature — async methods would exist only to serve callout-style filters (gRPC policy engines, database lookups), which the two-tier split places in the component tier; policy-as-code on the hot path has no await points. - The filters' `DecryptingReader` borrows the caller's decoded block map, buffer, BCB OperationSets, and key source — borrows that cannot cross a spawn boundary — so an invocation cannot migrate across tasks anyway. -"Parallel" Verifiers is therefore an **independence contract** — no ordering, no cross-talk between the Verifiers of one hook — not a spawning strategy, so the engine needs no task pool and no pool-deadlock reasoning. An empty chain costs one branch: nothing is parsed, nothing is allocated, so hooks nobody extends are near-free. Each Rewriter that edits costs a full copy of the wire form (its edits are materialised before the next link reads them), so M editing Rewriters hold up to M bundle copies at peak, per attempt — the whole-buffer interim; streamed egress (`streaming_pipeline_design.md` Phase B) materialises only the resident header region. +"Parallel" Verifiers is therefore an **independence contract** — no ordering, no cross-talk between the Verifiers of one hook — not a spawning strategy, so the engine needs no task pool and no pool-deadlock reasoning. An empty chain costs one branch: nothing is parsed, nothing is allocated, so hooks nobody extends are near-free. Each Rewriter that edits costs a snapshot of the attempt's staged edits for the next link to read: a header-sized copy of the block headers and the staged blocks' data, not a copy of the bundle. The bundle is materialised once per attempt, by the output door's rebuild. Two engine behaviours are load-bearing for the dispatcher's claim discipline: - **Every runner returns the bundle to the caller** — with every verdict (`ChainOutcome`; the Egress runner, which has none, returns the rewritten pair). No chain has an error path. The output hooks run inside a claimed window (`ForwardAckPending`/`DeliveryAckPending`), and the site that claimed the bundle resolves the claim with the same record on every exit — no re-fetch, no restore path, conditional swaps stay clean. -- **Each Rewriter sees its predecessors' edits**: the engine materialises every invocation's accepted edits into the wire form (and re-indexes the block map) before the next invocation reads it — the mutation-side mirror of the Classifier delta rule. +- **Each Rewriter sees its predecessors' edits**: the Rewriters edit the attempt's one editor, and each reads a snapshot of the edits staged before it (`hardy_bpv7::editor::StagedView`, re-taken only after a link that edits), never its own — the mutation-side mirror of the Classifier delta rule. The scheduled removals before them and the Egress per-hop writes after them edit the same editor, so the output door rebuilds the bundle once. -**A Rewriter execution failure is a fail-stop, not an error (ruling, 2026-09-09).** The engine has no recoverable error path: the input chains decode nothing themselves, and an output chain's decode failure is fatal too (below). A *Rewriter* failure — an edit the editor accepted but could not materialise, or an edit whose materialised bytes do not re-parse — is treated like a storage fault: the rewrite was meant to work and has not, so the correctness of every subsequent processing step is undefined, and there is no error a caller could react to appropriately (parking and retrying re-runs the same broken Rewriter forever; skipping it transmits a bundle the operator's own policy code failed to produce). The engine panics naming the failing link's pack-prefixed label, and the panic aborts the process (every pipeline task runs under the `TaskPool` watchdog); the message is formatted only on that path. The bundle stays stored and restart recovery re-queues it, so a deterministic failure recurs on every restart — the node crash-loops until the bundle is removed. That is the cost the ruling accepts, and why the call-time refusals below matter: they keep a correct Rewriter off this path. Rejected alternative, so it is not relitigated: a labelled error type (`RewriteError { label, source }`, the 2026-08-28 patch's `Error::Rewrite` shape) surfacing through `ChainOutcome`'s error path — it turns an undefined-state condition into a parkable status transition the dispatcher then has to have an answer for. Note the boundary: a Rewriter *refusal* (an edit rejected by the `ExtensionEditor` at call time) is the filter's own no-match path and entirely healthy; only `editor.finish()` failing or the materialised bytes failing to parse is the fault. The editor keeps that boundary where the bundle is attacker-chosen: whatever a receiver would reject — a `report_on_failure` flag on an administrative-record or null-source bundle and an unrecognised CRC type, which the parser rejects, and Previous Node, Bundle Age, or Hop Count data that does not decode, which BPA ingress rejects — is refused at call time with the rejecting check's own error, so a correct Rewriter meets a refusal, never the fail-stop. +**A Rewriter execution failure is a fail-stop, not an error (ruling, 2026-09-09).** The engine has no recoverable error path: the input chains decode nothing themselves, and an output chain's decode failure is fatal too (below). A *Rewriter* failure — an edit the editor accepted that the output door's rebuild then cannot materialise — is treated like a storage fault: the rewrite was meant to work and has not, so the correctness of every subsequent processing step is undefined, and there is no error a caller could react to appropriately (parking and retrying re-runs the same broken Rewriter forever; skipping it transmits a bundle the operator's own policy code failed to produce). The BPA panics, and the panic aborts the process (every pipeline task runs under the `TaskPool` watchdog). Every link's edits materialise together in the door's one rebuild, so the panic cannot name a link; the call-time refusals below are what keep a correct Rewriter off this path, and a rebuild failure past them is a BPA or bpv7 bug. The bundle stays stored and restart recovery re-queues it, so a deterministic failure recurs on every restart — the node crash-loops until the bundle is removed. That is the cost the ruling accepts, and why the call-time refusals below matter: they keep a correct Rewriter off this path. Rejected alternative, so it is not relitigated: a labelled error type (`RewriteError { label, source }`, the 2026-08-28 patch's `Error::Rewrite` shape) surfacing through `ChainOutcome`'s error path — it turns an undefined-state condition into a parkable status transition the dispatcher then has to have an answer for. Note the boundary: a Rewriter *refusal* (an edit rejected by the `ExtensionEditor` at call time) is the filter's own no-match path and entirely healthy; only the rebuild of the accepted edits failing is the fault. The editor keeps that boundary where the bundle is attacker-chosen: whatever a receiver would reject — a `report_on_failure` flag on an administrative-record or null-source bundle and an unrecognised CRC type, which the parser rejects, and Previous Node, Bundle Age, or Hop Count data that does not decode, which BPA ingress rejects — is refused at call time with the rejecting check's own error, so a correct Rewriter meets a refusal, never the fail-stop. **An output chain's decode failure is fatal too (ruling, 2026-10-02).** Anything that gets into the system is validated at the door, so a stored bundle the BPA later fails to process has met a BPA bug or storage corruption, and both are fatal, like a storage fault. The bytes the BPA stores without parsing are the `Builder`'s own, valid by construction — at the ADU originate door and in the status reports the BPA originates — so a `Builder` or `BuiltReceiver` defect surfaces as this fail-stop rather than as a refusal; the tests pin the streamed build byte for byte to `Builder::build`'s output. The Egress and Deliver chains decode bytes that were validated at ingress and read back from storage, so a decode failure there panics, as a Rewriter failure does. Parking the bundle instead would only re-run the same failure at every routing event or registration until the bundle expired. @@ -181,12 +181,14 @@ load from store (stored bundle = as received; source of **The scheduled removals come first (ruling, 2026-10-05).** The §5.1.1 failure-drops and `delete_block_on_failure` removals the ingress gate schedules apply ahead of the Egress Rewriters, as they apply ahead of the Deliver chain, so the filters see the bundle as it will travel or be delivered, and a removal can never delete a Rewriter's insert into a removed block's number. A removal the BPSec cascade cannot complete for want of a key is retained, and the bundle proceeds; any other failure applying them is a BPA bug or storage corruption, and fatal. -**The per-hop writes follow the Rewriters (ruling, 2026-10-02).** The per-hop writes are the BPA's RFC 9171 §5.4 duties for this hop, and Bundle Age is to be increased "at the last possible moment before the CLA initiates conveyance", so they run last, after every Rewriter and before the BPSec seam, which then protects their output as well as the Rewriters'. They therefore supersede a Rewriter's edits to the blocks they write: the Previous Node always; the Hop Count when the bundle arrived with one this node could read and can update; the Bundle Age when the bundle arrived with one or has no creation clock. A Rewriter can no longer strip a clockless bundle's Bundle Age, and the Rewriters read the stored per-hop blocks, matching the record's extension-field cache. A per-hop block a Rewriter adds where the BPA writes none travels as the Rewriter wrote it. The Rewriters still sit before the BPSec seam, so the blocks they add or modify can be signed per security policy. +**The per-hop writes follow the Rewriters (ruling, 2026-10-02).** The per-hop writes are the BPA's RFC 9171 §5.4 duties for this hop, and Bundle Age is to be increased "at the last possible moment before the CLA initiates conveyance", so they run last, after every Rewriter and before the BPSec seam, which then protects their output as well as the Rewriters'. They therefore supersede a Rewriter's edits to the blocks they write: the Previous Node always; the Hop Count when the bundle arrived from a peer with one this node could read and can update; the Bundle Age when the bundle arrived with one or has no creation clock. A Rewriter can no longer strip a clockless bundle's Bundle Age, and the Rewriters read the stored per-hop blocks, matching the record's extension-field cache. A per-hop block a Rewriter adds where the BPA writes none travels as the Rewriter wrote it. The Rewriters still sit before the BPSec seam, so the blocks they add or modify can be signed per security policy. -**The Hop Count increment yields to BPSec (ruling, 2026-10-05).** RFC 9171 §4.4.3 makes the increment a SHOULD, so where the per-hop write cannot safely update the Hop Count block — BCB-covered, beside an encrypted BIB that may cover it — or this node could not read it at ingress, the block travels unchanged with its security operations rather than holding the bundle back. Ingress and origination accept a Hop Count block this node cannot decrypt, enforcing no hop limit on it: a loop through nodes that all lack the key is bounded by the bundle's lifetime alone, while any key holder on the path still enforces the limit. A Hop Count block whose ciphertext fails authentication is scheduled for removal with its BCB (RFC 9172 §5.1.1), so the bundle travels on with no Hop Count and its lifetime alone bounds a loop on the rest of its path. The Previous Node and Bundle Age writes are MUSTs: a block of either kind under such coverage parks the bundle until the per-hop BPSec handling replaces the park. +**The Hop Count increment yields to BPSec (ruling, 2026-10-05).** RFC 9171 §4.4.3 makes the increment a SHOULD, so where the per-hop write cannot safely update the Hop Count block — one an encrypted BIB may cover (`BibCoverage::Maybe`: BCB-covered beside an encrypted BIB this node could not decrypt) or does cover — or this node could not read it at ingress, the block travels unchanged with its security operations rather than holding the bundle back. The per-hop writes judge coverage by the stamps ingress derived, keyed where this node holds the keys: a block a decrypted BIB provably does not cover is updated, stripped from any BCB and written in plaintext, this node being the acceptor of the operations on a block it must update. Ingress and origination accept a Hop Count block this node cannot decrypt, enforcing no hop limit on it: a loop through nodes that all lack the key is bounded by the bundle's lifetime alone, while any key holder on the path still enforces the limit. A Hop Count block whose ciphertext fails authentication is scheduled for removal with its BCB (RFC 9172 §5.1.1), so the bundle travels on with no Hop Count and its lifetime alone bounds a loop on the rest of its path. **Origination is not a hop (ruling, 2026-10-05):** the originating node never writes the Hop Count, so a locally originated bundle leaves with the count it was built with, and the next node makes the first increment. The Previous Node and Bundle Age writes are MUSTs: a block of either kind under such coverage parks the bundle until the per-hop BPSec handling replaces the park. The IPN legacy re-encode (3-element `ipn` EIDs re-encoded 2-element for peers that require the older form) is a built-in of this stage rather than a registered filter, by this design's own taxonomy: it rewrites the *primary block*, which is out of every Rewriter's scope by construction, and it is per-hop wire adaptation, not embedder policy. It is driven by `BpaBuilder::ipn_legacy_peers` (EID patterns matched against the resolved next hop) and — like every rewrite in this stage — applies only to the transmitted wire form: the record's own primary is never replaced, so id-keyed resolution after transmission (tombstones, dedup) always compares like with like. A BPSec operation that covers the primary block, as its target or through its scope (RFC 9173's default scope includes the primary in every operation's input), cannot survive the re-encode, so such a bundle is dropped at a legacy next hop with `UnexpectedSecurityOperation`, which RFC 9172 §7.1 notes may identify a security-policy misconfiguration. bpv7's editor decides it: it refuses the primary edit while a BIB targets the primary or any remaining operation's scope includes it, an unreadable encrypted BIB counting as one, and the stage maps that refusal to the drop, so the BPA reads no security context's parameters. The refusal is judged after the per-hop writes, so a BIB that covered only the blocks those writes replace, stripped and emptied by them, no longer stands in the way. +**Deliver makes no per-hop writes and no fixed strip.** At Deliver the BPA neither strips nor updates the RFC 9171 per-hop blocks (Previous Node, Hop Count, Bundle Age): those writes are this node's duties toward the next hop (RFC 9171 §5.4), and a delivery has no next hop. A raw `Service` receives the blocks as they arrived, less the scheduled §5.1.1 removals and any Deliver Rewriter's edits; an embedder that wants them gone registers a Deliver Rewriter, which may edit them like any other unprotected extension block. A fixed strip before the raw door was rejected: the service asked for the bundle, and the last hop's Previous Node and the Hop Count describe that bundle's journey, so removing them is policy, which is what the pluggable Rewriter is for. An `Application` receives the payload alone, so the question does not arise there. + The fixed byte-owners elsewhere are unchanged: - **Ingress**: the parser — canonical rejection (never rewrite incoming wire bytes), BPSec verify/decrypt, and the scheduling of the RFC 9172 §5.1.1 removals, which leaves the stored bytes as received: the output doors apply the removal cascade per attempt. @@ -204,7 +206,7 @@ The fixed BPSec machinery named above follows the RFC 9172 §1.4 role model, map | **Originate · Service trait** | caller = Source for payload; BPA = Source for extension blocks | structurally validate; verify caller's security | add BPA extension blocks; secure them at the BPSec-egress seam | reject to caller | | **Originate · Application trait** | BPA = Source | validate built bundle | build; BIB/BCB per policy at the BPSec-egress seam | reject to app | | **Receive** (CLA → BPA) | BPA = **Verifier** | parse; verify BIBs pre-drain; decrypt extension blocks to read; classify unsupported; payload BIBs incrementally during the drain (`deferred_verifiers`, begun in the header pass) | none to the stored bytes: *schedule* (`to_remove`) the failure-drop of corrupt non-payload blocks + their security blocks and the drop of `delete_block_on_failure` unknowns, applied per attempt at Forward/Deliver; keep valid encrypted | block → drop block (at the output doors); bundle-level → drop bundle; NoKey → keep | -| **Deliver · Application trait** | BPA = **Acceptor** | decrypt payload (`block_data`); optionally verify payload BIB | consume → transient plaintext to app; bundle deleted after | payload fail → drop bundle; NoKey → watch | +| **Deliver · Application trait** | BPA = **Acceptor** | decrypt payload (`block_data`); optionally verify payload BIB | consume → transient plaintext to app; bundle deleted after | payload decrypt fail → drop bundle (`FailedSecurityOperation`; an unrecognised context `UnknownSecurityOperation`); NoKey → park for the next registration | | **Deliver · Service trait** | service is the Acceptor; BPA passes through | none | none — hand raw encrypted bytes to service | service's responsibility | | **Forward** (BPA → CLA) | Forwarder (default); waypoint Acceptor/Source by policy | none by default; waypoint acceptor decrypts/verifies its op | per-hop blocks; optionally waypoint-accept at the BPSec seam → decrypt + strip + §5.1.1 | waypoint fail → §5.1.1 (drop block; payload → drop bundle) | @@ -240,7 +242,7 @@ The residual case externalisation cannot reach — post-decrypt plaintext at Del ## Registration: packs, frozen at `build()` -Filters ship in **`FilterPack`s** (`filter/pack`), the embedder's shipping unit reifying a filter pair's common construction code: a pack registers filters (per-hook methods, e.g. `ingress_classifier`, `egress_rewriter`), then `BpaBuilder::add_filters(pack)` splices it into the per-hook chains. The annotation slots a pair shares are declared, not registered (see [Annotation slots](#annotation-slots--embedder-private-metadata)). **Chain order is call order** — within a pack and across `add_filters` calls — lexically visible in the embedder's construction code. `build()` freezes each chain into a plain immutable slice the hot path iterates without synchronisation and fixes each input hook's payload-peek P; it has nothing to validate, so filter registration cannot fail it. +Filters ship in **`FilterPack`s** (`filter/pack`), the embedder's shipping unit reifying a filter pair's common construction code: a pack registers filters (per-hook methods, e.g. `ingress_classifier`, `egress_rewriter`), then `BpaBuilder::add_filters(pack)` splices it into the per-hook chains. The annotation slots a pair shares are declared, not registered (see [Annotation slots](#annotation-slots--embedder-private-metadata)). **Chain order is call order** — within a pack and across `add_filters` calls — lexically visible in the embedder's construction code. `build()` freezes each chain into a plain immutable slice the hot path iterates without synchronisation and fixes the payload-peek P of each input hook and of Deliver; it has nothing to validate, so filter registration cannot fail it. There is no registry object, no registration verb on a running BPA, and no name-keyed dependency resolution between filters: the component registries (CLA / Service / RoutingAgent, and AdminRecord once built) are lock-guarded dynamic collections because components genuinely come and go at runtime; filters do not. Registered filters have no lifecycle — the BPA owns them from `add_filters` until shutdown, with no unregistration and no early teardown; a filter that needs its own lifecycle is a component. A `label` argument survives only as a diagnostic for logs and metrics, prefixed `".