Skip to content

AS-IS.md

Latest commit

 

History

History
262 lines (215 loc) · 15.6 KB

AS-IS.md

File metadata and controls

262 lines (215 loc) · 15.6 KB
Raw

VectorMBE — AS-IS (Stage 0 baseline)

Supersedes the Phase 0 baseline taken at d7898e7 (commit 0ed9fe9); that text remains in git history. This is a re-baseline because HEAD advanced 13 commits and the surface changed.
Baseline commit 0a71b35 (feat(simulation): Phase 1 — simulate graph blocks with no FMU upload)
Working tree not clean — src/api/mod.rs, src/simulation/mod.rs, ui/src/graph-engine/GraphWorkspace.tsx, ui/src/styles.css modified by a concurrent author (see §9)
Binary under test target/release/vectormbed, built 05:40:33Z from the tree as of 05:35Z
Instance under test 127.0.0.1:18081, data dir /home/node/vmbe-audit2/data, VECTORMBE_STARTUP_GRAPH=demo
Untouched prod (:8080, docker), and the prior session's audit instance (:18080, pid 36458)

Observation only. No product code was changed in this stage.

1. What it claims to do

From Cargo.toml and README.md:

Neuro-symbolic MBSE substrate: OWL-backed graph + vectors + anchors + MCP … so engineers, CI pipelines, and LLM tools all reason over the same versioned structure.

Claimed standards alignment: INCOSE SE Handbook v4, OMG UML 2.5.1, SysML v1.6/v2, MOF 2.5.1, SAE AS5506 (AADL), DoDAF DM2, ARP4761.

Three claimed layers:

  • Symbolic — OWL TBox (src/tbox/vectormbe-tbox.ttl, 1014 lines), SHACL (src/shacl.rs, 844 lines, oxirs-shacl), SPARQL (POST /query/sparql), reasoning gate (src/reasoning/).
  • Neural — LLM generation (design / requirements / spec / code / diagrams) via VECTORMBE_LLM_PROVIDER (ollama by default), embeddings + HNSW/Qdrant retrieval.
  • Context — MCP server (mcp/, 18 tool modules) plus in-process MCP routes.

2. Surfaces that exist

Surface Entry point Reachable today?
HTTP API vectormbed (axum), 166 .route( registrations Yes
Web UI (dev) ui/ Vite+React, :5173 Yes — 200, currently running
Web UI (prod) Caddy serving www/ on :80, /api/* → vectormbed Yes — 200
CLI vectormbe-cli — thin HTTP client Yes
TUI vectormbe-tui (React Ink) Yes
MCP server mcp/ — 18 tools; node_modules is now present Framing still non-compliant (F-3): mcp/transport/stdio.ts emits Content-Length: (LSP) framing; MCP requires newline-delimited JSON-RPC
WASM / Tauri vectormbe-wasm, src-tauri Not exercised

3. Data model as actually seeded

VECTORMBE_STARTUP_GRAPH=demo on a clean data dir → 265 nodes / 529 edges.

Entity kinds: InputPin 54, OntologyClass 48, OutputPin 33, BomPart 23, Function 22, Signal 16, Component 15, Requirement 13, Guard 11, State 7, BomConfigGroup 6, Supplier 5, Interface 5, BomAssembly 4, Constraint 2, System 1.

Relation kinds: contains 173, flow 80, satisfy 60, allocated_to 55, sysml_specializes 52, option 20, satisfies 20, enables_transition 11, transition 11, has_guard 11, supplied_by 9, constrains 9, drives 7, connect 6, allocatedTo 4, feeds 1.

Two carried-forward observations, both reproduced on this baseline:

  • The seed still emits two spellings, in opposite directions, for both SysML relations that matter to traceability: satisfy/satisfies and allocated_to/allocatedTo (F-7, OPEN).
  • A snapshot edge is exactly {source, target, type}. There is no provenance field, so the 18 links the seeder infers at boot (inferred_links=18) are indistinguishable from asserted ones in the only whole-graph export the product offers (F-11, OPEN).

The startup log reports entities=174 relations=412 for the scenario, then the checkpoint two lines later reports entities=265. The two counts in the same boot disagree.

4. Baseline measurements

  • cargo check --release -p vectormbed: 54.5s warm. cargo build --release -p vectormbed: 5m17s.
  • Compiler warnings: 13, all in the vectormbe lib — 5 unused imports/variables, 6 dead functions/fields (model_file_path, node_label, two xmi.rs helpers, in_edges, is_gen), 2 never-read assignments. Zero in vectormbed.
  • Cold startup: 2.53s wall (t0 05:42:03.784Z → first 200 at 05:42:06.313Z), demo seed + inference included.
  • Test suite: see §7.
  • stderr at boot: MCP runtime initializing, MCP runtime running without data service, and Qdrant not configured are each emitted twice — the MCP runtime is initialised twice. One WARN that API-key auth is disabled and all requests are Admin. Nothing fatal.
  • No health endpoint. /health, /healthz, /status, /api/health, /readyz → all 404. Liveness has to be inferred from a data route.

5. API/documentation coverage

166 .route( registrations; GET /openapi.json documents 73 paths. README calls it the "Full OpenAPI catalog" (F-6, OPEN). The gap is one-directional — documented routes resolve; real routes are missing from the docs (/twin/*, /corpus/*, /simulation/*, /bom/*, /ci/*, /vc/*, /graph/impact/*, /ontology/drift, …).

6. Where the engineering semantics live

  • EARS: the token EARS still appears in exactly one place in the repo — ui/src/graph-engine/ui/RequirementsGeneratorTab.tsx, as a client-side prompt string. There is no server-side EARS grammar, validator or classifier. Whether generated text is EARS is unverified by the product (F-15, OPEN).
  • Traceability: GET /ontology/traceability. On this baseline it returns 13 rows, 0 orphans, 0 rows where satisfied_by == allocated_functions, and surfaces relation labels satisfy, satisfies, allocated_to. The F-1 and F-2 fixes hold at 0a71b35.
  • Ontology: TBox declares vmbe:Bus ("a shared communication bus … e.g. MIL-STD-1553, CAN") and a "communicates via shared interface or bus" relation. It declares none of NetworkNode, BusSegment, Link, hopCount, latencyBudget, bandwidthCapacity, zone, physicalDistance. Stage 3F is greenfield.

7. Test suite

cargo test --workspace --release, 05:42:34Z → 05:47:04Z (4m30s), exit 0:

518 passed · 0 failed · 2 ignored across 13 binaries.

Suite Tests
lib unit (vectormbe) 381
model_file_tests 73
api_acceptance 39
import_coverage_tests 6
agent_stream_tests 5
pdf_fixture_validation 5
fmi_tests 4
xmi_import_test 2
http_api_smoke 1
doctests 1 passed, 2 ignored
binaries (vectormbed, vectormbe-cli, vectormbe-keytool) 0–1 each

Green, but not a reproducible green: the tree at the start of the run was 0a71b35, and at the end it was 376f8c1. See §9.

Test-time warnings add one not seen in the release build: unused variable: eid.

8. Substrate already present for Stages 3–5

This matters because it changes what Stage 3 and Stage 5 are: extension in some places, replacement in others.

Stage 3 (geometry) — partial substrate, one real ARXML fixture.

  • src/import/arxml.rs (roxmltree) exists and maps AUTOSAR tags to entity kinds. It recognises can-cluster | lin-cluster | flexray-cluster | ethernet-cluster | j1939-cluster → kind Network, and ecu-instance → kind Component.
  • test-data/autosar-body-control.arxml (660 lines) carries genuine medium semantics — the discriminator Stage 3D requires:
    Cluster Declared type Rate
    BodyCANBus CAN-CLUSTER, <PROTOCOL>CAN 500 kbit/s
    InteriorLINBus LIN-CLUSTER, <PROTOCOL>LIN 19.2 kbit/s
    DiagnosticEthernetBus ETHERNET-CLUSTER, <PROTOCOL>ETHERNET 100 Mbit/s
    It also carries frame timing (CYCLE-TIME-VALUE 20 ms / 100 ms, CAN-ID, LENGTH,
    START-POSITION, BIT-LENGTH) — enough to compute CAN bus load.
  • But node-to-segment membership is mostly undeclared. Of 3 ECU-INSTANCEs, only BCM_ECU declares COM-CONTROLLERS (CAN + Ethernet). BCS_ECU and SJB_ECU declare none. There are no PHYSICAL-CHANNEL elements. The LIN cluster's only node is a LIN-SLAVE not tied to any ECU. ui/public/test-data/ev-powertrain-autosar.arxml (783 lines) has 1 ECU instance and no clusters at all. → On today's fixtures a faithful classifier yields mostly single-node segments plus UnknownGeometry. This is a Stage 3 gate risk and is raised as Q-3 below, not silently worked around.
  • src/topology/ (2046 lines) is graph analytics, not network topology: detect_cycles, find_critical_paths, find_orphans, detect_spofs, betweenness_centrality, pagerank, detect_subsystems, run_impact_analysis. Routes /topology/analysis, /topology/score, /topology/subsystems. Nothing here classifies a network. detect_spofs is the nearest neighbour to Stage 3G's minimum-cut work and should be read before writing it.

Stage 3C classification — an incumbent exists and does not conform. ui/src/graph-engine/layout/naturalTopologyLayout.ts:279 classifyTopology() returns exactly one of star | tree | mesh | geodesic | layered | hybrid. It is a weighted heuristic over mean clustering coefficient, degree centrality and community count, with unnamed constants (0.85, 0.2, 0.15). It has no bus, no ring, no line, no point-to-point, no full/partial mesh distinction; it returns a single winner rather than a lattice; it never consults medium type; and it classifies the whole graph rather than per-segment. Every one of those is a Stage 3 requirement it violates. Stage 3 replaces it rather than extending it.

Stage 5 (3D visualisation) — substantial substrate.

  • ui/ already depends on three + @react-three/fiber + @react-three/drei, and has graph-engine/gpu/GpuGraphCanvas.tsx, render/{scene,nodes,edges,LabelOverlay,LabelRenderer, CameraController,ViewportCuller,viewport,webglDebug}. WebGL is in place; the 5G instancing, GPU-picking and LOD requirements need verification against it, not from scratch.
  • 15 layout modules, 3164 lines: forceLayout (571), naturalTopologyLayout (633), embeddingLayout (302), topologyAnalysis (321), networkLayout (220), taxonomyLayout, ontologyLayout, hierarchicalLayout, dependencyLayout, louvainClustering, pcaTensor, cubeLattice, layoutNormalize. Plus ui/src/diagrams/layout.ts (1519 lines) with dagre and elkjs (elk-layered, elk-stress, elk-tree) for the 2D diagram surface.
  • Stage 5A already has a conflicting incumbent. src/topology/mod.rs:27 SemanticLayer defines 9 layers (Requirements, PhysicalArchitecture, LogicalArchitecture, RuntimeDependency, SoftwareDependency, Verification, OrganizationalOwnership, MissionThread, Unknown) with a hardcoded y_offset() z-stack. This is not the Stage 5A stack (no Behavioral, no Network/Communication, no Physical/Zonal), and membership is inferred by substring matching on the entity kind string (kind.contains("test") → Verification), which is the opposite of 5A's "derived from the ontology, never hardcoded per model". Reconciling these two stacks is a Stage 5 decision, flagged as Q-4.

9. Audit hygiene — the finding that shapes everything else

/home/node/vmbe-src is a live shared checkout with an actively committing concurrent author, and it moved twice during this stage alone:

Time (UTC) Event
05:32 Stage 0 begins. HEAD = 4ce36e6. Untracked src/simulation/mod.rs (31 KB), src/api/mod.rs +90
05:35–05:40 I build vectormbed from that tree
05:41 HEAD is now 0a71b35 — the simulation work was committed under me
05:41 Working tree dirty again: src/api/mod.rs +38/−11, src/simulation/mod.rs +115/−4
05:42–05:47 Full test suite runs. It starts at 0a71b35 and ends at 376f8c1
05:48 HEAD is now d9fbdbf (feat(canvas): clear the graph from the canvas toolbar)

Four commits landed under me during Stage 0 alone (4ce36e6 → 0a71b35 → 376f8c1 → d9fbdbf), one of them mid-test-run. Commit cadence over the preceding hour was one commit every 2–20 minutes. This is logged as F-17 and is the single most important thing to resolve before Stage 1, because it defeats Operating Rule 2 ("never claim a fix works without re-running the exact scenario") and Rule 4 ("small independently revertible commits") — I cannot attribute an observed behaviour to a known revision if the revision changes mid-journey.

Also carried forward: pkill -x vectormbed matches the prod container's process from the host PID namespace. The audit daemon is addressed only by recorded PID (73361), never by name.

10. Stage 0 findings

ID Severity Status Title
F-1 BLOCKER FIXED /ontology/traceability reports false orphan requirements
F-2 MAJOR FIXED allocated_functions populated from satisfy edges
F-3 BLOCKER OPEN MCP stdio transport uses LSP Content-Length framing
F-4 MAJOR OPEN /entities/{id}/neighborhood returns 200 for a nonexistent entity
F-5 MINOR OPEN Release binary cannot run standalone (no RPATH for libtorch)
F-6 MINOR OPEN OpenAPI documents 73 of 166 routes, described as "full"
F-7 MAJOR OPEN Seed emits two spellings and two directions for satisfy/allocate
F-8 MINOR OPEN First-run onboarding modal not dismissable by keyboard or backdrop
F-9 MAJOR OPEN /llm/status reports configured: true for a model it does not have
F-10 BLOCKER OPEN design/generate silently writes inferred edges onto the baseline
F-11 BLOCKER OPEN No edge provenance: inferred and asserted relations indistinguishable
F-12 MAJOR OPEN Design generator emits satisfies in the non-canonical direction
F-13 MAJOR OPEN Generated has_input/has_output contradict the function's own pins
F-14 MAJOR OPEN design/generate preview references element IDs it never defines
F-15 MAJOR OPEN Generated requirements are "shall" prose, not EARS, several unverifiable
F-16 MINOR OPEN /entities/search/semantic silently degrades to lexical
F-17 BLOCKER OPEN Checkout mutates under the audit — no reproducible baseline (new)
F-18 MINOR OPEN No health/readiness endpoint on any conventional path (new)
F-19 MINOR OPEN MCP runtime initialises twice at boot (new)
F-20 MINOR OPEN Boot log reports 174 entities; checkpoint 15 ms later reports 265 (new)

F-3 through F-16 were raised by the prior session and re-confirmed as still present at 0a71b35 by inspection this stage; each carries a live reproduction in REVIEW_LEDGER.md and will be re-run against this instance during Stage 1 rather than trusted.

11. Open QUESTIONs

  • Q-1 (F-7 direction). SysML v1.6: satisfy has source = satisfying element, target = requirement. The seed uses both directions. Confirm the canonical direction before I migrate the seed — this changes stored data, not just queries.
  • Q-2 (F-11 provenance). Should inferred edges be distinguishable from asserted ones in /graph/snapshot? Making them so is a schema change, so I am asking rather than assuming.
  • Q-3 (Stage 3 ground truth). The shipped ARXML declares medium type but not ECU-to-segment membership for 2 of 3 ECUs. Gate 3 requires classification to match "the actual known bus/ECU arrangement". Is there a richer ARXML I should be given, or is the correct Stage 3 output on these fixtures a mostly-UnknownGeometry gap report?
  • Q-4 (Stage 5A stack). SemanticLayer already defines a 9-layer z-stack that conflicts with the Stage 5A stack. Replace it, or map the new stack onto it?