+ Progress dossier
+
+
+ snapshot · 2026-07-31
+ ASTRA × Lightcone × HAPS
+
+ We turned a scientific record into a deterministic remote-execution
+ sequence where every boundary has an owner and emits something
+ inspectable. The experimental lightcone-haps backend
+ can carry the same resolved task to another HAPS-enabled trust
+ domain when a matching, approved application is deployed there;
+ MCP is an optional agent-facing interface, not the execution design.
+ A fresh
+ OpenQCD run reproducing the scheduler-template/SEB wire contract
+ completed on Jean Zay in 271 seconds and returned four locally
+ hashed outputs.
+ A scheduler-driver/MCP candidate passed focused offline tests, then
+ the backend and case bindings were restored to preserve the frozen
+ demo request IDs. The current combined checkout is therefore
+ intentionally not yet an integrated scheduler path.
+
+ This is the complete demo path, not only the intent and job objects. + It shows who authors scientific choices, who resolves + them, where the project-specific HAPS mapping lives, which component + owns remote lifecycle state, how a runtime reaches a centre scheduler, + and which evidence is returned or still missing. +
++ This page remains the provider-independent architecture and evidence + narrative. The notebook is its executable companion: it discovers the + workspace, calls the real CLIs, keeps network preflight read-only and + requires an explicit request-ID gate before remote mutation. The + step-by-step panel below links each general boundary to its notebook cue. +
++ Use the HTML to explain why ASTRA resolution, the project binding, + the durable lifecycle and optional MCP façade remain separate. No + case-specific notebook control flow defines this architecture. +
+ follow the ten architecture boundaries ↓ +astra-lightcone-haps.ipynb+ Use the notebook backstage to inspect concrete files, run Uppercase + or OpenQCD through the same generic lifecycle and show fail-loud + network behavior. It is an audit console, not a second execution engine. +
+ open operator notebook ↗ + read notebook roles ↗ +astra validate
+ lc resolve
+ lc-haps compile
+ lc-haps prepare
+ gated submit
+ status / wait
+ fetch
+ verify
+ report
+
+ Active .lightcone/ artefacts describe the current notebook
+ preparation; evidence/ directories preserve earlier runs.
+ Their request IDs are expected to differ when any resolved byte or
+ binding changes. The page never substitutes one identity for the other:
+ the preserved Uppercase record contains a 46-byte input, while the
+ current project input is 52 bytes and is rehashed before execution.
+
ASTRA decisions, one universe and verified input bytes are resolved before any execution provider is selected.
+ResolvedTask
+ The provider-neutral, content-addressed record is the stable input contract shared by every execution compiler.
+A declarative binding selects a catalog contract; the engine returns durable state, files, hashes and explicit gaps.
+
+
+ ResolvedTask, then expand
+ into declared execution modes, eligible resources and evidence.
+ Solid colored areas are implemented or recorded responsibilities.
+ Red dashed elements identify missing integration or provenance.
+ The gray dashed MCP path is optional and accesses the same HAPS APIs;
+ it does not replace ASTRA resolution, the binding or the persisted engine.
+ The frozen SVG is deliberately de-emphasized at rest; hover or focus
+ it for a moderately stronger inspection view.
+ On narrow screens, scroll the diagram horizontally or open the SVG.
+
+ + Select a component to reach its concrete example below. Uppercase + provides the complete persisted direct-execution record. OpenQCD is + introduced only where scheduler execution changes the contract. +
++ Every visible excerpt below is either verbatim from the linked file + or explicitly labelled as a projection joining unchanged recorded + values. No endpoint, credential or private profile value is embedded. +
++ The Uppercase case is deliberately small: one input, one + declared output and no decisions. The preserved completed run + used 46 bytes; the active notebook rehashes the current file + rather than reusing that historical value. It proves the transport + and verification sequence without pretending to be a scientific result. +
++ ASTRA owns scientific structure and semantics. Lightcone + receives this validated record later; neither HAPS nor MCP + interprets the project independently. +
+source_message resolves to data/message.txt.uppercase_message explicitly declares its input dependency.inputs:
+ - id: source_message
+ type: data
+ source: data/message.txt
+
+outputs:
+ - id: uppercase_message
+ type: data
+ inputs: [source_message]Exact excerpt. The recipe remains in the complete linked file.
+
+ ResolvedTask records what must run: selected
+ decisions, input hashes, output path and recipe. It contains
+ no HAPS application, buffer or credential concept.
+
+ Lightcone hashes the ASTRA record, universe and input bytes. + The same task can be consumed by the existing Snakemake + compiler or by the external HAPS compiler. +
+task_id is stable for this output and universe.task_sha256 covers the resolved material."task_id": "uppercase_message@quick",
+"inputs": [{
+ "bytes": 46,
+ "id": "source_message",
+ "logical_path": "data/message.txt",
+ "sha256": "8193306ae5f2…"
+}],
+"output": {"id": "uppercase_message"},
+"task_sha256": "730c477e2cae…"Exact values. Long digests are visually abbreviated; the linked JSON contains every byte.
++ The binding contains application-specific knowledge that must + remain reviewable data: MIME selector, ordered interface, + output capacity and receipt policy. Generic adapter code does + not branch on the case name. +
++ The restored experimental schema preserves the frozen demo + request IDs. It does not yet declare the direct versus + scheduler driver; that requires a versioned migration. +
+application/x-uppercase.out maps to ASTRA input source_message.binding_id: uppercase-aqmo
+target:
+ output: uppercase_message
+application:
+ selector:
+ fixed:
+ mime_type: application/x-uppercase
+ expected_interface:
+ arguments: []
+ inputs: [out]
+ outputs: [uppercase_message]Exact excerpt. Capacity and receipt policies continue in the linked file.
+
+ compile_to_haps() verifies identifiers and input
+ hashes, maps interface names and canonicalizes the complete
+ request. The resulting request ID is what the operator confirms.
+
+ Compilation is deterministic and fail-loud. It cannot read + credentials or query a catalog. Changing a mapped field + changes the canonical content and therefore the request ID. +
+"application": {
+ "mime_type": "application/x-uppercase"
+},
+"inputs": {"out": {
+ "bytes": 46,
+ "from_input": "source_message",
+ "sha256": "8193306ae5f2…"
+}},
+"request_id":
+ "sha256:d579aa960971d324…"Exact values. Digests are visually abbreviated only in this card.
++ Preparation queries the HAPS catalog and compares the complete + ordered interface. Submission remains impossible until the + operator supplies the exact compiled request ID. +
++ The MIME selector finds one application, then its argument, + input and output names must exactly match the binding. + Ambiguity or drift fails before an ebuffer is created. +
+uppercase.13e2e8f6-8091-461c-b4c4-aacb533d0ac6."application_id":
+ "13e2e8f6-8091-461c-b4c4-aacb533d0ac6",
+"interface": {
+ "arguments": [],
+ "inputs": ["out"],
+ "outputs": ["uppercase_message"]
+},
+"mime_type": "application/x-uppercase",
+"request_id":
+ "sha256:d579aa960971…"Exact values. The confirmation gate compares against the full unabridged request ID.
++ Staging creates the input buffer, submission creates the output + buffer and remote job, and the engine records the association. + A later notebook or agent can resume using the same job ID. +
++ Local state protects ordinary reruns, but the HAPS server + does not yet accept a request key. The engine therefore + records the submit/write ambiguity instead of claiming + exactly-once semantics. +
+4101ca7f-b788-49f3-ab7c-982fb12bcbdc.c04b4ba6-edb6-434e-8040-14d9145d5799.completed."idempotency": {
+ "scope": "local-state-only",
+ "server_request_key_supported": false,
+ "submit_write_gap": true
+},
+"job_id":
+ "c04b4ba6-edb6-434e-8040-14d9145d5799",
+"phase": "fetched",
+"remote_status": "completed"Exact excerpt. The gap is recorded rather than hidden behind a success badge.
+
+ ebservice owns application discovery and job
+ lifecycle. ebuffer moves ephemeral bytes. A
+ maintainer runtime claims the MIME contract and executes the application.
+
+ These identifiers are related but not interchangeable. The + request ID names immutable intent, the application ID names + a catalog entry, the job ID names one remote execution and + buffer IDs name ephemeral transport objects. +
+catalog.mime = application/x-uppercase
+catalog.application = 13e2e8f6-8091-461c-b4c4-aacb533d0ac6
+transport.ebin.out = 4101ca7f-b788-49f3-ab7c-982fb12bcbdc
+execution.job = c04b4ba6-edb6-434e-8040-14d9145d5799
+transport.ebout = d30e26ea-e4cc-4909-8b13-e64ab7c791aa
+execution.status = completedProjection, not a new file. Every value is copied unchanged from the linked catalog, staging and submission records.
+
+ OpenQCD is not submitted like Uppercase. The successful IDRIS
+ route created a scheduler-event buffer tagged
+ template::slurm and associated it with the job.
+ Bare generic submissions failed quickly and are preserved.
+
+ The centre, seed and trajectory parameters are still ASTRA
+ decisions. The HAPS binding maps those values to MIME and
+ wire arguments. The execution driver must be selected from
+ a versioned capability, never from an if openqcd branch.
+
The runtime used the SEB to obtain scheduler state.
Values are visible in the downloaded OpenQCD log.
id: quick
+description: "Low-cost pipeline validation; not a physics-quality run"
+decisions:
+ compute_centre: idris
+ seed: s73099
+ nth: nth2
+ nstep: nstep4Truth boundary. The 271-second run used the separate manual SEB reproducer, not a completed scheduler driver inside the restored adapter.
++ The client recomputed the hash of the fetched Uppercase file. + That verifies the returned bytes against the fetch record; it + does not prove which executable, host or scheduler allocation produced them. +
++ A fetch receipt records file path, byte count and SHA-256. + Verification composes the available evidence and reports the + absent runtime receipt explicitly instead of promoting local + integrity into remote attestation. +
+c105cf73105e7ec7….ok_with_missing_compute_receipt."files": {"uppercase_message": {
+ "bytes": 46,
+ "relative_path": "uppercase_message",
+ "sha256": "c105cf73105e…"
+}}
+
+"compute_receipt": "missing",
+"fetched_file_hashes": "verified",
+"verdict": "ok_with_missing_compute_receipt"Exact fields. The two source files remain separate; they are juxtaposed here for comparison.
++ Evidence directories are addressed by immutable request IDs. + MCP may expose the same lifecycle to an agent, but it does not + resolve ASTRA, compile the binding or become the durable job store. +
++ The MCP functions are a transport façade over the shared + intent lifecycle. In the current combined checkout, the + candidate façade expects a backend factory removed when the + frozen request-ID baseline was restored; integration remains open. +
+astra.yaml.prepare_intent(intent_path)
+submit_intent(
+ intent_path,
+ project_root,
+ confirm_request_id,
+)
+intent_status(intent_path)
+wait_intent(intent_path)
+fetch_intent(intent_path, project_root)Projection. Names and parameters follow the linked source; decorators, annotations and bodies are omitted.
++ Lightcone resolves ASTRA once. Its existing Snakemake path and the + experimental HAPS compiler consume the same task. MCP is useful, + but it is a parallel agent-facing interface—not a mandatory internal hop. +
+Scientific output, inputs, decisions, universes and rationale.
+ real source +ResolvedTaskOne concrete output for one universe, with content hashes and paths.
+ implemented +Reviewable ASTRA-to-catalog mapping; the restored demo binding does not yet declare the scheduler driver.
+ baseline preserved · backend interface open +HapsJobIntentPure compilation preserves the frozen request identity; the rejected candidate showed that adding a driver changes the hash.
+ baseline real · driver migration open +The restored engine owns the direct lifecycle; the tested scheduler client factory is no longer present in the backend checkout.
+ direct baseline · scheduler missing +Uppercase direct and the OpenQCD scheduler wire contract are live; the latter completed in 271 s with four outputs.
+ live modes separated +Compute receipt, split Lightcone/HAPS verdict and ASTRA-linked lifecycle.
+ not implemented +haps-mcp façade
+
+ Worktree tools prepare_intent,
+ submit_intent, intent_status,
+ wait_intent and fetch_intent accept the
+ same immutable intent file and are filesystem-scoped by
+ administrator-owned allowed roots. They are currently a candidate,
+ not a green cross-repository integration.
+
lightcone-haps engine
+
+ The façade is designed to delegate to HapsEngine and
+ client_for_intent. The restored backend no longer
+ contains that client factory, so the current MCP worktree fails
+ loudly rather than silently growing a second scheduler engine.
+
+ MCP is how an agent asks. The binding is what this project means.
+ lightcone-haps is how Lightcone executes it.
+
+ Validates the ASTRA project, selects a universe and freezes one + provider-neutral task. It owns decisions, paths and scientific + resolution—not HAPS wire mechanics. +
+ASTRA + universe → ResolvedTask
+
+ + Maps stable task, option, input and output IDs to one published + catalog contract. It is deliberately case-specific, versioned + and reviewable; adding a case changes YAML, not adapter code. +
+ResolvedTask IDs ↔ catalog interface
+
+ lightcone-haps+ Purely compiles task plus binding into an intent, preflights the + live catalog, dispatches the declared driver, persists lifecycle + state, fetches outputs and verifies what it can prove. +
+intent → prepare → submit → observe → fetch
+
+ haps-mcp+ Makes the same lifecycle callable by an agent or an MCP-only + client. It delegates to the backend; it does not interpret + ASTRA, invent mappings or become a second scheduler engine. +
+agent tool call → the same HapsEngine
+
+ + MCP only defines how a client calls tools. It does not + provide: +
+template::slurm selection;
+ Those responsibilities belong to the
+ HAPS binding and
+ lightcone-haps.
+
ASTRA → Lightcone → ResolvedTask
+ + HAPS binding
+ ↓
+ lightcone-haps
+ ↓
+ HAPSAgent → haps-mcp → the same lightcone-haps enginerun_openqcd.+ We could not use MCP instead of the binding and ++lightcone-haps.
+ We can use MCP in front of +lightcone-haps. +
.lightcone/resolved-task.json+ A persisted, content-addressed file—not a Python object shared by + private imports. Scientific choices are settled above it; provider + mechanics begin below it. +
+ +The existing local and shared-cluster backend remains unchanged.
+ +A peer compiler crosses identity, storage and scheduler boundaries through explicit artefacts.
+compile_to_haps
+ The JupyterHub use case can compile the same task without becoming a Lightcone special case.
+missing: compile_to_argo
+ + A shared JupyterHub cluster has one filesystem and one administrative + identity. The execution boundary becomes necessary when the next machine is Jean + Zay, NERSC or a remote GPU box with neither. +
++ Name, MIME selector, ordered arguments, named inputs and named + outputs describe one atomic callable application. +
+mime · arguments[] · ebin[] · ebout[]
+
+ + A job binds one application to input/output buffer handles. The + service owns remote job identity and coarse lifecycle state. +
+job UUID · pending → running → terminal
+
+ + Ephemeral buffers stage input bytes, status and results. Their UUIDs + are transport handles—not durable dataset identifiers. +
+ebin → runtime → ebout
+
+ + A runtime polls for a capability it serves, submits or executes the + application, then fills outputs. Scheduler-backed applications add + a declared template lifecycle; no inbound compute-node port is required. +
+application/x-openqcd-ym1-idris
+
+ | Application class | +Declared capability | +Correct client lifecycle | +Current evidence | +
|---|---|---|---|
| Echo / Uppercase | +Generic microservice | +submit_job(ebin, ebout, arguments) |
+ Uppercase completed and fetched on AQMO. | +
| OpenQCD → IDRIS | +template::slurm |
+ set_inputs → submit → monitoring → download_data through ClientSchedulerTemplate |
+ A manual wire-compatible reproducer produced job aa26bc0d… in 271 s; the public-client adapter has not yet reproduced it. |
+
+ Differential diagnosis · 31 July. Jobs
+ 6787f531, 6d56a857, 4e3b5d1d
+ and fafc1715 reached HAPS through a generic
+ submit_job() route that is invalid for OpenQCD. They do not
+ demonstrate an IDRIS regression. Repeating the same quick case with the
+ exact scheduler-template/SEB wire contract produced job
+ aa26bc0d…, which completed after 271 seconds. The failed
+ protocol—not the facility—was the discriminating variable. The
+ successful reproducer still builds that wire shape manually.
+
+ Integration stop sign · restoration completed. The
+ candidate driver field changed OpenQCD from
+ 7696f130… to 2c95a2bc… and Uppercase from
+ d579aa96… to 0ec67f68…. That experiment was
+ coherent, but incompatible with the frozen evidence. Commits
+ 88c92af in lightcone-haps and
+ db944b4 in the showcase restored the two historical IDs
+ and removed the candidate scheduler driver from those current trees.
+ The MCP façade worktree still expects the removed client factory, so
+ the combined checkout is deliberately red until an explicit
+ schema/version migration—or compatible absent-field design—is chosen.
+ Never point an old evidence directory at a new request.
+
+ Broadly, the proposed value is real: an ASTRA/Lightcone client can + send one resolved task to different HAPS-enabled trust domains + without installing Lightcone, an agent or a heavyweight workflow + engine inside every centre. “HAPS-enabled” is the essential + qualification: the site still deploys and governs a small runtime + adapter, catalog entry, network path and identity policy. +
++ ASTRA records the scientific choices. Lightcone resolves them. + Humans, notebooks or agents may initiate the same immutable intent. +
++ A vetted application contract is selected by capability; the job + lifecycle API records remote identity; bounded buffers move inputs, + status and outputs. Tokens and federation policy decide who may + call what. +
++ The runtime translates the HAPS contract into the execution + mechanism the site already trusts. HAPS reduces the integration + surface; it does not replace the scheduler or the centre’s policy. +
++ Uppercase reached AQMO directly; Docling previously ran on the DGX + Spark; OpenQCD crossed HAPS and a scheduler adapter to Jean Zay. + These runs prove heterogeneous reach, not yet production-grade + federation. +
++ A participating resource needs a HAPS runtime/application adapter + and approved catalog registration. It does not need ASTRA, + Lightcone, Snakemake, MCP or an agent installed on compute nodes. +
++ The same API-and-catalog pattern can support larger governance + domains. ILaaS is a useful French ESR analogy for federated, + API-accessible compute services—but this prototype is neither an + ILaaS integration nor a national HAPS deployment. +
+| Layer | +Question answered | +Reusable across | +Current truth | +
|---|---|---|---|
| Extended application descriptor | +What can this vetted service run, with which typed inputs, outputs, help and execution capability? | +Catalog registration, runtime skeleton, Galaxy/CWL and agent-facing projections. | +Galaxy/HAPS source format studied; our descriptor compiler remains a prototype. | +
| Live HAPS catalog entry | +Which deployed runtime currently serves this MIME/capability and exact ordered interface? | +Humans, Lightcone backends and optional MCP clients inside the same federation. | +Live matching exists; rich types, units, bounds, defaults and interface revisions are missing. | +
| Project HAPS binding | +How does this ASTRA output map onto that reusable service contract? | +Universes and reruns of the same scientific project. | +Explicit YAML exists for three cases; execution-driver migration still changes request identity. | +
+ Calibrated claim. HAPS avoids forcing each provider to + adopt the consumer’s workflow stack; it does not magically create a + federation. The upstream ebuffer design explicitly favors a minimal + data-logistics/API layer over a shared cross-centre filesystem, while + still naming the need for network reachability, a global namespace and + federated identity. For scale context, the official + ILaaS service description shows how + an ESR federation can expose trusted compute through APIs and keys; it + is an analogy, not evidence that ILaaS runs HAPS. +
++ A scientific case, a transport control and a private-platform proof + should not collapse into one green badge. Each card states exactly + what crossed the boundary—and what did not. +
+
+ The quick OpenQCD case ran on Jean Zay with the exact
+ scheduler-template tags, output capacities and SEB association.
+ HAPS reached completed; four files were downloaded
+ immediately and hashed locally.
+
+ The older fafc1715 tree remains useful negative
+ evidence for the wrong generic route. The live success is not yet a
+ run produced by ClientSchedulerTemplate inside the
+ candidate Lightcone–HAPS engine.
+
+ The same resolver, binding, catalog, staging, lifecycle and fetch + contracts completed. The 46-byte result hash verifies locally; the + deployed runtime did not author a compute receipt. +
++ This proves transport and fetched-byte integrity. It is deliberately + not presented as a scientific HPC result. +
+ ++ Historical evidence shows a 66 KB PDF converted to 14,371 Markdown + characters in 67 seconds on the private Spark deployment. Berkeley + connectivity is currently unavailable. +
++ The declared case proves a third adapter shape without changing + generic Python. It must not be described as a live Berkeley run. +
+ +
+ Read-only AQMO audit · 31 July. The private profile and
+ external password vault are both mode 0600 and excluded from Git. The
+ expected ebservice/ebuffer endpoints authenticated successfully; the
+ live catalog returned five applications and exactly one IDRIS OpenQCD
+ match (84fa4656-19f1-42ba-888f-d026b137eb60, reported
+ state 4). No buffer or job was created. The registered
+ profile remains external to the repository. The candidate adapter pins
+ ebstemplate==0.1.1, but that worktree has not passed its
+ final test/lint/type/build gate.
+
+ These were not nine versions of “the demo broke.” Each incident + occurred at a different layer and required a different remedy. The point + of the architecture is to make that layer—and the remaining + uncertainty—observable. +
+design rule
Connectivity is a deployment precondition and must fail before scientific mutation.
design rule
Version skew is evidence; content identity must change when authored bytes change.
HapsJobIntent retains typed values and separately freezes wire_arguments.design rule
Logical type and transport serialization are related, but not the same field.
ebstemplate==0.1.1 client creates fixed 6,000,000-byte output buffers; the candidate binding mirrors that invariant and the observed 4.7 MB files fit.design rule
Resource limits are reviewable data and part of request identity.
aa26bc0d… complete in 271 seconds.submit_job(), bypassing the template abstraction it claimed to reproduce.template::slurm in catalog/binding data, then let the adapter call the public template lifecycle instead of duplicating its wire algorithm. A local bytes-decoding bug obscured the SEB display, not the completed remote run.design rule
Select an execution template by declared capability—never by application name or copied tag folklore.
design rule
A remote terminal state is evidence only for the exact protocol path that produced it.
design rule
Never let one green “verified” badge collapse independent claims.
submitting state instead of retrying.design rule
Client replay protection is not a distributed idempotency guarantee.
push_input(file_path) can read any process-visible file; ephemeral outputs can also disappear before a polling client fetches them.design rule
Buffers are constrained transport, never a secret store or archive tier.
+ Evidence policy. The cards summarize the preserved run + artefacts and the dated continuity log. When a root cause was not + observable—or when the client used the wrong execution route—the page + says so. The authenticated 31 July catalog check proves endpoint, + credential and application discovery only; it does not prove scheduler + or runtime health. +
+ ++ Application values belong in scientific records and bindings—not in + connector branches. The schemas are explicitly experimental: one + producer and one consumer do not make a public standard. +
++ Lightcone interprets ASTRA once and emits one concrete output task + for one selected universe. Backend-specific runtime handles do not + enter the core record. +
+ +resolved.py
+ {
+ "schema_version": "lightcone.resolved-task/experimental",
+ "task_id": "lattice_products@quick",
+ "universe": {
+ "id": "quick",
+ "decisions": {
+ "compute_centre": "idris",
+ "seed": "s73099",
+ "nth": "nth2",
+ "nstep": "nstep4"
+ }
+ },
+ "inputs": [{
+ "id": "run_parameters",
+ "bytes": 742,
+ "sha256": "f2dde66d…"
+ }],
+ "task_sha256": "7d1ff848…"
+}+ The live HAPS catalog currently exposes ordered names. The target + contract adds types, units, requiredness, bounds, descriptions, + interface revision and a declared compute-receipt output. Extended + Galaxy Tool XML is the strongest existing source vocabulary. +
+ + +current live catalog:
+ mime_type: application/x-openqcd-ym1-idris
+ arguments: [seed, nth, nstep]
+ inputs: [ym1.in]
+ outputs: [log, n1, n2, par]
+
+target typed description:
+ interface_revision: explicit
+ arguments:
+ - name: seed
+ type: integer
+ required: true
+ description: HMC random seed
+ receipt:
+ output_name: compute-receipt.json
+
+truth boundary:
+ Galaxy XML vocabulary real upstream
+ enriched HAPS schema proposal+ The binding is intentionally case-specific data. It maps stable + ASTRA option IDs to one expected HAPS interface while keeping + endpoints and credentials elsewhere. A tested patch added the + driver and catalog capability under each selected application, + but it changed request identity and was removed from the current + binding to preserve frozen evidence. +
+ + +schema_version: lightcone.haps-binding/experimental
+binding_id: openqcd-lattice-products
+
+application:
+ selector:
+ from_decision: compute_centre
+ choices:
+ idris:
+ mime_type: application/x-openqcd-ym1-idris
+ execution:
+ driver: ebstemplate.scheduler
+ catalog_capability: template::slurm
+
+arguments:
+ seed:
+ from_decision: seed
+ choices: {s73099: 73099, s42: 42}
+
+outputs:
+ openqcd_ym1L8T8n1:
+ capacity_bytes: 6000000
+ capacity_provenance: fixed by ebstemplate 0.1.1; measured ≈ 4.7 MB
+
+truth boundary:
+ candidate field names tested, then removed
+ current demo binding preserves 7696f130… / d579aa96…+ Pure compilation freezes the selected catalog interface, typed + values, string wire values, hashed inputs, output capacities and + receipt policy before any remote mutation. The retired candidate + also froze the declared execution driver, proving that dispatch + belongs in request identity—but also changing every existing + request hash. +
+ + +{
+ "request_id": "sha256:2c95a2bc3994…",
+ "task_id": "lattice_products@quick",
+ "application": {
+ "mime_type": "application/x-openqcd-ym1-idris",
+ "execution": {
+ "driver": "ebstemplate.scheduler",
+ "catalog_capability": "template::slurm"
+ }
+ },
+ "arguments": {"seed": 73099, "nth": 2, "nstep": 4},
+ "wire_arguments": {
+ "seed": "73099",
+ "nth": "2",
+ "nstep": "4"
+ },
+ "receipt": {
+ "compute_receipt": {"policy": "report_missing"}
+ }
+}
+
+truth boundary:
+ restored schema does not contain the driver
+ retired candidate 2c95a2bc… proved the migration effect
+ old 7696f130… record generic-route evidence; never overwrite
+ engine reproduction pending
+ The state record links an immutable request to a
+ mutable asynchronous job. This particular OpenQCD record persists
+ a real remote association, but the associated request used the
+ wrong generic execution route. The successful
+ aa26bc0d… scheduler run has downloaded outputs, but
+ has not yet been ingested into this engine artefact sequence.
+
{
+ "schema_version": "lightcone.haps-remote-run-state/experimental",
+ "request_id": "sha256:7696f130d485…",
+ "application_id": "84fa4656-19f1-42ba-888f-d026b137eb60",
+ "job_id": "fafc1715-e8e6-4b04-8e27-817d0fe32a3c",
+ "phase": "failed",
+ "remote_status": "failed",
+ "staged_inputs": {
+ "ym1.in": {
+ "bytes": 742,
+ "sha256": "f2dde66d…",
+ "buffer_id": "b53cd2f1-…"
+ }
+ },
+ "idempotency": {
+ "scope": "local-state-only",
+ "server_request_key_supported": false,
+ "submit_write_gap": true
+ }
+}+ Fetch verification proves the local bytes match their recorded + hashes. A runtime-authored compute receipt is a different claim + and is currently missing from deployed applications. +
+ + +{
+ "schema_version": "lightcone.haps-verification/experimental",
+ "request_id": "sha256:d579aa960971…",
+ "fetched_file_hashes": "verified",
+ "compute_receipt": "missing",
+ "verdict": "ok_with_missing_compute_receipt"
+}
+
+claim boundary:
+ local bytes verified
+ runtime identity not attested
+ scientific result not evaluated here+ Remote events should return beside the scientific record through + an append-only execution artefact. HAPS receives opaque ASTRA + identities; it does not parse or rewrite the project. Promoting + an execution as evidence for a scientific finding remains a + separate reviewed action. +
+ +demo-spec.md
+ {
+ "schema_version": "astra.external-execution/experimental",
+ "analysis": {
+ "id": "openqcd-lattice",
+ "spec_sha256": "…"
+ },
+ "universe": {
+ "id": "quick",
+ "decisions_sha256": "…"
+ },
+ "task": {
+ "id": "lattice_products@quick",
+ "sha256": "…"
+ },
+ "remote_execution": {
+ "request_id": "sha256:…",
+ "job_id": "…",
+ "events": ["prepared", "submitted", "failed"]
+ },
+ "verification": {
+ "lightcone": "not-run",
+ "haps": "failed-no-receipt",
+ "scientific": "not-evaluated"
+ },
+ "finding_promotion": "requires-human-review"
+}
+
+not allowed:
+ mutate astra.yaml on every poll
+ auto-create a scientific finding+ The implementation is not “a notebook called an API.” It is a sequence of + authored, generated, observed and still-missing artefacts with named + producers and consumers. +
++ These are existing project images—not generated architecture claims. + The second is intentionally shown as historical because the design + evolved after it was drawn. +
+
+ 
+
+ The current correction is architectural, not cosmetic:
+ ResolvedTask → compile_to_haps → capability dispatch
+ must lead either to the generic direct client or to
+ ClientSchedulerTemplate. A candidate followed this
+ shape and passed focused offline compatibility tests, but it was
+ removed from the backend to restore frozen request IDs. The
+ still-uncommitted MCP façade expects that removed factory, so the
+ current combined checkout fails loudly. MCP remains beside the
+ deterministic execution path; it must delegate rather than become another
+ execution engine.
+
+ Sara’s compiler on main@35ea8df consumes a
+ Galaxy-shaped HAPS XML description. It is not a
+ Galaxy file plus a <haps> block: the root is
+ <tool>, HAPS-specific result projection lives in
+ <output_mapping>, and the centre is a declared
+ hpc_center select input. This task description still
+ does not replace an ASTRA project or its project-local HAPS binding.
+
+ The descriptor reuses Galaxy’s familiar
+ <inputs>/<param>,
+ <outputs>/<data>, labels, help and select
+ options. Its custom <output_mapping> maps HAPS
+ values such as remote_path and
+ status into one generated JSON metadata output.
+ Endpoints and scheduler capabilities are not declared in
+ this XML today.
+
<tool> + <inputs> + <outputs> + <output_mapping> + <help>
+ Exa_AToW_compiler.py
+ uppercase.xml
+
+ Galaxy mode is designed to generate one Galaxy tool XML and one
+ Python proxy from one HAPS description. The current checkout
+ imports galaxy_tool_generator.py, but that file is
+ absent, so this mode cannot presently be reproduced as committed.
+
+ CWL mode combines an abstract CWL DAG with one HAPS XML per step,
+ then intends to generate wf.yaml, one
+ CommandLineTool and one proxy per remote task.
+ Remote-to-remote edges carry local JSON metadata references.
+
+ Sara’s source does not generate ASTRA bindings, HAPS catalog
+ registration, runtime skeletons, execution-driver metadata or
+ MCP tools. Those are separate targets of our descriptor/catalog,
+ lightcone-haps and optional MCP work.
+
| Observed in source | +Concrete behavior | +Architectural consequence | +Current limitation | +
|---|---|---|---|
| HAPS XML vocabulary | +Parses tool id/name/version, description/help, typed parameters, select options, outputs and custom output mappings. | +A strong authored source for UI and proxy generation; a plausible input to a future typed catalog compiler. | +No XML schema validation, units, numeric bounds, interface revision or declared execution driver. | +
| Generated proxy contract | +Uses ebsclient, uploads local files, chooses <task>-<centre>, submits, polls and writes one JSON metadata output. |
+ Confirms that the descriptor can drive executable client code rather than only documentation. | +Endpoints, tag and 500,000-byte input buffers are hardcoded; credentials are CLI inputs; polling is unbounded. | +
| Workflow projection | +Shares username, password and hpc_center; connects tasks through JSON containing centre, remote path and status. |
+ Separates workflow composition from each atomic HAPS call and avoids moving some intermediate bytes back locally. | +Current main marks every step remote, requires exactly one JSON output, writes dataset_id: null and assumes an unconfirmed job.get_output_path(). |
+
| Reproducibility audit | +All checked Python files parse syntactically at commit 35ea8df. |
+ The design is concrete enough to study and align with our descriptor/catalog work. | +galaxy_tool_generator.py and legacy catalog_loader.py are absent; the committed tests call an obsolete CLI shape, so neither advertised mode is green in this checkout. |
+
+ Application descriptor: “what can HAPS run?” It is + reusable across projects, centres, humans, workflow engines and + agents, and should feed the vetted catalog. +
++ Project binding: “how do these ASTRA IDs map to that + application?” It stays explicit, reviewable and local to the case. +
+
+ lightcone-haps backend: “how is that
+ mapping executed safely?” It compiles and owns the lifecycle. MCP can
+ project the catalog or expose this backend; it replaces neither.
+
+ Truth boundary after reading Sara’s source. The
+ Galaxy-shaped HAPS XML and its Galaxy/CWL/proxy generation design are
+ upstream Exa-AToW work, but the checked-out main is
+ incomplete and was not represented as a passing compiler. MCP
+ generation is our separate extension; the current
+ haps_gen.py emits
+ registration/runtime/tool artefacts but does not yet supply the complete
+ execution-template contract required by the adapter.
+
+ The next interoperable input needs four separate contracts. Collapsing + DOI, Globus, DVC and ebuffer into one field would confuse identity, + discovery, movement and job staging. +
+A DOI or other PID plus a content digest identifies a versioned scientific object.
+DataReference.pid + digest
+ Metadata, access conditions and available replicas make the object findable.
+catalog → replicas[]
+ Globus or a site-supported peer moves bytes and emits a transfer receipt.
+source → task receipt → target
+ An ebuffer or local path supplies one job. Its UUID is not a durable dataset identity.
+replica → ebin → runtime
+
+ Status: proposed. The protocol-neutral ASTRA
+ DataReference, catalog resolution, transfer receipt and
+ runtime-recorded replica are specified, not implemented.
+
+ The current code is now split across real remotes. The showcase should + be the single review and demo entry point while pinning—not + duplicating—the adapter, MCP façade and Lightcone feature revision. + Frozen evidence must retain the exact code identities that produced it. +
+astra-haps-showcase/
+├── cases/
+│ ├── openqcd/
+│ ├── uppercase/
+│ └── docling/
+├── notebooks/ # operator + offline evidence
+├── evidence/ # immutable recorded outcomes
+├── docs/ # spec, talk, failure atlas
+├── components.lock.json
+│ ├── lightcone-haps@<commit>
+│ ├── haps-mcp@<commit>
+│ └── lightcone-cli@<resolved-task-commit>
+├── evidence-index.json # run → code → artefact digests
+├── pyproject.toml # reproducible demo environment
+├── LICENSE
+└── PROVENANCE.md # producers, imports and trust limits+ The notebook, three cases and evidence are reviewed together. + Component commits are pinned and installed from their own remotes, + avoiding unpublished sibling-path magic. +
+
+ lightcone-haps still does not parse ASTRA or depend on MCP; its
+ direct/scheduler dispatch remains internal to the adapter.
+ haps-mcp still does not own Lightcone paths.
+
+ lightcone-haps, haps-mcp and the
+ showcase already have GitHub remotes. The release gate is now
+ compatibility and provenance, not repository creation.
+
Sara’s HAPS XML already reuses the Galaxy-shaped tool/input/output/help vocabulary. Its custom output_mapping and future HAPS execution fields still need their own explicit schema.
Input and output JSON Schemas are the natural agent-facing projection of a typed application contract.
+ Official MCP tools spec ↗ +Asynchronous task IDs, lifecycle events and per-file checksums are a useful model for transfer evidence—not compute evidence.
+ Official Transfer API ↗ +Prospective workflow description and retrospective run provenance are already distinguished in a portable research-object profile.
+ Official profile collection ↗ +
+ Publication state · checked locally.
+ lightcone-haps, haps-mcp and
+ astra-haps-showcase point to
+ github.com/buildcore/*; lightcone-cli is on the
+ local feat/resolved-task-seam branch; the canonical
+ openqcd-lattice/ directory still has no Git history.
+ lightcone-haps and the showcase now contain additive
+ restoration commits preserving the demo request IDs, while the
+ haps-mcp worktree still contains the incompatible façade
+ candidate. Do not publish that candidate until its backend dependency,
+ tests, lint, strict typing, package builds, secret scan and
+ old-evidence hash compatibility all pass.
+
+ The experimental implementation deliberately fails loudly on missing + inputs, catalog drift, ambiguous submission state, oversized output + and request-ID mismatch. Several guarantees still require server, + runtime or upstream ownership. +
+lightcone-haps baseline passes 18 tests and 13 focused MCP façade tests pass, but those MCP tests mock engine construction; the real cross-repository client factory is absent.scheduler.py is absent.aa26bc0d… scheduler path.push_input tool.+ Recovery must happen on a dedicated branch with an explicit + experimental schema migration. Applying the stash directly would + combine incompatible request identities and leave required schema + changes undefined. +
+lightcone-haps scheduler stash
+ The stash contains changes to the direct client, engine, CLI,
+ report and dependency lock. Its untracked tree separately holds
+ clients.py, scheduler.py and two focused
+ test modules.
+
ebsclient.direct and ebstemplate.schedulertemplate::slurmstash@{0} + stash@{0}^3
+ + Fifteen cells show ASTRA validation, Lightcone resolution, + intent compilation, driver ownership, read-only preflight and + one explicitly confirmed submission through either the CLI or + optional MCP interface. +
+False
+ Driver selection belongs in the binding and content-addressed
+ intent, never in an if openqcd branch. That means a
+ new experimental contract revision and new request IDs for new
+ runs; preserved evidence retains its old schema and hashes.
+
Extract tracked and untracked stash contents, define a new experimental binding/intent revision, generate new request IDs and keep every frozen evidence directory immutable.
+Keep one persisted lifecycle while dispatching ebsclient.direct or ebstemplate.scheduler from the hashed intent; make the optional MCP tools delegate to that exact engine.
Pass schemas, complete cross-repository tests, lint, strict typing, builds and secret scan; then confirm a fresh request ID and reproduce OpenQCD through lc-haps, not the manual launcher.
Persist scheduler observations and split verification in an ASTRA-linked sidecar; then align the Lightcone backend API, server idempotency, typed catalog, compute receipts and durable data handoff with their owners.
++ Post-demo implementation authority. The focused + execution-driver specification defines the recovery order, resumption + boundary, rejected alternatives and acceptance gates. It is a proposed + engineering contract—not evidence that the scheduler driver is present + in the restored backend. +
+ ++ The dossier is a reading surface, not a second source of truth. + Follow these links to the actual specifications, code, schemas, + notebook and evidence directories in this workspace checkout. A + standalone hosted copy may not include those sibling repositories. +
+Current/target architecture, ownership, contracts, anti-hardcoding rules and closure ledger.
+ + + evidence narrative↗ + evidence-first-execution.md +The concise story of the execution boundary, three cases, failures, lessons and next steps.
+ + + operator surface↗ + astra-lightcone-haps.ipynb +CLI-first notebook with visible gates, exact request confirmation and no MCP hop.
+ + + provider-neutral task contract↗ + resolved.py +Experimental ResolvedTask records and deterministic list serialization.
+ + + pure compiler↗ + compiler.py +ResolvedTask plus binding to content-addressed HapsJobIntent.
+ + + backend ownership contract↗ + lightcone-haps/README.md +What the generic extension owns, what belongs in a project binding, and what remains outside it.
+ + + optional agent façade · candidate↗ + haps-mcp/README.md +The intended shared-intent tools do not interpret ASTRA or select a scientific case; the current façade worktree expects a backend factory that was removed by the request-ID restoration.
+ + + persisted lifecycle · restored baseline↗ + engine.py +The current direct lifecycle and durable state machine; it does not contain the retired scheduler-driver candidate.
+ + + direct client · restored baseline↗ + direct.py +The current generic ebsclient route used by ordinary catalog applications; it is not the OpenQCD scheduler-template lifecycle.
+ + + driver migration decision · open↗ + Execution-driver contract +The focused post-demo specification dispatches by a declared, hashed capability rather than by application name and defines migration, recovery, security and acceptance gates.
+ + + live scheduler-wire reproducer↗ + repro-openqcd-idris.py +The separate manual reproducer whose SEB/template route completed on Jean Zay; it is evidence for the missing backend capability, not the backend implementation.
+ + + real case binding↗ + openqcd/haps-binding.yaml +Explicit IDs, MIME selectors, arguments, outputs, capacities and receipt policy.
+ + + preserved evidence↗ + showcase/evidence/ +Content-addressed OpenQCD and Uppercase artefact sets.
+ + + fresh scientific output↗ + openqcd-live-aa26bc0d/ +Four files downloaded after the 271-second scheduler run; local hashes independently recomputed.
+ + + design fixtures↗ + interoperability-reference/ +Inspectible proposed schemas, receipts, manifests and verification composition.
+ + + descriptor compiler prototype↗ + haps_gen.py +Galaxy-shaped XML or catalog entry to an experimental registration, runtime skeleton and MCP tool schema; it is separate from Sara’s compiler.
+ +Sara’s Galaxy-shaped HAPS XML parser and intended Galaxy/CWL/proxy projections; the audited checkout is not published with this page.
+The minimalist cross-infrastructure data-logistics proposal, including its network, namespace and IAM conditions.
+ + + historical reproducer↗ + client-idris-repro.ipynb +Useful audit history, but the fresh success came from a separate full wire-compatible reproducer; public-client adapter execution remains pending.
+ +
+ The contribution is not merely “an agent can submit HPC.” It is a
+ an execution record where scientific intent, deployment mapping, remote identity,
+ execution mode, returned bytes and uncertainty each have an explicit
+ owner. The failed generic runs and the successful 271-second
+ scheduler run together identified the missing contract without
+ blaming the facility. HAPS provides the minimal job, catalog and
+ buffer APIs that carry this contract across approved resources; the binding and
+ lightcone-haps backend make that reach deterministic,
+ while MCP only provides an additional controlled client interface.
+