Consume the editor from GitHub release tarballs instead of staged file: copies #6

Summary

Replace the staged-file: consumption mechanism that PT-4500's open PRs implement with GitHub release assets: the publish workflow here packs platform-editor and scripture-utilities into tarballs and attaches them to the release; consumers depend on the tarball URLs. yalc returns for the inner development loop only, as an opt-in step rather than something every npm install does.

This would delete roughly 1,450 of the ~2,480 non-dist, non-lockfile lines across the eleven open PRs (the install-lifecycle scripts, the committed dist/, the lockfile-sync guard, the platform-yalc mover, and every extension repo's CI staging step) and add about 300. Sizing is below.

Status: proposed, not yet decided. TJ leans toward it but ran out of time (2026-09-09). Nothing is merged, so switching now costs only the rework listed here.

Context

PT-4500 (Jira) moves this repo into the paranext org. Consumption had to change because the old registry pin was carrying the editor's dependency closure, not its code (yalc replaced the code seconds later). The current PRs solve that by having paranext-core stage each package into dev-packages/staging/ and depend on it with file: specifiers, so npm reads the staged manifest and installs the closure. To make staging a copy instead of a pnpm + nx build, this repo commits dist/ (#4). The rest of the machinery follows from those two choices.

Open PRs implementing the current design:

Full handoff for the current design, including the npm facts that shaped it: ~/source/repos/workspaces/pt-4500-file-link/HANDOFF.md on TJ's machine (not committed anywhere). The relevant parts are restated below.

What is already verified (2026-08-31, fixture tests)

• A dependency given as a remote tarball URL installs the package and its full dependency closure; package-lock.json records resolved (the URL) and integrity, so npm ci is reproducible.
• A tarball can itself depend on another unpublished tarball by URL: consumer → editor (URL) → utilities (URL) resolved the whole graph and deduped to a single copy of utilities.
• No stale-cache trap for remote URLs, because the URL changes per release. (A file: tarball at a stable local path is never re-read by npm, which is what rules out local tarballs for the dev loop and why yalc stays for that.)
• npm silently ignores subdirectory fragments in git dependencies, so github:paranext/scripture-editors#... cannot target packages/platform. Release assets are the no-registry option that works.
• With a lockfile present, npm install keeps the locked version whenever it satisfies the range; only npm update moves within a range. So losing semver ranges costs npm update / npm outdated / Dependabot visibility and cross-consumer dedupe, not automatic freshness.

Not yet verified: an install from an actual GitHub release asset URL (GitHub redirects to objects.githubusercontent.com; npm follows redirects and the docs support URL tarballs, but the fixture used a local HTTP server). Do this first with a throwaway prerelease.

Why the current design was chosen instead

The design session recommended release-asset URLs on 2026-08-31 and listed these cons: exact URLs only (no semver), dedupe keyed on the exact URL so all consumers must pin identical URLs, npm outdated and Dependabot blind, and the repo must be public. None was decisive. The pivot came from a preference for not hosting built artifacts, for mirroring the existing platform-bible-react / platform-bible-utils sibling-file: pattern, and for dependency changes flowing into core with no release step. The ticket's own implementation ideas list "a prebuilt artifact" as an acceptable mechanism, so this is in scope.

Trade-offs

Release-asset design gains

• Consumers are decoupled. Each pins a URL and upgrades when it chooses; an editor dependency change can never break another repo's build. The lockfile-sync guard, move-platform-yalc, and verify-platform-yalc.yml have no job.
• No committed dist/, no verify-committed-dist, no per-PR dist churn or conflicts, and Separate the published artifact directory from the toolchain's scratch space (both are currently dist/) #5 stops mattering to consumers.
• No preinstall / postinstall scripts in core, no second-repo clone on ordinary installs, no self-heal, no .staged-from marker states. Fresh-clone npm install is one pass; CI needs no git access to this repo and no pnpm.
• Extension repos need nothing: their existing npm ci --ignore-scripts in the core checkout downloads the tarballs, and the editor resolves through platform-bible-react's realpath into core's node_modules exactly as today. The six CI-only PRs close unmerged.
• Standard npm shape that anyone can read.

Release-asset design costs

• Every integration of new editor code into core needs a release (Bump Versions + Publish dispatch) and a core PR touching six package.json files plus the lockfile. During heavy editor work that is several releases a week. An automatic prerelease on every push to main (tag like v0.9.0-main.<shortsha>) removes the manual dispatch but is a small workflow of its own.
• The inner loop is yalc over an installed tarball again. A dependency added in the editor does not reach core until a release; the developer runs npm i --no-save <dep> in core or cuts a prerelease. The staged design flowed that automatically. (editor:link could install missing deps from the built manifest with --no-save, ~20 lines, optional.)
• Dedupe is by exact URL. All six core package.json entries must carry the identical URL or npm nests a second copy and Lexical instanceof checks fail across the boundary. A root overrides entry or a ten-line consistency check closes this.
• The publish workflow must rewrite the editor's "@eten-tech-foundation/scripture-utilities": "workspace:~" to the utilities asset URL before packing. pnpm pack would otherwise rewrite it to ~0.1.x, which resolves to ETEN's registry copy.
• Repos that depend on the editor directly (paratext-bible-extensions' text-collection, paratext-assistant) get their own extracted copy while platform-bible-react resolves core's. Two directories, so two module instances in a bundle that uses both. That is the status quo on main today; the staged design accidentally fixed it by pointing them at one realpath. Alternative: file:../paranext-core/node_modules/@eten-tech-foundation/platform-editor (out-of-tree symlink into core's installed copy). Implementer's call; flag it either way.
• Release assets must be treated as immutable. Re-uploading an asset under the same name changes its integrity and every consumer's npm ci fails with EINTEGRITY. That is a safety property, but document it.
• The repo must stay public (already decided) or consumers need a token.

Design under the release-asset strategy

scripture-editors (revise PR #1)

Keep: README ownership/relationship/remotes sections, LICENSE line, nx-cloud removal, bump-versions workflow + action + script, test.yml, docs/transferring-work-from-eten-tech-foundation.md, removal of manual-publish.yml / test-publish.yml.

Drop: committed packages/*/dist, the dist .gitignore changes, scripts/verify-committed-dist.mjs and its steps in test.yml / publish.yml, scripts/verify-consumer-lockfile-sync.mjs, scripts/move-platform-yalc.mjs, .github/workflows/verify-platform-yalc.yml, the README's "committed dist" and lockfile-coupling sections.

Add to publish.yml, before creating the release:

1. pnpm nx run-many -t extract-api (the real build; --skip-nx-cache is still wise given Separate the published artifact directory from the toolchain's scratch space (both are currently dist/) #5).
2. For each package, produce a publish-shaped manifest the way core's stage-dev-packages.ts prepareStagedManifest does today (it is the reference implementation): delete exports['.'].development, delete devDependencies, and rewrite every workspace: specifier. Here the rewrite target is the sibling's asset URL, which is deterministic before upload: https://github.com/paranext/scripture-editors/releases/download/v<version>/<npm-pack-filename>.tgz. Pack utilities first so its filename is known.
3. npm pack (or pnpm pack) each package. files already includes dist and src; check whether src should stay in the tarball.
4. Attach both .tgz files to the release (ncipollo/release-action accepts artifacts).
5. Decide whether to also add an automatic prerelease workflow on push to main.

Version-alignment note: the current publish.yml refuses to tag unless the input version matches packages/platform/package.json; keep that. Tags stay v<platform-editor version>.

paranext-core (revise PR #2745)

Keep: dev-packages.json clone URL flip, chromatic.yml hardcoded-checkout removal, README updates, the source-comment edits, bump-versions.yml tweak.

Replace the six file:dev-packages/staging/... specifiers with the asset URLs in: root package.json, extensions/src/platform-scripture-editor/package.json, extensions/src/platform-scripture/package.json, extensions/src/legacy-comment-manager/package.json, lib/platform-bible-react/package.json, lib/platform-bible-utils/package.json. Consider a root overrides entry or a check script so they cannot drift.

Restore from main: .erb/scripts/link-dev-packages.ts, dev-package-utils.ts, unlink-dev-packages.ts, the editor:link / utils:link / *:unlink scripts, the yalc devDependency, and main's pre-commit .yalc-in-lockfile guard. Make linking opt-in: remove npm run link-dev-packages from postinstall so ordinary installs and CI consume the release. dev-packages.json's revision then only matters for the opt-in link.

Delete: preinstall, .erb/scripts/stage-dev-packages.ts, .erb/scripts/postinstall.ts (fold postinstall:build back into postinstall), .claude/rules/install-path-scripts.md, the provisional-build pre-commit guard, the tsconfig.erb.json install-script gate and moduleDetection change, build:editor / stage-dev-packages scripts. The engines.node >=22.18.0 bump existed for native TS stripping in preinstall; revisit. The extensions/lib/add-remotes.ts native-includes change is harmless and can stay.

ADR: adr-dev-packages-staged-file-deps and adr-staged-closure-owned-by-core in .context/standards/Architecture-Decisions.md are on the branch, not on main, so rewrite them rather than superseding.

paratext-10-studio (revise PR #179)

repo-patches/scripture-editors.patch is empty, so studio's release build has no reason to clone this repo once linking is opt-in. Options: remove the scripture-editors entry from productInfo.json devPackageRepos (and decide what happens to the branch: false / disabled-repo logic #179 removed), or leave the entry for the dev link only. Regenerate repo-patches/paranext-core.patch with save-repo-patches after core's package.json settles, since the patch rewrites core's postinstall line.

Consumers

• paratext-bible-extensions #119 and paratext-assistant #130: point their editor / utilities dependencies at the same URLs core uses, refresh lockfiles, drop the CI staging steps.
• Close #163, #99, #195, PT-4540: Make a paragraph's gutter marker selectable #22, #62, #69 unmerged.

platform-yalc

Its role as "what core consumes" ends; core pins a release. Decide whether it survives as the branch releases are cut from, or retires in favour of main + prerelease tags for breaking-change coordination. The earlier "keep it" decision was made under the staged design.

Verify before building

1. Cut a throwaway prerelease here with both tarballs attached (a hand-run npm pack is fine) and npm install it in a scratch project: closure installed, resolved + integrity in the lock, npm ci from that lock succeeds offline-from-cache.
2. Same, with the editor tarball's utilities dep pointing at the utilities asset URL: one copy of utilities.
3. Fresh clone of core with the URL deps: npm install one pass, npm ci, npm run typecheck && npm run lint && npm test, npm run build.
4. Extension-repo CI shape: npm ci in paranext-extension-template then npm ci --ignore-scripts in core beside it; lint and typecheck pass (PBR imports the editor in 28 files, PBU in 10).
5. npm run editor:link then unlink round-trips without leaving .yalc entries in the lockfile.

Merge order

1. scripture-editors Adopt this repository into the paranext organization #1 (revised) → cut the first real release (0.8.16 or 0.9.0) so a URL exists.
2. core #2745 (revised) and studio #179 (revised) together.
3. paratext-bible-extensions #119 and paratext-assistant #130 after core, with lockfiles refreshed against the merged core.
4. Close the six CI-only PRs.

Prompt for a new chat

Continue PT-4500 (scripture-editors into the paranext org) by switching the consumption
mechanism to GitHub release tarballs, per paranext/scripture-editors issue #6.

Read first, in this order:
1. This issue, top to bottom. It has the verified npm facts, the per-repo design, the
   trade-offs, the open decisions, the verification list, and the merge order.
2. ~/source/repos/workspaces/pt-4500-file-link/HANDOFF.md, for the CURRENT (staged file:)
   design you are replacing: the eleven open PRs, worktree locations, and the non-obvious
   npm/nx constraints. Constraints 1, 2, 3, 5, 7, 8 and 9 there are consequences of the
   staged design and stop applying; constraint 4 (platform-yalc is force-pushed) still
   holds while that branch exists.
3. Claude memory files `pt-4500-repo-move.md` and `editor-consumption-file-deps.md` in
   ~/.claude/projects/-home-tj-co-source-repos-paranext-core/memory/ (npm resolution facts,
   all fixture-verified).
4. `.erb/scripts/stage-dev-packages.ts` on core branch `pt-4500-editor-file-link`, function
   `prepareStagedManifest`: the reference for how to shape each package's manifest before
   packing (drop the `development` export condition and devDependencies, rewrite
   `workspace:` specifiers). The publish workflow here needs the same shaping with asset
   URLs as the rewrite target.

Step 0: confirm with TJ that the switch is a go. It was proposed on 2026-09-09 but not
finally decided. Then ask for the open decisions in the issue that need TJ: automatic
prereleases on push to main or manual dispatch only; whether platform-yalc survives;
whether studio keeps a devPackageRepos entry for the editor; how the two direct consumers
(text-collection, paratext-assistant) reference the editor.

Step 1: run the "Verify before building" list in the issue against a throwaway prerelease
before rewriting any PR. If an install from a real GitHub release asset URL does not carry
the closure or does not record integrity, stop and report; the whole design rests on it.

Step 2: revise the PRs in the merge order given. Work in the existing worktrees under
~/source/repos/workspaces/pt-4500-file-link/ (core worktree has node_modules and a
dev-packages/scripture-editors checkout pointed at paranext). scripture-editors was worked
in a disposable scratch clone; re-clone. Keep the org-move parts of each PR (URL flips,
README, LICENSE, workflows, nx-cloud removal); remove the staged-file: machinery and the
committed dist; add the pack-and-upload step and the workspace:-to-URL rewrite.

Rules of the road: check PR CI per commit with
`gh api repos/<owner>/<repo>/commits/<sha>/check-runs`, not the PR status rollup, which
shows stale results. core#2745 conflicts with main roughly daily on the append-only
Architecture-Decisions.md; keep both sides. Never `git stash` bare in the worktrees. Run
`npm run typecheck && npm run lint && npm test` in core before pushing. Report failures as
failures. Ask rather than guess where the issue marks something as TJ's call.