The audit cannot see prose references — 10 {download}/link refs to files the migration will delete

[mmcky]

scripts/build_audit.py classifies a data reference by finding the code that reads it — read_csv, read_excel, np.load and friends. Lectures also link to the same files in prose, and those references are invisible to the scan. Ten of them exist today, every one pointing at a file the migration is going to delete.

Found while scoping repoint set 2. Not hypothetical: had I repointed the code and deleted intro's copy as in set 1, the strict audit would have gone green while two published lectures carried a 404 download link.

The ten

All point at lecture-python-intro's committed copies, including the ones in lecture-wasm — the same sibling coupling that made repoint rule 1 necessary, in a form the audit cannot see.

Data file	Where	Kind	Migration set
chapter_3.xlsx	intro inflation_history.md:228	{download}	set 2
chapter_3.xlsx	wasm inflation_history.md:220	{download}	set 2
fig_3.xlsx	intro french_rev.md:60	prose link	set 3
dette.xlsx	intro french_rev.md:61	prose link	set 3
assignat.xlsx	intro french_rev.md:62	prose link	set 3
fig_3.xlsx	wasm french_rev.md:60	prose link	set 3
dette.xlsx	wasm french_rev.md:61	prose link	set 3
assignat.xlsx	wasm french_rev.md:62	prose link	set 3
life-expectancy-vs-gdp-per-capita.csv	intro simple_linear_regression.md:411	{download}	wave 4
life-expectancy-vs-gdp-per-capita.csv	wasm simple_linear_regression.md:421	{download}	wave 4

Why this is worse than an undercount

A missing read would show up as a dataset with fewer consumers than it has — visible, and probably noticed. A missing prose link is different in kind: the tracker can record a dataset as fully repointed, the strict build can pass, and a reader can still click a dead link. The guardrail reports success on a broken state, which is the failure mode it exists to prevent.

It also interacts badly with the deletion rule. "Delete the lecture repo's own copy in the repoint PR" is safe when every consumer has moved — and the audit is what tells us they have. If it cannot see prose references, it cannot answer that question, so the rule rests on a check that does not cover it.

Scope, honestly

This is a reader-facing break, not a build break. A dead {download} link does not fail a lecture build, so nothing in either repo's CI catches it either. It surfaces when someone clicks.

Both kinds also use a URL form the scan has no reason to look at: the {download} directives use /raw/, the french_rev links use /blob/, and none of them sit near a read_* call.

Suggested fix

Scan for the file paths, not only the read idioms. Once a dataset is known — every migrated file has a manifest, and every unmigrated one has an annotation — the audit can grep each lecture repo for that filename and flag any occurrence not already accounted for by a classified read. That catches {download}, markdown links, and prose mentions of a path, without inventing a new taxonomy: they are references to a known dataset, in a new hosting pattern (call it prose), and they belong in the same refs list.

Two things fall out of doing it that way. fully_migrated starts telling the truth, so a dataset with a stale prose link cannot be marked repointed — the migration consistency check begins covering the case that motivated it. And the strict build starts failing on exactly the condition that is silently true right now.

A cheaper interim, if the full scan is more than is wanted: a targeted check that no lecture references github.com/QuantEcon/lecture-*/(raw|blob)/main/lectures/(datasets|_static)/… for any file that has a manifest here. Narrower, but it would have caught all ten.

Meanwhile

Each repoint set fixes its own prose references by hand, and the PR says so explicitly. Set 2 (chapter_3.xlsx) does this. Recording the general problem here so the manual step is a known stopgap rather than something rediscovered when a set forgets it.

Part of #8. Related: #20 (the dashboard this scans for), and PLAN "Repoint rules", whose deletion rule depends on this coverage.

[mmcky]

Correction: the count is 12, not 10 — and the sweep that produced it had the same blind spot it was documenting

Copilot, reviewing QuantEcon/lecture-python-intro#824, flagged a reference this issue's inventory had missed:

Let us bring the data into pandas from a spreadsheet that is hosted on github.

A link to the directory, not to a file. The inventory above was built by searching for links ending in a data-file extension — so a directory link has nothing to match, and was invisible to the very sweep written to catch invisible references. Worth stating plainly: the method used to find the audit's blind spot shared it.

The failure mode is also worse than the ones already listed. That directory survives this migration holding other files, so the link keeps returning 200 while quietly pointing somewhere that no longer contains the spreadsheet it introduces. A 404 gets noticed; a link that silently means the wrong thing does not.

The two additional references

Data reference	Where	Kind	Set
lectures/datasets/ directory	intro inflation_history.md:82	directory link	set 2 — fixed
lectures/datasets/ directory	wasm inflation_history.md:71	directory link	set 2 — fixed

Both now point at CATALOG.md instead: a reader following "hosted on github" gets the dataset registry with source, licence, integrity status and consuming lectures, which is more than the folder gave them, and it is generated so it cannot drift.

Two more of the same shape, for a later wave

Re-sweeping properly also found these, which are not data files but are the same class of invisible reference into a directory the migration will touch:

Reference	Where	Relevant to
_static/lecture_specific/inequality/data.ipynb	intro inequality.md:618	wave 4 — this is the usa-gini builder
same	wasm inequality.md:616	wave 4

That one matters beyond a broken link: usa-gini-nwealth-tincome-lincome.csv is constructed by that notebook, from SCF_plus_mini. When it migrates, the builder migrates with it, and these links have to follow.

What this changes about the fix

It sharpens the argument for scanning by filename against the manifests rather than by URL pattern. Any regex over link syntax will keep missing shapes nobody anticipated — this issue is now its own evidence for that. Matching known dataset paths and directories against every lecture repo does not depend on guessing the form a reference takes.

Two additions to the suggested check: it should cover directory references to a migrated file's former location, not only file references; and it should treat a reference that still resolves as a finding, since a stale-but-200 link is the harder case to catch by any other means.

[mmcky]

Re-swept today: eight of the ten are already fixed, and the inventory is missing a whole repo

Two corrections, in opposite directions. The count of outstanding references is much smaller than the table suggests, and the scope of the inventory is one repo narrower than the problem.

Eight of the ten are done. All six french_rev prose links now point at data-lectures/blob/main/lectures/… (intro and wasm french_rev.md:60-62 each), and both chapter_3.xlsx {download} directives point at data-lectures/raw/main/lectures/chapter_3.xlsx (intro inflation_history.md:228, wasm :220). Both directory links from the correction comment are on CATALOG.md (intro :82, wasm :71). What is left in the workspace repos is four, not ten:

Reference	Where	Kind
life-expectancy-vs-gdp-per-capita.csv	intro simple_linear_regression.md:411	{download}, /raw/
same	wasm simple_linear_regression.md:421	{download}, /raw/
_static/lecture_specific/inequality/data.ipynb	intro inequality.md:618	/tree/ link
same	wasm inequality.md:616	/tree/ link

The inventory is workspace-scoped, and the class is not. lecture-intro.zh-cn carries seven members of it. Five are already repointed, because the sync workflow copies whole cells and the English fix rode along: french_rev.md:67,68,69 (all three spreadsheets, on data-lectures), inflation_history.md:98 (the CATALOG.md directory link) and :245 (the chapter_3.xlsx {download}). Two are unfixed, and they are unfixed for exactly the reason the four above are — their English originals still are: simple_linear_regression.md:421 ({download}) and inequality.md:610 (the /tree/ link into lecture-python-intro). So the real remaining set is six across three repos, and fixing the four upstream ones clears the zh-cn pair on the next sync — for .md sources only, and only for editions that have a sync workflow (lecture-python-programming.ml has none).

That also bites the suggested fix. build_audit.py sets SCAN_REPOS to the eight workspace repos and excludes translations "by decision" (:45-57), so a filename-against-manifests scan built there would reproduce this issue's blind spot in a new form: correct method, wrong repo list.

The data.ipynb link is worse than a link. That notebook is excluded from the scan twice over — build_audit.py takes lectures/**.md only and skips /_static/ (:215-216, :222-224) — and it reads media.githubusercontent.com/media/QuantEcon/high_dim_data/main/SCF_plus/SCF_plus_mini.csv at line 37 in all three repos (intro, wasm, zh-cn), byte-identical. The sync orchestrator is .md-only, so zh-cn's twin can never receive a sync PR for it. That is three hand edits in three repos, and it belongs on the same work item as the two inequality.md prose links rather than being tracked separately.

Sequencing. None of this gates the high_dim_data fold. A naming hazard worth flagging: this issue's "set 3" is french_rev (done), while step 3 of QuantEcon/workspace-lectures#23 is the six-file high_dim_data fold — and step 3 has zero prose exposure. Neither intro nor wasm holds a copy of those six files, every read is a cross-repo URL, and no {download} or prose link names any of them. All six remaining references, plus the three data.ipynb:37 builder reads, land in step 4 (life-expectancy, usa-gini and its builder). Worth stating on the fold's PR so the manual step is not re-litigated there.