Add per-file options to extra source files

[andreabedini]

Adds per-file options to the five extra-source fields

This is one of the last things the Cabal library needs before it can build GHC; Hadrian makes the same per-file split today. This patch has been developed for https://github.com/stable-haskell/ghc.

cabal-version: 3.20

library
  c-sources:   foo.c (-DFOO -O2) bar.c
  cmm-sources: Jumps_V32.cmm (-mavx2)

The options are applied only to the relative file.

Based on #12275 which needs to go in first.

Design

ExtraSourceKind tags each source with its kind, and addExtraSourceOpts maps that kind to the one GhcOptions field GHC consults for it:

field	GhcOptions field	GHC flag
c-sources	ghcOptCcOptions	-optc
cxx-sources	ghcOptCxxOptions	-optcxx
asm-sources	ghcOptAsmOptions	-opta
cmm-sources	ghcOptExtra	plain GHC options
js-sources	ghcOptJSppOptions	-optJSP

Only C-- options are plain GHC options, because GHC compiles C-- itself and there is no -optcmm. The other four are options for a tool GHC invokes, so they cannot share ghcOptExtra: ghc -DFOO -c foo.c defines a Haskell CPP macro instead of passing -DFOO to the C compiler.

Keeping the kind — rather than a string the user has already prefixed — leaves the spelling to Cabal at render time, which it has to be, because the prefix depends on the compiler version: -optcxx only exists since GHC 8.10 (below that, C++ options go through -optc), -optJSP only since 9.12 (below that they are dropped, with a warning).

Each extra source is already compiled by its own ghc -c run, so per-file options are just appended to that run's component-wide ones. They come last, so they win over cc-options and its siblings.

Interface change

The five fields hold ExtraSource (a path plus its options) instead of SymbolicPath Pkg File. extraSourceFromPath builds one with no options. Distribution.Simple.SetupHooks re-exports both, so a Hooks package that sets cSources needs no direct Cabal-syntax dependency.

Why the cabal-version gate is an error, and why not 3.18

Gated behind cabal-version: 3.20 per #9331. Worth flagging explicitly: an older Cabal does not reject this syntax, it misreads it. A path is any non-space token, so Cabal-syntax 3.14.1.0 reads

c-sources: foo.c (-DFOO -O2) bar.c

as four source files — foo.c, (-DFOO, -O2), bar.c — with no diagnostic, failing only later when it tries to compile (-DFOO. That rules out a warning, and it rules out the already-published 3.18 as the gate.

Tests

• Parser regression test over all five kinds at 3.20, with quoted options and the pretty-print round trip.
• Error test pinning the rejection below 3.20.
• QuickCheck round-trip property for ExtraSource, with a generator that produces options needing quotes.
• renderGhcOptions unit tests: -optJSP rendered at 9.12 and dropped below it; C-- options plain at every version.
• ExtraSources setup test with a C source that only compiles if its per-file -D arrives.

QA Notes

With cabal-version: 3.20:

• Build a package whose C source #errors unless a macro is defined, and define it per-file: c-sources: greet.c (-DGREETING=hi) other.c. It should build, and -DGREETING must not reach other.c.
• An option containing whitespace has to be quoted: c-sources: greet.c ("-DGREETING=\"hi there\""). Unquoted, it should be read as two options.
• The same file with cabal-version: 3.18 should fail with a parse error naming 3.20 — not a missing-file error about (-DGREETING=hi.
• cabal sdist on a package using per-file options should ship the sources and round-trip the options through the regenerated .cabal.
• With GHC < 9.12: js-sources: foo.js (-DBAR) should warn that the options are ignored and still build; jspp-options: -DBAR should now build rather than failing with an unrecognised-flag error from GHC.

Checklist

Template A: this PR modifies behaviour or interface.

• Patches conform to the coding conventions.
• Changes relevant to users have been recorded in the changelog (significance: significant for the feature).
• The documentation has been updated (doc/cabal-package-description-file.rst, doc/file-format-changelog.rst).
• Manual QA notes have been included.
• Tests have been added.

[Copilot]

Pull request overview

This PR extends Cabal’s .cabal file format and internal APIs to support per-file options on the five “extra source” fields (c-sources, cxx-sources, asm-sources, cmm-sources, js-sources) gated behind cabal-version: 3.20, enabling per-source compilation flags (e.g. foo.c (-DFOO)).

Changes:

• Introduces ExtraSource (path + per-file opts) and updates parsing/pretty-printing/field grammars so the five extra-source fields carry per-file options (guarded at cabal-version >= 3.20).
• Wires per-file options into GHC invocation by routing options into the appropriate GhcOptions fields (including version-gating -optJSP to GHC ≥ 9.12).
• Adds/updates documentation, changelog entries, and tests (parser regression + error test, QuickCheck roundtrip, rendering unit tests, and a setup testsuite case).

Reviewed changes

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
doc/file-format-changelog.rst	Documents the 3.20 format change for per-file extra-source options.
doc/cabal-package-description-file.rst	Adds user-facing documentation for per-file options and updates field docs.
changelog.d/per-file-extra-source-options.md	Changelog entry describing the new per-file options feature and API change.
changelog.d/jspp-options-ghc-912.md	Changelog entry for omitting -optJSP on GHC < 9.12.
changelog.d/cabal-spec-latest-3-20.md	Changelog entry for making 3.20 the latest known spec (PR #12275).
Cabal/src/Distribution/Simple/SrcDist.hs	Ensures sdist source collection includes ExtraSource file paths.
Cabal/src/Distribution/Simple/SetupHooks/Internal.hs	Updates autogen extra-source handling to extract ExtraSource file paths.
Cabal/src/Distribution/Simple/Program/GHC.hs	Gates -optJSP rendering to GHC ≥ 9.12 and updates related documentation.
Cabal/src/Distribution/Simple/GHCJS.hs	Adapts GHCJS build/link handling to ExtraSource-typed extra sources.
Cabal/src/Distribution/Simple/GHC/Internal.hs	Adds ExtraSourceKind + routing of per-file opts into correct GhcOptions.
Cabal/src/Distribution/Simple/GHC/Build/ExtraSources.hs	Switches extra-source builds to ExtraSource, applies opts per file, warns when JS opts are dropped (< 9.12).
Cabal/src/Distribution/Simple/BuildTarget.hs	Updates component file listing to extract file paths from ExtraSource.
Cabal/src/Distribution/Simple/Build.hs	Updates build-info augmentation helpers to wrap paths with extraSourceFromPath.
Cabal/src/Distribution/PackageDescription/Check/Target.hs	Updates path well-formedness checks to validate extraSourceFile.
Cabal-tree-diff/src/Data/TreeDiff/Instances/Cabal.hs	Adds ToExpr ExtraSource instance for tree-diff output.
cabal-testsuite/PackageTests/SetupHooks/SetupHooksNonHs/SetupHooks.hs	Updates setup hooks test to supply ExtraSource values.
cabal-testsuite/PackageTests/ExtraSources/src/MyLib.hs	Adds a testsuite library module for the new extra-sources setup test.
cabal-testsuite/PackageTests/ExtraSources/setup.test.hs	Adds a setup-based integration test for compiling extra sources.
cabal-testsuite/PackageTests/ExtraSources/setup.out	Expected output for the ExtraSources setup test.
cabal-testsuite/PackageTests/ExtraSources/extra-sources.cabal	Test package exercising per-file options in c-sources / cmm-sources.
cabal-testsuite/PackageTests/ExtraSources/cbits/testcmm.cmm	Test C-- file requiring a macro defined via per-file options.
cabal-testsuite/PackageTests/ExtraSources/cbits/test.c	Test C file requiring a macro defined via per-file options.
Cabal-tests/tests/UnitTests/Distribution/Utils/Structured.hs	Updates structured-hash golden values after type/API changes.
Cabal-tests/tests/UnitTests/Distribution/Types/ExtraSource.hs	Adds QuickCheck roundtrip property for ExtraSource parse/pretty.
Cabal-tests/tests/UnitTests/Distribution/Simple/Program/GHC.hs	Adds unit tests for -optJSP gating and C-- option rendering behavior.
Cabal-tests/tests/UnitTests.hs	Registers new ExtraSource unit tests.
Cabal-tests/tests/ParserTests/regressions/libpq2.expr	Updates expected parse AST output for cSources now being ExtraSource.
Cabal-tests/tests/ParserTests/regressions/libpq1.expr	Updates expected parse AST output for cSources now being ExtraSource.
Cabal-tests/tests/ParserTests/regressions/extra-source-opts.format	Adds formatting regression case for per-file options across all five kinds.
Cabal-tests/tests/ParserTests/regressions/extra-source-opts.expr	Adds AST expectation for the per-file extra-source options regression test.
Cabal-tests/tests/ParserTests/regressions/extra-source-opts.cabal	Adds .cabal input exercising per-file options parsing/printing.
Cabal-tests/tests/ParserTests/errors/extra-source-opts-old-spec.errors	Adds expected error output when using per-file opts below spec 3.20.
Cabal-tests/tests/ParserTests/errors/extra-source-opts-old-spec.cabal	Adds error fixture using per-file opts with cabal-version: 3.4.
Cabal-tests/tests/ParserTests.hs	Registers new parser regression and error tests.
Cabal-tests/tests/NoThunks.hs	Adds NoThunks ExtraSource instance for no-thunks testing.
Cabal-tests/Cabal-tests.cabal	Adds new unit test module to the Cabal-tests test-suite stanza.
Cabal-syntax/src/Distribution/Types/ExtraSource.hs	Introduces the ExtraSource type plus parsing/pretty-printing and version guard.
Cabal-syntax/src/Distribution/Types/BuildInfo/Lens.hs	Updates build-info lenses to expose extra-source fields as [ExtraSource].
Cabal-syntax/src/Distribution/Types/BuildInfo.hs	Updates BuildInfo extra-source fields to [ExtraSource] with clarified comments.
Cabal-syntax/src/Distribution/SPDX/LicenseListVersion.hs	Maps CabalSpecV3_20 to SPDX license list 3.28.
Cabal-syntax/src/Distribution/PackageDescription/FieldGrammar.hs	Updates field grammars to parse/format extra-source fields as ExtraSource.
Cabal-syntax/src/Distribution/PackageDescription.hs	Re-exports Distribution.Types.ExtraSource from Distribution.PackageDescription.
Cabal-syntax/src/Distribution/CabalSpecVersion.hs	Sets cabalSpecLatest to CabalSpecV3_20.
Cabal-syntax/Cabal-syntax.cabal	Exposes the new Distribution.Types.ExtraSource module.
Cabal-QuickCheck/src/Test/QuickCheck/Instances/Cabal.hs	Adds Arbitrary ExtraSource generator/shrinker.
cabal-install/tests/IntegrationTests2.hs	Updates integration test fixture construction for cSources :: [ExtraSource].
cabal-install/src/Distribution/Client/TargetSelector.hs	Updates known-component file collection to extract extraSourceFile.
cabal-install/src/Distribution/Client/SourceFiles.hs	Updates source file existence tracking to extract extraSourceFile.
Cabal-hooks/src/Distribution/Simple/SetupHooks.hs	Re-exports ExtraSource and extraSourceFromPath for hooks packages.
Cabal-described/src/Distribution/Described.hs	Adds Described ExtraSource and updates regex charsets for options syntax.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[andreabedini]

Thank you for the review @zlonast

[zlonast]

I don't like the current syntax, so I'll suggest two other options that I think fit better into the ecosystem design.

Option A — file as a section with a nested field

library
  c-source: cbits/foo.c
    c-options: -DFOO -O2

  c-source: cbits/bar.c

Option B — options as a separate map field

library
  c-sources:
    cbits/foo.c
    cbits/bar.c

  c-options:
    cbits/foo.c: -DFOO -O2

[andreabedini]

I don't think either of those can work, but the objection did make me change the syntax — just not to A or B.

A isn't expressible. There are no nested fields in .cabal files. fieldLayout in Cabal-syntax/src/Distribution/Fields/Parser.hs reads a field as an optional first line plus many (do _ <- indentOfAtLeast ilevel; fieldContent), so every more-indented line is more content of the same field. Your example parses as one field c-source whose value is the two lines cbits/foo.c and c-options: -DFOO -O2 — the colon is just text there. That needs a new production in the grammar, not a new field.

B parses, but it splits something that is currently one value. An ExtraSource is self-contained, so [ExtraSource] merges monoidally and there is nothing to keep in sync. A separate map keyed by a path gives two independently merged fields:

library
  c-sources: cbits/foo.c
  if os(windows)
    c-sources: cbits/win.c
    c-options: cbits/foo.c: -DWIN

Both are monoidalFields and accumulate separately, so nothing guarantees a key names a file that is in c-sources in that configuration, and a typo in the path silently does nothing. common stanzas make it worse. Keeping the options on the file makes that unrepresentable.

What I did change: the text between the parentheses is now taken verbatim and split by splitArgs — the same function --PROG-options uses, so the quoting rules are the command line's. Only ( and ) are structural; they nest as long as they balance, and \ escapes (, ) and \. A backslash anywhere else stands for itself, so Windows paths need no doubling.

c-sources:
  cbits/wide.c ("-DMESSAGE=hello there")
  cbits/size.c (-DSIZE=f(1))
  cbits/smiley.c (-DSMILEY=\))

That also killed a bug: (-DX="a b") used to parse as two options, -DX= and a b, and pretty-printing rendered it back as (-DX= "a b") — making the misreading permanent.

[AndreasPK]

The (-DSMILEY=\)) syntax seems very unintuitive to me given how cabal syntax works in general. If there is a need for this feature some of the options @zlonast offered seem far more reasonable.

A isn't expressible. There are no nested fields in .cabal files.

We already have nested blocks, and even if there were no nested blocks we we could still just add them anyway if everyone agrees this is the best way to do it.

[sheaf]

I think this change requires buy-in from the community and the Cabal maintainers before we can proceed to technical review. I say this for two main reasons:

• We need to get some kind of consensus on the syntax. I know this is painful as it will invite a lot of bike-shedding, but I don't think we can skip it.
• It's not clear to me that we want this feature at all. To me it looks like trying to replicate parts of Make in Cabal and that seems wrong to me. Can't we solve the issues Hadrian faces by using build-type: Configure or build-type: Hooks to programmatically change the flags passed when compiling extra sources instead of requiring new syntax in Cabal files?

The Cabal proposal repository can be found here.