Skip to content

codegen: choose generated names once and enforce generated contracts - #3969

Open
raphael wants to merge 43 commits into
v3from
fix/shared-union-package-names
Open

codegen: choose generated names once and enforce generated contracts#3969
raphael wants to merge 43 commits into
v3from
fix/shared-union-package-names

Conversation

@raphael

@raphael raphael commented Aug 21, 2026

Copy link
Copy Markdown
Member

What this fixes

Goa could choose a generated Go name in one local naming scope and later look it up in another. When two services or transports wanted the same name, those scopes did not always see conflicts in the same order. Goa could therefore declare ValidateLifecycle and emit a call to ValidateLifecycle2. AURA exposed the result as generated source that did not compile.

This change makes one generation run choose every generated package name, import name, type name, function name, and constant name before writing source. Service, HTTP, gRPC, JSON-RPC, OpenAPI, example, and plugin generation all read those same final choices. Templates write the selected code; they do not choose names or rediscover static type shapes while the generated program runs.

The result is repeatable generated source whose declarations and references agree across services, transports, shared packages, and plugins.

Before and after

Before:

  1. Each generator built mutable service or transport data of its own.
  2. Several render paths asked a local NameScope to choose a name again.
  3. Shared packages, relocated union types, selected result views, and plugins could discover conflicts in a different order.
  4. A template could call a function that was never declared.

After:

  1. Goa evaluates and prepares the design.
  2. One generation object records every package, import, and declaration requested by built-in generators and plugins.
  3. Goa rejects duplicate declarations, fixes all names, and links the service and transport data that use them.
  4. Templates emit only the already selected names, types, imports, and branches.
  5. Goa verifies that generators and plugins did not change the prepared design while generating source.

Union names use the union's design identity and owning Go package. They do not use decorated strings, file-path registry keys, or changed Hash behavior. AGENTS.md and codegen/ARCHITECTURE.md record this lifecycle so later generator work follows the same ownership rules.

Other incorrect generated contracts fixed here

The shared generation model exposed several places where generated declarations, validation, examples, or transport code disagreed. These fixes belong in Goa because application code cannot safely repair generated wire contracts.

  • A required protobuf OneOf now requires exactly one non-nil branch. Selecting an empty-message branch is valid. Selecting no branch, or selecting a typed wrapper whose value is nil, returns a precise validation error.
  • ArrayOfRequired works for primitive aliases. Goa no longer emits element == nil when the Go element is a value such as a string alias.
  • Incoming HTTP and gRPC transport arrays use pointer elements only where validation must distinguish null from a primitive zero value. Service types retain Goa's existing rules: optional primitive fields use pointers; required primitives and non-primitive service values do not become pointers merely for transport validation.
  • A result fixed to one view uses that view's fields, conversions, imports, and validator everywhere.
  • Multipart decoders fill the generated HTTP request body. Generated validation runs on that body before conversion to the service payload.
  • Server-sent event writers emit code specific to the configured event shape and return write or flush failures.
  • Ordinary HTTP server streaming supports a method with a normal result and streamed results. Interceptors receive the streamed item for stream calls and the normal result for the final call.
  • Generated command-line clients place global flags where their parser accepts them and print a result once.

The final example review found four additional transport defects:

  • gRPC command-line clients had used Go's ordinary JSON decoder for protobuf request messages since 2019. A documented field such as tenantID did not populate the generated protobuf field tagged tenant_id, while unknown fields were silently ignored. Generated gRPC commands now use protobuf JSON. Help shows the exact .proto field names, standard lower-camel protobuf aliases remain accepted, and unknown fields return an error.
  • Goa's whole-query MapParams() handling was internally inconsistent in v3.30.0: generated clients sent key=value, while generated servers searched for query[key]. Servers now decode the flat query entries that generated clients and the DSL contract use.
  • OpenAPI byte examples and security scope lists produced different JSON values depending on whether the document was written as JSON or YAML. Byte examples are now base64 strings in both formats, and a security scheme with no scopes is an empty list rather than null.
  • A server that closed a WebSocket result stream before sending its first item returned an ordinary empty HTTP 200 response. The client then failed while decoding EOF. The first Close now performs the WebSocket handshake and sends the normal close frame. A separate one-time close record preserves the behavior introduced in 2018: later and concurrent Close calls return the first result without writing again.

The gRPC JSON, whole-query map, OpenAPI, and empty-stream problems all exist in v3.30.0; they were not invented by the new planning architecture. The diff corrects them because the complete example regeneration made their caller-visible failures observable.

JSON-RPC contract

This PR supports two clear JSON-RPC transports:

  • HTTP POST for one request and one response.
  • HTTP with server-sent events when the server streams results.

Goa now rejects JSON-RPC methods that request WebSocket transport, stream client payloads, stream in both directions, combine Result with StreamingResult, or declare server streaming without server-sent events. The DSL reports these errors before code generation. Ordinary Goa HTTP WebSockets and gRPC streaming are unchanged.

Every JSON-RPC response contains either result or error; a successful result is present even when its value is null. A malformed object without an ID is an invalid request and receives error -32600 with a null ID. Only a valid request object without an ID is a notification and receives no response.

For a viewed Goa result, the standard JSON-RPC result member contains this JSON value:

{
  "view": "detailed",
  "body": {}
}

The view and body fields are emitted only for results that use Goa views. Results without views keep their normal JSON value.

The removed JSON-RPC WebSocket path was not used by goa-ai's MCP implementation. goa-ai mounts MCP over JSON-RPC HTTP and server-sent events. The dedicated goa-ai generation-migration branch passes its generation, runtime, and race-enabled integration tests against this Goa branch. Current goa-ai main still calls the removed internal NormalizeRoot and service.NewServicesData constructors, so it must merge a dedicated migration before updating its Goa dependency.

Plugin compatibility

The new per-run plugin factory API lets a plugin declare names before Goa fixes them, then generate files using those exact names.

The released extension API remains available:

  • codegen.RegisterPlugin, RegisterPluginFirst, and RegisterPluginLast keep their four-argument signatures and ordering.
  • generator.Generators remains replaceable.
  • generator.Service, Transport, OpenAPI, and Example remain exported with the released Genfunc signature.
  • Built-in functions returned through Generators join the shared naming run. Additional generator functions run with the prepared roots after names are fixed.
  • Generated extension names that can remain stable, including MountHandler and HandlerInit, are preserved.

Plugins that use the released registration API and add or modify generated files should continue to compile. Plugins that depended on mutable internal generator data, changed the design after preparation, or called removed internal constructors must move that work into the new planning API.

External plugin repositories reviewed for follow-up:

  • Likely to need a migration PR: keboola/keboola-as-code, tchssk/goaplugins, xeger/goa-vcr, and kitagry/goaplugin.
  • Current released-API usage appears compatible: pgEdge/control-plane and NagayamaRyoga/goalint.

Breaking changes and required action

Regeneration is required. Generated source from the old and new generator must not be mixed in one module. Current goa-ai main must merge its generator migration before updating to this Goa revision. There is no stored-data migration.

Area Generated change Consumer action
Interceptors A parameter such as info *LoggingInfo becomes info LoggingInfo. The public name is an interface because unary calls, stream sends, and stream receives use separate specific implementations. Update handwritten interceptor signatures to accept the generated interface value. Accessors such as Service(), Method(), CallType(), and typed payload or result methods remain available.
Multipart decoders The callback receives the generated HTTP request-body type instead of the service payload, including cases that formerly used a double pointer. Update custom decoder signatures and fill the request body. Generated validation and conversion do the rest.
JSON-RPC WebSockets JSON-RPC WebSocket designs are rejected and jsonrpc.WebSocketConfig is removed. Use JSON-RPC HTTP POST for unary methods, JSON-RPC server-sent events for server-result streams, or ordinary HTTP WebSockets/gRPC for two-way streaming.
JSON-RPC viewed results Viewed results carry view and body inside result. Successful null results include "result": null. Update clients that decode viewed JSON-RPC results or assumed a successful result member could be absent.
Required primitive transport arrays Some incoming HTTP or gRPC wire fields change from value elements to pointer elements so null can be rejected. Service method types remain values under the existing Goa rules. Regenerate transport adapters and avoid depending on generated wire structs in service code.
Generated names Names that previously depended on collision or traversal order can change. Redundant gRPC view helper names are shortened, and an accidental Request2 may become Request. Regenerate together and call generated declarations instead of copying numbered helper names into handwritten code.
Whole-query maps Servers accept the documented flat key=value entries rather than the accidental query[key]=value form. Handwritten clients using the bracketed form must send flat entries. Generated clients already do.
gRPC command JSON Protobuf request messages use protobuf JSON, reject unknown fields, and show .proto field names. Primitive, array, and map wrapper messages are shown as JSON objects such as {"field": ...}. Update scripts that relied on unknown-field acceptance or supplied an undecodable scalar for a protobuf wrapper.
Server-sent event sends Send methods can return write and flush errors that were previously lost. Handle the returned error. Existing code that already returns or checks it needs no change.
Generated CLI layout Global flags appear before the service and method command, and examples no longer print a result twice. Update scripts that placed a generated global flag after the method command.
Internal plugin data Mutable internal generator data and removed internal constructors are no longer supported extension points. Use the released compatibility API or the new plugin planning API.

The generator and regenerated application code should be deployed together. Rolling back means reverting the Goa revision and regenerating with the prior generator; no database or message conversion is involved.

Verification performed

  • make lint in Goa: zero issues.
  • go test ./... -count=1 in Goa: all packages pass.
  • make integration-test in Goa: the separate JSON-RPC protocol suite passes.
  • go test ./... -count=1 and make itest on goa-ai commit 758a500 with this Goa revision: all package, MCP, and race-enabled integration tests pass. Its lint target reports 33 findings, so this PR does not claim a clean goa-ai lint result.
  • A clean clone of current goa-ai main (26b57ee) was also tested. It does not compile against this Goa branch because it still calls NormalizeRoot and service.NewServicesData. That companion migration remains required before dependency rollout.
  • All 18 designs in goa.design/examples were deleted and regenerated twice with this exact Goa binary. Both runs produced the same digest across 493 generated files.
  • Every one of the 19 example modules passes go test -count=1 ./... and go build ./... against this Goa branch. The nineteenth module is the tracing example, which has no design directory to regenerate.
  • All 493 generated example files were compared with fetched examples origin/main: 238 are unchanged, 252 are modified, and 3 are newly generated. No generated file was omitted from the comparison.
  • All 54 OpenAPI JSON/YAML pairs parse to the same JSON values.
  • Public Go API comparison with apidiff was run for every example module and the affected httpstatus packages; the breaking changes are listed above.
  • Live example checks cover HTTP, gRPC, server-sent events, ordinary HTTP WebSockets, multipart upload, TUS upload, and authentication. The final empty WebSocket history check completes cleanly with no EOF or decode error.
  • AURA's generated directory was recreated with this Goa branch and the service suites that exposed the invalid validator name pass.

No production deployment or stored-data migration was performed.

Reviewer focus

The most useful review order is:

  1. codegen/generation.go, codegen/generated_types.go, and import naming: one owner chooses final names.
  2. codegen/service planning and linking: service types, views, validation, and interceptor call shapes.
  3. HTTP, gRPC, and JSON-RPC plan objects: transport source reads those service decisions.
  4. codegen/generator plugin compatibility tests: released entry points still work while new plugins can declare names before they become final.
  5. Focused golden and integration tests for required unions, primitive aliases, fixed views, multipart requests, CLI protobuf JSON, map queries, empty WebSocket streams, and JSON-RPC errors.

raphael added 30 commits August 20, 2026 21:19
@raphael raphael changed the title codegen: keep union names consistent in shared packages codegen: choose generated names once and enforce transport contracts Aug 24, 2026
@raphael
raphael marked this pull request as ready for review August 24, 2026 04:11
@raphael raphael changed the title codegen: choose generated names once and enforce transport contracts codegen: choose generated names once and enforce generated contracts Aug 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant