Skip to content

fix(docgen): probe member converters to document their real JSON kind - #12869

Open
hudem1 wants to merge 2 commits into
masterfrom
fix/12838-docgen-integer-types
Open

fix(docgen): probe member converters to document their real JSON kind#12869
hudem1 wants to merge 2 commits into
masterfrom
fix/12838-docgen-integer-types

Conversation

@hudem1

@hudem1 hudem1 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Resolves #12838

Follow-up to #12868 (now merged), addressing the per-member converter case raised in review (#12868 (comment)).

Changes

#12868 derives JSON-RPC docs from the serializer contract and already fixes the plain-int case (int/uint_integer_). The remaining #12838 gap is that a member's CLR type alone doesn't determine its wire form: a field with an explicit [JsonConverter] can emit something other than the hex-quantity string its type implies — the classic case being two ulong fields where one carries a raw-number converter (struct-log gas vs transaction gas).

Rather than hard-code the raw-number converter types, this probes the member's converter: it serializes a sample value through that converter and reads back the JSON token.

  • Number/True/False → override the type-based label (_integer_/_number_/_boolean_).
  • String → fall through and keep the editorial flavour (hex data, hash, …).

It works for any converter, not a fixed list — the same "ask the serializer, don't guess" principle as #12868. Probing is deliberately scoped to explicit member converters: probing a type's default serialization is unsound for value-dependent unions (e.g. eth_syncing returns false or an object; a default-value probe would collapse it to _boolean_).

Result / verification

Diffed the generated docs against master. The only changes are the intended fields flipping hex-string → _integer_:

  • trace_* result blockNumber
  • debug_* Geth struct-log entries: gas, gasCost, pc, refund, step

eth_syncing, byte-buffer fields, hashes, addresses and all other shapes are unchanged. Together with #12868, every field reported in #12838 is now correct and matches the wire format (Geth/OpenEthereum/Erigon).

No RPC-module code changes: the implementation already matched the reference clients; only the generated docs were wrong.

Types of changes

  • Bugfix (a non-breaking change that fixes an issue)
  • Documentation update

Testing

Requires testing

  • No

Notes on testing

The DocGen tool has no test project (consistent with #12858/#12868). Verified by regenerating the docs on this branch and diffing against master, confirming only the raw-number-converter fields change.

Documentation

Requires documentation update

  • Yes

Regenerating the docs from this branch updates the NethermindEth/docs JSON-RPC pages.

Requires explanation in Release Notes

  • No

🤖 Generated with Claude Code

@claude

claude Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Claude finished @hudem1's task in 4m 6s —— View job


Review: docgen integer types

  • Read the diff and surrounding generator code
  • Verify the claimed serialization behaviour of int, [ULongRawJsonConverter], [LongRawJsonConverter]
  • Check for missed cases (other raw-number converters, class-level converters, nullables)
  • Post review findings

The diagnosis is correct and the fix is in the right place. I verified each claim against the serializer:

  • IntConverter.Writewriter.WriteNumberValue(value) (Nethermind.Serialization.Json/IntConverter.cs:36), and it is globally registered (EthereumJsonSerializer.cs:109), with NullableIntConverter wrapping it for int? — so Int32_integer_ is right, and the Nullable unwrap at the top of GetJsonTypeName(Type) makes int? land on the same arm.
  • LongRawJsonConverter / ULongRawJsonConverter do write raw numbers, and the fields named in the PR body check out: PortsInfo.Discovery/Listener (int), ParityNetPeers.Active/Connected/Max (int), ParityTxTraceFromStore.BlockNumber ([ULongRawJsonConverter]), Subtraces/TransactionPosition (int/int?), and the Geth struct-log entries in GethTxTraceEntry.cs:20,26,29,37.
  • Checking the attribute before unwrapping Nullable<T> is deliberate-looking and correct — GethTxTraceEntry.Refund is long? with [LongRawJsonConverter] and would otherwise be missed.
  • No regression risk on the Int32 arm: I found no int property in the RPC surface carrying a converter that emits hex, so nothing that legitimately renders as a hex quantity gets downgraded.

Findings

Medium — two raw-number converters are still unhandled (tools/DocGen/JsonRpcGenerator.cs:307-310)
NullableRawLongConverter and NullableRawULongConverter (Nethermind.Serialization.Json/NullableLongConvertercs.cs:16,39) also WriteNumberValue, and NullableRawULongConverter is applied to step in Nethermind.Facade/Eth/BlockForRpc.cs:139 and Nethermind.Facade/Eth/BlockHeaderForRpc.cs:77. Both types are expanded by the generator, so step in the eth_getBlockBy* docs will still be published as _string_ (hex integer) despite being a raw number — the identical bug, one converter name away. Suggested patch posted inline. Fix this →

Low — converter matching by Name string literal
DocGen project-references Nethermind.Runner and already consumes Nethermind.JsonRpc.Modules types directly, so typeof(ULongRawJsonConverter) is available and compile-checked. Repo style prefers type/nameof references over string literals, and since this tool has no test project a rename or move would silently regress the docs with no signal. See the inline suggestion.

Low — nonce has the same class of mismatch
BlockNonceConverter (Nethermind.Facade/Eth/BlockNonceConverter.cs:36-40) writes a zero-padded 8-byte hex string, but nonce is documented as _string_ (hex integer). Pre-existing and strictly out of scope, but the new property-aware hook is exactly where a _string_ (hex data) mapping would go if you want to close it in the same pass.

Low — no regression test
.agents/rules / AGENTS.md ask for a regression test with every bug fix; the PR acknowledges DocGen has no test project (consistent with #12858). Reasonable, but the string-literal coupling above is the kind of thing that only a test would catch — worth a follow-up issue rather than more scope here.

Notes

  • The comment on lines 305-306 and 328-329 explain why, are concise and reference the converters — in line with the repo comment guidelines.
  • I could not compile locally (build command not permitted in this environment), so the change is reviewed by reading only; JsonConverterAttribute.ConverterType and the is "A" or "B" pattern are valid against the existing using System.Text.Json.Serialization; / System.Reflection imports.
  • No production RPC behaviour is touched — I agree with the decision not to "fix" the modules to emit hex, which would diverge from Geth/Parity/Erigon.

Nothing here is a correctness or security risk in the client itself; the Medium is an incomplete-fix gap in the generated docs.
• branch fix/12838-docgen-integer-types

Comment thread tools/DocGen/JsonRpcGenerator.cs Outdated
Comment on lines +305 to +310
// A property may override the default numeric serialization with a converter that writes a raw JSON
// number (e.g. trace/debug block numbers) instead of the hex-quantity string implied by its CLR type.
private static string GetJsonTypeName(PropertyInfo prop) =>
prop.GetCustomAttribute<JsonConverterAttribute>()?.ConverterType?.Name is "ULongRawJsonConverter" or "LongRawJsonConverter"
? "_integer_"
: GetJsonTypeName(prop.PropertyType);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium — this covers only two of the four raw-number converters in the repo. NullableRawLongConverter / NullableRawULongConverter (src/Nethermind/Nethermind.Serialization.Json/NullableLongConvertercs.cs:16,39) also WriteNumberValue, and they are used on step in both BlockForRpc.cs:139 and BlockHeaderForRpc.cs:77 — both of which the generator expands (they are eth_getBlockBy* / AuRa results). Those fields will still be documented as _string_ (hex integer) while the wire value is a raw number, i.e. exactly the bug this PR fixes.

Second, matching converters by Name string literal is fragile: DocGen already project-references Nethermind.Runner (and uses Nethermind.JsonRpc.Modules types directly), so Nethermind.Serialization.Json types are reachable and a typeof comparison would be compile-checked. With no test project behind this tool, a rename/move would silently reintroduce wrong docs.

Suggested change
// A property may override the default numeric serialization with a converter that writes a raw JSON
// number (e.g. trace/debug block numbers) instead of the hex-quantity string implied by its CLR type.
private static string GetJsonTypeName(PropertyInfo prop) =>
prop.GetCustomAttribute<JsonConverterAttribute>()?.ConverterType?.Name is "ULongRawJsonConverter" or "LongRawJsonConverter"
? "_integer_"
: GetJsonTypeName(prop.PropertyType);
// A property may override the default numeric serialization with a converter that writes a raw JSON
// number (e.g. trace/debug block numbers) instead of the hex-quantity string implied by its CLR type.
private static readonly Type[] _rawNumberConverters = [
typeof(LongRawJsonConverter),
typeof(ULongRawJsonConverter),
typeof(NullableRawLongConverter),
typeof(NullableRawULongConverter)
];
private static string GetJsonTypeName(PropertyInfo prop) =>
prop.GetCustomAttribute<JsonConverterAttribute>()?.ConverterType is Type converterType &&
Array.IndexOf(_rawNumberConverters, converterType) >= 0
? "_integer_"
: GetJsonTypeName(prop.PropertyType);

(needs using Nethermind.Serialization.Json; at the top; the field would read better next to _assemblies/_objectTypeName at the top of the class.)

AnkushinDaniil
AnkushinDaniil previously approved these changes Aug 18, 2026
@hudem1
hudem1 force-pushed the fix/12838-docgen-integer-types branch from 6a44210 to 8f93b50 Compare August 18, 2026 09:12
@hudem1 hudem1 changed the title fix(docgen): document int and raw-number RPC fields as JSON integers, not hex strings fix(docgen): document raw-number-converter fields as JSON integers Aug 18, 2026
@hudem1
hudem1 changed the base branch from master to fix/docgen-serializer-contract August 18, 2026 09:13
dipkakwani
dipkakwani previously approved these changes Aug 18, 2026
@hudem1 hudem1 changed the title fix(docgen): document raw-number-converter fields as JSON integers fix(docgen): probe member converters to document their real JSON kind Aug 18, 2026
Base automatically changed from fix/docgen-serializer-contract to master August 18, 2026 10:44
@dipkakwani
dipkakwani dismissed stale reviews from AnkushinDaniil and themself August 18, 2026 10:44

The base branch was changed.

@dipkakwani dipkakwani left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to rebase, since the base PR got merged before the stacked PR 😅

A member's CLR type does not determine its wire form. A field carrying an
explicit [JsonConverter] can emit something other than the hex-quantity string
its type implies - e.g. two `ulong` fields where one has a raw-number converter
(struct-log gas vs tx gas).

Instead of hard-coding the raw-number converter types, serialize a sample value
through the member's converter and read back the JSON token: a Number/Boolean
overrides the type-based label, while a String keeps its editorial flavour
(hex data, hash, ...). Probing is scoped to explicit member converters; probing
a type's default serialization is unsound for value-dependent unions (e.g.
eth_syncing returns `false` or an object).

Completes #12838 on top of #12868: trace_* result blockNumber and debug_*
struct-log entries (gas/gasCost/pc/refund/step) now render as JSON integers,
matching the wire format and Geth/OpenEthereum/Erigon.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@hudem1
hudem1 force-pushed the fix/12838-docgen-integer-types branch from 0bd5c8c to 3747b96 Compare August 18, 2026 11:15
}
}

private static void InvokeConverterWrite(Type converterType, Type valueType, object sample, Utf8JsonWriter writer)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InvokeConverterWrite looks up Write using the member's CLR type after the nullable unwrap, so for BlockForRpc.Step it asks for Write(..., ulong, ...) against a JsonConverter<ulong?> and succeeds only because the reflection binder widens. A JsonConverterFactory declares no Write, so GetMethod returns null and the ! throws into the blanket catch. Taking the value type from the converter's own JsonConverter<T> base would make both explicit.

private static Type? ConverterValueType(Type converterType)
{
    for (Type? t = converterType; t is not null; t = t.BaseType)
        if (t.IsGenericType && t.GetGenericTypeDefinition() == typeof(JsonConverter<>))
            return t.GetGenericArguments()[0];

    return null;
}

Address review (wurdum): the sample write looked up `Write` by the member's
CLR type after nullable unwrap, so a `JsonConverter<ulong?>` (e.g.
NullableRawULongConverter on BlockForRpc.Step) matched only via binder widening,
and a JsonConverterFactory - which declares no `Write` - would null-deref into
the blanket catch and silently mislabel the field.

Take the value type from the converter's own `JsonConverter<T>` base instead:
it makes the nullable case explicit and returns cleanly for factories. Output
is unchanged (verified by regenerating and diffing).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

numeric fields serialize as raw JSON numbers instead of the documented hexadecimal-quantity strings

4 participants