Problem Description
When running the full test suite, three tests in test_cli.py failed with Pydantic serialization errors: test_convert_header, test_convert_device_name and test_reconvert
However, these same tests passed when run individually.
Error messages indicated that Pydantic models like ToggleButton, TextWrite, Include and Group had corrupted serialization schemas:
PydanticSerializationError: Error calling function `serialize_sequence_via_list`:
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected `ToggleButton` -
serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
Error Messages
Those messages can be observed when running 'tox -p' on the branch fix-group-label-presevation.
The cumulative code changes on the file screen.py trigger a different import/initialisation sequence of the formatter module and cause the formatter module to rebuild Pydantic models in an unpredictable order; multiple redundant model_rebuild() calls corrupt the serializer state.
Run all tests will return:
FAILED tests/test_cli.py::test_convert_header - pydantic_core._pydantic_core.PydanticSerializationError: Error calling function serialize_sequence_via_list: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected ToggleButton - serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
PydanticSerializationUnexpectedValue(Expected Include - serialized value may not be as expected [field_name='children', input_value=Group(name='SimDetector',...e='Group', type='Group'), input_type=Group])
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected ToggleButton - serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected ToggleButton - serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
FAILED tests/test_cli.py::test_convert_device_name - pydantic_core._pydantic_core.PydanticSerializationError: Error calling function serialize_sequence_via_list: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
PydanticSerializationUnexpectedValue(Expected Include - serialized value may not be as expected [field_name='children', input_value=Group(name='Mako125B', la...e='Group', type='Group'), input_type=Group])
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
FAILED tests/test_cli.py::test_reconvert - pydantic_core._pydantic_core.PydanticSerializationError: Error calling function serialize_sequence_via_list: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
PydanticSerializationUnexpectedValue(Expected Include - serialized value may not be as expected [field_name='children', input_value=Group(name='SimDetector',...e='Group', type='Group'), input_type=Group])
===================================== 3 failed, 62 passed, 8 skipped in 5.39s ======================================
Run each test individually will return:
tests/test_cli.py::test_convert_device_name PASSED [100%]
tests/test_cli.py::test_convert_header PASSED [100%]
tests/test_cli.py::test_reconvert PASSED [100%]
Cause of the problem and suggested solution were reached using Github Copilot.
Root Cause
The TypedModel base class uses a models_typed class variable to guard against redundant model_rebuild() calls. However, this variable was never being set to True after rebuilding, causing model_rebuild(force=True) to be invoked repeatedly on the same Pydantic models during test execution.
When Pydantic's model_rebuild(force=True) is called multiple times on models with complex union types, the internal serializer state becomes corrupted causing validation errors. This happens because:
- Models with union fields (like
write_widget: WriteWidgetUnion) have their serializer state cached
- Multiple force-rebuilds regenerate the schema but don't properly invalidate the cached serializer
- The cached serializer becomes out of sync with the actual model schema
- Serialization attempts fail because the expected field count doesn't match
Solution
A three-part fix implementation is required:
typed_model.py:
Added models_typed = True assignment after rebuild_child_models() in model_json_schema()
Added _rebuilt_models tracking to prevent rebuilding the same model class multiple times
device.py:
Added module-level TypedModel.rebuild_child_models() and TypedModel.models_typed = True call at the end of the file to force a single rebuild at import time, before any tests run
__init__.py:
Added unconditional call to Formatter.rebuild_child_models() after all Formatter subclasses are defined
Use unconditional rebuild (not checking models_typed) because the shared class variable may have been set by Device models
Schema files (regenerated):
pvi.device.schema.json
pvi.formatter.schema.json
Now properly include type fields for all model classes
These changes ensure that Pydantic models are rebuilt exactly once, with proper tracking to prevent the serializer state corruption that was causing the test failures.
See commit 2e2352f "Fixes #194: Pydantic serialization error" in branch fix_test_interdependency
Warning: ignore the suggested reference to commit aeeee92 below as it was an erroneous commit
Problem Description
When running the full test suite, three tests in
test_cli.pyfailed with Pydantic serialization errors:test_convert_header,test_convert_device_nameandtest_reconvertHowever, these same tests passed when run individually.
Error messages indicated that Pydantic models like
ToggleButton,TextWrite,IncludeandGrouphad corrupted serialization schemas:Error Messages
Those messages can be observed when running 'tox -p' on the branch
fix-group-label-presevation.The cumulative code changes on the file
screen.pytrigger a different import/initialisation sequence of the formatter module and cause the formatter module to rebuild Pydantic models in an unpredictable order; multiple redundantmodel_rebuild()calls corrupt the serializer state.Run all tests will return:
Run each test individually will return:
Cause of the problem and suggested solution were reached using Github Copilot.
Root Cause
The
TypedModelbase class uses amodels_typedclass variable to guard against redundantmodel_rebuild()calls. However, this variable was never being set to True after rebuilding, causingmodel_rebuild(force=True)to be invoked repeatedly on the same Pydantic models during test execution.When Pydantic's
model_rebuild(force=True)is called multiple times on models with complex union types, the internal serializer state becomes corrupted causing validation errors. This happens because:write_widget: WriteWidgetUnion) have their serializer state cachedSolution
A three-part fix implementation is required:
typed_model.py:Added
models_typed = Trueassignment afterrebuild_child_models()inmodel_json_schema()Added
_rebuilt_modelstracking to prevent rebuilding the same model class multiple timesdevice.py:Added module-level
TypedModel.rebuild_child_models()andTypedModel.models_typed = Truecall at the end of the file to force a single rebuild at import time, before any tests run__init__.py:Added unconditional call to
Formatter.rebuild_child_models()after all Formatter subclasses are definedUse unconditional rebuild (not checking
models_typed) because the shared class variable may have been set by Device modelsSchema files (regenerated):
pvi.device.schema.jsonpvi.formatter.schema.jsonNow properly include
typefields for all model classesThese changes ensure that Pydantic models are rebuilt exactly once, with proper tracking to prevent the serializer state corruption that was causing the test failures.
See commit 2e2352f "Fixes #194: Pydantic serialization error" in branch fix_test_interdependency
Warning: ignore the suggested reference to commit aeeee92 below as it was an erroneous commit