Skip to content

Test interdependency which makes some tests fail #194

Description

@ggayDiamond

Problem Description
When running the full test suite, three tests in test_cli.py failed with Pydantic serialization errors: test_convert_header, test_convert_device_name and test_reconvert
However, these same tests passed when run individually.
Error messages indicated that Pydantic models like ToggleButton, TextWrite, Include and Group had corrupted serialization schemas:

PydanticSerializationError: Error calling function `serialize_sequence_via_list`:
  PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected `ToggleButton` - 
  serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])

Error Messages
Those messages can be observed when running 'tox -p' on the branch fix-group-label-presevation.
The cumulative code changes on the file screen.py trigger a different import/initialisation sequence of the formatter module and cause the formatter module to rebuild Pydantic models in an unpredictable order; multiple redundant model_rebuild() calls corrupt the serializer state.

Run all tests will return:

FAILED tests/test_cli.py::test_convert_header - pydantic_core._pydantic_core.PydanticSerializationError: Error calling function serialize_sequence_via_list: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected ToggleButton - serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
PydanticSerializationUnexpectedValue(Expected Include - serialized value may not be as expected [field_name='children', input_value=Group(name='SimDetector',...e='Group', type='Group'), input_type=Group])
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected ToggleButton - serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
PydanticSerializationUnexpectedValue(Expected 1 fields but got 0: Expected ToggleButton - serialized value may not be as expected [field_name='write_widget', input_value=ToggleButton(type='ToggleButton'), input_type=ToggleButton])
FAILED tests/test_cli.py::test_convert_device_name - pydantic_core._pydantic_core.PydanticSerializationError: Error calling function serialize_sequence_via_list: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
PydanticSerializationUnexpectedValue(Expected Include - serialized value may not be as expected [field_name='children', input_value=Group(name='Mako125B', la...e='Group', type='Group'), input_type=Group])
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
FAILED tests/test_cli.py::test_reconvert - pydantic_core._pydantic_core.PydanticSerializationError: Error calling function serialize_sequence_via_list: UserWarning: Pydantic serializer warnings:
PydanticSerializationUnexpectedValue(Expected 4 fields but got 3: Expected TextWrite - serialized value may not be as expected [field_name='write_widget', input_value=TextWrite(lines=None, for...=None, type='TextWrite'), input_type=TextWrite])
PydanticSerializationUnexpectedValue(Expected Include - serialized value may not be as expected [field_name='children', input_value=Group(name='SimDetector',...e='Group', type='Group'), input_type=Group])
===================================== 3 failed, 62 passed, 8 skipped in 5.39s ======================================

Run each test individually will return:

tests/test_cli.py::test_convert_device_name PASSED                       [100%]
tests/test_cli.py::test_convert_header PASSED                            [100%]
tests/test_cli.py::test_reconvert PASSED                                 [100%]

Cause of the problem and suggested solution were reached using Github Copilot.

Root Cause
The TypedModel base class uses a models_typed class variable to guard against redundant model_rebuild() calls. However, this variable was never being set to True after rebuilding, causing model_rebuild(force=True) to be invoked repeatedly on the same Pydantic models during test execution.
When Pydantic's model_rebuild(force=True) is called multiple times on models with complex union types, the internal serializer state becomes corrupted causing validation errors. This happens because:

  1. Models with union fields (like write_widget: WriteWidgetUnion) have their serializer state cached
  2. Multiple force-rebuilds regenerate the schema but don't properly invalidate the cached serializer
  3. The cached serializer becomes out of sync with the actual model schema
  4. Serialization attempts fail because the expected field count doesn't match

Solution
A three-part fix implementation is required:

typed_model.py:
Added models_typed = True assignment after rebuild_child_models() in model_json_schema()
Added _rebuilt_models tracking to prevent rebuilding the same model class multiple times

device.py:
Added module-level TypedModel.rebuild_child_models() and TypedModel.models_typed = True call at the end of the file to force a single rebuild at import time, before any tests run

__init__.py:
Added unconditional call to Formatter.rebuild_child_models() after all Formatter subclasses are defined
Use unconditional rebuild (not checking models_typed) because the shared class variable may have been set by Device models

Schema files (regenerated):
pvi.device.schema.json
pvi.formatter.schema.json
Now properly include type fields for all model classes

These changes ensure that Pydantic models are rebuilt exactly once, with proper tracking to prevent the serializer state corruption that was causing the test failures.
See commit 2e2352f "Fixes #194: Pydantic serialization error" in branch fix_test_interdependency

Warning: ignore the suggested reference to commit aeeee92 below as it was an erroneous commit

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions