fix: clear space caches on reset for generated agents (#229) - #271
Open
teddytennant wants to merge 1 commit into
Open
fix: clear space caches on reset for generated agents (#229)#271teddytennant wants to merge 1 commit into
teddytennant wants to merge 1 commit into
Conversation
Procedurally generated agent envs can change observation/action space shapes across resets. SuperSuit wrappers cached spaces with lru_cache and reused per-agent modifiers, so after reset the cached space no longer matched live observations (out-of-bounds, broadcast errors, bad actions). Clear observation/action space caches (and rebuild shared-wrapper modifiers) on reset. Re-enable generated_agents_test for wrappers that now pass PettingZoo API tests. Leave AEC frame_skip skipped with a clear reason: it still does not fully implement the dead-agent protocol when agents are created or destroyed mid-episode (parallel frame_skip passes). Fixes Farama-Foundation#229
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
test/generated_agents_test.pyhad three entire PettingZoo API test suites skipped with TODOs (issue #229). The failures looked like out-of-bounds observations, action-space mismatches, and numpy broadcast errors when wrapping PettingZoogenerated_agents_*environments.Root cause
Those example envs regenerate agent types (and therefore observation/action shapes) on every
reset(). Several SuperSuit wrappers cache per-agent spaces with@functools.lru_cacheand keep long-lived per-agent modifiers:observation_lambda/action_lambdashared_wrapper(frame stack, sticky actions, delay obs, max obs, nan_random, etc.)After a second reset, the cached space for e.g.
type0_0still had the previous episode shape, while the live observation used the new shape. That produced the reported failures.Fix
observation_space/action_spaceLRU caches onreset()in the AEC lambda wrappers.shared_wrapper(AEC and Parallel), drop all per-agent modifiers and clear space caches onreset(), then rebuild modifiers for the new agent set.generated_agents_testfor the wrappers that now pass.frame_skip_v0out of the enabled suites (documented skip): mid-episode agent birth/death still breaks the PettingZoo dead-agent protocol under AEC frame skip. Parallelframe_skipis tested and passes.Test plan
pytest test/generated_agents_test.py -o addopts=→ 45 passed, 1 skippedpytest test/aec_mock_test.py -o addopts=→ 36 passedpytest test/parallel_env_test.py -o addopts=→ 5 passedFixes #229