Keep the scaled action space valid for a negative scale - #274
Open
DenisDrobyshev wants to merge 1 commit into
Open
Keep the scaled action space valid for a negative scale#274DenisDrobyshev wants to merge 1 commit into
DenisDrobyshev wants to merge 1 commit into
Conversation
scale_actions_v0 built its Box as Box(low=low * scale, high=high * scale).
A negative scale maps [low, high] onto [high * scale, low * scale], so the two
bounds swap places and the low ends up above the high. Gymnasium then rejects
the space from inside its own constructor:
ValueError: Box all low values must be less than or equal to high
which names neither this wrapper nor its scale argument. The bounds are now
ordered with an element-wise minimum and maximum, which is the image of the
interval under the scaling.
The existing test only covers positive factors, 2, 0.5 and 0.3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
scale_actions_v0builds the scaled action space by multiplying both bounds inplace:
A negative scale maps
[low, high]onto[high * scale, low * scale], so thetwo bounds swap places and the low ends up above the high. Gymnasium then
refuses the space from inside its own constructor:
The message names neither
scale_actions_v0nor itsscaleargument, so thereport points at Gymnasium rather than at the wrapper that built the space.
The bounds are now ordered with an element-wise minimum and maximum, which is
the image of the interval under the scaling. Nothing changes for a positive
scale, where
low * scaleis already the smaller of the two.Box(-4.0, 4.0)Box(-4.0, 4.0)Box(-1.0, 1.0)Box(-1.0, 1.0)Box(0.0, 0.0)Box(0.0, 0.0)ValueErrorBox(-2.0, 2.0)ValueErrorBox(-5.0, 5.0)The dtype is left to
Boxto infer exactly as before, so integer action spacesare unaffected by this change.
Tests
test_scale_action_wrapper_negative_scaleintest/aec_mock_test.py, next tothe existing
test_scale_action_wrapper. It uses an asymmetric action space,Box(1.0, 5.0), so the swap is visible: withscale=-2.0the wrapped space isBox(-10.0, -2.0), and the action[2, 1, 3]arrives at the environment as[-4, -2, -6]. It fails onmainwith theValueErrorabove and passes here.The existing test only covers positive factors — 2, 0.5 and 0.3 — which is why
this went unnoticed.
Full suite locally: 106 passed against 105 on
main, the difference being thistest; the one failure and six collection errors are identical on both sides and
come from optional dependencies missing in my environment.
pre-commit runisclean on both files.
Type of change
Checklist
pre-commitchecks withpre-commit run --all-files