Add valid fill_value to Uint16 range reduction scale_offset codec example - #68
Add valid fill_value to Uint16 range reduction scale_offset codec example#68bogovicj wants to merge 3 commits into
Conversation
* also add brief explanatory sentence
| ### Uint16 range reduction | ||
|
|
||
| In this example, a `uint16` array with values in the range `[1000, 1255]` is shifted down by `1000` so that values fall in the range `[0, 255]`, then cast to `uint8` via the `cast_value` codec. | ||
| In this example, a `uint16` array with values in the range `[1000, 1255]` is shifted down by `1000` so that values fall in the range `[0, 255]`, then cast to `uint8` via the `cast_value` codec. Note that with these codecs,`fill_value` is constrained to the range `[1000, 1255]`. |
There was a problem hiding this comment.
we can handle fill values outside the decoded range of [1000, 1255] by prepending a cast_value codec with a scalar map that sends fill_value to -> 1256 on the encode path, and sends 1256 -> fill_value on the decode path. Would that example be helpful here?
There was a problem hiding this comment.
this would make the transformation lossy because we would be squeezing 257 input values (256 consecutive numbers + fill value) into 256 outputs, so maybe we need to reduce the number of input values by 1
There was a problem hiding this comment.
Would that example be helpful here?
Yes, I agree that would be useful too. I think it'd be useful to include two examples
- this one - with a "simple"
cast_valuecodec that emphasizes the constraints onfill_value - a more involved
cast_valuelike what you suggest to makefill_valueless constrained.'
If you agree, I can add (2).
There was a problem hiding this comment.
so maybe we need to reduce the number of input values by 1
agreed. maybe we show that for what I'm calling example (2) above.
There was a problem hiding this comment.
Okay, latest commit adds such an example, though as you'll see I decided to just map 0 -> 1000 without shrinkng the range of values as I feel it gets the point across, but feel free to edit if you prefer.
* prepend with cast_value with scalar_map enables use of any fill_value
Co-authored-by: Davis Bennett <davis.v.bennett.git@gmail.com>
2 participants
I think it would be informative to readers to emphasize the valid range that
fill_valuecan take for the "Uint16 range reduction" example.