downsample pipelines reject pixel types the rest of the package handles

[vboussot]

The three downsample pipelines reject some input pixel types that the rest of the package handles, and the gap is in the compiled modules rather than in a binding: @itk-wasm/downsample 2.0.2 on Node and itkwasm-downsample 2.0.2 with itkwasm-downsample-wasi 2.0.2 on Python fail on exactly the same inputs, with Unsupported image type: {"dimension":2,"componentType":"int32","pixelType":"VariableLengthVector","components":3} and its variants.

Measured on a 2D 8x8 image with 3 components for the vector rows, shrinkFactors: [2, 2], no other options:

pipeline	Scalar	VariableLengthVector
downsample	all 8 types ok	int8, int32, uint32 rejected
downsampleBinShrink	all 8 types ok	int8, int32, uint32 rejected
downsampleLabelImage	float64 rejected	every type rejected

The lists in packages/downsample on main (480a6d6) explain the table. downsample.cxx and downsample-bin-shrink.cxx instantiate every integer and floating scalar type but only five vector types:

itk::wasm::SupportInputImageTypes<PipelineFunctor,
                                  uint8_t, int8_t, uint16_t, int16_t, uint32_t, int32_t, uint64_t, int64_t, float, double,
                                  itk::VariableLengthVector<uint8_t>,
                                  itk::VariableLengthVector<uint16_t>,
                                  itk::VariableLengthVector<int16_t>,
                                  itk::VariableLengthVector<float>,
                                  itk::VariableLengthVector<double>>::Dimensions<2U, 3U, 4U, 5U>("input", pipeline);

downsample-label-image.cxx stops at float and has no vector entry:

itk::wasm::SupportInputImageTypes<PipelineFunctor,
                                  int8_t, uint8_t, int16_t, uint16_t, int32_t, uint32_t, int64_t, uint64_t,
                                  float>::Dimensions<2U, 3U, 4U, 5U>("input", pipeline);

Adding VariableLengthVector<int8_t>, <uint32_t> and <int32_t> to the first two lists would make the vector rows match the scalar rows, and double in the label image list would complete that one. Whether a vector label image should be supported at all is a separate question, since a label is a scalar and a multi-channel label image is several label maps, but a caller cannot tell that from the error message today.

Reproduction, Python:

import numpy as np
import itkwasm
from itkwasm_downsample import downsample, downsample_bin_shrink, downsample_label_image

vector = itkwasm.image_from_array(np.zeros((8, 8, 3), dtype=np.int32), is_vector=True)
downsample(vector, shrink_factors=[2, 2])              # exits with Unsupported image type
downsample_bin_shrink(vector, shrink_factors=[2, 2])   # same
downsample_label_image(itkwasm.image_from_array(np.zeros((8, 8), dtype=np.float64)), shrink_factors=[2, 2])  # same

Node:

import { downsampleNode } from "@itk-wasm/downsample";

const image = {
  imageType: { dimension: 2, componentType: "int32", pixelType: "VariableLengthVector", components: 3 },
  name: "image", origin: [0, 0], spacing: [1, 1], direction: new Float64Array([1, 0, 0, 1]),
  size: [8, 8], metadata: new Map(), data: new Int32Array(8 * 8 * 3),
};
await downsampleNode(image, { shrinkFactors: [2, 2] });  // Unsupported image type

Where it shows up: ngff-zarr sends images with up to 8 channels through the vector path, so a 3-channel int32 image downsampled with bin shrink, or any multi-channel label image, stops on these messages (fideus-labs/ngff-zarr#672 has the details on the caller side).

[thewtex]

There is a tradeoff with wasm binary size and pixel support, we need to account for. If there are use cases with the need for the particular types, then we could add that type.