`region_name` must now be specified when reading from S3 buckets not in us-east-1

[IgorTatarnikov]

Prior to 0.44 running the below resolved successfully with s3fs dealing with the redirect gracefully under the hood:

import ngff_zarr as nz

multiscale = from_ome_zarr("s3://brainglobe/atlas/annotation-sets/allen_mouse-annotation/3_0/annotations_compressed.ome.zarr", storage_options={"anon": True})

Running it with ngff-zarr>=0.44.0 now leads to the below stack trace, where the us-east-1 region is used by default. It's now mandatory to pass the correct region the bucket is in, adding "region_name": "us-west-2" to storage_options resolves this error.

Stack trace

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/from_ngff_zarr.py:264, in from_ome_zarr(store, validate, version, storage_options)
    259     store = RemoteZarrStore(store, storage_options=storage_options)
    261 # Open the root node with helpful error messages, dispatching on the
    262 # store type: local paths -> compat layer, bytes mappings -> store
    263 # reader, remote handles -> zarrista/obstore.
--> 264 root = _open_root_node(store, version)
    266 # Check root-level attributes first to see if this is an HCS plate
    267 root_attrs_initial = root.attrs.asdict()

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/from_ngff_zarr.py:158, in _open_root_node(store, version)
    150 """Open the root node of *store* for reading via the best backend.
    151
    152 Local directory paths go through the zarrista compat layer (direct
   (...)    155 reader. Any other store object raises ``TypeError``.
    156 """
    157 if isinstance(store, RemoteZarrStore):
--> 158     return _open_remote_root(store, version)
    159 if _is_bytes_mapping(store):
    160     from ._v2_store_reader import V2Group, V3Group, open_store_node

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/from_ngff_zarr.py:141, in _open_remote_root(store, version)
    133 def _open_remote_root(store: RemoteZarrStore, version: str | None):
    134     """Open the root group of a remote store via zarrista's async API.
    135
    136     zarrista auto-detects the zarr format (preferring format 3, with the
   (...)    139     needs no format pinning here.
    140     """
--> 141     node = open_remote_node(store)
    142     if node is None:
    143         raise _group_not_found_error(store)

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/_remote_reader.py:420, in open_remote_node(store, component)
    417             return None
    418         # Transport/store errors repeat identically for both node types;
    419         # propagate the group attempt's, which callers opened first.
--> 420         raise group_error from array_error
    421     return RemoteZarrArray(store, normalized, arr)
    422 node = RemoteZarrGroup(store, normalized, group)

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/_remote_reader.py:409, in open_remote_node(store, component)
    407 normalized = path.strip("/")
    408 try:
--> 409     group = _run(lambda: AsyncGroup.open(store._store, path))
    410 except GroupCreateError as group_error:
    411     # Either the node is an array (missing or non-group metadata) or the
    412     # store itself is unreachable; the array attempt distinguishes them.
    413     try:

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/_remote_reader.py:97, in _run(factory)
     94 async def _invoke():
     95     return await factory()
---> 97 return asyncio.run_coroutine_threadsafe(_invoke(), _io_loop()).result()

File ~/miniforge3/envs/env-test/lib/python3.13/concurrent/futures/_base.py:460, in Future.result(self, timeout)
    458     raise CancelledError()
    459 elif self._state == FINISHED:
--> 460     return self.__get_result()
    461 else:
    462     raise TimeoutError()

File ~/miniforge3/envs/env-test/lib/python3.13/concurrent/futures/_base.py:402, in Future.__get_result(self)
    400 if self._exception is not None:
    401     try:
--> 402         raise self._exception
    403     finally:
    404         # Break a reference cycle with the exception in self._exception
    405         self = None

File ~/miniforge3/envs/env-test/lib/python3.13/site-packages/ngff_zarr/_remote_reader.py:95, in _run.<locals>._invoke()
     94 async def _invoke():
---> 95     return await factory()

GroupCreateError: Generic S3 error: Error performing GET https://s3.us-east-1.amazonaws.com/brainglobe/atlas/annotation-sets/allen_mouse-annotation/3_0/annotations_compressed.ome.zarr/zarr.json in 238.890234ms - Received redirect without LOCATION, this normally indicates an incorrectly configured region

[thewtex]

@IgorTatarnikov thanks for the report 🙏

Please check whether #713 addressed your use case. This will be published in a release later this week.

[IgorTatarnikov]

This is resolved after #713, thanks!