isolate dpnp sua offset fix - #3330
Conversation
|
/intelci: run |
Codecov Report❌ Patch coverage is
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
icfaust
left a comment
There was a problem hiding this comment.
@ethanglaser quick question, wouldn't this have thrown off all use_raw_input results?
Yes. The reason it was not caught is there are minimal slicing operations in the spmd test suite. But any slicing operation in these device-only dpnp arrays would not account for the indicated offset. |
|
Pretty big bug and equally large test gap |
|
/intelci: run |
There was a problem hiding this comment.
Pull request overview
This PR fixes a latent bug in the DPC (SYCL USM) zero-copy conversion path where dpnp slice views were converted to oneDAL tables without applying the __sycl_usm_array_interface__['offset'], causing oneDAL to read from the wrong starting element for views.
Changes:
- Add
get_sua_offset()helper to read the optional SUAoffsetfield (defaulting to 0). - Apply the SUA offset during
convert_to_homogen_implpointer construction so slice views point at the correct starting element. - Add a regression test covering dpnp slice-view roundtrips through
to_table/from_table.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| onedal/datatypes/tests/test_data.py | Adds regression test for dpnp slice-view offset handling in SUA-backed table conversions. |
| onedal/datatypes/sycl_usm/sycl_usm_utils.hpp | Declares new get_sua_offset() helper in the SUA utilities API. |
| onedal/datatypes/sycl_usm/sycl_usm_utils.cpp | Implements get_sua_offset() to retrieve the optional SUA offset field. |
| onedal/datatypes/sycl_usm/data_conversion.cpp | Applies SUA offset when building the oneDAL homogen table pointer from SUA metadata. |
| std::int64_t get_sua_offset(const py::dict& sua) { | ||
| if (!sua.contains("offset") || sua["offset"].is_none()) { | ||
| return 0l; | ||
| } | ||
| return sua["offset"].cast<std::int64_t>(); | ||
| } |
| X_view = X_dp[start:] | ||
| X_roundtrip = from_table(to_table(X_view), like=X_dp) | ||
|
|
|
/intelci: run |
|
/intelci: run |
Description
Isolated from #3322 where it was initially discovered. Basically any dpnp non-host offloaded array slicing operation would unsuccessfully apply an offset (ie X_dpnp_device[5:10] would actually pull X_dpnp_device[0:5]). Per Claude:
Root cause
The C++
convert_to_homogen_implinonedal/datatypes/sycl_usm/data_conversion.cppbuilt the oneDAL table from__sycl_usm_array_interface__['data'][0](the base allocation pointer) but never applied theoffsetfield. A dpnp slice view (e.g.X[a:b]) keeps the samedata[0]base pointer and encodes its start position viaoffset(in elements, not bytes). Without applying it, oneDAL read from row 0 of the allocation for every view rather than from the view's actual start.Symptom
Under
array_api_dispatch=True,IncrementalEmpiricalCovariance.fit(dpnp_X)produced a wronglocation_/score — surfaced astest_whitened_toy_score[dpnp-*]returning ≈ -13.58 vs expected ≈ -14.18 (identical on CPU and GPU).IncrementalEmpiricalCovariance.fitbatches its input via slice views (X[batch_start:batch_end]), so every batch after the first read the wrong rows. Single-batch fits (batch_size ≥ n_samples) and explicitly copied batches were unaffected.Minimal repro, independent of covariance: a dpnp
X[5:10]view round-tripped throughonedal.datatypes.to_table/from_tablereturns row-0 data instead of row-5.Why it wasn't caught before
On
main, dpnp inputs withoutarray_api_dispatchare routed through_transfer_to_host(→asnumpy) before reaching the SUA→table path, so slice views never hit this conversion. This is a latent backend bug; #3322 (removing non-array-API dpnp support) only exposes it by passing dpnp slice views directly toto_table.Fix
Added
get_sua_offset()insycl_usm_utils.cpp/.hpp(reads the optionaloffset, defaults to 0) and applied it:ptr = get_sua_ptr(...) + offset.Checklist:
Completeness and readability
Testing
Performance