Skip to content

[GPU] create tensor from file - #37153

Open
michal-miotk wants to merge 87 commits into
openvinotoolkit:masterfrom
michal-miotk:tensor_file
Open

[GPU] create tensor from file#37153
michal-miotk wants to merge 87 commits into
openvinotoolkit:masterfrom
michal-miotk:tensor_file

Conversation

@michal-miotk

@michal-miotk michal-miotk commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Details:

  • as it is possible to create tensor from cpu memory([GPU] Support CPU pointers on remote tensor #36539) , it is good to have also possibility to read/write data from file and create tensor in one function - this PR encapsulates mmaping and creating remote tensor from cpu memory.
  • file permissions will decide if tensor will be created with read or read write capabilities
  • it is possible to create tensor from file with offset

Tickets:

  • 191076

AI Assistance:

  • AI assistance used: yes

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@github-actions github-actions Bot added the category: Core OpenVINO Core (aka ngraph) label Aug 10, 2026
@michal-miotk
michal-miotk requested a review from a team as a code owner August 10, 2026 14:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

0,
0,
ov::intel_gpu::SharedBufferHandle{},
VirtualAddressMemory{mapped_memory->data(), static_cast<int64_t>(*byte_size)},

@javier-intel javier-intel left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@olpipi olpipi self-assigned this Aug 10, 2026
Comment thread src/core/tests/mmap_object.cpp Outdated
Comment on lines +272 to +275
ASSERT_NE(mm, nullptr);
ASSERT_EQ(mm->size(), k_size);
ASSERT_THAT(std::vector<uint8_t>(m_content.begin() + k_offset, m_content.begin() + k_offset + k_size),
ElementsAreArray(reinterpret_cast<const uint8_t*>(mm->data()), mm->size()));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whether load_mmap_object does its job is tested elsewhere. This test precondition is that before and after values differ.

Comment thread src/common/util/include/openvino/util/mmap_object.hpp
Comment thread src/common/util/src/os/lin/lin_mmap_object.cpp
@maxnick maxnick self-assigned this Aug 10, 2026
@maxnick

maxnick commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

@michal-miotk , could you please fill out the PR description?


/** @brief Legacy single-call MapViewOfFile path (no partial-release support). */
void legacy_setup(size_t aligned_offset, size_t head_pad, size_t size);
void legacy_setup(size_t aligned_offset, size_t head_pad, size_t size, MmapMode mode = MmapMode::READ);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needn't be defaulted - legacy_setup is private and called only once.

Suggested change
void legacy_setup(size_t aligned_offset, size_t head_pad, size_t size, MmapMode mode = MmapMode::READ);
void legacy_setup(size_t aligned_offset, size_t head_pad, size_t size, MmapMode mode);


/** @brief Core setup shared by set() and set_from_handle(). */
void setup(HANDLE file_handle, size_t offset, size_t size, bool no_placeholder);
void setup(HANDLE file_handle, size_t offset, size_t size, bool no_placeholder, MmapMode mode = MmapMode::READ);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd avoid default parameter and explicitly set mode when called.

* so the file must not be modified until the returned tensor is destroyed.
* @ingroup ov_runtime_ocl_gpu_cpp_api
*/
struct FileDescriptor { // need to be merged with ov::intel_npu::FileDescriptor in future

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: Core OpenVINO Core (aka ngraph) category: CPP API OpenVINO CPP API bindings category: docs_snippets OpenVINO docs snippets (docs/snippets) category: docs OpenVINO documentation category: GPU OpenVINO GPU plugin category: inference OpenVINO Runtime library - Inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants