Publish serialized FlatBuffers from the tracker API

docs/source/device/add_device.rst

@@ -65,9 +65,6 @@ Reference schema: :code-file:`src/core/schema/fbs/pedals.fbs`
 
 - **Output table** — The primary payload type (e.g. ``Generic3AxisPedalOutput``) with the
   device fields. This is what the plugin serializes and pushes.
-- **Tracked wrapper** — A table that wraps the output in an optional ``data`` field
-  (e.g. ``Generic3AxisPedalOutputTracked``). Used by the in-memory tracker API so that
-  ``data`` can be null when no sample is available.
 - **Record wrapper** — A table that wraps the output plus ``DeviceDataTimestamp``
   (e.g. ``Generic3AxisPedalOutputRecord``). This is the root type written to MCAP channels
   by the recorder; trackers serialize into this type in ``serialize_all()``.
@@ -132,9 +129,9 @@ tensor samples from OpenXR. Implement a concrete tracker class (e.g.
 In the **Impl**:
 
 - **update()** — Call ``m_schema_reader.read_all_samples(pending_records)``. If the
-  collection is not present, clear the tracked state (e.g. set ``m_tracked.data = nullptr``).
-  Otherwise, deserialize the latest sample (or all samples) into your tracked type and
-  keep the last one for ``get_data()``.
+  collection is not present, clear the published state (e.g. assign an empty
+  ``Serialized<Generic3AxisPedalOutput>``). Otherwise, keep the latest sample's buffer and
+  publish it as a ``Serialized<...>`` from ``get_data()``.
 - **serialize_all()** — For each sample in the pending batch, deserialize, build the
   Record FlatBuffer (output table + ``DeviceDataTimestamp``), and invoke the callback with
   ``(log_time_ns, buffer_ptr, size)``. The buffer is only valid during the callback. If the
@@ -146,7 +143,7 @@ Reference implementation — split across facade and live backend:
 - **Tracker facade** — :code-file:`src/core/deviceio_trackers/cpp/generic_3axis_pedal_tracker.cpp`
   (class ``Generic3AxisPedalTracker``): holds collection configuration, implements ``ITracker``, and
   exposes ``get_data(session)`` returning
-  ``Generic3AxisPedalOutputTrackedT`` by dispatching to the session’s
+  ``Serialized<Generic3AxisPedalOutput>`` by dispatching to the session’s
   ``IGeneric3AxisPedalTrackerImpl`` (see :code-file:`src/core/deviceio_base/cpp/inc/deviceio_base/generic_3axis_pedal_tracker_base.hpp`).
 - **Live backend** — :code-file:`src/core/live_trackers/cpp/live_generic_3axis_pedal_tracker_impl.cpp`
   (``LiveGeneric3AxisPedalTrackerImpl``): composes ``SchemaTracker``, implements ``update()`` and
@@ -166,8 +163,8 @@ the collection and prints samples. Pattern (see :code-file:`examples/schemaio/pe
 2. Get required extensions with ``DeviceIOSession::get_required_extensions(trackers)`` and
    create an ``OpenXRSession``.
 3. Create a ``DeviceIOSession`` with ``DeviceIOSession::run(trackers, oxr_session->get_handles())``.
-4. Loop: call ``session->update()``, then read ``tracker->get_data(*session)``. If
-   ``tracked.data`` is non-null, use the latest sample; otherwise sleep briefly and repeat.
+4. Loop: call ``session->update()``, then read ``tracker->get_data(*session)``. If the
+   returned handle is non-empty, use the latest sample; otherwise sleep briefly and repeat.
 
 Use the same ``collection_id`` (and optionally ``tensor_identifier``) as the plugin. See
 :ref:`Schema IO example: build and run <schema-io-example>` above for building and running
@@ -219,7 +216,7 @@ Both exit after 100 samples, or press Ctrl+C to exit early.
   compose a ``SchemaTracker`` and implement ``ITrackerImpl::update()`` / ``serialize_all()``.
 - **Generic3AxisPedalTracker** (tracker facade in ``deviceio_trackers``) — Concrete ``ITracker`` for
   ``Generic3AxisPedalOutput``: holds configuration and
-  ``get_data(session)`` returning ``Generic3AxisPedalOutputTrackedT`` via the session’s
+  ``get_data(session)`` returning ``Serialized<Generic3AxisPedalOutput>`` via the session’s
   ``IGeneric3AxisPedalTrackerImpl``.
 - **DeviceIOSession** — Session manager: collects required OpenXR extensions from registered
   trackers, creates tracker implementations with session handles, and calls ``update()`` on all

examples/lerobot/record.py

@@ -120,41 +120,37 @@ def main():
                         session.update()
 
                         # Get hand data
-                        left_tracked: schema.HandPoseTrackedT = (
-                            hand_tracker.get_left_hand(session)
-                        )
-                        right_tracked: schema.HandPoseTrackedT = (
-                            hand_tracker.get_right_hand(session)
+                        left_tracked: schema.HandPose = hand_tracker.get_left_hand(
+                            session
                         )
-                        head_tracked: schema.HeadPoseTrackedT = head_tracker.get_head(
+                        right_tracked: schema.HandPose = hand_tracker.get_right_hand(
                             session
                         )
+                        head_tracked: schema.HeadPose = head_tracker.get_head(session)
 
                         # Extract positions and orientations (with defaults for invalid data)
                         left_pos = np.zeros(3, dtype=np.float32)
                         right_pos = np.zeros(3, dtype=np.float32)
 
-                        if left_tracked.data is not None and left_tracked.data.joints:
-                            wrist = left_tracked.data.joints.poses(deviceio.JOINT_WRIST)
+                        if left_tracked and left_tracked.joints:
+                            wrist = left_tracked.joints.poses(deviceio.JOINT_WRIST)
                             if wrist.is_valid:
                                 pos = wrist.pose.position
                                 left_pos = np.array(
                                     [pos.x, pos.y, pos.z], dtype=np.float32
                                 )
 
-                        if right_tracked.data is not None and right_tracked.data.joints:
-                            wrist = right_tracked.data.joints.poses(
-                                deviceio.JOINT_WRIST
-                            )
+                        if right_tracked and right_tracked.joints:
+                            wrist = right_tracked.joints.poses(deviceio.JOINT_WRIST)
                             if wrist.is_valid:
                                 pos = wrist.pose.position
                                 right_pos = np.array(
                                     [pos.x, pos.y, pos.z], dtype=np.float32
                                 )
 
                         head_pos = np.zeros(3, dtype=np.float32)
-                        if head_tracked.data is not None and head_tracked.data.is_valid:
-                            pos = head_tracked.data.pose.position
+                        if head_tracked and head_tracked.is_valid:
+                            pos = head_tracked.pose.position
                             head_pos = np.array([pos.x, pos.y, pos.z], dtype=np.float32)
 
                         # STEP 3: Record frame to dataset

examples/mcap_record_replay/cpp/record_full_body.cpp

@@ -26,6 +26,7 @@
 #include <deviceio_trackers/full_body_tracker.hpp>
 #include <oxr/oxr_session.hpp>
 #include <schema/full_body_generated.h>
+#include <schema/serialized.hpp>
 
 #include <chrono>
 #include <cstdint>
@@ -70,12 +71,12 @@ std::string resolve_output_path(const std::string& arg)
     return arg;
 }
 
-uint32_t count_valid_joints(const core::FullBodyPoseT& data)
+uint32_t count_valid_joints(const core::FullBodyPose& data)
 {
     uint32_t valid_count = 0;
     for (uint32_t i = 0; i < core::FullBodyTracker::JOINT_COUNT; ++i)
     {
-        if ((*data.joints->joints())[i]->is_valid())
+        if ((*data.joints()->joints())[i]->is_valid())
         {
             ++valid_count;
         }
@@ -123,12 +124,11 @@ try
 
         if (frame_count % 60 == 0)
         {
-            const auto& tracked = tracker->get_body_pose(*session);
+            const auto* body = tracker->get_body_pose(*session).get();
             std::cout << "[record] t=" << std::fixed << std::setprecision(2) << elapsed_s << "s  frame=" << frame_count;
-            if (tracked.data)
+            if (body != nullptr)
             {
-                std::cout << "  joints=" << count_valid_joints(*tracked.data) << "/"
-                          << core::FullBodyTracker::JOINT_COUNT;
+                std::cout << "  joints=" << count_valid_joints(*body) << "/" << core::FullBodyTracker::JOINT_COUNT;
             }
             else
             {

examples/oglo_tactile/oglo_teleop_record.py

@@ -140,9 +140,9 @@ def _make_overlay_layer(
 
 
 def _taxels(tracked) -> np.ndarray | None:
-    if tracked is None or tracked.data is None:
+    if not tracked:
         return None
-    t = tracked.data.taxels
+    t = tracked.taxels
     if not t or len(t) < NUM_TAXELS:
         return None
     return np.asarray(t, dtype=np.float32)[:NUM_TAXELS]

examples/oxr/cpp/oxr_session_sharing.cpp

@@ -5,6 +5,7 @@
 #include <deviceio_trackers/hand_tracker.hpp>
 #include <deviceio_trackers/head_tracker.hpp>
 #include <oxr/oxr_session.hpp>
+#include <schema/serialized.hpp>
 
 #include <chrono>
 #include <iostream>
@@ -85,12 +86,13 @@ try
 
         if (i % 3 == 0)
         {
+            const bool head_valid = head_tracked && head_tracked->is_valid();
             std::cout << "Frame " << i << ": "
-                      << "Hands=" << (left_tracked.data ? "ACTIVE" : "INACTIVE") << " | "
-                      << "Head=" << ((head_tracked.data && head_tracked.data->is_valid) ? "VALID" : "INVALID");
-            if (head_tracked.data && head_tracked.data->is_valid && head_tracked.data->pose)
+                      << "Hands=" << (left_tracked ? "ACTIVE" : "INACTIVE") << " | "
+                      << "Head=" << (head_valid ? "VALID" : "INVALID");
+            if (head_valid && head_tracked->pose() != nullptr)
             {
-                const auto& pos = head_tracked.data->pose->position();
+                const auto& pos = head_tracked->pose()->position();
                 std::cout << " [" << pos.x() << ", " << pos.y() << ", " << pos.z() << "]";
             }
             std::cout << std::endl;

examples/oxr/cpp/oxr_simple_api_demo.cpp

@@ -1,10 +1,11 @@
-// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 // SPDX-License-Identifier: Apache-2.0
 
 #include <deviceio_session/deviceio_session.hpp>
 #include <deviceio_trackers/hand_tracker.hpp>
 #include <deviceio_trackers/head_tracker.hpp>
 #include <oxr/oxr_session.hpp>
+#include <schema/serialized.hpp>
 
 #include <iostream>
 #include <memory>
@@ -84,14 +85,13 @@ try
         const auto& head_tracked = head_tracker->get_head(*session);
 
         std::cout << "Frame " << i << ":" << std::endl;
-        std::cout << "  Left hand:  " << (left_tracked.data ? "ACTIVE" : "INACTIVE") << std::endl;
-        std::cout << "  Right hand: " << (right_tracked.data ? "ACTIVE" : "INACTIVE") << std::endl;
-        std::cout << "  Head pose:  " << ((head_tracked.data && head_tracked.data->is_valid) ? "VALID" : "INVALID")
-                  << std::endl;
+        std::cout << "  Left hand:  " << (left_tracked ? "ACTIVE" : "INACTIVE") << std::endl;
+        std::cout << "  Right hand: " << (right_tracked ? "ACTIVE" : "INACTIVE") << std::endl;
+        std::cout << "  Head pose:  " << ((head_tracked && head_tracked->is_valid()) ? "VALID" : "INVALID") << std::endl;
 
-        if (head_tracked.data && head_tracked.data->is_valid)
+        if (head_tracked && head_tracked->is_valid())
         {
-            const auto& pos = head_tracked.data->pose->position();
+            const auto& pos = head_tracked->pose()->position();
             std::cout << "    Position: [" << pos.x() << ", " << pos.y() << ", " << pos.z() << "]" << std::endl;
         }
         std::cout << std::endl;

examples/oxr/python/modular_example.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 
 """
@@ -75,15 +75,15 @@ def main():
                         print(f"[{elapsed:4.1f}s] Frame {frame_count}")
 
                         # Get hand data
-                        left_tracked: schema.HandPoseTrackedT = (
-                            hand_tracker.get_left_hand(session)
+                        left_tracked: schema.HandPose = hand_tracker.get_left_hand(
+                            session
                         )
-                        right_tracked: schema.HandPoseTrackedT = (
-                            hand_tracker.get_right_hand(session)
+                        right_tracked: schema.HandPose = hand_tracker.get_right_hand(
+                            session
                         )
 
-                        if left_tracked.data is not None:
-                            pos = left_tracked.data.joints.poses(
+                        if left_tracked:
+                            pos = left_tracked.joints.poses(
                                 deviceio.JOINT_WRIST
                             ).pose.position
                             print(
@@ -92,8 +92,8 @@ def main():
                         else:
                             print("  Left hand:   inactive")
 
-                        if right_tracked.data is not None:
-                            pos = right_tracked.data.joints.poses(
+                        if right_tracked:
+                            pos = right_tracked.joints.poses(
                                 deviceio.JOINT_WRIST
                             ).pose.position
                             print(
@@ -103,11 +103,9 @@ def main():
                             print("  Right hand:  inactive")
 
                         # Get head data
-                        head_tracked: schema.HeadPoseTrackedT = head_tracker.get_head(
-                            session
-                        )
-                        if head_tracked.data is not None:
-                            pos = head_tracked.data.pose.position
+                        head_tracked: schema.HeadPose = head_tracker.get_head(session)
+                        if head_tracked:
+                            pos = head_tracked.pose.position
                             print(
                                 f"  Head pos:    [{pos.x:6.3f}, {pos.y:6.3f}, {pos.z:6.3f}]"
                             )

examples/oxr/python/test_controller_tracker.py

@@ -111,8 +111,8 @@ def assert_trackers_consistent(label, ta, tb):
 
                 print(f"  [{elapsed:5.2f}s] Frame {frame_count:4d}")
 
-                left_data = left_tracked.data
-                if left_data is not None:
+                left_data = left_tracked
+                if left_data:
                     li = left_data.inputs
                     print(
                         f"    L: Trig={li.trigger_value:.2f} Sq={li.squeeze_value:.2f}"
@@ -122,8 +122,8 @@ def assert_trackers_consistent(label, ta, tb):
                 else:
                     print("    L: INACTIVE")
 
-                right_data = right_tracked.data
-                if right_data is not None:
+                right_data = right_tracked
+                if right_data:
                     ri = right_data.inputs
                     print(
                         f"    R: Trig={ri.trigger_value:.2f} Sq={ri.squeeze_value:.2f}"
@@ -146,12 +146,12 @@ def assert_trackers_consistent(label, ta, tb):
 
         def print_controller_summary(hand_name, tracked):
             print(f"  {hand_name} Controller:")
-            if tracked.data is not None:
-                pos = tracked.data.grip_pose.pose.position
+            if tracked:
+                pos = tracked.grip_pose.pose.position
                 print(f"    Grip position: [{pos.x:+.3f}, {pos.y:+.3f}, {pos.z:+.3f}]")
-                pos = tracked.data.aim_pose.pose.position
+                pos = tracked.aim_pose.pose.position
                 print(f"    Aim position:  [{pos.x:+.3f}, {pos.y:+.3f}, {pos.z:+.3f}]")
-                inputs = tracked.data.inputs
+                inputs = tracked.inputs
                 print(f"    Trigger: {inputs.trigger_value:.2f}")
                 print(f"    Squeeze: {inputs.squeeze_value:.2f}")
                 print(

examples/oxr/python/test_extensions.py

@@ -1,5 +1,5 @@
 #!/usr/bin/env python3
-# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: Apache-2.0
 
 """
@@ -91,13 +91,13 @@
         left_tracked = hand.get_left_hand(session)
         head_tracked = head.get_head(session)
         print("  ✅ Update successful")
-        if left_tracked.data is not None:
-            pos = left_tracked.data.joints.poses(deviceio.JOINT_WRIST).pose.position
+        if left_tracked:
+            pos = left_tracked.joints.poses(deviceio.JOINT_WRIST).pose.position
             print(f"    Left wrist: [{pos.x:.3f}, {pos.y:.3f}, {pos.z:.3f}]")
         else:
             print("    Left hand:  inactive")
-        if head_tracked.data is not None:
-            pos = head_tracked.data.pose.position
+        if head_tracked:
+            pos = head_tracked.pose.position
             print(f"    Head pos:   [{pos.x:.3f}, {pos.y:.3f}, {pos.z:.3f}]")
         else:
             print("    Head:       inactive")

examples/oxr/python/test_full_body_tracker.py

@@ -63,15 +63,13 @@
         # Test 6: Check initial body tracking state
         print("[Test 6] Checking body tracking state...")
         body_tracked = body_tracker.get_body_pose(session)
-        print(
-            f"  Body tracking active: {'YES' if body_tracked.data is not None else 'NO'}"
-        )
+        print(f"  Body tracking active: {'YES' if body_tracked else 'NO'}")
 
-        if body_tracked.data is not None:
+        if body_tracked:
             valid_count = sum(
                 1
                 for i in range(schema.BodyJoint.NUM_JOINTS)
-                if body_tracked.data.joints.joints(i).is_valid
+                if body_tracked.joints.joints(i).is_valid
             )
             print(f"  Valid joints: {valid_count}/{schema.BodyJoint.NUM_JOINTS}")
         print()
@@ -94,11 +92,11 @@
                 elapsed = current_time - start_time
                 body_tracked = body_tracker.get_body_pose(session)
 
-                if body_tracked.data is not None:
-                    pelvis_pos = body_tracked.data.joints.joints(
+                if body_tracked:
+                    pelvis_pos = body_tracked.joints.joints(
                         int(schema.BodyJoint.PELVIS)
                     ).pose.position
-                    head_pos = body_tracked.data.joints.joints(
+                    head_pos = body_tracked.joints.joints(
                         int(schema.BodyJoint.HEAD)
                     ).pose.position
                     print(
@@ -121,15 +119,13 @@
         print("[Test 8] Final body pose state...")
         body_tracked = body_tracker.get_body_pose(session)
 
-        print(
-            f"  Body tracking active: {'YES' if body_tracked.data is not None else 'NO'}"
-        )
+        print(f"  Body tracking active: {'YES' if body_tracked else 'NO'}")
 
-        if body_tracked.data is not None:
+        if body_tracked:
             print()
             print("  Joint positions:")
             for i in range(schema.BodyJoint.NUM_JOINTS):
-                joint = body_tracked.data.joints.joints(i)
+                joint = body_tracked.joints.joints(i)
                 name = schema.BodyJoint(i).name
                 pos = joint.pose.position
                 rot = joint.pose.orientation