Implementation of custom bytetrack demo for real time stream analysis [GSoC'2026] - #4420
Open
Vishwa2684 wants to merge 14 commits into
Open
Implementation of custom bytetrack demo for real time stream analysis [GSoC'2026]#4420Vishwa2684 wants to merge 14 commits into
Vishwa2684 wants to merge 14 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new MediaPipe ByteTrack demo that integrates OVMS + the existing real-time stream analysis Python client to run object detection + multi-object tracking on RTSP streams. It also updates the MediaPipe build integration and adjusts streaming timestamp handling in the KFS graph executor.
Changes:
- Adds a new
demos/mediapipe/bytetrackdemo (graph, OVMS config, and setup README). - Extends the real-time stream analysis Python demo CLI to pass FFmpeg output dimensions through to
StreamClient. - Updates MediaPipe/Bazel integration (new ByteTrack calculators dependency; changes MediaPipe git_repository source) and adds stricter timestamp monotonicity checks for streaming.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
WORKSPACE |
Switches the MediaPipe repository source (currently to a fork/branch). |
third_party/mediapipe_calculators/BUILD |
Adds ByteTrack calculators target to the OVMS MediaPipe calculators aggregation library. |
src/kfs_frontend/kfs_graph_executor_impl.cpp |
Changes timestamp deserialization/defaulting and adds stream-level monotonic timestamp enforcement. |
demos/real_time_stream_analysis/python/client.py |
Adds CLI args for FFmpeg output width/height and forwards them to StreamClient. |
demos/mediapipe/bytetrack/README.md |
Introduces demo setup/run instructions for ByteTrack RTSP streaming demo. |
demos/mediapipe/bytetrack/config.json |
Adds OVMS config for the YOLOX model + ByteTrack MediaPipe graph. |
demos/mediapipe/bytetrack/bytetrack_ovms.pbtxt |
Adds the MediaPipe graph wiring OpenVINO/OVMS inference + ByteTrack tracking + overlay. |
Suppressed comments (1)
src/kfs_frontend/kfs_graph_executor_impl.cpp:1147
- The inline comment "// this method" is not descriptive. Either remove it or replace with a comment that explains what the function does.
// this method
Status createAndPushPacketsImpl(
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+1051
to
1062
| // steady_clock never jumps backward; system_clock can | ||
| auto now = std::chrono::steady_clock::now().time_since_epoch(); | ||
| int64_t candidate = std::chrono::duration_cast<std::chrono::microseconds>(now).count(); | ||
| timestamp = ::mediapipe::Timestamp::CreateNoErrorChecking(candidate); | ||
| } | ||
|
|
||
| // Enforce strict monotonicity regardless of source | ||
| if (lastPushedTimestamp != ::mediapipe::Timestamp::Unset() && timestamp <= lastPushedTimestamp) { | ||
| SPDLOG_DEBUG("Non-monotonic timestamp detected: new={} <= last={}; for request to: {}", | ||
| timestamp.DebugString(), lastPushedTimestamp.DebugString(), request.model_name()); | ||
| return Status(StatusCode::MEDIAPIPE_INVALID_TIMESTAMP, "Timestamp did not increase relative to previous packet in stream"); | ||
| } |
Comment on lines
+1030
to
1031
| // this method | ||
| static Status deserializeTimestampIfAvailable( |
Comment on lines
+1057
to
+1061
| // Enforce strict monotonicity regardless of source | ||
| if (lastPushedTimestamp != ::mediapipe::Timestamp::Unset() && timestamp <= lastPushedTimestamp) { | ||
| SPDLOG_DEBUG("Non-monotonic timestamp detected: new={} <= last={}; for request to: {}", | ||
| timestamp.DebugString(), lastPushedTimestamp.DebugString(), request.model_name()); | ||
| return Status(StatusCode::MEDIAPIPE_INVALID_TIMESTAMP, "Timestamp did not increase relative to previous packet in stream"); |
dtrawins
reviewed
Aug 11, 2026
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Collaborator
|
make sdl-check is failing with error: |
Author
|
@atobiszei @dtrawins I've written code for custom calculator |
Vishwa2684
force-pushed
the
gsoc_bytetrack
branch
from
August 12, 2026 14:51
0a2781e to
fdb2dac
Compare
Vishwa2684
force-pushed
the
gsoc_bytetrack
branch
2 times, most recently
from
August 13, 2026 18:13
4a0b8f8 to
bf1c022
Compare
Vishwa2684
force-pushed
the
gsoc_bytetrack
branch
from
August 13, 2026 18:19
bf1c022 to
7524ee6
Compare
Author
|
@dtrawins Why is the ci failing? Could you share the logs? |
Vishwa2684
force-pushed
the
gsoc_bytetrack
branch
7 times, most recently
from
August 19, 2026 22:07
7d36dd6 to
2b2ba29
Compare
Vishwa2684
force-pushed
the
gsoc_bytetrack
branch
from
August 20, 2026 08:53
af4bf26 to
d2d0703
Compare
Collaborator
|
Earlier on there was CI hiccup with network connectivity. Currently tests are ongoing but we already have documentation test failing: |
Vishwa2684
force-pushed
the
gsoc_bytetrack
branch
from
August 20, 2026 16:36
7652811 to
e0e9d98
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
@atobiszei @dtrawins
Summary
This PR adds a ByteTrack Real-Time Stream Analysis demo for OpenVINO Model Server (OVMS).
The demo reuses the MediaPipe ByteTrack graph from
demos/mediapipe/bytetrackwith theyolox_tinymodel and integrates it with thereal_time_stream_analysisPython client. This enables object detection and multi-object tracking on RTSP streams using OVMS.Key Changes
Implementation of custom bytetrack graph [GSoC'26] mediapipe#131
OVYoloXTensorsToDetectionsCalculatorto process YOLOX output tensors from the YOLOX-Tiny model hosted on Hugging Face.Directory Structure
demos/mediapipe/bytetracksrc/bytetracksrc/yoloxWhat's Included
demos/mediapipe/bytetrack.client.pyexample for running inference with: