Skip to content

Restructure edit section in GUI guide #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
anna-teruel merged 1 commit into from
Oct 9, 2026
Merged

Restructure edit section in GUI guide #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
232 changes: 102 additions & 130 deletions docs/source/user_guide/gui.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,8 @@ Dragging and dropping the image file onto the `napari` window
(or opening it via the `File` menu) will load the image
as a single 2D frame without a slider.

## Load the tracked dataset
(target-load-tracked-data)=
## Load tracked data

Now you are ready to load some motion tracks over your chosen background layer.

Expand Down Expand Up @@ -203,7 +204,7 @@ And for a bounding boxes dataset, you will see a view more like the one below:

Note the additional bounding boxes layer that is loaded for bounding boxes datasets. For both poses and bounding boxes datasets, you can toggle the visibility of any of these layers by clicking on the eye icon.


(target-points-layer)=
### The points layer
The points layer shows the data for the current frame.

Expand Down Expand Up @@ -249,7 +250,7 @@ You can find all the [keyboard shortcuts](napari:guides/preferences.html#shortcu

:::


(target-tracks-layer)=
### The tracks layer

The tracks layer allows us to visualise data before and after the current frame.
Expand Down Expand Up @@ -316,180 +317,151 @@ You can find all the [keyboard shortcuts](napari:guides/preferences.html#shortcu
(target-edit-tracked-data)=
## Edit tracked data

Pose estimation models occasionally produce errors that automated
{ref}`filtering <sphx_glr_examples_filter_and_interpolate.py>` cannot fully fix, such as keypoints
jumping to the wrong body part or spurious detections in the background.
The `movement` GUI lets you correct these errors by hand, directly on
the video frames, and save the corrected data as a
[movement dataset](target-poses-and-bboxes-dataset).

Currently, you can:

- **move** keypoints that were placed in the wrong position;
- **remove** keypoints that should not be there (false positives).

:::{note}
Manual editing is currently supported for poses datasets only.
Bounding boxes datasets can be loaded and viewed, but not saved after editing.
Pose estimation models occasionally produce errors such as mis-localised
keypoints and false-positive detections in the background, which automated
post-processing (e.g. {ref}`filtering or interpolation <sphx_glr_examples_filter_and_interpolate.py>`)
cannot fully resolve.
The `movement` GUI lets you correct these prediction errors by interactively
[editing the keypoints](target-edit-types) on the
[points layer](target-points-layer) created when you
[load the tracked dataset](target-load-tracked-data).

Editing involves the following parts of the GUI:

- **The viewer**, where you select and edit keypoints on the points layer.
- **The `Edit tracked data` menu** on the right-hand side of the window,
which holds the [editing options](target-edit-types). Expanding or collapsing it shows or
hides the edit timeline.
- **The [edit timeline](target-edit-timeline)** at the bottom of the
window, which provides a visual summary of all edited frames and lets
you navigate to them.
- **The `Save tracked data` menu**, which lets you
[save your changes](target-save-edits) to a file.

<!-- TODO: (optional) insert screenshot of the GUI, highlighting relevant elements, similar to the ones above, napari_bboxes_layers.png -->

:::{warning}
Editing is currently supported for **poses datasets** only.
While the keypoints in bounding boxes datasets are editable, saving any
changes is not yet supported and will result in an error.
:::

### Move and remove keypoints
### Select keypoints

Editing happens on the [points layer](napari:howtos/layers/points.html)
created when you [load the tracked dataset](#load-the-tracked-dataset).
To start editing:

1. Select the points layer in the layer list.
2. Use the frame slider to go to the frame you want to correct.
3. In the layer controls panel, activate the select points tool
(the arrow icon, or press `S` or `3`).
4. To **move** a keypoint, click on it and drag it to the correct position.
5. To **remove** one or more keypoints, select them (click, `Shift`+click,
or drag a selection box around them) and press `Delete` or `Backspace`,
or click the delete button (the ✕ icon) in the layer controls panel.
3. In the layer controls panel, activate the *Select points* tool
(the arrowhead icon, or press `S` or `3`).

![Moving and removing keypoints in napari](../_static/napari_edit_keypoints.gif)
With the *Select points* tool active, you can select keypoints directly in the
viewer using one of the following methods:

Only the points in the current frame are affected, so you can
move through the video with the frame slider and correct errors
frame by frame.
- Click a keypoint to select it.
- Hold `Shift` while clicking to select multiple keypoints.
- Drag a selection box to select a group of keypoints.

:::{admonition} Editing is only possible when browsing through time
:class: warning
<!-- TODO: insert GIF of the 3 selection methods -->

Points can only be edited while the frame slider controls time,
which is the default view. If you roll or transpose the viewer dimensions,
or switch to 3D display, the edit tools in the points layer
controls panel are greyed out until you return to the default view.
:::{note}
Editing is disabled whenever the frame slider is controlling a non-time
dimension, e.g. after changing the order of the viewer axes or switching to 3D
view. Return to the 2D view with time as the leading axis to re-enable editing.
:::

When you move or remove a keypoint:
(target-edit-types)=
### Edit types

- **The trajectories update automatically.** The
[tracks layer](#the-tracks-layer) is kept in sync with the points layer,
so a moved keypoint's trajectory passes through its new position,
and a removed keypoint disappears from the trajectory.
When you edit a keypoint, the `Edit tracked data` menu expands and the
[edit timeline](target-edit-timeline) appears at the bottom of the window,
showing a bar for the current frame. The [tracks layer](target-tracks-layer)
updates automatically to stay in sync with the points layer, so trajectories
always reflect the current keypoint positions.

![Tracks layer updating after keypoints are moved and removed](../_static/napari_tracks_update.gif)
#### Move keypoints

- **Moved keypoints are shown as rings.** Edited keypoints change from a
filled disc to a hollow ring, so you can tell at a glance which
predictions have been corrected by hand.
To **move** the selected keypoint(s), drag them to the correct position.
Moved keypoints are shown as rings. Their confidence is set to `NaN`
(hover over a point to check), because the model's score no longer applies.

![Edited keypoints shown as rings](../_static/napari_edited_points_rings.gif)
<!-- TODO: insert GIF showing keypoint(s) being dragged, converted to ring(s); hover over to show confidence is NaN; can be entire GUI window to show the timeline changes -->

- **The confidence of moved keypoints is set to `NaN`.** The confidence
score produced by the pose estimation model no longer describes a
position that was set by hand, so it is discarded. You can check this by
hovering over an edited point to show its tooltip.
#### Remove keypoints

### Find edited frames with the timeline
To **remove** the selected keypoint(s), click the ✕ icon in the layer controls
panel, or press `Delete`, `Backspace`, or `1`.
Removed keypoints disappear from the viewer, and their confidence scores are
set to `NaN`.

As soon as you edit a keypoint, the `Edit tracked data` menu on the
right-hand side of the window expands, and an `edited frames` timeline
is docked at the bottom of the window. You can also show or hide
the timeline at any time by expanding or collapsing the
`Edit tracked data` menu.
<!-- TODO: insert GIF showing keypoint(s) being removed; can be entire GUI window to show the timeline changes -->

The timeline spans the whole recording and shows a vertical bar
for every frame containing at least one moved or removed keypoint.
A dashed line marks the frame currently shown in the viewer
and follows the frame slider as you move through the video.
(target-edit-timeline)=
### Edit timeline

The edit timeline spans all frames of the dataset in the currently selected
`movement` points layer, and shows a vertical bar for every frame
containing at least one edited keypoint.
A dashed line marks the current frame displayed in the viewer and
moves with the frame slider as you navigate through the frames.

You can interact with the timeline as follows:

| Action | Effect |
|---|---|
| Click on a bar | Jump to that edited frame |
| Scroll up / down | Zoom in / out around the cursor |
| Click a bar | Go to the corresponding edited frame |
| Scroll up / down | Zoom in / out, centred on the cursor (e.g. to distinguish between bars of adjacent frames) |
| Click and drag | Pan along the timeline (when zoomed in) |
| Double-click | Reset the view to the full recording |

![Edited frames timeline docked at the bottom of the napari window](../_static/napari_edit_timeline.gif)
| Double-click | Reset the view to show all frames |

<!-- TODO: insert GIF showing the 4 timeline interactions; e.g. start with 3 frames already edited (1 that's further away from the other 2 to demo clicking; the other 2 edited frames should be adjacent to demo zooming in, reset -->

Zooming in is useful for long recordings, where edits made in
neighbouring frames would otherwise overlap into a single bar.
![Edited frames timeline docked at the bottom of the napari window](../_static/napari_edit_timeline.gif)

For datasets with multiple individuals, tick the `Display individuals`
checkbox in the `Edit tracked data` menu to split the timeline into
one row per individual. Each row's bars take the colour of that
individual's points, so you can see which animals were corrected and when.
For single-individual datasets, this checkbox is disabled.
checkbox in the `Edit tracked data` menu to split the timeline into one
row per individual. Each row's bars match the colour of that individual's
points, showing which individuals were corrected in which frames.
The checkbox is disabled for single-individual datasets.

<!-- TODO: trim GIF to show only the action of checking the box and the timeline changes -->
![Edited frames timeline split into one row per individual](../_static/napari_edit_timeline_individuals.gif)

(target-save-edits)=
### Save edits

To save your edits to a file:

The timeline always shows the edits of the currently selected
`movement` points layer. If you have loaded several datasets,
select a different points layer in the layer list to see its edits.

### Save the edited data

To save your corrections:

1. Select the edited points layer in the layer list.
2. Expand the `Save tracked data` menu and click `Save`.
3. Choose a destination file. The data is saved in `movement`'s native
[netCDF](target-netcdf) format, and a `.nc` extension is added
to the file name if missing.

The `Save` button is only enabled when a `movement` points layer is selected.
1. Select the `movement` points layer that contains the edits in the layer list.
2. Expand the `Save tracked data` menu and click `Save`
(enabled only when a `movement` points layer is selected).
3. In the dialog that opens, choose a location and enter a file name.
The data is currently saved in `movement`'s native
[netCDF](target-netcdf) format.

The saved file contains a valid
[movement poses dataset](target-poses-and-bboxes-dataset) with:

- the corrected `position` values; removed keypoints are stored as `NaN`;
- the `confidence` values, with moved and removed keypoints set to `NaN`;
- an additional boolean `edited` data variable, with dimensions
`(time, keypoint, individual)`, which is `True` for every keypoint
that was moved or removed.
- the `confidence` values, with moved and removed keypoints stored as `NaN`;
- a boolean `edited` data variable, with dimensions
`(time, keypoint, individual)`, storing `True` for every edited keypoint and `False` for all others.

The `edited` variable is only added if at least one keypoint was edited.
The `edited` variable is included only if at least one keypoint was edited.
Keypoints or individuals that end up with no data in any frame
(e.g. because all their points were removed) are dropped from the saved
dataset. Frames are never dropped: a frame whose points were all removed
is kept, with `NaN` values.

### Resume an editing session

To continue editing later, load the saved file in the GUI as described in
[Load the tracked dataset](#load-the-tracked-dataset),
choosing `movement (netCDF)` as the `source software`.
Your previous edits are restored:

- previously moved keypoints are shown as rings;
- the timeline is populated with all previously edited frames,
including those where keypoints were removed;
- any new corrections are added to the existing ones, and saving again
keeps both.

::: {dropdown} Using edited data in Python
:color: info
:icon: info

Because the edited data is saved as a netCDF file, you can open it in Python
and use the `edited` variable in your analysis, for example
to count how many keypoints were corrected by hand:

```python
import xarray as xr

ds = xr.open_dataset("path/to/my_data_edited.nc")

# Total number of edited keypoints
n_edited = int(ds["edited"].sum())

# Frames containing at least one edited keypoint
edited_frames = ds["time"].where(
ds["edited"].any(dim=["keypoint", "individual"]), drop=True
)
```
To resume editing later, [load](target-load-tracked-data) the saved
`.nc` file in the GUI: all previous edits are restored, so you can pick
up exactly where you left off.

As usual, you can continue processing the corrected dataset with any
`movement` function, such as
{func}`~movement.filtering.interpolate_over_time` to fill in the positions
of removed keypoints.
:::{tip}
Because the edits are saved in the `edited` variable, you can use them in
your analysis, for example to report how many keypoints were corrected by
hand, or to compare results with and without the corrected frames.
See [](target-netcdf) for how to load the file in Python.
:::

(target-define-rois)=
Expand Down
Loading