From 1f5c6a8102fd6e8387e1fd6ebbc81f655d465321 Mon Sep 17 00:00:00 2001 From: lochhh Date: Fri, 9 Oct 2026 12:40:05 +0100 Subject: [PATCH] Restructure edit section in GUI guide --- docs/source/user_guide/gui.md | 232 +++++++++++++++------------------- 1 file changed, 102 insertions(+), 130 deletions(-) diff --git a/docs/source/user_guide/gui.md b/docs/source/user_guide/gui.md index f95325dd15..28cded0553 100644 --- a/docs/source/user_guide/gui.md +++ b/docs/source/user_guide/gui.md @@ -120,7 +120,8 @@ Dragging and dropping the image file onto the `napari` window (or opening it via the `File` menu) will load the image as a single 2D frame without a slider. -## Load the tracked dataset +(target-load-tracked-data)= +## Load tracked data Now you are ready to load some motion tracks over your chosen background layer. @@ -203,7 +204,7 @@ And for a bounding boxes dataset, you will see a view more like the one below: Note the additional bounding boxes layer that is loaded for bounding boxes datasets. For both poses and bounding boxes datasets, you can toggle the visibility of any of these layers by clicking on the eye icon. - +(target-points-layer)= ### The points layer The points layer shows the data for the current frame. @@ -249,7 +250,7 @@ You can find all the [keyboard shortcuts](napari:guides/preferences.html#shortcu ::: - +(target-tracks-layer)= ### The tracks layer The tracks layer allows us to visualise data before and after the current frame. @@ -316,180 +317,151 @@ You can find all the [keyboard shortcuts](napari:guides/preferences.html#shortcu (target-edit-tracked-data)= ## Edit tracked data -Pose estimation models occasionally produce errors that automated -{ref}`filtering ` cannot fully fix, such as keypoints -jumping to the wrong body part or spurious detections in the background. -The `movement` GUI lets you correct these errors by hand, directly on -the video frames, and save the corrected data as a -[movement dataset](target-poses-and-bboxes-dataset). - -Currently, you can: - -- **move** keypoints that were placed in the wrong position; -- **remove** keypoints that should not be there (false positives). - -:::{note} -Manual editing is currently supported for poses datasets only. -Bounding boxes datasets can be loaded and viewed, but not saved after editing. +Pose estimation models occasionally produce errors such as mis-localised +keypoints and false-positive detections in the background, which automated +post-processing (e.g. {ref}`filtering or interpolation `) +cannot fully resolve. +The `movement` GUI lets you correct these prediction errors by interactively +[editing the keypoints](target-edit-types) on the +[points layer](target-points-layer) created when you +[load the tracked dataset](target-load-tracked-data). + +Editing involves the following parts of the GUI: + +- **The viewer**, where you select and edit keypoints on the points layer. +- **The `Edit tracked data` menu** on the right-hand side of the window, + which holds the [editing options](target-edit-types). Expanding or collapsing it shows or + hides the edit timeline. +- **The [edit timeline](target-edit-timeline)** at the bottom of the + window, which provides a visual summary of all edited frames and lets + you navigate to them. +- **The `Save tracked data` menu**, which lets you + [save your changes](target-save-edits) to a file. + + + +:::{warning} +Editing is currently supported for **poses datasets** only. +While the keypoints in bounding boxes datasets are editable, saving any +changes is not yet supported and will result in an error. ::: -### Move and remove keypoints +### Select keypoints -Editing happens on the [points layer](napari:howtos/layers/points.html) -created when you [load the tracked dataset](#load-the-tracked-dataset). +To start editing: 1. Select the points layer in the layer list. 2. Use the frame slider to go to the frame you want to correct. -3. In the layer controls panel, activate the select points tool - (the arrow icon, or press `S` or `3`). -4. To **move** a keypoint, click on it and drag it to the correct position. -5. To **remove** one or more keypoints, select them (click, `Shift`+click, - or drag a selection box around them) and press `Delete` or `Backspace`, - or click the delete button (the ✕ icon) in the layer controls panel. +3. In the layer controls panel, activate the *Select points* tool + (the arrowhead icon, or press `S` or `3`). -![Moving and removing keypoints in napari](../_static/napari_edit_keypoints.gif) +With the *Select points* tool active, you can select keypoints directly in the +viewer using one of the following methods: -Only the points in the current frame are affected, so you can -move through the video with the frame slider and correct errors -frame by frame. +- Click a keypoint to select it. +- Hold `Shift` while clicking to select multiple keypoints. +- Drag a selection box to select a group of keypoints. -:::{admonition} Editing is only possible when browsing through time -:class: warning + -Points can only be edited while the frame slider controls time, -which is the default view. If you roll or transpose the viewer dimensions, -or switch to 3D display, the edit tools in the points layer -controls panel are greyed out until you return to the default view. +:::{note} +Editing is disabled whenever the frame slider is controlling a non-time +dimension, e.g. after changing the order of the viewer axes or switching to 3D +view. Return to the 2D view with time as the leading axis to re-enable editing. ::: -When you move or remove a keypoint: +(target-edit-types)= +### Edit types -- **The trajectories update automatically.** The - [tracks layer](#the-tracks-layer) is kept in sync with the points layer, - so a moved keypoint's trajectory passes through its new position, - and a removed keypoint disappears from the trajectory. +When you edit a keypoint, the `Edit tracked data` menu expands and the +[edit timeline](target-edit-timeline) appears at the bottom of the window, +showing a bar for the current frame. The [tracks layer](target-tracks-layer) +updates automatically to stay in sync with the points layer, so trajectories +always reflect the current keypoint positions. - ![Tracks layer updating after keypoints are moved and removed](../_static/napari_tracks_update.gif) +#### Move keypoints -- **Moved keypoints are shown as rings.** Edited keypoints change from a - filled disc to a hollow ring, so you can tell at a glance which - predictions have been corrected by hand. +To **move** the selected keypoint(s), drag them to the correct position. +Moved keypoints are shown as rings. Their confidence is set to `NaN` +(hover over a point to check), because the model's score no longer applies. - ![Edited keypoints shown as rings](../_static/napari_edited_points_rings.gif) + -- **The confidence of moved keypoints is set to `NaN`.** The confidence - score produced by the pose estimation model no longer describes a - position that was set by hand, so it is discarded. You can check this by - hovering over an edited point to show its tooltip. +#### Remove keypoints -### Find edited frames with the timeline +To **remove** the selected keypoint(s), click the ✕ icon in the layer controls +panel, or press `Delete`, `Backspace`, or `1`. +Removed keypoints disappear from the viewer, and their confidence scores are +set to `NaN`. -As soon as you edit a keypoint, the `Edit tracked data` menu on the -right-hand side of the window expands, and an `edited frames` timeline -is docked at the bottom of the window. You can also show or hide -the timeline at any time by expanding or collapsing the -`Edit tracked data` menu. + -The timeline spans the whole recording and shows a vertical bar -for every frame containing at least one moved or removed keypoint. -A dashed line marks the frame currently shown in the viewer -and follows the frame slider as you move through the video. +(target-edit-timeline)= +### Edit timeline + +The edit timeline spans all frames of the dataset in the currently selected +`movement` points layer, and shows a vertical bar for every frame +containing at least one edited keypoint. +A dashed line marks the current frame displayed in the viewer and +moves with the frame slider as you navigate through the frames. You can interact with the timeline as follows: | Action | Effect | |---|---| -| Click on a bar | Jump to that edited frame | -| Scroll up / down | Zoom in / out around the cursor | +| Click a bar | Go to the corresponding edited frame | +| Scroll up / down | Zoom in / out, centred on the cursor (e.g. to distinguish between bars of adjacent frames) | | Click and drag | Pan along the timeline (when zoomed in) | -| Double-click | Reset the view to the full recording | - -![Edited frames timeline docked at the bottom of the napari window](../_static/napari_edit_timeline.gif) +| Double-click | Reset the view to show all frames | + -Zooming in is useful for long recordings, where edits made in -neighbouring frames would otherwise overlap into a single bar. +![Edited frames timeline docked at the bottom of the napari window](../_static/napari_edit_timeline.gif) For datasets with multiple individuals, tick the `Display individuals` -checkbox in the `Edit tracked data` menu to split the timeline into -one row per individual. Each row's bars take the colour of that -individual's points, so you can see which animals were corrected and when. -For single-individual datasets, this checkbox is disabled. +checkbox in the `Edit tracked data` menu to split the timeline into one +row per individual. Each row's bars match the colour of that individual's +points, showing which individuals were corrected in which frames. +The checkbox is disabled for single-individual datasets. + ![Edited frames timeline split into one row per individual](../_static/napari_edit_timeline_individuals.gif) +(target-save-edits)= +### Save edits +To save your edits to a file: -The timeline always shows the edits of the currently selected -`movement` points layer. If you have loaded several datasets, -select a different points layer in the layer list to see its edits. - -### Save the edited data - -To save your corrections: - -1. Select the edited points layer in the layer list. -2. Expand the `Save tracked data` menu and click `Save`. -3. Choose a destination file. The data is saved in `movement`'s native - [netCDF](target-netcdf) format, and a `.nc` extension is added - to the file name if missing. - -The `Save` button is only enabled when a `movement` points layer is selected. +1. Select the `movement` points layer that contains the edits in the layer list. +2. Expand the `Save tracked data` menu and click `Save` + (enabled only when a `movement` points layer is selected). +3. In the dialog that opens, choose a location and enter a file name. + The data is currently saved in `movement`'s native + [netCDF](target-netcdf) format. The saved file contains a valid [movement poses dataset](target-poses-and-bboxes-dataset) with: - the corrected `position` values; removed keypoints are stored as `NaN`; -- the `confidence` values, with moved and removed keypoints set to `NaN`; -- an additional boolean `edited` data variable, with dimensions - `(time, keypoint, individual)`, which is `True` for every keypoint - that was moved or removed. +- the `confidence` values, with moved and removed keypoints stored as `NaN`; +- a boolean `edited` data variable, with dimensions + `(time, keypoint, individual)`, storing `True` for every edited keypoint and `False` for all others. -The `edited` variable is only added if at least one keypoint was edited. +The `edited` variable is included only if at least one keypoint was edited. Keypoints or individuals that end up with no data in any frame (e.g. because all their points were removed) are dropped from the saved dataset. Frames are never dropped: a frame whose points were all removed is kept, with `NaN` values. -### Resume an editing session - -To continue editing later, load the saved file in the GUI as described in -[Load the tracked dataset](#load-the-tracked-dataset), -choosing `movement (netCDF)` as the `source software`. -Your previous edits are restored: - -- previously moved keypoints are shown as rings; -- the timeline is populated with all previously edited frames, - including those where keypoints were removed; -- any new corrections are added to the existing ones, and saving again - keeps both. - -::: {dropdown} Using edited data in Python -:color: info -:icon: info - -Because the edited data is saved as a netCDF file, you can open it in Python -and use the `edited` variable in your analysis, for example -to count how many keypoints were corrected by hand: - -```python -import xarray as xr - -ds = xr.open_dataset("path/to/my_data_edited.nc") - -# Total number of edited keypoints -n_edited = int(ds["edited"].sum()) - -# Frames containing at least one edited keypoint -edited_frames = ds["time"].where( - ds["edited"].any(dim=["keypoint", "individual"]), drop=True -) -``` +To resume editing later, [load](target-load-tracked-data) the saved +`.nc` file in the GUI: all previous edits are restored, so you can pick +up exactly where you left off. -As usual, you can continue processing the corrected dataset with any -`movement` function, such as -{func}`~movement.filtering.interpolate_over_time` to fill in the positions -of removed keypoints. +:::{tip} +Because the edits are saved in the `edited` variable, you can use them in +your analysis, for example to report how many keypoints were corrected by +hand, or to compare results with and without the corrected frames. +See [](target-netcdf) for how to load the file in Python. ::: (target-define-rois)=