Rescale and LOT correction - #424
Open
Soorya19Pradeep wants to merge 89 commits into
Open
Conversation
edyoshikun
force-pushed
the
rescale-lot-correction
branch
from
June 29, 2026 17:59
a93d071 to
37b81ff
Compare
…staging Scale-aware patch extraction (packages/viscy-data): - Add _read_pixel_size() helper to read X pixel size from OME-Zarr metadata - Add reference_pixel_size parameter to TripletDataModule: when set, computes initial_yx_patch_size from the pixel-size ratio so the same physical area is covered at inference time - Replace BatchedRescaleYXd (removed per review) with existing BatchedZoomd using scale_factor=(1.0, final_y/initial_y, final_x/initial_x) and mode="bilinear" with antialias; import is lazy to avoid a hard dep LOT batch correction (applications/dynaclr): - Add dynaclr.evaluation.lot_correction submodule with core logic (fit, apply, save, load), Pydantic configs, and Click CLI entry points - Register fit-lot-correction and apply-lot-correction in cli.py via LazyCommand - Add pot and joblib to optional-dependencies.eval in pyproject.toml Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Plots n cell patches side-by-side: - Left: raw patch at initial_yx_patch_size (larger physical area) - Right: same patch bilinearly downscaled to final_yx_patch_size (model input) Both columns share the same percentile contrast window so differences are spatial. Physical scale (µm × µm) is shown in each panel title. Usage: python visualize_triplet_rescaling.py --data-path /path/to/data.zarr --tracks-path /path/to/tracks --source-channel Phase3D --z-range 0 5 --final-yx-patch-size 224 224 --reference-pixel-size 0.325 --output rescaling_comparison.png Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…meta
_collate_norm_meta iterated every normalization level and called
torch.stack on each stat, but timepoint_statistics is nested
{timepoint: {stat: tensor}} rather than flat {stat: tensor}. Any zarr
carrying timepoint_statistics (alongside fov/dataset stats) crashed
batch collation in TripletDataModule with "expected Tensor as element 0
... but got dict". Stack within each timepoint sub-dict instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…etDataModule
Lets a 3D OME-Zarr feed a 2D model without materializing a separate MIP
dataset, and centers the extracted Z window on each FOV's focus plane.
- z_reduction ("mip"/"center"): collapse the extracted z_range to one
slice via BatchedChannelWiseZReductiond. Label-free channels (resolved
by parse_channel_name) take the center slice; others are max-projected.
on_after_batch_transfer expects Z=1 when reduction is on.
- z_extraction_window/z_focus_offset/focus_channel: resolve a per-FOV
focus-centered window from each position's
focus_slice[ch].fov_statistics.z_focus_mean (fallback z_total//2), all
windows the same width. z_range stays as an explicit override; exactly
one of z_range / z_extraction_window must be given. Per-FOV windows are
resolved at setup() and looked up per patch in the dataset.
Tests cover both reduction strategies (discriminating center vs MIP),
the normalize-then-reduce order, per-FOV focus resolution, and the
z_range/z_extraction_window XOR guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Documents the TripletDataModule predict path (zarr + tracking, not parquet) and adds a runnable sample config demonstrating z_reduction + reference_pixel_size to feed a 3D dataset to a 2D model. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Notebook-style script (no CLI) that loads a 3D OME-Zarr + tracking, extracts a per-FOV focus-centered Z window, collapses it via z_reduction, applies a random affine so anchor/positive diverge, and visualizes a couple of batches to a PNG. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
edyoshikun
force-pushed
the
rescale-lot-correction
branch
from
June 29, 2026 20:35
901d5b7 to
dc39073
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR expands the Triplet-based data loading path to better support cross-dataset inference (pixel-size aware patch extraction + on-the-fly Z reduction + focus-centered per-FOV Z windows), and adds a LOT (Linear Optimal Transport) batch-correction pipeline with CLI entrypoints for embedding-space correction.
Changes:
- Extend
TripletDataModule/TripletDatasetto support per-FOVz_rangeresolution viaz_extraction_window+focus_slicemetadata, plus optionalz_reductionand pixel-size aware rescaling. - Add LOT correction core functions + Pydantic config models +
dynaclrCLI commands for fitting/applying correction pipelines. - Add tests for the new Triplet Z-window/Z-reduction behavior and update docs/config examples for inference.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/viscy-data/src/viscy_data/triplet.py | Adds pixel-size rescaling, per-FOV focus-centered Z windows, and optional Z-reduction in the Triplet datamodule/dataset. |
| packages/viscy-data/tests/test_triplet.py | Adds tests for z_extraction_window XOR validation, per-FOV focus-centered windows, and Z-reduction behavior/order. |
| packages/viscy-data/src/viscy_data/_utils.py | Extends norm-meta collation to correctly stack nested timepoint_statistics. |
| applications/dynaclr/src/dynaclr/evaluation/lot_correction/lot_correction.py | Implements LOT correction fit/apply + pipeline save/load helpers. |
| applications/dynaclr/src/dynaclr/evaluation/lot_correction/config.py | Adds Pydantic models for fit/apply configs and filter specs. |
| applications/dynaclr/src/dynaclr/evaluation/lot_correction/fit_lot_correction.py | CLI wrapper for fitting and saving a LOT pipeline from YAML config. |
| applications/dynaclr/src/dynaclr/evaluation/lot_correction/apply_lot_correction.py | CLI wrapper for applying a saved LOT pipeline to an embedding zarr. |
| applications/dynaclr/src/dynaclr/evaluation/lot_correction/init.py | Exposes LOT correction functions at the package level. |
| applications/dynaclr/src/dynaclr/cli.py | Registers new fit-lot-correction and apply-lot-correction CLI subcommands. |
| applications/dynaclr/pyproject.toml | Adds joblib + pot to eval extras for LOT correction dependencies. |
| applications/dynaclr/scripts/dataloader_inspection/visualize_triplet_rescaling.py | Adds a visualization utility for the new pixel-size rescaling behavior. |
| applications/dynaclr/scripts/dataloader_inspection/triplet_dataloader_zprojection.py | Adds an inspection notebook/script for focus-centered Z windows + Z-reduction. |
| applications/dynaclr/docs/DAGs/inference_triplet.md | Documents Triplet inference path and new options (needs a small wording update for z-window semantics). |
| applications/dynaclr/configs/prediction/predict_triplet_2d_from_3d.yml | Provides a sample prediction config demonstrating reference_pixel_size + z_reduction. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
- Declare viscy-transforms in viscy-data's triplet extra and hoist the BatchedZoomd/BatchedChannelWiseZReductiond imports to module top; the inline imports were masking a missing runtime dependency that would ImportError for anyone using z_reduction or reference_pixel_size. - Raise in _resolve_per_fov_z_ranges when a FOV's Z is smaller than z_extraction_window, instead of silently emitting a narrower window that breaks cross-FOV batch stacking. - Remove a dead duplicate break in test_focus_centered_z_range. - Correct the inference DAG doc: embeddings live in .X (the embedding_key array), mirrored to obsm["X_backbone"]/["X_projections"]. uv.lock intentionally omitted: the workspace glob currently entangles the untracked applications/eet package, so a clean regen of the single viscy-transforms edge is not possible until eet is committed or removed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the NumPy/scipy.cdist backend with PyTorch so the pooled RBF kernel matrix is built once on the available device (CUDA if usable, else CPU) and reused across all permutations. Device selection probes a trivial kernel launch and falls back to CPU, so a present-but-unusable GPU (e.g. compute capability older than the torch build) does not crash. Public API is unchanged: median_heuristic/compute_mmd_unbiased return the same bandwidth-convention values, and mmd_permutation_test still returns (mmd2, p_value, null_distribution) so dynaclr's effect-size and activity z-score computations keep working. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add witness_function to the MMD module: the per-point empirical witness w(z) = mean_i k(z, x_i) - mean_j k(z, y_j), the RKHS direction along which distributions P (X) and Q (Y) differ. Positive scores lean toward X, negative toward Y. Reuses the GPU RBF-kernel helpers and bandwidth (median-heuristic) convention already in the module, and chunks over query points to bound the intermediate kernel matrices. Tests cover distribution separation, X<->Y antisymmetry, and chunk-size invariance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds `label_source: witness` to the linear-classifier eval as a second option to annotation-CSV labels. Per marker, cells are pooled across experiments, control/perturbed references are built from per-experiment wells, the MMD witness is fit and scored per cell, and scores are gated (sign + dead-zone) into control/perturbed pseudo-labels. Everything downstream (train_linear_classifier, publish, append-predictions, plots) is unchanged — only the label source differs. - evaluate_config.py: WitnessLabelSource, WitnessSettings, label_source switch on LinearClassifiersStepConfig (annotations stays the default). - witness_labels.py: well->reference mask (path-component match so C/1 != C/10), witness fit/score, dead-zone gating. - orchestrated.py: branch label assembly into _annotation_run_specs / _witness_run_specs / _build_labeled_adata; training loop unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Five integration tests exercising the real code path: - well-prefix match rejects C/1 vs C/10 confusion - witness scores separate control/perturbed clusters - dead-zone drops ambiguous near-origin cells - empty AnnData when a reference group has no cells - end-to-end run_linear_classifiers witness mode → metrics + summary PDF Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirrors linear_classifiers_infectomics.yml but with label_source: witness. Per-experiment control/perturbed wells replace annotation CSVs; wells are templates to edit per plate layout. One classifier per marker (G3BP1, SEC61B, Phase3D, viral_sensor), dead_zone 0.1, track-level split. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- witness_score_classifiers.md: full DAG (why, step-by-step, gating rule, config, notes) plus embedded visuals. - visuals/: three Graphviz flow diagrams (data flow, gating, pipeline) as .dot + png + pdf, and four mock example-output plots (witness-score histogram, per-marker metrics bar, ROC, F1-over-time) with a reproducible generator (mock_witness_plots.py). Mock plots are watermarked synthetic. - Cross-links from evaluation.md and linear_classifiers/README.md so the new DAG is discoverable from the annotation path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The witness label is a deterministic function of the embedding, so
evaluating a witness-trained classifier against that same label is
circular — it reports a trivial ~1.0 (verified on real infectomics
embeddings: 1.000 vs witness label, 0.712 vs true infection_state).
WitnessSettings gains eval_against (default "infection_state") and
eval_class_map ({control: uninfected, perturbed: infected}). In witness
mode, val metrics are recomputed on the val split against the ground-truth
obs column via the class map; the summary records eval_source. When the
column is absent, metrics fall back to the witness label flagged as
eval_source="witness_label". Adds two tests (annotation eval + fallback).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Score the witness-trained classifier against infection_state (not the self-referential witness label) with the control→uninfected / perturbed→infected class map, matching the leakage fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- witness_score_classifiers.md: add "Evaluation: avoid the circularity trap" section with the real per-marker metrics table (scored vs infection_state), eval_against/eval_class_map in the config block, and step 7 in the flow; refresh the Notes caveat (metrics are honest now, check eval_source; note class imbalance). - witness_dataflow.dot/png/pdf: add the EVALUATE-vs-annotations node. - mock_witness_plots.py + bar/ROC png/pdf: use real run values (viral_sensor 0.87, SEC61B 0.76, Phase3D 0.58, G3BP1 0.49) instead of invented all-high numbers; watermark clarifies bar/ROC are real, hist/F1 synthetic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Weak channels (TOMM20/Phase3D) have a real but graded control-vs-perturbed shift, not a distinct bimodal split — the perturbed-only GMM gate abstains on them. Add a control-anchored alternative that produces labels with a controlled false-positive rate. - fit_control_anchored_labels (witness_gmm.py): model perturbed scores as pi*N(control) + (1-pi)*N(remodel) with the baseline Gaussian FROZEN to the (time-matched) control distribution; fit only the remodel component + mixing weight via constrained EM. "Remodel" = excess over baseline, so a subtle proportion shift still yields labels. Posterior threshold calibrated so the control false-positive rate equals control_fp_target. - config: gate (gmm | control_anchored) + control_fp_target (default 0.05) - label_marker branches on gate; sidecar records gate/remodel_fraction/control_fp/ posterior_threshold for control_anchored, GMM summary otherwise - control-FP diagnostic line uses the calibrated threshold; plot_witness_gmm is GMM-only Validated: TOMM20 (time-matched witness + control_anchored, 5% target) yields 6617 remodel / 9123 noremodel with realized control FP 0.050 (vs ~22% under the pooled GMM gate). Tests 6/6. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extract the GroupShuffleSplit-vs-train_test_split decision and the split_groups_by group-id construction into group_val_split() and group_ids_from_obs() in linear_classifier.py. train_linear_classifier and orchestrated.py's val-index replay now call them, removing three copies of the same split logic. Pure extract-method: same (seed, groups, y) yields identical indices, so training and val-metric behavior are unchanged (verified: Phase3D val AUROC 0.915, SEC61B 1.000, and a bit-for-bit idx_val match to the prior inline split). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
End-to-end two-stage recipe on 2026_07_01_A549_SEC61B_TOMM20_G3BP1_ZIKV: Stage A (witness-gmm-labels on SEC61B organelle) then Stage B (run-linear-classifiers). Two Stage-B configs share the same labels and params, differing only by input channel: 02_stage_b_train_phase (teacher/student, trains on Phase3D) and 02_stage_b_train_sec61b (same-modality). run_*.sh drives both stages (CPU-only, bash or sbatch). Reproduces Soorya's per-timepoint AUROC band. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…odeled) plot_timecourse_and_prob_grids.py produces three diagnostics for the ZIKV plate: D1 per-timepoint AUROC overlay (SEC61B witness vs Phase3D classifier), D2 P(remodel)-split sample grids (HPI bins x 5 probability buckets) per marker, and D3 %-remodeled-vs-time (witness-GMM gate + SEC61B/Phase3D logistic classifiers, ZIKV vs control false-positive). Reuses the shared LC split helpers and the real Stage-A compute_marker_scores path so the plotted scores match Stage A/B exactly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Raw per-marker embedding zarrs now write to
{dataset}/2-phenotyping/predictions/{family}/{run}/{ckpt}/embeddings/{marker}.zarr
so the {ckpt} dir is a container for downstream analysis (labels, plots, umap,
...) as sibling folders. embedding_store + eval_launch glob both updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…bs enrichment
Expand the per-reporter triplet predict path (dynaclr predict-triplet):
- foundation model_type: instantiate the model: block of a LightningCLI
training_config (frozen encoder loads its own weights) instead of loading a
ContrastiveModule checkpoint; --checkpoint / --training-config gate by type.
- focus-centered Z: --z-window WIDTH centered per-FOV on the focus plane
(--focus-channel / --z-focus-offset), mutually exclusive with fixed --z-range;
--reference-pixel-size-z-um converts the reference-grid slice count to the
native count covering the same physical depth.
- obs enrichment: append collection metadata (experiment/marker/perturbation/
organelle/microscope/hours_post_perturbation/interval_minutes) to each
embedding's obs so triplet and parquet embeddings are consumed identically;
--no-enrich-obs opts out. New enrich-obs-from-collection subcommand backfills
obs onto already-written embeddings.
- exclude_fovs from the collection experiment is forwarded to predict.
Update the {ckpt}/embeddings/{marker}.zarr path expectation in
test_output_path_is_dataset_centric (embeddings/ nesting from d89f9ed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…lections
run-matrix / predict-batch gain:
- foundation rows (model_type: foundation): resolve from explicit identity +
training_config (no train_sbatch parsing), never emit a train stage, and
forward --model-type/--training-config through predict-batch and predict.sbatch
instead of a checkpoint.
- focus-centered Z + physical Z reference (z_window/focus_channel/z_focus_offset/
reference_pixel_size_z_um) forwarded as the trailing predict positionals;
z_range and z_window validated mutually exclusive.
- skip-existing (default) via resolve_datasets_to_run: prune datasets whose
per-marker embeddings already exist for a row's (family, run, ckpt_name);
drop fully-done rows, write a pruned temp collection for partial ones. A dir
without group metadata (crash mid-write) counts as incomplete and re-runs.
--overwrite bypasses the check; default --stages is predict,eval.
Point _mark_done at the {ckpt}/embeddings/ path so completion detection matches
the nested layout (d89f9ed).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add the merged organelle_remodeling collection (9 experiments: SEC61B/TOMM20/ G3BP1 reporters under ZIKV/DENV, one experiment per dataset, all prepared on VAST with normalization + focus_slice). Includes 2026_03_24_A549_SEC61_ZIKV (re-concatenated + QC'd clean, mantis_v2 at 0.1133/0.16995 um); 04_21 G3BP1 DENV stays deferred. Repoint the matrix configs at the new progressive-collection workflow: - organelle_remodeling.yml matrix: one row per MODEL (DynaCLR-2D-MIP-BoC single marker + MorphEm-frozen foundation baseline) over the shared collection; focus-centered Z (z_window 16, focus_channel Phase3D, reference_pixel_size_z_um 0.174); output root on intracellular_dashboard. - example.yml: document model_type foundation rows, focus-centered vs fixed Z, and skip-existing defaults. - 07_01 collection: add the Phase3D channel. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ription 2026_04_21_A549_G3BP1_DENV was re-concatenated to a flat store on VAST (odd 5-assemble layout resolved) and verified predict-ready. Note the residual z_focus=0 frames (12/54 FOVs, mean 2.4 timepoints each). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add audit_focus_slice() + a `qc audit-focus` CLI command that reads already-
written focus_slice.{channel}.per_timepoint metadata and reports where in-focus
detection landed at a stack edge (z == 0 or z == Z-1), the signature of a failed
search on a 3D stack. Z-depth-aware: a 2D acquisition (Z == 1) trivially focuses
at slice 0 for every timepoint, so it is reported as 2D with zero flags rather
than misclassifying every frame as failed.
Returns per-FOV suspect counts, dataset totals, and the valid (non-edge) focus
distribution — the summary previously computed ad hoc while vetting datasets for
focus-centered predict.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
embedding_store writes .../{ckpt}/embeddings/{marker}.zarr since the embeddings/
nesting landed (d89f9ed), but iter_embeddings still globbed .../{ckpt}/*.zarr —
so it returned [] and ALL cross-dataset pooling (embedding-consistency-qc, MMD,
linear classifiers) silently found zero embeddings. Add the missing EMBEDDINGS_DIR
segment; fix the test_paths fixtures that encoded the pre-nesting layout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ling, temporal-std Extend the per-marker embedding-consistency QC: - split_by: partition datasets by a per-dataset obs column (e.g. microscope) into within-group + cross-group blocks; degenerate blocks (single-dataset within, or empty cross side) are skipped with a log line. - metrics knob: choose any of pearson/mmd/frechet (default all three). MMD pass is skipped when not requested. - Fréchet (mean+covariance) matrix: 2-Wasserstein between Gaussian summaries of each dataset's control cells — richer than Pearson-of-means, cheaper than full MMD. - Time-aware Pearson (pearson_hpi_bin_hours): mean per hours_post_perturbation bin then mean-of-means, so uneven time sampling can't masquerade as a batch effect; plus a per-dataset temporal-std drift diagnostic (std of the control centroid across HPI bins). - Annotated, auto-scaled correlation heatmaps. Organelle-box recipe (metrics: [pearson], split_by: microscope, 2 h HPI bins). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…tions; build-collection skill Add per-experiment microscope (mantis_v1 for the 0.1494 um/px dataset, mantis_v2 for the 0.1133 ones) to organelle_remodeling so obs enrichment writes it and the consistency QC can split v1/v2. Normalize the 2024_11_07 control label mock→uninfected in both organelle_remodeling and the source organelle-box collection so a single uninfected filter catches every dataset's controls. Update the airtable-build-collection skill to always populate microscope (field table, schema, derivation rule). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
plot_corr_matrix auto-scaled vmin to each matrix's own off-diagonal minimum, so
heatmaps for different markers used different colour scales and couldn't be
compared by eye. Use a shared fixed vmin (default 0.8). Fix the title
("mean-embedding" -> "control-cell summary correlation").
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add control_normalization.py: robust z-score of embeddings against the control (uninfected) population of the same plate and HPI bin. Stores only per-(plate, bin) median/IQR statistics and applies (x - median) / IQR on the fly rather than materializing a normalized array. - control_reference_stats: per-experiment median + IQR of control cells, pooled across control wells, in fixed-width HPI bins anchored at 0h. - control_reference_stats_multi: same at several bin widths (default 1h + 2h). - ControlReference.apply: robust z-score against the matched bin, nearest occupied bin as fallback; zero-IQR dimensions floored to 1.0. - Raises if a plate has cells but no controls (no silent drop). 8 unit tests cover recovery of the control center, control/perturbed separation, nearest-bin fallback, zero-IQR flooring, multi-bin widths, and the control-less-plate guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Standalone README next to control_normalization.py documenting the empirical finding so a teammate knows when NOT to use it: it flattens control temporal drift (−32 to −53% control-centroid std across 3 plates) but LOWERS held-out infection/remodeling classifier AUROC on every ZIKV DynaCLR classifier tested (−0.008 to −0.032), because flattening control-relative time removes infection-trajectory signal. Per-dimension only (not multivariate batch correction). Full write-up under .ed_planning/dynaclr/batch_correction/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ssifiers Add a control_normalize flag (+ control_normalize_bin_hours, default 2.0) to LinearClassifiersStepConfig and wire it into run-linear-classifiers: when set, embeddings are robustly z-scored per plate/HPI-bin against control cells before training/scoring, via viscy_utils.evaluation.control_normalization. Off by default and backward-compatible. It is off because the held-out A/B showed it lowers infection/remodeling AUROC (see control_normalization_README); the flag exists to reproduce that result and for cases where matched-time cross-condition comparability is the actual goal. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This reverts commit 13d0547.
edyoshikun
pushed a commit
that referenced
this pull request
Aug 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The following have been implemented in this branch:
TripletDataModulethat rescales the patches to a reference pixel and a uses the Z-MIP projection transform