DynaCLR models staging: requeue (#478) + rescale/LOT (#424) + temporal straightening (#493) - #486

Open

edyoshikun wants to merge 98 commits into

mainfrom

Commits on Jun 29, 2026

Port scale-aware inference and LOT batch correction to modular-viscy-staging

authored and
edyoshikun
committed
0b529dd
Add visualization script for TripletDataModule scale-aware rescaling

authored and
edyoshikun
committed
5e6407a
fix(viscy-data): handle nested timepoint_statistics in _collate_norm_meta

edyoshikun
and
claude
committed
378a4ac
feat(viscy-data): add Z-reduction and focus-centered z_range to TripletDataModule

edyoshikun
and
claude
committed
19a3301
docs(dynaclr): triplet inference DAG + sample 2D-from-3D predict config

edyoshikun
and
claude
committed
365fc42
docs(dynaclr): add triplet z-projection dataloader inspection script

edyoshikun
and
claude
committed
dc39073
Potential fix for pull request finding

edyoshikun
and
Copilot
authored
a14a0ac
Potential fix for pull request finding

edyoshikun
and
Copilot
authored
52c72a1
Potential fix for pull request finding

edyoshikun
and
Copilot
authored
6d320ae
Potential fix for pull request finding

edyoshikun
and
Copilot
authored
2a1b327
Potential fix for pull request finding

edyoshikun
and
Copilot
authored
dbe22f2
Potential fix for pull request finding

edyoshikun
and
Copilot
authored
4a2de14

Commits on Jun 30, 2026

fix(viscy-data): address triplet z-reduction/rescaling review findings

edyoshikun
and
claude
committed
5b81c2c

Commits on Jul 1, 2026

lightning config for requeing jobs on a preempted slurm partition
edyoshikun
committed
6f37d55

Commits on Jul 6, 2026

CLI to split ann data by a column to group by
edyoshikun
committed
c7f8a16

Commits on Jul 10, 2026

perf(viscy-utils): GPU-accelerate MMD via PyTorch

Soorya19Pradeep
and
claude
committed
b39205f
feat(viscy-utils): add MMD witness function

Soorya19Pradeep
and
claude
committed
f85cc7e

Commits on Jul 14, 2026

feat(dynaclr): add MMD-witness weak-label source for linear classifiers

edyoshikun
and
claude
committed
fa09832
test(dynaclr): cover MMD-witness weak labeling for linear classifiers

edyoshikun
and
claude
committed
c9d67ec
config(dynaclr): add witness-labeled LC recipe for infectomics

edyoshikun
and
claude
committed
c643e95
docs(dynaclr): witness-score LC DAG with flow diagrams and mock plots

edyoshikun
and
claude
committed
bfc1936
fix(dynaclr): evaluate witness LC against ground-truth annotations

edyoshikun
and
claude
committed
361a8a5
config(dynaclr): add eval_against to witness LC recipe

edyoshikun
and
claude
committed
1d8bffa
docs(dynaclr): reflect annotation-based eval in witness DAG + visuals

edyoshikun
and
claude
committed
1af4f03

Commits on Jul 15, 2026

feat(dynaclr): add witness gating diagnostic page to LC summary PDF

edyoshikun
and
claude
committed
faf741d
feat(dynaclr): filter-based witness references + annotation-scored eval

edyoshikun
and
claude
committed
0a17c20
fix(dynaclr): witness ROC/F1 plots use annotation-scored eval, not witness label

edyoshikun
and
claude
committed
b6015a9
feat(dynaclr): per-marker eval target for witness classifiers

edyoshikun
and
claude
committed
401a38a

Commits on Jul 17, 2026

feat(dynaclr): witness-GMM annotations replace the witness label source

edyoshikun
and
claude
committed
b769bc0
chore(dynaclr): untrack mock DAG visuals

edyoshikun
and
claude
committed
d01e749
refactor(mmd): debloat MMD utilities

edyoshikun
and
claude
committed
68f3543
fix(triplet): reject non-positive pixel size in rescaling

edyoshikun
and
claude
committed
a3d7a5f
fix(data): clear error when batch mixes FOV timepoint stats

edyoshikun
and
claude
committed
9b0a114
fix(triplet): pre-resolve timepoint_statistics norm_meta to sample t

edyoshikun
and
claude
committed
4c6aa9f
refactor(triplet): lazy-import viscy_transforms in rescale/z_reduction

edyoshikun
and
claude
committed
4a28b20

Commits on Jul 20, 2026

Apply suggestions from code review

edyoshikun
and
Copilot
authored
f64ed40

Commits on Jul 21, 2026

fix(dynaclr): scope SLURM checkpoints by job id for clean requeue

Eduardo Hirata-Miyasaki
and
claude
committed
c7ef9ac
docs(clusters): add Reef guide and reorganize DynaCLR-2D configs by cluster

Eduardo Hirata-Miyasaki
and
claude
committed
6070f6f
fix(dynaclr): scale patch border margin by per-experiment pixel size

Eduardo Hirata-Miyasaki
and
claude
committed
d40f900
Merge branch 'rescale-lot-correction' into dynaclr_models
Eduardo Hirata-Miyasaki
committed
3b5d4cf

Commits on Jul 22, 2026

feat(dynaclr): per-marker embedding-consistency QC (MMD + correlation matrices)

edyoshikun
and
claude
committed
7fa02de
docs(dynaclr): gate-free witness reference default + rationale

edyoshikun
and
claude
committed
10bea7b
chore(ruff): ignore D and E501 under scripts/

edyoshikun
and
claude
committed
6306aa3
feat(dynaclr): occlusion-attribution report script for classifier eval

edyoshikun
and
claude
committed
06f6655
feat(dynaclr): add clustering PoP eval configs (infection/remodel/velocity)

edyoshikun
and
claude
committed
8549598
feat(dynaclr): add embedding clustering proof-of-principle scripts

edyoshikun
and
claude
committed
5b18944
feat(dynaclr): MMD significance gate before witness-GMM labeling

edyoshikun
and
claude
committed
7cf6674
feat(dynaclr): add canonical per-marker prediction paths
edyoshikun
committed
f86b9e5
feat(dynaclr): evaluate reusable dataset-scoped embeddings
edyoshikun
committed
b28e38a
feat(dynaclr): add train-predict-eval model matrix launcher
edyoshikun
committed
85374c6
config(dynaclr): add organelle-box remodeling matrix
edyoshikun
committed
cd03f60
feat(data): support parquet focus planes and cell-level splits
edyoshikun
committed
1650293
config(dynaclr): add Zuben gut preparation and training recipes
edyoshikun
committed
576c124
docs(dynaclr): document run-once eval-many pipeline
edyoshikun
committed
8192d75
feat(dynaclr): Benjamini-Yekutieli FDR control for the MMD significance gate

edyoshikun
and
claude
committed
111af49
feat(viscy-utils): model-agnostic visualization module (occlusion + PCA-RGB)

edyoshikun
and
claude
committed
87763f9
feat(dynaclr): expose occluder fill in occlusion report CONFIG

edyoshikun
and
claude
committed
1ac6b49

Commits on Jul 23, 2026

Revert "config(dynaclr): add Zuben gut preparation and training recipes"
edyoshikun
committed
32bf4f0
Revert "feat(data): support parquet focus planes and cell-level splits"
edyoshikun
committed
a405e30
qc for the embeddings via pearson and mmd
edyoshikun
committed
6ac217e
plotting for mmd
edyoshikun
committed
1897b8f
markdown cleanup
edyoshikun
committed
214b398
cleanp the markdowns dags
edyoshikun
committed
0396452
feat(dynaclr): witness-GMM label provenance + diagnostic hardening

edyoshikun
and
claude
committed
55b559d

Commits on Jul 24, 2026

remove eval matrix from gitignore
edyoshikun
committed
7fe82c1
feat(dynaclr): time-matched MMD-vs-HPI diagnostic for witness-GMM

edyoshikun
and
claude
committed
7498c50
feat(dynaclr): control-anchored gate for weak-channel witness labels

edyoshikun
and
claude
committed
f6018f9
fix(dynaclr): make LOT MMD comparisons consistent
edyoshikun
committed
a63d374
fix(dynaclr): repeat marker options in batch prediction
edyoshikun
committed
13bb401
fix(dynaclr): forward matrix prediction settings
edyoshikun
committed
c854754

Commits on Jul 27, 2026

fix(viscy-data): resize rescaled patches exactly
edyoshikun
committed
7f3ceca
fix(dynaclr): make LOT output replacement safe
edyoshikun
committed
b9952de
feat(viscy-models): add TemporalStraighteningLoss and Predictor

edyoshikun
and
claude
committed
bfd0854
feat(dynaclr): wire temporal straightening + predictor into training

edyoshikun
and
claude
committed
b0fc7cc
fix(dynaclr): preserve temporal sequence integrity
edyoshikun
committed
061d6e3
fix(dynaclr): keep temporal predictor DDP-safe
edyoshikun
committed
651d693
fix(dynaclr): validate and close HPI bins
edyoshikun
committed
f9426fa
docs(dynaclr): record temporal hardening
edyoshikun
committed
ac1f9e7
refactor(viscy-utils): unify LC train/val split behind one helper

edyoshikun
and
claude
committed
89c17a0
feat(dynaclr): witness-GMM phase-classifier configs + driver

edyoshikun
and
claude
committed
eb970dc
feat(dynaclr): witness-GMM diagnostic plots (AUROC, prob grids, %-remodeled)

edyoshikun
and
claude
committed
5ff2c28

Commits on Jul 28, 2026

fix(data): normalize MIP depth across z sampling
edyoshikun
committed
7b03e1b
feat(dynaclr): nest embedding zarrs under {ckpt}/embeddings/

edyoshikun
and
claude
committed
d89f9ed
feat(dynaclr): predict-triplet foundation models, focus-centered Z, obs enrichment

edyoshikun
and
claude
committed
37e3312
feat(dynaclr): matrix foundation rows + skip-existing progressive collections

edyoshikun
and
claude
committed
1fd4def
feat(dynaclr): organelle_remodeling collection + DynaCLR/MorphEm matrix

edyoshikun
and
claude
committed
1c6455d
docs(dynaclr): un-defer 04_21 in organelle_remodeling collection description

edyoshikun
and
claude
committed
f676a9a
feat(qc): audit_focus_slice — flag suspect focus detections (Z-aware)

edyoshikun
and
claude
committed
655a18f

Commits on Jul 31, 2026

fix(dynaclr): iter_embeddings glob missing embeddings/ segment

edyoshikun
and
claude
committed
3f2a500
feat(dynaclr): embedding-consistency-qc v1/v2 split, Fréchet, HPI pooling, temporal-std

edyoshikun
and
claude
committed
3ec6934
feat(dynaclr): microscope field + mock→uninfected on organelle collections; build-collection skill

edyoshikun
and
claude
committed
7aa2c3d
fix(dynaclr): fixed Pearson-r heatmap scale (vmin=0.8) across markers

edyoshikun
and
claude
committed
e3fd8f7

Commits on Aug 3, 2026

feat(viscy-utils): per-plate HPI-binned control-reference normalization

edyoshikun
and
claude
committed
d3af40e

Commits on Aug 4, 2026

docs(viscy-utils): when to use control-reference normalization

edyoshikun
and
claude
committed
13d0547
feat(dynaclr): optional control-reference normalization in linear classifiers

edyoshikun
and
claude
committed
8ea80b7
Revert "docs(viscy-utils): when to use control-reference normalization"
edyoshikun
committed
79a2634

Commits on Aug 7, 2026

Merge PR #424 into dynaclr_models
Eduardo Hirata-Miyasaki
committed
b64283c
Merge PR #493 into dynaclr_models
Eduardo Hirata-Miyasaki
committed
fa6cffd