Use Case: ViT vs CNN for LIGO Gravitational Wave Glitch Classification (No Physics Background)

[JesseRWeigel]

Hi everyone,

I wanted to share a research project I just completed using GPD — with what I think is an unusual starting point: I have no formal physics training. I'm a software engineer with experience in AI/ML and TypeScript/React, with a childhood interest in physics but zero academic background in the field.

GPD made it possible for me to conduct a rigorous, multi-phase physics research project that I could never have structured or verified on my own.

The Project

A controlled comparison of Vision Transformer (ViT-B/16) vs CNN (ResNet-50v2 BiT) for classifying gravitational wave detector glitches in LIGO data, using the Gravity Spy O3 dataset (325,634 glitches across 23 classes).

How GPD Helped

GPD guided the entire workflow across 7 phases and 2 milestones:

Milestone v1.0: Core Research (5 phases)

• Phase 1: Data Pipeline & Experimental Design — GPD's questioning protocol helped me scope the project, lock conventions (Q-transform parameters, normalization, temporal splitting with 60s gap), and avoid common pitfalls like temporal data leakage that would have invalidated all results.

• Phase 2: CNN Baseline Reproduction — Trained ResNet-50v2 with a modern recipe (AdamW, cosine schedule, focal loss). GPD's verification caught that our 91.8% accuracy was lower than Gravity Spy's published ~97% — but correctly identified this as honest methodology (temporal split vs their random split), not a bug.

• Phase 3: ViT Training & Rare-Class Optimization — This is where the core hypothesis was tested. The rethink trigger fired: ViT improved overall macro-F1 (0.723 vs 0.679, p=0.0002) but failed on rare classes (0.241 vs 0.303). GPD's contract system correctly classified this as a claim failure and presented structured pivot options. We re-scoped from "ViT beats CNN" to "architecture preference is class-dependent."

• Phase 4: O4 Validation & CW Sensitivity — GPD's verification framework caught a critical np.interp boundary clamping bug that inflated the ViT's CW advantage from ~1.01× to 3.4×. Without this catch, that wrong number would have made it into the paper.

• Phase 5: Paper & Model Packaging — GPD's extract_numbers.py → paper_numbers.json → LaTeX \newcommand pipeline meant zero manual number transcription. The verifier caught a second bug: positional CSV indexing misreported Power_Line F1 diff as +0.005 instead of +0.507.

Milestone v1.1: Referee Revision (2 phases)

GPD's mock peer review (/gpd:peer-review) identified 4 major and 5 minor issues before submission. Key catches:

• "Class-morphology-dependent" was unsupported (we hadn't measured morphological complexity) → changed to "class-dependent"
• Rare-class framing said "ViT underperforms" despite power analysis showing insufficient evidence → reframed as "insufficient statistical evidence"
• Missing canonical citation (Cuoco et al. 2025, Living Reviews in Relativity)

Key Results

Metric	CNN	ViT	Δ	p-value
Overall macro-F1	0.679	0.723	+0.044	0.0002
Rare-class macro-F1	0.303	0.241	−0.062	0.884
Power_Line F1	0.235	0.742	+0.507	—

• Architecture preference is class-dependent, not sample-size-dependent (Spearman ρ = −0.034 on O4)
• Temporal splitting reveals ~3.6pp accuracy inflation in prior random-split evaluations
• Both models generalize to O4 within 20% degradation threshold
• CW veto efficiency approximately equal at matched operating points

What GPD Got Right

1. Verification caught 2 real bugs that would have made it into the paper
2. The contract system forced honest reporting — when the rare-class hypothesis failed, GPD presented structured pivot options instead of letting me quietly drop the claim
3. Convention locking prevented subtle errors from inconsistent normalization or metric definitions across phases
4. The forbidden proxy system correctly identified that overall accuracy improvement masked rare-class regression
5. Mock peer review caught framing issues that a real reviewer would have flagged

What Could Be Improved

• Filed Issue #24: write-paper generates conflicting \bibliographystyle with inline \thebibliography, causing natbib compilation errors
• The paper verification step skips LaTeX compilation if no TeX toolchain is detected — would be great if it attempted compilation as an automated check when pdflatex is available
• Phase execution sometimes stalls between waves, requiring manual "continue" prompts

Links

• Code: github.com/JesseRWeigel/ligo-glitch-vit-cnn
• Models: huggingface.co/JesseWeigel/ligo-glitch-vit-cnn
• Paper: Manuscript targeting Classical and Quantum Gravity (pending arXiv submission)

Looking for an arXiv Endorser

Since this is my first physics paper, I need an endorser for arXiv gr-qc. If anyone in the GW community would be willing to review the paper and endorse, I'd be very grateful. The full PDF, code, and trained models are all publicly available at the links above.

Thanks to the GPD team for building a tool that made this possible for someone outside the physics community. Happy to answer questions about the workflow or results.

[SergioHC95]

@JesseRWeigel Thanks so much for sharing your project and your experience using GPD. It is very rewarding for us to see GPD being used to get physics done!

We will definitely look into your suggested improvements, and we would greatly appreciate any other feedback you’d be willing to share.

[JesseRWeigel]

Thank you! I hope to be able to put my GPU to good use with GPD to contribute meaningfully to physics and other areas of science.