Skip to content

[Feature] Add SigLIP loss and forward template - #421

Open
24GUNV wants to merge 1 commit into
galilai-group:mainfrom
24GUNV:add-siglip
Open

[Feature] Add SigLIP loss and forward template#421
24GUNV wants to merge 1 commit into
galilai-group:mainfrom
24GUNV:add-siglip

Conversation

@24GUNV

@24GUNV 24GUNV commented May 10, 2026

Copy link
Copy Markdown

Description

Closes #284

This pull request adds SigLIP support for image-text pretraining as described in Zhai et al., 2023.

I referenced the SigLIP loss implementations from Google Research Big Vision and OpenCLIP.

Changes

  • Added SigLIPLoss class in stable_pretraining/losses/multimodal.py.
  • Implements the pairwise sigmoid image-text objective with learnable logit scale and logit
    bias.
  • Supports distributed feature gathering so DDP training uses cross-rank negatives.

Added forward template

  • Added siglip_forward in stable_pretraining/forward.py.
  • Supports paired image/text batches with vision_model, text_model, and siglip_loss.
  • Accepts model outputs exposing either image_embeds / text_embeds or pooler_output.

Added example

  • Created benchmarks/diffusiondb2m/siglip-vitb.py.
  • Uses Hugging Face SigLIP vision/text towers with the reusable forward template.

Added tests

  • Created stable_pretraining/tests/unit/test_siglip_loss.py.
  • Added siglip_forward coverage in stable_pretraining/tests/unit/test_forward_functions.py.
  • Tests cover SigLIP loss behavior, reference formula, normalization, learnable parameter
    gradients, unpaired batch validation, and forward template outputs.

Docs

  • Added API docs for SigLIPLoss and siglip_forward.
  • Added the SigLIP citation to docs/source/references.bib.

Checklist

  • I have read the Contributing document.
  • The documentation is up-to-date with the changes I made (check build artifacts).
  • All tests passed, and additional code has been covered with new tests.
  • I have added the PR to the RELEASES.rst file.

@24GUNV 24GUNV changed the title [FEATURE] Add SigLIP loss and forward template [Feature] Add SigLIP loss and forward template May 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement SigLIP

1 participant