Skip to content
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .ai/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(curiosity): why is this needed?

"name": "diffusers",
"description": "Conventions and task skills for contributing to diffusers",
"author": {
"name": "The Hugging Face team"
},
"homepage": "https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents",
"repository": "https://github.com/huggingface/diffusers",
"license": "Apache-2.0",
"keywords": [
"diffusers",
"diffusion-models",
"contributing"
]
}
6 changes: 6 additions & 0 deletions .ai/.codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"name": "diffusers",
"version": "0.1.0",
"description": "Conventions and task skills for contributing to diffusers",
"skills": "./skills/"
}
43 changes: 36 additions & 7 deletions .ai/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,37 @@

## Setup

- Local Claude Code agents: run `make claude` after cloning to wire the [skills](#skills) under `.claude/`.
- Local OpenAI Codex agents: run `make codex` after cloning to wire the [skills](#skills) under `.agents/`.
We recommend developing in a virtual environment managed by [uv](https://docs.astral.sh/uv/):

```bash
uv venv && source .venv/bin/activate
uv pip install -e . # provides diffusers-cli
```

List the available skills, and what each one is for, with:

```bash
diffusers-cli skills list
```

Install them with:

```bash
diffusers-cli skills add <skill name> # install one, or --all for every skill
# --claude / --codex / --cursor to pick one target
```

`diffusers-cli skills update` refreshes what you installed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this handled?


Claude Code and Codex can also install via plugins

```bash
claude plugin marketplace add huggingface/diffusers
claude plugin install diffusers@diffusers-skills --scope project

codex plugin marketplace add huggingface/diffusers # then install from the Plugins Directory
```


## Coding style

Expand All @@ -27,10 +56,10 @@ Strive to write code as simple and explicit as possible.

## Reference guides

- **Models** — see [models.md](models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](./skills/model-integration/SKILL.md) skill.
- **Pipelines** — see [pipelines.md](pipelines.md) for pipeline conventions, patterns, and gotchas.
- **Modular pipelines** — see [modular.md](modular.md) for modular pipeline conventions, patterns, and gotchas.
- **Tests** — see [testing.md](testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.
- **Models** — see [models.md](references/models.md) for model conventions, attention pattern, implementation rules, dependencies, and gotchas. For adding or converting a model, use the [model-integration](./skills/model-integration/SKILL.md) skill.
- **Pipelines** — see [pipelines.md](references/pipelines.md) for pipeline conventions, patterns, and gotchas.
- **Modular pipelines** — see [modular.md](references/modular.md) for modular pipeline conventions, patterns, and gotchas.
- **Tests** — see [testing.md](references/testing.md) for test conventions: required test layers, tester mixins, and dummy-component rules.
Comment on lines +66 to +69

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it solve the discovery problem?


## Skills

Expand All @@ -43,4 +72,4 @@ Task-specific guides live in `.ai/skills/` and are loaded on demand by AI agents

## Self-review before a PR

Before opening a PR, run self-review against [review-rules.md](review-rules.md). The [self-review skill](skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.
Before opening a PR, run self-review against [review-rules.md](references/review-rules.md). The [self-review skill](./skills/self-review/SKILL.md) runs this as the same pass the `@claude` CI reviewer uses. Share the final report on the PR (description or comment) — see the skill for details.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would the agents be able to discover it after the necessary skills are installed?

16 changes: 16 additions & 0 deletions .ai/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
"name": "diffusers",
"description": "Conventions and task skills for contributing to diffusers",
"author": {
"name": "The Hugging Face team"
},
"homepage": "https://huggingface.co/docs/diffusers/main/en/conceptual/contribution#coding-with-ai-agents",
"repository": "https://github.com/huggingface/diffusers",
"license": "Apache-2.0",
"keywords": [
"diffusers",
"diffusion-models",
"contributing"
]
}
1 change: 0 additions & 1 deletion .ai/models.md → .ai/references/models.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Model conventions and rules

Shared reference for model-related conventions, patterns, and gotchas.
Linked from `AGENTS.md`, `skills/model-integration/SKILL.md`, and `review-rules.md`.

## Coding style

Expand Down
File renamed without changes.
1 change: 0 additions & 1 deletion .ai/pipelines.md → .ai/references/pipelines.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
# Pipeline conventions and rules

Shared reference for pipeline-related conventions, patterns, and gotchas.
Linked from `AGENTS.md`, `skills/model-integration/SKILL.md`, and `review-rules.md`.

> **Prefer modular for new pipelines.** [Modular Diffusers](modular.md) is the preferred way to add a new pipeline; the standard `DiffusionPipeline` covered below is still supported but is no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (e.g. modality baked into the checkpoint) or that are actively evolving. The conventions below apply when you do build or review a standard pipeline.

Expand Down
8 changes: 4 additions & 4 deletions .ai/review-rules.md → .ai/references/review-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@
Review-specific rules for Claude. Focus on correctness — style is handled by ruff.

Before reviewing, read and apply the guidelines in:
- [AGENTS.md](AGENTS.md) — coding style, copied code
- the repo's `AGENTS.md` — coding style, copied code
- [models.md](models.md) — model conventions, attention pattern, implementation rules, dependencies, gotchas
- [pipelines.md](pipelines.md) — pipeline conventions, coding style, gotchas
- [modular.md](modular.md) — modular pipeline conventions, patterns, common mistakes
- [testing.md](testing.md) — test conventions: required test layers, tester mixins, dummy-component rules. When a PR adds or changes tests, check them against this guide.
- [skills/model-integration/pitfalls.md](skills/model-integration/pitfalls.md) — known pitfalls causing numerical discrepancies between the reference implementation and the diffusers port (dtype mismatches, config assumptions, etc.)
- [skills/model-integration/pitfalls.md](../skills/model-integration/pitfalls.md) — known pitfalls causing numerical discrepancies between the reference implementation and the diffusers port (dtype mismatches, config assumptions, etc.)

## Common mistakes

Common mistakes are covered in the common-mistakes / gotcha sections in [AGENTS.md](AGENTS.md), [models.md](models.md), [pipelines.md](pipelines.md), and [modular.md](modular.md). Additionally, watch for below patterns that aren't covered there:
Common mistakes are covered in the common-mistakes / gotcha sections in `AGENTS.md`, [models.md](models.md), [pipelines.md](pipelines.md), and [modular.md](modular.md). Additionally, watch for below patterns that aren't covered there:

- **Ephemeral context.** Comments, docstrings, and files that only made sense to the current PR's author or reviewer don't help a future reader/user/developer. Examples: `# per reviewer comment on PR #NNNN`, `# as discussed in review`, `# TODO from offline chat`, debug printouts. Same for files: parity harnesses, comparison scripts, anything in `scripts/` with hardcoded developer paths or imports from the reference repo. State the *reason* so the comment stands alone, or drop it.

Expand All @@ -21,7 +21,7 @@ Common mistakes are covered in the common-mistakes / gotcha sections in [AGENTS.
A PR can leave existing docs stale or surface a pattern worth recording. Scan the docs related to what the PR touches and flag updates as a **suggestions / additional info** section (not blocking):

- **Usage docs.** New or changed public behavior — a new pipeline/model, a new argument, changed defaults, a renamed API — should have matching updates in `docs/`, docstrings, and examples. Flag any that now describe outdated behavior or that are missing for the new surface.
- **Agent docs.** If the review turns up a rule, pattern, or common gotcha that isn't written down yet — especially one the author got wrong or that you had to reason out — propose adding it to the relevant agent guide ([AGENTS.md](AGENTS.md), [models.md](models.md), [pipelines.md](pipelines.md), [modular.md](modular.md), a skill, or this file) so the next contributor/agent gets it for free instead of repeating the mistake. Human review comments on the PR are a good source for these: if a human reviewer pointed something out and your review missed it, that usually indicates a doc gap — figure out what's missing and propose the addition.
- **Agent docs.** If the review turns up a rule, pattern, or common gotcha that isn't written down yet — especially one the author got wrong or that you had to reason out — propose adding it to the relevant agent guide (`AGENTS.md`, [models.md](models.md), [pipelines.md](pipelines.md), [modular.md](modular.md), a skill, or this file) so the next contributor/agent gets it for free instead of repeating the mistake. Human review comments on the PR are a good source for these: if a human reviewer pointed something out and your review missed it, that usually indicates a doc gap — figure out what's missing and propose the addition.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we just removing it from AGENT.md? Additionally, wouldn't we have a similar problem like https://huggingface.slack.com/archives/C065E480NN9/p1786544533688239?


## Dead code analysis (new models)

Expand Down
File renamed without changes.
7 changes: 3 additions & 4 deletions .ai/skills/custom-blocks/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,12 +146,11 @@ diffusers-cli run --model my-user/my-denoise-block --trust-remote-code \

## Verifying the install

If `diffusers-cli` isn't on PATH, see the install verification section of
[`../diffusers-cli/SKILL.md`](../diffusers-cli/SKILL.md#verifying-the-cli-is-installed).
If `diffusers-cli` isn't on PATH, see the install verification section of the `diffusers-cli` skill.

## Related

- [`diffusers-cli` skill](../diffusers-cli/SKILL.md) — once your block is uploaded, `schema`/`run`
- the `diffusers-cli` skill — once your block is uploaded, `schema`/`run`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without the link or reference, would it not leave the agent to guess?

let you call it from the terminal without writing Python.
- diffusers' [modular pipelines docs](../../../docs/source/en/modular_diffusers) — for writing the block
- diffusers' [modular pipelines docs](https://huggingface.co/docs/diffusers/main/en/modular_diffusers/overview) — for writing the block
class itself.
16 changes: 8 additions & 8 deletions .ai/skills/model-integration/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
name: integrating-models
name: model-integration
description: >
Use when adding a new model or pipeline to diffusers, setting up file
structure for a new model, converting a pipeline to modular format, or
Expand All @@ -16,7 +16,7 @@ Before writing any code, gather info in this order:

1. **Reference repo** — ask for the github link. If they've already set it up locally, ask for the path. Otherwise, ask what setup steps are needed (install deps, download checkpoints, set env vars, etc.) and run through them before proceeding.
2. **Inference script** — ask for a runnable end-to-end script for a basic workflow first (e.g. T2V). Then ask what other workflows they want to support (I2V, V2V, etc.) and agree on the full implementation order together.
3. **Standard vs modular** — **default to modular.** [Modular Diffusers](../../modular.md) is the preferred implementation for new pipelines; the standard `DiffusionPipeline` is still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.
3. **Standard vs modular** — **default to modular.** [Modular Diffusers](references/modular.md) is the preferred implementation for new pipelines; the standard `DiffusionPipeline` is still supported but no longer the default. We prefer modular especially for models that don't fit a fixed task-based structure (modality baked into the checkpoint) or that are actively evolving.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this means we cannot click through to the linked guides (like the [Modular Diffusers] here) anymore when we browse the skill file on github? I don't think it's a deal breaker if it's the case, but wonder if there is an easy fix

@DN6 DN6 Aug 13, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it's set up this way so that we maintain a single set of reference guides. We could duplicate the guide files and place them as needed in the individual skill folders. Then the links would work when browsing on github.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another point is the discoverability of the agents. Without this mechanism in place, they won't be able to do so. Cc: https://huggingface.slack.com/archives/C065E480NN9/p1786544533688239


Ask step 3 as an `AskUserQuestion`, with modular marked as the recommended default.

Expand All @@ -29,15 +29,15 @@ Then work through the **Integration checklist** below
A pipeline in Diffusers (be it standard or modular) will have multiple components. These components can be models, schedulers, processors, etc.

- [ ] **Transformer model**
- [ ] Implement the model with `from_pretrained` support (conventions: [models.md](../../models.md))
- [ ] Implement the model with `from_pretrained` support (conventions: [models.md](references/models.md))
- [ ] Convert weights (see **Weight / Checkpoint Conversion**)
- [ ] Parity test against the reference (internal, not shipped — see **Model parity test**)
- [ ] Register in the relevant `__init__.py` files (lazy imports)
- [ ] Model-level tests (see **Testing**)
- [ ] **VAE** (if applicable) — reuse an existing `AutoencoderKL*` if possible; if a new one is needed, follow the same sub-steps as the transformer
- [ ] **Scheduler** — reuse an existing scheduler, or add a custom one
- [ ] **Pipeline**
- [ ] Implement the pipeline — see [modular.md](../../modular.md) for modular pipeline, or [pipelines.md](../../pipelines.md) for standard pipeline
- [ ] Implement the pipeline — see [modular.md](references/modular.md) for modular pipeline, or [pipelines.md](references/pipelines.md) for standard pipeline
- [ ] Add a LoRA mixin if applicable
- [ ] Register in the relevant `__init__.py` files (lazy imports)
- [ ] Pipeline-level tests (see **Testing**)
Expand Down Expand Up @@ -66,7 +66,7 @@ docs/source/en/

## Model integration specific rules

**Match the reference's numerical logic.** Restructuring code to fit diffusers APIs (`ModelMixin`, `ConfigMixin`, blocks for modular, etc.) is expected, and required diffusers conventions (e.g. the attention pattern in [models.md](../../models.md)) take precedence. Beyond those, keep the actual computation as close to the reference as possible — don't reorder operations, change the math, or rename internals for aesthetics, even if it looks unclean. Small deviations make output mismatches very hard to track down.
**Match the reference's numerical logic.** Restructuring code to fit diffusers APIs (`ModelMixin`, `ConfigMixin`, blocks for modular, etc.) is expected, and required diffusers conventions (e.g. the attention pattern in [models.md](references/models.md)) take precedence. Beyond those, keep the actual computation as close to the reference as possible — don't reorder operations, change the math, or rename internals for aesthetics, even if it looks unclean. Small deviations make output mismatches very hard to track down.

## Weight / Checkpoint Conversion

Expand All @@ -76,7 +76,7 @@ Convert the original checkpoint into diffusers format with a standalone script u
2. Instantiate the diffusers model from its config and load the converted state dict.
3. `save_pretrained(...)` to a local path, then load it back with `from_pretrained` to confirm it round-trips.

All weights load through the standard paths — `from_pretrained`, or `from_single_file` (add `FromSingleFileMixin` + a weight-mapping) for an original-format single checkpoint. No custom `from_pretrained`, no manual runtime loading. See the loading rule in [models.md](../../models.md).
All weights load through the standard paths — `from_pretrained`, or `from_single_file` (add `FromSingleFileMixin` + a weight-mapping) for an original-format single checkpoint. No custom `from_pretrained`, no manual runtime loading. See the loading rule in [models.md](references/models.md).

Common conversion patterns to watch for model-level components:
- Fused QKV weights that need splitting into separate Q, K, V
Expand All @@ -87,13 +87,13 @@ Common conversion patterns to watch for model-level components:

## Testing

Two test layers must be added for any new pipeline: pipeline-level tests, and (if a new model is introduced) model-level tests. Conventions for both layers — file locations, tester mixins, dummy-component rules — live in [testing.md](../../testing.md); follow it when writing the tests.
Two test layers must be added for any new pipeline: pipeline-level tests, and (if a new model is introduced) model-level tests. Conventions for both layers — file locations, tester mixins, dummy-component rules — live in [testing.md](references/testing.md); follow it when writing the tests.

## Model parity test

Confirm the diffusers implementation matches the reference. Test each component on **CPU/float32** with a strict tolerance (`max_diff < 1e-3`), comparing the **freshly converted** weights against the reference in a single script — both sides side by side, nothing saved to disk in between. See [pitfalls.md](pitfalls.md) for the common sources of numerical discrepancy.

This is an **internal verification tool for integration — it should not be shipped in the PR** (it imports the reference repo). The tests that ship with the PR are the model-level and pipeline-level tests in [testing.md](../../testing.md).
This is an **internal verification tool for integration — it should not be shipped in the PR** (it imports the reference repo). The tests that ship with the PR are the model-level and pipeline-level tests in [testing.md](references/testing.md).

The example below is schematic (placeholder names). `ReferenceModel` is the component **imported from the original repo**, and `convert_my_component` is **the same conversion function you wrote for the conversion script for the component**. You should make sure both load the *same* checkpoint weights and run the *same* input, so any difference is a conversion or implementation bug — not a difference in inputs.

Expand Down
8 changes: 4 additions & 4 deletions .ai/skills/self-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: self-review
description: >
Use before opening a PR, or whenever asked to self-review a diffusers
contribution. Applies the same rubric as the `@claude` CI (checks the diff
against .ai/review-rules.md, traces call paths for dead code). Reports findings grouped by
against references/review-rules.md, traces call paths for dead code). Reports findings grouped by
severity, flagging what to fix before submitting (blocking issues + dead code)
vs what to leave for the actual review. Report-only — does not edit files.
---
Expand All @@ -30,14 +30,14 @@ files, scope to your own commits: `git log main..HEAD --oneline`, then

## 2. Read the rubric

`.ai/review-rules.md` is the canonical rubric (the CI pins it from `main`) — read
`references/review-rules.md` is the canonical rubric (the CI pins it from `main`) — read
it and review against it; don't rely on a remembered copy. For the areas you
touched, also read `.ai/models.md`, `.ai/pipelines.md`, or `.ai/modular.md`.
touched, also read `references/models.md`, `references/pipelines.md`, `references/modular.md`, or `references/testing.md`.

## 3. Report

- **Blocking issues** — numbered. Each: title → explanation → `file.py:line` →
impact. Cite the rule, e.g. *Per `.ai/models.md`: "…only keep the inference path."*
impact. Cite the rule, e.g. *Per `references/models.md`: "…only keep the inference path."*
- **Non-blocking issues** — same format, lower severity.
- **Dead code (advisory)** — a table: `path:line` · Likely-dead / Used · reason.
- **Summary** — short synthesis and a verdict (**READY** / **NEEDS CHANGES**),
Expand Down
14 changes: 14 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be one for Codex too?

"name": "diffusers-skills",
"description": "Agent plugins published from the diffusers repository",
"owner": {
"name": "The Hugging Face team"
},
"plugins": [
{
"name": "diffusers",
"source": "./.ai",
"description": "Conventions and task skills for contributing to diffusers"
}
]
}
4 changes: 2 additions & 2 deletions .github/workflows/claude_review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -62,8 +62,8 @@ jobs:
EOF_DELIMITER="GITHUB_ENV_$(openssl rand -hex 8)"
{
echo "REVIEW_RULES<<${EOF_DELIMITER}"
git show "origin/${DEFAULT_BRANCH}:.ai/review-rules.md" 2>/dev/null \
|| echo "No .ai/review-rules.md found. Apply Python correctness standards."
git show "origin/${DEFAULT_BRANCH}:.ai/references/review-rules.md" 2>/dev/null \
|| echo "No .ai/references/review-rules.md found. Apply Python correctness standards."
echo "${EOF_DELIMITER}"
Comment on lines +65 to 67

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm this affects how https://serge.huggingface.tech/ fetches the reviewing rules btw. Cc: @tarekziade

} >> "$GITHUB_ENV"

Expand Down
Loading
Loading