A Rust-native bioinformatics pipeline engine with AI Companion — built from first principles for performance, reproducibility, and developer experience.
Documentation · Workflow Gallery · Community · Roadmap · Contributing · Security
oxo-flow is a high-performance bioinformatics pipeline engine built in Rust. It compiles workflows into Directed Acyclic Graphs and orchestrates execution with native concurrency, per-rule environment isolation, and AI-powered assistance — all from a single binary.
🧬 Community: browse curated, rated, ready-to-run workflows — ports of popular nf-core & Snakemake pipelines, original designs, and community submissions — at oxo-flow-community.
- 🤖 AI Companion — Natural language pipeline generation, intelligent refinement, failure diagnosis, and results interpretation. Powered by Claude, OpenAI, DeepSeek, or local Ollama.
- 🔀 DAG engine — Automatic dependency resolution, topological ordering, and parallel execution with resource-aware scheduling (CPU, memory, GPU, disk) across local and cluster backends (SLURM, PBS, SGE, LSF)
- 📦 8 environment backends — conda, mamba, pixi, docker, singularity, venv, system, and HPC modules — with per-rule isolation
- ⚡ Rust performance — Fearless concurrency, zero-cost abstractions,
#![forbid(unsafe_code)]in every workspace crate - 🌐 Professional Web UI — React 19 SPA with DAG visualization (React Flow + d3-dag), TOML editor (CodeMirror 6), and AI chat
- 🧩 VS Code extension — first-party
traitome.oxo-flowIDE for.oxoflow: schema-driven completion, background diagnostics, canonical formatting, one-click run / dry-run / graph / status / clean / resume / AI-generate (Open VSX + release VSIX) - 📊 Built-in reporting — HTML/JSON/Markdown/PDF reports with execution summaries, failure diagnosis, resource metrics, and checkpoint-verified file manifests; QC metrics parsed from real tool outputs (fastp, flagstat, STAR, featureCounts, bcftools, kraken2), R-friendly TSV export (
--r-data), and a JSON report snapshot auto-written after every run - 🔒 Security hardened — Shell injection prevention, path traversal protection, secret scanning, and per-IP rate limiting
- 🗄️ Checkpoint & resume — JSON-persisted execution state; resume interrupted workflows from the last completed rule
- 🚀 Three deployment modes — Personal workstation, team server with OAuth2, or HPC submit panel — same binary
| Feature | oxo-flow | Snakemake | Nextflow |
|---|---|---|---|
| Language | Rust — compiled, type-safe, #![forbid(unsafe_code)] in the core, AI, CLI, and web crates |
Python | Groovy/JVM |
| Performance | Native binary, instant startup | Python + JIT overhead | JVM startup overhead |
| Workflow format | TOML (.oxoflow) — declarative, composable |
Snakefile (Python DSL) | Nextflow DSL (Groovy) |
| Environment support | 8 backends — conda, mamba, pixi, docker, singularity, venv, system, modules — per-rule | conda, singularity, docker | conda, docker, singularity, modules |
| Web interface | Built-in React 19 SPA + REST API | External Snakemake-UI | Nextflow Tower (commercial) |
| Reporting | Built-in HTML/JSON/Markdown/PDF reports from checkpoint data (execution truth, failure diagnosis, file checksums) | Via MultiQC | Via Nextflow Tower |
| Cluster backends | SLURM, PBS, SGE, LSF | SLURM, PBS, SGE, LSF | SLURM, PBS, SGE, LSF, k8s |
| Security | Shell sanitization, path traversal prevention, rate limiting | Limited | Limited |
| AI Companion | Built-in — generate, refine, diagnose, interpret | Not built-in | Not built-in |
| Testing | 3,200+ tests (unit, integration, doc) | pytest-based | Varied |
oxo-flow is built on five principles:
- DAG is the fundamental abstraction — Every workflow is a directed acyclic graph. The engine constructs, validates, and executes DAGs with maximum parallelism.
- Environment isolation is non-negotiable — Each task runs in its own isolated environment via one of 8 backends.
- Reproducible by design — Config checksums, execution provenance, and container pinning guarantee identical outputs from identical inputs.
- Performance through Rust — Zero-cost abstractions and fearless concurrency for orchestrating thousands of concurrent tasks.
- Outcome-driven — The DAG engine's target-aware execution (
-tflag) computes the minimal rule set needed to produce specific deliverables.
Learn oxo-flow incrementally with curated, validated example workflows — from a one-rule hello-world to production-grade pipelines. Every workflow passes oxo-flow validate and is tested in CI. See the Workflow Gallery for the full catalog with detailed explanations, DAG visualizations, and CLI output.
Download the latest release for your platform from GitHub Releases:
# Linux (x86_64)
curl -LO https://github.com/Traitome/oxo-flow/releases/latest/download/oxo-flow-latest-x86_64-unknown-linux-gnu.tar.gz
tar xzf oxo-flow-latest-x86_64-unknown-linux-gnu.tar.gz
sudo mv oxo-flow /usr/local/bin/
# macOS (Apple Silicon)
curl -LO https://github.com/Traitome/oxo-flow/releases/latest/download/oxo-flow-latest-aarch64-apple-darwin.tar.gz
tar xzf oxo-flow-latest-aarch64-apple-darwin.tar.gz
sudo mv oxo-flow /usr/local/bin/cargo install oxo-flow-cliconda install -c bioconda oxo-flow-cliVersion caveat — the bioconda recipe can lag a freshly cut release while the bot recipe update lands. Check
conda search -c bioconda oxo-flow-clifor the currently delivered version; if it is behind the release notes you need, install from the pre-built binaries above orcargo install oxo-flow-cli(crates.io).
Pre-built images are published to GitHub Container Registry on every release (and every push to main). Release images are multi-arch (linux/amd64 + linux/arm64, so Apple silicon and ARM servers pull a native image). :latest moves only after the release image passes a health smoke test; a rolling :<major.minor> tag (e.g. :0.19) tracks the newest patch of each minor line, and :main is a multi-arch dev build from source:
# Web UI at http://localhost:3000 (CLI is inside the same image)
docker run -d --name oxo-flow -p 3000:3000 ghcr.io/traitome/oxo-flow:latest
# Pin a release (or track a minor line)
docker run -d -p 3000:3000 ghcr.io/traitome/oxo-flow:0.23.2
docker run -d -p 3000:3000 ghcr.io/traitome/oxo-flow:0.23
# CLI one-shot usage (workflow files mounted from the host)
docker run --rm -v "$PWD:/work" -w /work ghcr.io/traitome/oxo-flow:latest \
oxo-flow run my-pipeline.oxoflowHealth endpoint: GET /api/health returns "status":"ok" when the database is reachable.
Container configuration is env-only: the image sets OXO_FLOW_MODE=team, OXO_FLOW_HOST=0.0.0.0, and OXO_FLOW_PORT=3000 as defaults, and the CMD carries no CLI flags — so docker run -e OXO_FLOW_PORT=9090 … changes the actual bind port (the built-in healthcheck follows it). Note the published image is built without the postgres compile feature: pointing DATABASE_URL at a postgres:// URL makes the container exit at startup (see the docker-compose.yml comment for how to build a feature-enabled image).
git clone https://github.com/Traitome/oxo-flow.git
cd oxo-flow
cargo build --release --workspace
# Install the CLI to your local cargo bin directory:
cargo install --path crates/oxo-flow-cli
# Binaries are in target/release/
# - oxo-flow (CLI, includes `oxo-flow serve` for web UI)
# - oxo-flow-web (Standalone web server)# Create a new pipeline project (creates my-pipeline/ directory and my-pipeline.oxoflow)
oxo-flow init my-pipeline
cd my-pipeline
# Validate the workflow
oxo-flow validate my-pipeline.oxoflow
# Preview execution plan
oxo-flow dry-run my-pipeline.oxoflow
# Execute with 8 parallel jobs
oxo-flow run my-pipeline.oxoflow -j 8
# Visualize the DAG (use -f dot for Graphviz DOT output)
oxo-flow graph my-pipeline.oxoflow -f dot > dag.dot
dot -Tpng dag.dot -o dag.png
# Export a transit-map-style diagram via nf-metro
oxo-flow graph my-pipeline.oxoflow -f metro -o pipeline.mmd
# Render locally (requires nf-metro) or paste the .mmd content into the
# online playground: https://seqeralabs.github.io/nf-metro/latest/playground/
nf-metro render pipeline.mmd -o pipeline.svg
# Generate an HTML report
oxo-flow report my-pipeline.oxoflow -f html -o report.htmloxo-flow supports job submission to HPC cluster schedulers including SLURM, PBS/PBS Pro, SGE/UGE, and LSF. The oxo-flow cluster subcommand manages the full submission lifecycle.
# Submit a workflow to a SLURM cluster
oxo-flow cluster submit workflow.oxoflow --backend slurm --queue short -o jobs/
# Check submission status (no job ids: lists the user's jobs; --backend optional)
oxo-flow cluster status --backend slurm <job-id>
oxo-flow cluster status
# Cancel a submitted job
oxo-flow cluster cancel --backend slurm <job-id>Supported backends: slurm, pbs, sge, lsf. Cluster submissions are configured inline via oxo-flow cluster submit flags.
Workflows can also carry reusable config supplements (e.g. per-cluster thread
and memory defaults) in profiles/<NAME>.toml next to the workflow:
# Run a workflow with a config supplement profile
oxo-flow run workflow.oxoflow --profile slurmoxo-flow uses a TOML-based workflow format that is human-readable, composable, and declarative:
[workflow]
name = "variant-calling"
version = "1.0.0"
[config]
reference = "/data/ref/GRCh38.fa"
[[rules]]
name = "fastp"
input = ["raw/{sample}_R1.fastq.gz", "raw/{sample}_R2.fastq.gz"]
output = ["trimmed/{sample}_R1.fastq.gz", "trimmed/{sample}_R2.fastq.gz"]
threads = 8
shell = "fastp -i {input[0]} -I {input[1]} -o {output[0]} -O {output[1]}"
[rules.environment]
conda = "envs/fastp.yaml"
[[rules]]
name = "bwa_align"
input = ["trimmed/{sample}_R1.fastq.gz", "trimmed/{sample}_R2.fastq.gz"]
output = ["aligned/{sample}.bam"]
threads = 16
memory = "32G"
shell = "bwa-mem2 mem -t {threads} {config.reference} {input[0]} {input[1]} | samtools sort -o {output[0]}"
[rules.environment]
docker = "biocontainers/bwa-mem2:2.2.1"Wildcards like {sample} expand automatically via input file discovery. Features include reference directory conventions, environment groups, optional rules, and directory inputs. See the full Workflow Format Specification for details.
The first-party oxo-flow Pipeline extension (traitome.oxo-flow) makes VS Code a full .oxoflow IDE — and it is released in lockstep with the engine (VSIX with checksum attached to every GitHub release, published to Open VSX):
Note — the VS Code Marketplace is not currently supported (requires an Azure DevOps publisher account the project does not maintain yet). VSCodium, Cursor, Windsurf and most forks get the extension from Open VSX (searched by default); stock VS Code users can install the release VSIX offline:
codium --install-extension traitome.oxo-flow # Open VSX (forks)
code --install-extension oxo-flow-vscode-v<version>.vsix # offline VSIX (any VS Code)It provides [[rules]]-aware syntax highlighting (wildcards and {input[0]} placeholders), completion and hover docs generated from the canonical workflow JSON Schema, background validate/lint diagnostics anchored to the failing rule, canonical formatting, and one-click run / dry-run / graph / resume / AI-generate commands. See the VS Code Extension reference, or Editor Setup for other editors (Zed, Helix, Neovim, JetBrains, …).
The oxo-flow binary provides 30 subcommands covering the complete workflow lifecycle. See the full CLI Reference for details.
| Category | Commands |
|---|---|
| Execution | run, resume, dry-run (alias plan), test, batch |
| Development | init, validate, format, lint, debug, template |
| Inspection | graph, info, report, status, config (alias check), diff, provenance, schema |
| Environment | env, export, clean, touch |
| Deployment | serve, cluster, publish, pull |
| AI & System | ai, completions, license |
See the full CLI Reference for detailed usage of each subcommand.
The oxo-flow serve command starts an axum-powered REST server with 100+ endpoints across 9 domains (workflow, execution, dag, AI, auth, collaboration, observability, chat, clusters). Full API reference at the Web API reference (GET /api/openapi.json serves the live OpenAPI 3.1 spec).
oxo-flow is organized as a Cargo workspace with four crates, plus a fifth
(oxo-flow-desktop) that lives in crates/ but is excluded from the
workspace so headless builds never need GUI toolchains:
oxo-flow/
├── crates/
│ ├── oxo-flow-core/ # Core library: DAG engine, executor, environment mgmt,
│ │ # config parsing, scheduler, wildcard expansion, reporting
│ ├── oxo-flow-ai/ # AI companion: provider abstraction, skill system, agents
│ ├── oxo-flow-cli/ # CLI binary ("oxo-flow") — Clap-based, 30 subcommands
│ ├── oxo-flow-web/ # Web server ("oxo-flow-web") — axum REST API + frontend
│ └── oxo-flow-desktop/ # Native desktop shell ("oxo-flow-desktop", wry + tao) —
│ # excluded from the workspace (own Cargo.toml)
├── examples/ # Example .oxoflow workflows
├── tests/ # Integration tests
└── docs/ # Documentation (MkDocs)
| Crate | Type | Binary | License |
|---|---|---|---|
oxo-flow-core |
Library | — | Apache-2.0 |
oxo-flow-ai |
Library | — | Apache-2.0 |
oxo-flow-cli |
Binary | oxo-flow |
Apache-2.0 |
oxo-flow-web |
Binary | oxo-flow-web |
Dual Academic / Commercial |
oxo-flow-desktop |
Binary | oxo-flow-desktop |
Apache-2.0 |
| Module | Crate | Responsibility |
|---|---|---|
dag.rs |
core | DAG construction, validation, topological sort |
executor/ |
core | Task execution (local, cluster, cloud), checkpointing, staging |
environment.rs |
core | Environment management (conda, mamba, pixi, docker, singularity, venv, system, modules) |
config/ |
core | Workflow configuration and .oxoflow file parsing |
rule.rs |
core | Rule/step definitions with inputs, outputs, shell, resources |
scheduler.rs |
core | Job scheduling with resource constraints |
wildcard.rs |
core | Wildcard pattern expansion ({sample}, {chr}, etc.) |
report.rs |
core | Modular report generation (HTML, JSON, PDF via wkhtmltopdf) |
format.rs |
core | Workflow formatting, linting, and 30+ diagnostic patterns |
plugin.rs |
core | Plugin system for MCP servers and custom tools |
ai_provider.rs |
web | Multi-provider AI abstraction (Claude, OpenAI, DeepSeek, Ollama) |
domains/ai/ |
web | AI translation, chat SSE streaming, agent orchestration |
domains/execution/ |
web | Run lifecycle, diagnostics, pause/resume, retry |
domains/workflow/ |
web | Pipeline CRUD, validation, DAG building, templating |
provider.rs |
ai | Multi-provider AI client abstraction with auto-detection |
skill.rs |
ai | Skill registry and discovery system |
agent/ |
ai | Agent orchestration and tool-calling framework |
mcp.rs |
ai | MCP (Model Context Protocol) server integration |
oxo-flow serve starts the web interface with an embedded REST API and React SPA. Run it without arguments for local experimentation:
# Mode 1: Personal workstation (default) — SQLite, localhost:8080, no auth
oxo-flow serve
# → Open http://127.0.0.1:8080 in your browser
# Mode 2: Team server — multi-user (env-var passwords + optional ORCID/GitHub OAuth2)
oxo-flow serve --mode team --host 0.0.0.0
# Mode 3: HPC submit panel — the scheduler (SLURM/PBS/SGE/LSF) is probed at
# startup when the server itself runs on a cluster login node
oxo-flow serve --mode hpcServe flags: --mode (personal | team | hpc), --host (default
127.0.0.1), -p/--port (default 8080), --base-path, and --open
(open the browser on startup). Each flag also has an OXO_FLOW_*
environment-variable form, plus a platform config file layer — see
Deployment Modes.
PostgreSQL: both oxo-flow serve and the standalone oxo-flow-web
binary read the DATABASE_URL environment variable — a postgres:// URL
(on a build with the postgres feature) enables the PostgreSQL backend for
the library/AI/auth domains; the default (sqlite://oxo-flow.db in the
working directory) stays SQLite. Note that the published Docker image is
built WITHOUT the postgres feature — rebuild with
cargo build --features postgres to enable it. Run execution stays
SQLite-only either way: on a PostgreSQL server every /api/runs* endpoint
answers 503 RUNS_REQUIRE_SQLITE.
Full documentation is available at traitome.github.io/oxo-flow/latest/.
| If you are... | Recommended Start |
|---|---|
| New to oxo-flow | Quick Start · First Workflow |
| A Bioinformatician | Workflow Gallery |
| A Pipeline Engineer | Workflow Format Specification · CLI Reference |
| A DevOps/Cloud Admin | Environment Management · Running on Cluster |
| A Bioinformatics Core | Workflow Gallery · Environment Management |
MkDocs source lives under docs/guide/src/.
# Build all workspace crates
cargo build --workspace
# Run all tests (unit + integration)
cargo test --workspace
# Run the full CI suite (fmt + clippy + build + test + schema-drift + version-check + audit + frontend lint)
make ci
# Individual CI steps
cargo fmt -- --check # Check formatting
cargo clippy --workspace --all-targets -- -D warnings # Lint (zero warnings)
cargo build --workspace # Compile
cargo test --workspace # Test
# Format code
cargo fmtA bare
cargo buildat the repo root builds only the rootoxo-flowlibrary package (the integration-test crate) — not the CLI or web binaries. Pass--workspacewhenever you wanttarget/*/oxo-flowandtarget/*/oxo-flow-web.
| Component | Technology |
|---|---|
| Language | Rust (2024 edition) |
| Async runtime | tokio |
| CLI framework | clap (derive) |
| Web framework | axum |
| Serialization | serde + TOML |
| Graph library | petgraph |
| Templating | tera |
| Error handling | thiserror (lib) / anyhow (bin) |
| Logging | tracing |
This project uses a split licensing model:
| Crate | License | Details |
|---|---|---|
oxo-flow-core |
Apache-2.0 | Free and open-source |
oxo-flow-ai |
Apache-2.0 | Free and open-source |
oxo-flow-cli |
Apache-2.0 | Free and open-source |
oxo-flow-web |
Academic / Commercial | Free for academic and non-commercial use; commercial use requires a separate license |
The core library, AI companion, and CLI are licensed under the Apache License 2.0 — you are free to use, modify, and distribute them without restriction.
The web interface (oxo-flow-web) is available under a dual license: free for academic and non-commercial use under the Academic License, and requiring a commercial license for commercial deployments. See LICENSE-ACADEMIC and LICENSE-COMMERCIAL for details.
Contributions are welcome! Please see:
- CONTRIBUTING.md — Contribution guidelines
- ROADMAP.md — Project roadmap and areas where help is needed
- CODE_OF_CONDUCT.md — Community standards
- GOVERNANCE.md — Project governance and decision-making
- SECURITY.md — Security vulnerability reporting
Before submitting a PR, ensure all checks pass:
make ciIf you use oxo-flow in academic research, please cite:
Shixiang Wang, oxo-flow: compiled, memory-safe bioinformatics workflow orchestration, bioRxiv, 2026, https://doi.org/10.64898/2026.06.11.731578
Jia Ding, Yun Peng, Ruochen Wei, Boquan Wang, Jian-Guo Zhou, Shixiang Wang, BLIT: an R package for seamless integration of command-line bioinformatics tool universe, Bioinformatics Advances, Volume 6, Issue 1, 2026, vbag088, https://doi.org/10.1093/bioadv/vbag088
- Bug reports — GitHub Issues (use bug report template)
- Feature requests — GitHub Issues (use feature request template)
- Documentation — traitome.github.io/oxo-flow/latest/
- Questions — Ask DeepWiki
oxo-flow aims to work reliably across diverse computing environments — laptops, HPC clusters, GPU nodes. We cannot replicate every deployment scenario in CI. If you use oxo-flow, please share your experience (success or failure) as a GitHub Issue with the prefix [Real-World Testing]. Your feedback directly shapes our priorities.
- LIMITATIONS.md — Known limitations and constraints
- REPRODUCIBILITY.md — Reproducibility guarantees and methodology
- RELEASING.md — Release process and versioning policy
- TRADEMARK.md — Trademark usage guidelines
- docs/CHANGE_CONTROL.md — Change control for regulated environments
- docs/VALIDATION_PROTOCOL.md — IQ/OQ/PQ validation protocol
Built with 🧬 by Traitome
Thanks goes to these wonderful people:
Run
make contributorsto refresh the contributor list from git history.