data-orchestration

Event Driven Orchestration & Scheduling Platform for Mission Critical Applications

Alluxio, data orchestration for analytics and machine learning in the cloud

cloud-native distributed storage

Open-source ETL and ELT tool on DuckDB. No-Code/Low-code visual data pipelines or SQL: 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents.

An open source, standard data file format for graph data storage and retrieval.

A collection of AI skills for working with Dagster

A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for cloud data warehouse and Metabase to serve the needs of data visualizations such as analytical dashboards.

Build ML pipelines you can trust — visually on a canvas or in Python. Self-hosted platform + Apache-2.0 library. Catches data leakage, keeps scores honest, and takes you from raw data to monitored model.

Best practices for data workflows, integrations with the Modern Data Stack (MDS), Infrastructure as Code (IaC), Cloud Provider Services

Data Engineering - Metropolitan Transportation Authority (MTA) Subway Data Analysis

Data-aware orchestration with dagster, dbt, and airbyte

Asset-first data orchestration for Elixir/BEAM. Dagster-inspired with OTP fault tolerance, LiveView dashboard, lineage tracking, checkpoint gates, and distributed execution via Oban.

This repo contains a dataset, exercises, and sample code for an end-to-end SAP BTP data-to-value bootcamp covering SAP HANA Cloud, SAP Data Warehouse Cloud, SAP Data Intelligence Cloud, and SAP Analytics Cloud.

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

Get started with Dagster ASAP

CI/CD repository template to automate deployments of your production flows

An operator for managing Alluxio system on Kubernetes cluster

A simple pipeline infrastructure with ETL pipeline contained in a Docker environment on Apache Airflow for orchestration and Postgres for data warehousing

A TCG-style web game for mastering Apache Airflow SQL check operators. Battle the DQ Monster, deploy Astro Observe, and learn about data quality patterns through a simple card game.

End-to-end Azure Databricks retail data engineering project using Medallion Architecture (Bronze, Silver, Gold). Implements Auto Loader, Unity Catalog, Delta Lake, SCD Type 1 & 2 dimensions, and Fact Orders for analytics-ready star schema modeling.