README.md

# Awesome-LLMOps [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)

🎉 An awesome & curated list of best LLMOps tools.

> More than welcome to add a new project by simply opening an issue.

## Table of Contents

- [Inference](#inference)
    - [Inference Engine](#inference-engine)
    - [Inference Platform](#inference-platform)
    - [Middleware](#middleware)
    - [LLM Router](#llm-router)
    - [AI Gateway](#ai-gateway)
    - [Output](#output)
    - [Simulator](#simulator)
    - [Benchmark](#benchmark)
- [Orchestration](#orchestration)
    - [Agent Framework](#agent-framework)
    - [Application Framework](#application-framework)
    - [RAG](#rag)
    - [Workflow](#workflow)
    - [Evolutionary Framework](#evolutionary-framework)
- [Runtime](#runtime)
    - [AI Terminal](#ai-terminal)
    - [AI Agent](#ai-agent)
    - [Chatbot](#chatbot)
    - [Code Agent](#code-agent)
    - [Evolve Agent](#evolve-agent)
    - [Database](#database)
    - [Observation](#observation)
    - [Sandbox](#sandbox)
    - [Tool](#tool)
- [Training](#training)
    - [Framework](#framework)
    - [FineTune](#finetune)
    - [RLHF](#rlhf)
    - [Agentic RL](#agentic-rl)
    - [Benchmark](#benchmark)
    - [Workflow](#workflow)

## Inference

### Inference Engine

* **[Cortex.cpp](https://github.com/janhq/cortex.cpp)**: Local AI API Platform. ![Stars](https://img.shields.io/github/stars/janhq/cortex.cpp.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/janhq/cortex.cpp?color=green) ![LastCommit](https://img.shields.io/github/last-commit/janhq/cortex.cpp?color=green)
* **[DeepSpeed-MII](https://github.com/microsoft/DeepSpeed-MII)**: MII makes low-latency and high-throughput inference possible, powered by DeepSpeed. ![Stars](https://img.shields.io/github/stars/microsoft/deepspeed-mii.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/deepspeed-mii?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/deepspeed-mii?color=green)
* **[llama-box](https://github.com/gpustack/llama-box)**: LM inference server implementation based on *.cpp. ![Stars](https://img.shields.io/github/stars/gpustack/llama-box.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/gpustack/llama-box?color=green) ![LastCommit](https://img.shields.io/github/last-commit/gpustack/llama-box?color=green)
* **[Nvidia Dynamo](https://github.com/ai-dynamo/dynamo)**: A Datacenter Scale Distributed Inference Serving Framework. ![Stars](https://img.shields.io/github/stars/ai-dynamo/dynamo.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ai-dynamo/dynamo?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ai-dynamo/dynamo?color=green)
* **[ipex-llm](https://github.com/intel-analytics/ipex-llm)**: Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc. ![Stars](https://img.shields.io/github/stars/intel-analytics/ipex-llm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/intel-analytics/ipex-llm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/intel-analytics/ipex-llm?color=green)
* **[LMDeploy](https://github.com/InternLM/lmdeploy)**: LMDeploy is a toolkit for compressing, deploying, and serving LLMs. ![Stars](https://img.shields.io/github/stars/internlm/lmdeploy.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/internlm/lmdeploy?color=green) ![LastCommit](https://img.shields.io/github/last-commit/internlm/lmdeploy?color=green)
* **[LoRAX](https://github.com/predibase/lorax)**: Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs. ![Stars](https://img.shields.io/github/stars/predibase/lorax.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/predibase/lorax?color=green) ![LastCommit](https://img.shields.io/github/last-commit/predibase/lorax?color=green) ![Tag](https://img.shields.io/badge/lora-orange)
* **[llama.cpp](https://github.com/ggerganov/llama.cpp)**: LLM inference in C/C++. ![Stars](https://img.shields.io/github/stars/ggerganov/llama.cpp.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ggerganov/llama.cpp?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ggerganov/llama.cpp?color=green)
* **[Llumnix](https://github.com/AlibabaPAI/llumnix)**: Efficient and easy multi-instance LLM serving. ![Stars](https://img.shields.io/github/stars/alibabapai/llumnix.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/alibabapai/llumnix?color=green) ![LastCommit](https://img.shields.io/github/last-commit/alibabapai/llumnix?color=green)
* **[MInference](https://github.com/microsoft/minference)**: [NeurIPS'24 Spotlight, ICLR'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy. ![Stars](https://img.shields.io/github/stars/microsoft/minference.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/minference?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/minference?color=green) ![Tag](https://img.shields.io/badge/long_context-orange)
* **[MLC LLM](https://github.com/mlc-ai/mlc-llm)**: Universal LLM Deployment Engine with ML Compilation. ![Stars](https://img.shields.io/github/stars/mlc-ai/mlc-llm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mlc-ai/mlc-llm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mlc-ai/mlc-llm?color=green)
* **[MLServer](https://github.com/SeldonIO/MLServer)**: An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more. ![Stars](https://img.shields.io/github/stars/seldonio/mlserver.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/seldonio/mlserver?color=green) ![LastCommit](https://img.shields.io/github/last-commit/seldonio/mlserver?color=green)
* **[Ollama](https://github.com/ollama/ollama)**: Get up and running with Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, and other large language models. ![Stars](https://img.shields.io/github/stars/ollama/ollama.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ollama/ollama?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ollama/ollama?color=green)
* **[OpenLLM](https://github.com/bentoml/OpenLLM)**: Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud. ![Stars](https://img.shields.io/github/stars/bentoml/openllm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/bentoml/openllm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/bentoml/openllm?color=green)
* **[OpenVINO](https://github.com/openvinotoolkit/openvino)**: OpenVINO™ is an open source toolkit for optimizing and deploying AI inference. ![Stars](https://img.shields.io/github/stars/openvinotoolkit/openvino.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openvinotoolkit/openvino?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openvinotoolkit/openvino?color=green)
* **[Petals](https://github.com/bigscience-workshop/petals)**: 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading ![Stars](https://img.shields.io/github/stars/bigscience-workshop/petals.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/bigscience-workshop/petals?color=green) ![LastCommit](https://img.shields.io/github/last-commit/bigscience-workshop/petals?color=green)
* **[Ratchet](https://github.com/huggingface/ratchet)**: A cross-platform browser ML framework. ![Stars](https://img.shields.io/github/stars/huggingface/ratchet.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/huggingface/ratchet?color=green) ![LastCommit](https://img.shields.io/github/last-commit/huggingface/ratchet?color=green) ![Tag](https://img.shields.io/badge/browser-orange)
* **[SGLang](https://github.com/sgl-project/sglang)**: SGLang is a fast serving framework for large language models and vision language models. ![Stars](https://img.shields.io/github/stars/sgl-project/sglang.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/sgl-project/sglang?color=green) ![LastCommit](https://img.shields.io/github/last-commit/sgl-project/sglang?color=green)
* **[TinyGrad](https://github.com/tinygrad/tinygrad)**: You like pytorch? You like micrograd? You love tinygrad! ❤️  ![Stars](https://img.shields.io/github/stars/tinygrad/tinygrad.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/tinygrad/tinygrad?color=green) ![LastCommit](https://img.shields.io/github/last-commit/tinygrad/tinygrad?color=green)
* **[transformers.js](https://github.com/huggingface/transformers.js)**: State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server! ![Stars](https://img.shields.io/github/stars/huggingface/transformers.js.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/huggingface/transformers.js?color=green) ![LastCommit](https://img.shields.io/github/last-commit/huggingface/transformers.js?color=green) ![Tag](https://img.shields.io/badge/browser-orange)
* **[Triton Inference Server](https://github.com/triton-inference-server/server)**: The Triton Inference Server provides an optimized cloud and edge inferencing solution. ![Stars](https://img.shields.io/github/stars/triton-inference-server/server.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/triton-inference-server/server?color=green) ![LastCommit](https://img.shields.io/github/last-commit/triton-inference-server/server?color=green)
* **[Text Generation Inference](https://github.com/huggingface/text-generation-inference)**: Large Language Model Text Generation Inference. ![Stars](https://img.shields.io/github/stars/huggingface/text-generation-inference.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/huggingface/text-generation-inference?color=green) ![LastCommit](https://img.shields.io/github/last-commit/huggingface/text-generation-inference?color=green)
* **[vLLM](https://github.com/vllm-project/vllm)**: A high-throughput and memory-efficient inference and serving engine for LLMs. ![Stars](https://img.shields.io/github/stars/vllm-project/vllm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vllm-project/vllm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vllm-project/vllm?color=green)
* **[web-llm](https://github.com/mlc-ai/web-llm)**: High-performance In-browser LLM Inference Engine. ![Stars](https://img.shields.io/github/stars/mlc-ai/web-llm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mlc-ai/web-llm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mlc-ai/web-llm?color=green) ![Tag](https://img.shields.io/badge/browser-orange)
* **[Xinference](https://github.com/xorbitsai/inference)**: Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop. ![Stars](https://img.shields.io/github/stars/xorbitsai/inference.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/xorbitsai/inference?color=green) ![LastCommit](https://img.shields.io/github/last-commit/xorbitsai/inference?color=green)
* **[zml](https://github.com/zml/zml)**: Any model. Any hardware. Zero compromise. Built with @ziglang / @openxla / MLIR / @bazelbuild. ![Stars](https://img.shields.io/github/stars/zml/zml.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/zml/zml?color=green) ![LastCommit](https://img.shields.io/github/last-commit/zml/zml?color=green)

### Inference Platform

* **[AIBrix](https://github.com/vllm-project/aibrix)**: Cost-efficient and pluggable Infrastructure components for GenAI inference. ![Stars](https://img.shields.io/github/stars/vllm-project/aibrix.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vllm-project/aibrix?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vllm-project/aibrix?color=green)
* **[BentoML](https://github.com/bentoml/BentoML)**: The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more! ![Stars](https://img.shields.io/github/stars/bentoml/bentoml.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/bentoml/bentoml?color=green) ![LastCommit](https://img.shields.io/github/last-commit/bentoml/bentoml?color=green)
* **[beta9](https://github.com/beam-cloud/beta9)**: Ultrafast serverless GPU inference, sandboxes, and background jobs ![Stars](https://img.shields.io/github/stars/beam-cloud/beta9.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/beam-cloud/beta9?color=green) ![LastCommit](https://img.shields.io/github/last-commit/beam-cloud/beta9?color=green)
* **[Kaito](https://github.com/kaito-project/Kaito)**: Kubernetes operator for large-model inference and fine-tuning, with GPU auto-provisioning, container-based hosting, and CRD-based orchestration. ![Stars](https://img.shields.io/github/stars/kaito-project/Kaito.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kaito-project/Kaito?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kaito-project/Kaito?color=green)
* **[Kserve](https://github.com/kserve/kserve)**: Standardized Serverless ML Inference Platform on Kubernetes. ![Stars](https://img.shields.io/github/stars/kserve/kserve.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kserve/kserve?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kserve/kserve?color=green)
* **[KubeAI](https://github.com/substratusai/kubeai)**: AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text. ![Stars](https://img.shields.io/github/stars/substratusai/kubeai.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/substratusai/kubeai?color=green) ![LastCommit](https://img.shields.io/github/last-commit/substratusai/kubeai?color=green)
* **[llm-d](https://github.com/llm-d/llm-d)**: llm-d is a Kubernetes-native high-performance distributed LLM inference framework ![Stars](https://img.shields.io/github/stars/llm-d/llm-d.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/llm-d/llm-d?color=green) ![LastCommit](https://img.shields.io/github/last-commit/llm-d/llm-d?color=green)
* **[llmaz](https://github.com/InftyAI/llmaz)**: ☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work! ![Stars](https://img.shields.io/github/stars/inftyai/llmaz.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/inftyai/llmaz?color=green) ![LastCommit](https://img.shields.io/github/last-commit/inftyai/llmaz?color=green)
* **[Modular](https://github.com/modular/modular)**: The Modular Platform (includes MAX & Mojo) ![Stars](https://img.shields.io/github/stars/modular/modular.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/modular/modular?color=green) ![LastCommit](https://img.shields.io/github/last-commit/modular/modular?color=green)
* **[Mooncake](https://github.com/kvcache-ai/Mooncake)**: Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI. ![Stars](https://img.shields.io/github/stars/kvcache-ai/mooncake.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kvcache-ai/mooncake?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kvcache-ai/mooncake?color=green)
* **[OME](https://github.com/sgl-project/ome)**: OME is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs) ![Stars](https://img.shields.io/github/stars/sgl-project/ome.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/sgl-project/ome?color=green) ![LastCommit](https://img.shields.io/github/last-commit/sgl-project/ome?color=green)
* **[Paralleliq](https://www.paralleliq.ai)**: Model-aware GPU optimization layer for AI inference clusters. Understands which model runs on which GPU to identify waste and risk — tier misplacement, dark capacity, OOM risk, and CPU:GPU imbalance — with human-in-the-loop approval workflows and a full audit trail.

### Middleware

* **[Checkpoint Engine](https://github.com/MoonshotAI/checkpoint-engine)**: Checkpoint-engine is a simple middleware to update model weights in LLM inference engines ![Stars](https://img.shields.io/github/stars/MoonshotAI/checkpoint-engine.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/MoonshotAI/checkpoint-engine?color=green) ![LastCommit](https://img.shields.io/github/last-commit/MoonshotAI/checkpoint-engine?color=green)
* **[kvcached](https://github.com/ovg-project/kvcached)**: Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond ![Stars](https://img.shields.io/github/stars/ovg-project/kvcached.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ovg-project/kvcached?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ovg-project/kvcached?color=green)
* **[LMCache](https://github.com/LMCache/LMCache)**: 10x Faster Long-Context LLM By Smart KV Cache Optimizations. ![Stars](https://img.shields.io/github/stars/lmcache/lmcache.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lmcache/lmcache?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lmcache/lmcache?color=green) ![Tag](https://img.shields.io/badge/kvcache-orange)

* **[KV Cache Store](https://kvcachestore.com/)**: Hosted KV-cache artifact registry plus an open-source Rust CLI. Precompute, verify, quantize, and share attention states across RAG and long-context prompts to cut prefill cost and latency. ![Website](https://img.shields.io/badge/Website-kvcachestore.com-blue) ([CLI source](https://github.com/kvcachestore/kvcdn))
* **[warmctl](https://github.com/TensorRails/warmctl)**: GPU checkpoint/restore for vLLM: snapshot a warmed-up engine (compiled kernels, CUDA graphs, weights, 1-8 GPUs) into an OCI image and restore it in seconds instead of a multi-minute cold start. Scale to zero, spot GPUs, Kubernetes optional. ![Stars](https://img.shields.io/github/stars/TensorRails/warmctl.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/TensorRails/warmctl?color=green) ![LastCommit](https://img.shields.io/github/last-commit/TensorRails/warmctl?color=green)


### LLM Router

* **[AI Gateway](https://github.com/Portkey-AI/gateway)**: A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. ![Stars](https://img.shields.io/github/stars/portkey-ai/gateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/portkey-ai/gateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/portkey-ai/gateway?color=green)
* **[bifrost](https://github.com/maximhq/bifrost)**: Fastest LLM gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS. ![Stars](https://img.shields.io/github/stars/maximhq/bifrost.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/maximhq/bifrost?color=green) ![LastCommit](https://img.shields.io/github/last-commit/maximhq/bifrost?color=green)
* **[Doubleword Control Layer](https://github.com/doublewordai/control-layer)**: The world’s fastest AI model gateway (450x less overhead than LiteLLM). Unified access to LLMs across endpoints (openAI, self-hosted, etc.) behind a single authentication layer - with API key generation, user management, request logging, and more ![Stars](https://img.shields.io/github/stars/doublewordai/control-layer.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/doublewordai/control-layer?color=green) ![LastCommit](https://img.shields.io/github/last-commit/doublewordai/control-layer?color=green)
* **[Ferro Labs AI Gateway](https://github.com/ferro-labs/ai-gateway)**: One API for 25+ LLMs, OpenAI, Anthropic, Bedrock, Azure. Caching, guardrails & cost controls. Go-native LiteLLM & Kong AI Gateway alternative. ![Stars](https://img.shields.io/github/stars/ferro-labs/ai-gateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ferro-labs/ai-gateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ferro-labs/ai-gateway?color=green)
* **[Hebo AI Gateway](https://github.com/8monkey-ai/hebo-gateway/)**: OpenAI-compatible /chat/completions, /embeddings & /models endpoints ![Stars](https://img.shields.io/github/stars/8monkey-ai/hebo-gateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/8monkey-ai/hebo-gateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/8monkey-ai/hebo-gateway?color=green)
* **[LiteLLM](https://github.com/BerriAI/litellm)**: Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]. ![Stars](https://img.shields.io/github/stars/berriai/litellm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/berriai/litellm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/berriai/litellm?color=green)
* **[llm-cache-router](https://github.com/svalench/llm-cache-router)**: A lightweight, async-first Python library combining semantic caching, multi-provider routing (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen) and cost tracking. Cuts LLM spend 30-70% via vector-similarity cache with in-memory/Redis/Qdrant backends. ![Stars](https://img.shields.io/github/stars/svalench/llm-cache-router.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/svalench/llm-cache-router?color=green) ![LastCommit](https://img.shields.io/github/last-commit/svalench/llm-cache-router?color=green)
* **[llm-swarm-router](https://github.com/matthewdcage/llm-swarm-router)**: Self-hosted mesh router for local LLM backends — OpenAI `/v1` + Anthropic Messages on `:11400`, mDNS LAN peers. Complements cloud gateways with **local LAN mesh** for Cursor, Claude Code, Codex, and Honcho. ![Stars](https://img.shields.io/github/stars/matthewdcage/llm-swarm-router.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/matthewdcage/llm-swarm-router?color=green) ![LastCommit](https://img.shields.io/github/last-commit/matthewdcage/llm-swarm-router?color=green)
* **[LLMRouter](https://github.com/ulab-uiuc/LLMRouter)**: LLMRouter: An Open-Source Library for LLM Routing ![Stars](https://img.shields.io/github/stars/ulab-uiuc/LLMRouter.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ulab-uiuc/LLMRouter?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ulab-uiuc/LLMRouter?color=green)
* **[Ollama Herd](https://github.com/geeks-accelerator/ollama-herd)**: Routes LLM, image, speech-to-text and embedding requests across a fleet of machines you own. Auto-discovers nodes over mDNS and scores each on memory fit, thermal state, queue depth and already-loaded models. OpenAI-, Ollama- and Anthropic-compatible. ![Stars](https://img.shields.io/github/stars/geeks-accelerator/ollama-herd.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/geeks-accelerator/ollama-herd?color=green) ![LastCommit](https://img.shields.io/github/last-commit/geeks-accelerator/ollama-herd?color=green)
* **[OpenPaths](https://github.com/lee101/openpaths)**: Open-source, OpenAI-compatible model router/AI gateway routing chat, image, video, music, speech, transcription, embeddings and reasoning models to the lowest-latency provider via a unified API. ![Stars](https://img.shields.io/github/stars/lee101/openpaths.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lee101/openpaths?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lee101/openpaths?color=green)
* **[Otari](https://github.com/mozilla-ai/otari)**: Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking. ![Stars](https://img.shields.io/github/stars/mozilla-ai/otari.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mozilla-ai/otari?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mozilla-ai/otari?color=green)
* **[Respan Gateway](https://github.com/respanai/respan-gateway)**: One endpoint for every LLM vendor. Any SDK, any model, any tool. One Rust binary. ![Stars](https://img.shields.io/github/stars/respanai/respan-gateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/respanai/respan-gateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/respanai/respan-gateway?color=green)
* **[RouteLLM](https://github.com/lm-sys/routellm)**: A framework for serving and evaluating LLM routers - save LLM costs without compromising quality. ![Stars](https://img.shields.io/github/stars/lm-sys/routellm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lm-sys/routellm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lm-sys/routellm?color=green)
* **[Swobu](https://github.com/swobuforge/swobu)**: Local AI gateway for routing clients across providers, regions, accounts, and local models. ![Stars](https://img.shields.io/github/stars/swobuforge/swobu.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/swobuforge/swobu?color=green) ![LastCommit](https://img.shields.io/github/last-commit/swobuforge/swobu?color=green)
* **[vLLM Semantic Router](https://github.com/vllm-project/semantic-router)**: Intelligent Mixture-of-Models Router for Efficient LLM Inference  ![Stars](https://img.shields.io/github/stars/vllm-project/semantic-router.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vllm-project/semantic-router?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vllm-project/semantic-router?color=green)

### AI Gateway
* **[Cognipeer Console](https://github.com/Cognipeer/console)**: Open-source, self-hosted AI gateway and control plane: OpenAI-compatible LLM routing, RAG, an MCP hub, guardrails and agent tracing for multi-tenant teams. ![Stars](https://img.shields.io/github/stars/Cognipeer/console.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Cognipeer/console?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Cognipeer/console?color=green)
* **[TrustGate](https://github.com/NeuralTrust/TrustGate)**: Self-hosted Go Agent Gateway (Apache-2.0) — OpenAI-compatible LLM proxy + MCP aggregation with per-consumer auth/audit. ![Stars](https://img.shields.io/github/stars/NeuralTrust/TrustGate.svg?style=flat&color=green)

* **[agentgateway](https://github.com/agentgateway/agentgateway)**: Next Generation Agentic Proxy for AI Agents and MCP servers ![Stars](https://img.shields.io/github/stars/agentgateway/agentgateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/agentgateway/agentgateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/agentgateway/agentgateway?color=green)
* **[APISIX](https://github.com/apache/apisix)**: The Cloud-Native API Gateway and AI Gateway with extensive plugin system and AI capabilities. ![Stars](https://img.shields.io/github/stars/apache/apisix.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/apache/apisix?color=green) ![LastCommit](https://img.shields.io/github/last-commit/apache/apisix?color=green)
* **[Envoy AI Gateway](https://github.com/envoyproxy/ai-gateway)**: Envoy AI Gateway is an open source project for using Envoy Gateway to handle request traffic from application clients to Generative AI services. ![Stars](https://img.shields.io/github/stars/envoyproxy/ai-gateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/envoyproxy/ai-gateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/envoyproxy/ai-gateway?color=green)
* **[Higress](https://github.com/alibaba/higress)**: 🤖 AI Gateway | AI Native API Gateway. ![Stars](https://img.shields.io/github/stars/alibaba/higress.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/alibaba/higress?color=green) ![LastCommit](https://img.shields.io/github/last-commit/alibaba/higress?color=green)
* **[kgateway](https://github.com/kgateway-dev/kgateway)**: The Cloud-Native API Gateway and AI Gateway. ![Stars](https://img.shields.io/github/stars/kgateway-dev/kgateway.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kgateway-dev/kgateway?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kgateway-dev/kgateway?color=green)
* **[Kong](https://github.com/Kong/kong)**: 🦍 The Cloud-Native API Gateway and AI Gateway. ![Stars](https://img.shields.io/github/stars/Kong/kong.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Kong/kong?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Kong/kong?color=green)
* **[gateway-api-inference-extension](https://github.com/kubernetes-sigs/gateway-api-inference-extension)**: Gateway API Inference Extension. ![Stars](https://img.shields.io/github/stars/kubernetes-sigs/gateway-api-inference-extension.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kubernetes-sigs/gateway-api-inference-extension?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kubernetes-sigs/gateway-api-inference-extension?color=green)
* **[Gram](https://github.com/speakeasy-api/gram)**: Open-source AI control plane for connecting agents to MCPs with role-scoped access, policy enforcement, threat detection, and observability. ![Stars](https://img.shields.io/github/stars/speakeasy-api/gram.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/speakeasy-api/gram?color=green) ![LastCommit](https://img.shields.io/github/last-commit/speakeasy-api/gram?color=green)

### Output

* **[BAML](https://github.com/boundaryml/baml)**: The AI framework that adds the engineering to prompt engineering (Python/TS/Ruby/Java/C#/Rust/Go compatible) ![Stars](https://img.shields.io/github/stars/boundaryml/baml.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/boundaryml/baml?color=green) ![LastCommit](https://img.shields.io/github/last-commit/boundaryml/baml?color=green)
* **[Instructor](https://github.com/instructor-ai/instructor)**: structured outputs for llms. ![Stars](https://img.shields.io/github/stars/instructor-ai/instructor.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/instructor-ai/instructor?color=green) ![LastCommit](https://img.shields.io/github/last-commit/instructor-ai/instructor?color=green)
* **[Outlines](https://github.com/dottxt-ai/outlines)**: Structured Text Generation. ![Stars](https://img.shields.io/github/stars/dottxt-ai/outlines.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/dottxt-ai/outlines?color=green) ![LastCommit](https://img.shields.io/github/last-commit/dottxt-ai/outlines?color=green)
* **[XGrammar](https://github.com/mlc-ai/xgrammar)**: Fast, Flexible and Portable Structured Generation ![Stars](https://img.shields.io/github/stars/mlc-ai/xgrammar.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mlc-ai/xgrammar?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mlc-ai/xgrammar?color=green)

### Simulator

* **[Vidur](https://github.com/microsoft/vidur)**: A large-scale simulation framework for LLM inference ![Stars](https://img.shields.io/github/stars/microsoft/vidur.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/vidur?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/vidur?color=green)

### Benchmark

* **[genai-bench](https://github.com/sgl-project/genai-bench)**: Genai-bench is a powerful benchmark tool designed for comprehensive token-level performance evaluation of large language model (LLM) serving systems. ![Stars](https://img.shields.io/github/stars/sgl-project/genai-bench.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/sgl-project/genai-bench?color=green) ![LastCommit](https://img.shields.io/github/last-commit/sgl-project/genai-bench?color=green)
* **[Inference Benchmark](https://github.com/AI-Hypercomputer/inference-benchmark)**: A model server agnostic inference benchmarking tool that can be used to benchmark LLMs running on differet infrastructure like GPU and TPU. It can also be run on a GKE cluster as a container. ![Stars](https://img.shields.io/github/stars/AI-Hypercomputer/inference-benchmark.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/AI-Hypercomputer/inference-benchmark?color=green) ![LastCommit](https://img.shields.io/github/last-commit/AI-Hypercomputer/inference-benchmark?color=green)
* **[Inference Perf](https://github.com/kubernetes-sigs/inference-perf)**: GenAI inference performance benchmarking tool ![Stars](https://img.shields.io/github/stars/kubernetes-sigs/inference-perf.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kubernetes-sigs/inference-perf?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kubernetes-sigs/inference-perf?color=green)


## Orchestration

### Application Framework

* **[AgentMark](https://github.com/agentmark-ai/agentmark)**: Open-source, Git-native platform for building and observing reliable AI agents. Prompts and datasets in your repo, evals in CI, and OpenTelemetry tracing. ![Stars](https://img.shields.io/github/stars/agentmark-ai/agentmark.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/agentmark-ai/agentmark?color=green) ![LastCommit](https://img.shields.io/github/last-commit/agentmark-ai/agentmark?color=green)
* **[Cordum.io](https://github.com/cordum-io/cordum)**: Cordum (cordum.io) is a platform-only control plane for autonomous AI Agents and external workers. It uses NATS for the bus, Redis for state and payload pointers, and CAP v2 wire contracts for jobs, results, and heartbeats. Workers and product packs live outside this repo.Core cordum  ![Stars](https://img.shields.io/github/stars/cordum-io/cordum.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/cordum-io/cordum?color=green) ![LastCommit](https://img.shields.io/github/last-commit/cordum-io/cordum?color=green)
* **[DeepEval](https://github.com/confident-ai/deepeval)**: The LLM Evaluation Framework ![Stars](https://img.shields.io/github/stars/confident-ai/deepeval.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/confident-ai/deepeval?color=green) ![LastCommit](https://img.shields.io/github/last-commit/confident-ai/deepeval?color=green)
* **[Evidently](https://github.com/evidentlyai/evidently)**: Evidently is ​​an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics. ![Stars](https://img.shields.io/github/stars/evidentlyai/evidently.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/evidentlyai/evidently?color=green) ![LastCommit](https://img.shields.io/github/last-commit/evidentlyai/evidently?color=green)
* **[Future AGI](https://github.com/future-agi/future-agi)**: Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0. ![Stars](https://img.shields.io/github/stars/future-agi/future-agi.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/future-agi/future-agi?color=green) ![LastCommit](https://img.shields.io/github/last-commit/future-agi/future-agi?color=green)
* **[Langfuse](https://github.com/langfuse/langfuse)**: 🪢 Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets. Integrates with OpenTelemetry, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23  ![Stars](https://img.shields.io/github/stars/langfuse/langfuse.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/langfuse/langfuse?color=green) ![LastCommit](https://img.shields.io/github/last-commit/langfuse/langfuse?color=green)
* **[Helicone](https://github.com/helicone/helicone)**: 🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓 ![Stars](https://img.shields.io/github/stars/helicone/helicone.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/helicone/helicone?color=green) ![LastCommit](https://img.shields.io/github/last-commit/helicone/helicone?color=green)
* **[lunaary](https://github.com/lunary-ai/lunary)**: The production toolkit for LLMs. Observability, prompt management and evaluations.  ![Stars](https://img.shields.io/github/stars/lunary-ai/lunary.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lunary-ai/lunary?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lunary-ai/lunary?color=green)
* **[Neurolink](https://github.com/juspay/neurolink)**: Universal AI Development Platform with MCP server integration, multi-provider support, and professional CLI. Build, test, and deploy AI applications with multiple ai providers. ![Stars](https://img.shields.io/github/stars/juspay/neurolink.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/juspay/neurolink?color=green) ![LastCommit](https://img.shields.io/github/last-commit/juspay/neurolink?color=green)
* **[OpenLIT](https://github.com/openlit/openlit)**: Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs. ![Stars](https://img.shields.io/github/stars/openlit/openlit.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openlit/openlit?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openlit/openlit?color=green)
* **[phoenix](https://github.com/arize-ai/phoenix)**: AI Observability & Evaluation. ![Stars](https://img.shields.io/github/stars/arize-ai/phoenix.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/arize-ai/phoenix?color=green) ![LastCommit](https://img.shields.io/github/last-commit/arize-ai/phoenix?color=green)
* **[PostHog](https://github.com/PostHog/posthog)**: 🦔 PostHog provides open-source web & product analytics, session recording, feature flagging and A/B testing that you can self-host. Get started - free. ![Stars](https://img.shields.io/github/stars/PostHog/posthog.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/PostHog/posthog?color=green) ![LastCommit](https://img.shields.io/github/last-commit/PostHog/posthog?color=green)
* **[ragas](https://github.com/explodinggradients/ragas)**: Supercharge Your LLM Application Evaluations 🚀 ![Stars](https://img.shields.io/github/stars/explodinggradients/ragas.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/explodinggradients/ragas?color=green) ![LastCommit](https://img.shields.io/github/last-commit/explodinggradients/ragas?color=green)
* **[Weave](https://github.com/wandb/weave)**: Weave is a toolkit for developing AI-powered applications, built by Weights & Biases. ![Stars](https://img.shields.io/github/stars/wandb/weave.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/wandb/weave?color=green) ![LastCommit](https://img.shields.io/github/last-commit/wandb/weave?color=green)

### Agent Framework

* **[Agent Development Kit (ADK)](https://github.com/google/adk-python)**: An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control. ![Stars](https://img.shields.io/github/stars/google/adk-python.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/google/adk-python?color=green) ![LastCommit](https://img.shields.io/github/last-commit/google/adk-python?color=green)
* **[Agent Shadow Brain](https://github.com/theihtisham/agent-shadow-brain)**: Self-evolving AI coding intelligence with infinite memory (TurboQuant), genetic algorithm evolution, predictive bug detection, PageRank knowledge graphs, and swarm intelligence. The world's first autonomous coding brain. ![Stars](https://img.shields.io/github/stars/theihtisham/agent-shadow-brain.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/theihtisham/agent-shadow-brain?color=green) ![LastCommit](https://img.shields.io/github/last-commit/theihtisham/agent-shadow-brain?color=green)
* **[AgentField](https://github.com/Agent-Field/agentfield)**: Framework for AI Backend. Build and run AI agents like microservices - scalable, observable, and identity-aware from day one. ![Stars](https://img.shields.io/github/stars/Agent-Field/agentfield.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Agent-Field/agentfield?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Agent-Field/agentfield?color=green)
* **[Agno](https://github.com/agno-agi/agno)**: Open-source framework for building multi-agent systems with memory, knowledge and reasoning. ![Stars](https://img.shields.io/github/stars/agno-agi/agno.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/agno-agi/agno?color=green) ![LastCommit](https://img.shields.io/github/last-commit/agno-agi/agno?color=green)
* **[autogen](https://github.com/microsoft/autogen)**: A programming framework for agentic AI ![Stars](https://img.shields.io/github/stars/microsoft/autogen.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/autogen?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/autogen?color=green)
* **[AutoGPT](https://github.com/Significant-Gravitas/AutoGPT)**: AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters. ![Stars](https://img.shields.io/github/stars/significant-gravitas/autogpt.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/significant-gravitas/autogpt?color=green) ![LastCommit](https://img.shields.io/github/last-commit/significant-gravitas/autogpt?color=green)
* **[CAMEL](https://github.com/camel-ai/camel)**: 🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. ![Stars](https://img.shields.io/github/stars/camel-ai/camel.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/camel-ai/camel?color=green) ![LastCommit](https://img.shields.io/github/last-commit/camel-ai/camel?color=green)
* **[Cognipeer Agent SDK](https://github.com/Cognipeer/agent-sdk)**: Lightweight, message-first TypeScript agent runtime with planning, sub-agents, human-in-the-loop approvals, MCP tools and OpenTelemetry tracing. ![Stars](https://img.shields.io/github/stars/Cognipeer/agent-sdk.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Cognipeer/agent-sdk?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Cognipeer/agent-sdk?color=green)
* **[crewAI](https://github.com/crewAIInc/crewAI)**: Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks. ![Stars](https://img.shields.io/github/stars/crewAIInc/crewAI.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/crewAIInc/crewAI?color=green) ![LastCommit](https://img.shields.io/github/last-commit/crewAIInc/crewAI?color=green)
* **[CUGA](https://github.com/cuga-project/cuga-agent)**: CUGA is an open-source generalist agent harness for the enterprise, supporting complex task execution on web and APIs, OpenAPI/MCP integrations, composable architecture, reasoning modes, and policy-aware features. ![Stars](https://img.shields.io/github/stars/cuga-project/cuga-agent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/cuga-project/cuga-agent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/cuga-project/cuga-agent?color=green)
* **[fast-agent](https://github.com/evalstate/fast-agent)**: Define, Prompt and Test MCP enabled Agents and Workflows ![Stars](https://img.shields.io/github/stars/evalstate/fast-agent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/evalstate/fast-agent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/evalstate/fast-agent?color=green)
* **[Flowise](https://github.com/FlowiseAI/Flowise)**: Build AI Agents, Visually ![Stars](https://img.shields.io/github/stars/flowiseai/flowise.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/flowiseai/flowise?color=green) ![LastCommit](https://img.shields.io/github/last-commit/flowiseai/flowise?color=green)
* **[kagent](https://github.com/kagent-dev/kagent)**: kagent is a kubernetes native framework for building AI agents. ![Stars](https://img.shields.io/github/stars/kagent-dev/kagent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kagent-dev/kagent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kagent-dev/kagent?color=green) ![Tag](https://img.shields.io/badge/kubernetes-orange)
* **[LangGraph](https://github.com/langchain-ai/langgraph)**: Build resilient language agents as graphs. ![Stars](https://img.shields.io/github/stars/langchain-ai/langgraph.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/langchain-ai/langgraph?color=green) ![LastCommit](https://img.shields.io/github/last-commit/langchain-ai/langgraph?color=green)
* **[MetaGPT](https://github.com/geekan/MetaGPT)**: 🌟 The Multi-Agent Framework: First AI Software Company, Towards Natural Language Programming. ![Stars](https://img.shields.io/github/stars/geekan/metagpt.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/geekan/metagpt?color=green) ![LastCommit](https://img.shields.io/github/last-commit/geekan/metagpt?color=green)
* **[OpenAI Agents SDK](https://github.com/openai/openai-agents-python)**: A lightweight, powerful framework for multi-agent workflows. ![Stars](https://img.shields.io/github/stars/openai/openai-agents-python.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openai/openai-agents-python?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openai/openai-agents-python?color=green)
* **[PydanticAI](https://github.com/pydantic/pydantic-ai)**: GenAI Agent Framework, the Pydantic way ![Stars](https://img.shields.io/github/stars/pydantic/pydantic-ai.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/pydantic/pydantic-ai?color=green) ![LastCommit](https://img.shields.io/github/last-commit/pydantic/pydantic-ai?color=green)
* **[Qwen-Agent](https://github.com/QwenLM/Qwen-Agent)**: Agent framework and applications built upon Qwen>=3.0, featuring Function Calling, MCP, Code Interpreter, RAG, Chrome extension, etc. ![Stars](https://img.shields.io/github/stars/QwenLM/Qwen-Agent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/QwenLM/Qwen-Agent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/QwenLM/Qwen-Agent?color=green)
* **[Semantic Kernel](https://github.com/microsoft/semantic-kernel)**: Integrate cutting-edge LLM technology quickly and easily into your apps. ![Stars](https://img.shields.io/github/stars/microsoft/semantic-kernel.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/semantic-kernel?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/semantic-kernel?color=green)
* **[Suna](https://github.com/kortix-ai/suna)**: Suna - Open Source Generalist AI Agent ![Stars](https://img.shields.io/github/stars/kortix-ai/suna.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kortix-ai/suna?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kortix-ai/suna?color=green)
* **[Swarm](https://github.com/openai/swarm)**: Educational framework exploring ergonomic, lightweight multi-agent orchestration. Managed by OpenAI Solution team. ![Stars](https://img.shields.io/github/stars/openai/swarm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openai/swarm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openai/swarm?color=green) ![Tag](https://img.shields.io/badge/experimental-slategray)

### Evolutionary Framework

* **[AI-Researcher](https://github.com/HKUDS/AI-Researcher)**: [NeurIPS2025] "AI-Researcher: Autonomous Scientific Innovation" -- A production-ready version: https://novix.science/chat ![Stars](https://img.shields.io/github/stars/HKUDS/AI-Researcher.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/HKUDS/AI-Researcher?color=green) ![LastCommit](https://img.shields.io/github/last-commit/HKUDS/AI-Researcher?color=green)
* **[AIDE ML](https://github.com/WecoAI/aideml)**: AIDE: AI-Driven Exploration in the Space of Code. The machine Learning engineering agent that automates AI R&D. ![Stars](https://img.shields.io/github/stars/WecoAI/aideml.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/WecoAI/aideml?color=green) ![LastCommit](https://img.shields.io/github/last-commit/WecoAI/aideml?color=green)
* **[Kapso](https://github.com/Leeroo-AI/kapso)**: Long-running agents that optimize AI and Data systems, and learn from every experience. \#1 open-source on MLE-Bench; ALE-Bench; RelBench. ![Stars](https://img.shields.io/github/stars/Leeroo-AI/kapso.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Leeroo-AI/kapso?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Leeroo-AI/kapso?color=green)
* **[LoongFlow](https://github.com/baidu-baige/LoongFlow)**: LoongFlow: A Thinking & Learning Framework for Expert-Grade AI Agents. ![Stars](https://img.shields.io/github/stars/baidu-baige/LoongFlow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/baidu-baige/LoongFlow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/baidu-baige/LoongFlow?color=green)
* **[OpenEvolve](https://github.com/codelion/openevolve)**: Open-source implementation of AlphaEvolve ![Stars](https://img.shields.io/github/stars/codelion/openevolve.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/codelion/openevolve?color=green) ![LastCommit](https://img.shields.io/github/last-commit/codelion/openevolve?color=green)
* **[ShinkaEvolve](https://github.com/SakanaAI/ShinkaEvolve)**: ShinkaEvolve: Towards Open-Ended and Sample-Efficient Program Evolution ![Stars](https://img.shields.io/github/stars/SakanaAI/ShinkaEvolve.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/SakanaAI/ShinkaEvolve?color=green) ![LastCommit](https://img.shields.io/github/last-commit/SakanaAI/ShinkaEvolve?color=green)
* **[SkyDiscover](https://github.com/skydiscover-ai/skydiscover#-benchmark-performance)**: AI-Driven Scientific and Algorithmic Discovery ![Stars](https://img.shields.io/github/stars/skydiscover-ai/skydiscover.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/skydiscover-ai/skydiscover?color=green) ![LastCommit](https://img.shields.io/github/last-commit/skydiscover-ai/skydiscover?color=green)

### RAG

* **[EmbedGuard](https://github.com/neerazz/embedguard)**: Cross-layer detection and provenance attestation for adversarial embedding attacks in RAG systems; published in IJCESEN 2026 (DOI 10.22399/ijcesen.4869). ![Stars](https://img.shields.io/github/stars/neerazz/embedguard.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/neerazz/embedguard?color=green) ![LastCommit](https://img.shields.io/github/last-commit/neerazz/embedguard?color=green)
* **[GraphRAG](https://github.com/microsoft/graphrag)**: A modular graph-based Retrieval-Augmented Generation (RAG) system. ![Stars](https://img.shields.io/github/stars/microsoft/graphrag.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/graphrag?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/graphrag?color=green)
* **[LightRAG](https://github.com/HKUDS/LightRAG)**: "LightRAG: Simple and Fast Retrieval-Augmented Generation" ![Stars](https://img.shields.io/github/stars/HKUDS/LightRAG.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/HKUDS/LightRAG?color=green)  ![LastCommit](https://img.shields.io/github/last-commit/HKUDS/LightRAG?color=green)
* **[quivr](https://github.com/QuivrHQ/quivr)**: Opiniated RAG for integrating GenAI in your apps 🧠 Focus on your product rather than the RAG. Easy integration in existing products with customisation! Any LLM: GPT4, Groq, Llama. Any Vectorstore: PGVector, Faiss. Any Files. Anyway you want. ![Stars](https://img.shields.io/github/stars/QuivrHQ/quivr.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/QuivrHQ/quivr?color=green)  ![LastCommit](https://img.shields.io/github/last-commit/QuivrHQ/quivr?color=green)
* **[RAG-Anything](https://github.com/HKUDS/RAG-Anything)**: "RAG-Anything: All-in-One RAG Framework" ![Stars](https://img.shields.io/github/stars/HKUDS/RAG-Anything.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/HKUDS/RAG-Anything?color=green) ![LastCommit](https://img.shields.io/github/last-commit/HKUDS/RAG-Anything?color=green)
* **[RAGFlow](https://github.com/infiniflow/ragflow)**: RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. ![Stars](https://img.shields.io/github/stars/infiniflow/ragflow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/infiniflow/ragflow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/infiniflow/ragflow?color=green)
* **[Verbatim Citation Gate](https://github.com/Palo-Alto-AI-Research-Lab/verbatim-citation-gate)**: A zero-token verbatim check plus a burden-of-proof judge that catches fabricated RAG citations before they reach the user; framework-agnostic. ![Stars](https://img.shields.io/github/stars/Palo-Alto-AI-Research-Lab/verbatim-citation-gate.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Palo-Alto-AI-Research-Lab/verbatim-citation-gate?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Palo-Alto-AI-Research-Lab/verbatim-citation-gate?color=green)

### Workflow

* **[Dify](https://github.com/langgenius/dify)**: Production-ready platform for agentic workflow development. ![Stars](https://img.shields.io/github/stars/langgenius/dify.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/langgenius/dify?color=green) ![LastCommit](https://img.shields.io/github/last-commit/langgenius/dify?color=green)
* **[FastGPT](https://github.com/labring/FastGPT)**: FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration. ![Stars](https://img.shields.io/github/stars/labring/FastGPT.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/labring/FastGPT?color=green) ![LastCommit](https://img.shields.io/github/last-commit/labring/FastGPT?color=green)
* **[Haystack](https://github.com/deepset-ai/haystack)**: AI orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots. ![Stars](https://img.shields.io/github/stars/deepset-ai/haystack.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/deepset-ai/haystack?color=green) ![LastCommit](https://img.shields.io/github/last-commit/deepset-ai/haystack?color=green)
* **[Heym](https://github.com/heymrun/heym)**: Source-available, self-hosted visual platform for building, running, evaluating, and observing AI workflows with agents, RAG, and MCP. ![Stars](https://img.shields.io/github/stars/heymrun/heym.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/heymrun/heym?color=green) ![LastCommit](https://img.shields.io/github/last-commit/heymrun/heym?color=green)
* **[Inference](https://github.com/roboflow/inference)**: Turn any computer or edge device into a command center for your computer vision projects. ![Stars](https://img.shields.io/github/stars/roboflow/inference.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/roboflow/inference?color=green) ![LastCommit](https://img.shields.io/github/last-commit/roboflow/inference?color=green) ![Tag](https://img.shields.io/badge/computer_vision-orange)
* **[LangChain](https://github.com/langchain-ai/langchain)**: 🦜🔗 Build context-aware reasoning applications. ![Stars](https://img.shields.io/github/stars/langchain-ai/langchain.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/langchain-ai/langchain?color=green)  ![LastCommit](https://img.shields.io/github/last-commit/langchain-ai/langchain?color=green)
* **[LlamaIndex](https://github.com/run-llama/llama_index)**: LlamaIndex is the leading framework for building LLM-powered agents over your data. ![Stars](https://img.shields.io/github/stars/run-llama/llama_index.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/run-llama/llama_index?color=green) ![LastCommit](https://img.shields.io/github/last-commit/run-llama/llama_index?color=green)
* **[Nika](https://github.com/supernovae-st/nika)**: Intent-as-code AI workflow engine in a single Rust binary — reviewable YAML DAGs statically checked (schema, permits, honest cost floor) before any token is spent, tamper-evident traces after. ![Stars](https://img.shields.io/github/stars/supernovae-st/nika.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/supernovae-st/nika?color=green) ![LastCommit](https://img.shields.io/github/last-commit/supernovae-st/nika?color=green)
* **[Tale](https://github.com/tale-project/tale)**: Open-source project workspace for teams and AI agents. Assign tasks, coordinate agents in persistent sandboxes, and review results together. Self-hosted or managed cloud. ![Stars](https://img.shields.io/github/stars/tale-project/tale.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/tale-project/tale?color=green) ![LastCommit](https://img.shields.io/github/last-commit/tale-project/tale?color=green)


## Runtime

### AI Terminal

* **[aider](https://github.com/Aider-AI/aider)**: aider is AI pair programming in your terminal ![Stars](https://img.shields.io/github/stars/Aider-AI/aider.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Aider-AI/aider?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Aider-AI/aider?color=green)
* **[Atomic Agent](https://github.com/AtomicBot-ai/atomic-agent)**: Local-first CLI and TUI coding agent that runs open-weight models entirely on your machine through a llama.cpp fork, with no account or API key required. 56 built-in tools, MCP support, and a five-layer local memory system. Currently a developer preview. ![Stars](https://img.shields.io/github/stars/AtomicBot-ai/atomic-agent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/AtomicBot-ai/atomic-agent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/AtomicBot-ai/atomic-agent?color=green) ![Tag](https://img.shields.io/badge/coding-orange)
* **[Codex](https://github.com/openai/codex)**: Lightweight coding agent that runs in your terminal ![Stars](https://img.shields.io/github/stars/openai/codex.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openai/codex?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openai/codex?color=green) ![Tag](https://img.shields.io/badge/coding-orange)
* **[Crush](https://github.com/charmbracelet/crush)**: The glamourous AI coding agent for your favourite terminal 💘 ![Stars](https://img.shields.io/github/stars/charmbracelet/crush.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/charmbracelet/crush?color=green) ![LastCommit](https://img.shields.io/github/last-commit/charmbracelet/crush?color=green)
* **[Gemini CLI](https://github.com/google-gemini/gemini-cli)**: An open-source AI agent that brings the power of Gemini directly into your terminal. ![Stars](https://img.shields.io/github/stars/google-gemini/gemini-cli.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/google-gemini/gemini-cli?color=green) ![LastCommit](https://img.shields.io/github/last-commit/google-gemini/gemini-cli?color=green)
* **[OpenCode](https://github.com/sst/opencode)**: The AI coding agent built for the terminal. ![Stars](https://img.shields.io/github/stars/sst/opencode.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/sst/opencode?color=green) ![LastCommit](https://img.shields.io/github/last-commit/sst/opencode?color=green)
* **[Pydantic Deep Agents](https://github.com/vstorm-co/pydantic-deepagents)**: Terminal coding agent and Python agent framework built on Pydantic AI, with planning, sub-agents, MCP and optional Docker-sandboxed execution; works with any model, including local ones. ![Stars](https://img.shields.io/github/stars/vstorm-co/pydantic-deepagents.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vstorm-co/pydantic-deepagents?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vstorm-co/pydantic-deepagents?color=green)
* **[Stakpak](https://github.com/stakpak/agent)**: DevOps agent that won't accidentally tweet your AWS credentials 🦀 ![Stars](https://img.shields.io/github/stars/stakpak/agent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/stakpak/agent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/stakpak/agent?color=green)

### AI Agent

* **[Agent QA](https://github.com/vostride/agent-qa)**: Open-source self-improving QA agent for software teams. A test harness with memory. Write tests in natural language for web and mobile. agent-qa learns from every run, adapts to UI changes, and catches regressions before you ship. ![Stars](https://img.shields.io/github/stars/vostride/agent-qa.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vostride/agent-qa?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vostride/agent-qa?color=green)
* **[goose](https://github.com/block/goose)**: an open source, extensible AI agent that goes beyond code suggestions - install, execute, edit, and test with any LLM ![Stars](https://img.shields.io/github/stars/block/goose.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/block/goose?color=green) ![LastCommit](https://img.shields.io/github/last-commit/block/goose?color=green)
* **[Kwant](https://github.com/alexsergivan/kwant)**: Self-hosted team of AI agents in one Rust binary with @mention handoffs, scheduled routines, approval prompts, and spaces. ![Stars](https://img.shields.io/github/stars/alexsergivan/kwant.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/alexsergivan/kwant?color=green) ![LastCommit](https://img.shields.io/github/last-commit/alexsergivan/kwant?color=green)
* **[Magentic-UI](https://github.com/microsoft/magentic-ui)**: A research prototype of a human-centered web agent ![Stars](https://img.shields.io/github/stars/microsoft/magentic-ui.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/magentic-ui?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/magentic-ui?color=green)
* **[OpenManus](https://github.com/FoundationAgents/OpenManus)**: No fortress, purely open ground. OpenManus is Coming. ![Stars](https://img.shields.io/github/stars/FoundationAgents/openmanus.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/FoundationAgents/openmanus?color=green) ![LastCommit](https://img.shields.io/github/last-commit/FoundationAgents/openmanus?color=green)
* **[Tholos](https://github.com/mertkayacs/tholos)**: Local AI assistants that run on a schedule and share tables, notes and a task board, with rules that allow, ask about or deny each action. Works with llama.cpp, Ollama and OpenAI-compatible servers. ![Stars](https://img.shields.io/github/stars/mertkayacs/tholos.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mertkayacs/tholos?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mertkayacs/tholos?color=green)
* **[Tongyi Deep Research](https://github.com/Alibaba-NLP/DeepResearch)**: Tongyi DeepResearch, the Leading Open-source DeepResearch Agent ![Stars](https://img.shields.io/github/stars/Alibaba-NLP/DeepResearch.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Alibaba-NLP/DeepResearch?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Alibaba-NLP/DeepResearch?color=green)

### Code Agent

* **[Continue](https://github.com/continuedev/continue)**: ⏩ Create, share, and use custom AI code assistants with our open-source IDE extensions and hub of models, rules, prompts, docs, and other building blocks. ![Stars](https://img.shields.io/github/stars/continuedev/continue.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/continuedev/continue?color=green) ![LastCommit](https://img.shields.io/github/last-commit/continuedev/continue?color=green)
* **[Open SWE](https://github.com/langchain-ai/open-swe)**: An Open-Source Asynchronous Coding Agent ![Stars](https://img.shields.io/github/stars/langchain-ai/open-swe.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/langchain-ai/open-swe?color=green) ![LastCommit](https://img.shields.io/github/last-commit/langchain-ai/open-swe?color=green)
* **[Orbi](https://github.com/orbi-build/orbi)**: Open-source (AGPL-3.0), self-hostable coding agent for GitHub: label an Issue `ai-ready` and it implements the change, runs tests, opens a PR, and merges once a separate AI review and CI pass. ![Stars](https://img.shields.io/github/stars/orbi-build/orbi.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/orbi-build/orbi?color=green) ![LastCommit](https://img.shields.io/github/last-commit/orbi-build/orbi?color=green)
* **[Sno Station](https://github.com/sno-ai/sno-station)**: When Claude Code hits its usage limit, Codex picks the task up from the same memory. Sno Station puts Codex, Claude Code, Cursor, Hermes Agent and OpenClaw on one squad: they hand work to each other and review each other's code. Open source, runs on your computer. ![Stars](https://img.shields.io/github/stars/sno-ai/sno-station.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/sno-ai/sno-station?color=green) ![LastCommit](https://img.shields.io/github/last-commit/sno-ai/sno-station?color=green)
* **[SWE-agent](https://github.com/SWE-agent/SWE-agent)**: SWE-agent takes a GitHub issue and tries to automatically fix it, using your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges. [NeurIPS 2024]  ![Stars](https://img.shields.io/github/stars/SWE-agent/SWE-agent.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/SWE-agent/SWE-agent?color=green) ![LastCommit](https://img.shields.io/github/last-commit/SWE-agent/SWE-agent?color=green)
* **[Tabby](https://github.com/TabbyML/tabby)**: Self-hosted AI coding assistant. ![Stars](https://img.shields.io/github/stars/tabbyml/tabby.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/tabbyml/tabby?color=green) ![LastCommit](https://img.shields.io/github/last-commit/tabbyml/tabby?color=green)
* **[Kolega Code](https://github.com/kolega-ai/kolega-code)**: Terminal coding agent where the model writes its own multi-agent workflows (Gigacode); provider-agnostic with MCP support and journaled resume. ![Stars](https://img.shields.io/github/stars/kolega-ai/kolega-code.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kolega-ai/kolega-code?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kolega-ai/kolega-code?color=green)

### Tool

* **[Agent-Ready Repository Auditor](https://github.com/wrightops-ai/agent-ready-repo-auditor)**: Free GitHub Action that scores public repositories for Codex, Claude Code, Copilot, and Cursor readiness—without cloning or executing code. ![Stars](https://img.shields.io/github/stars/wrightops-ai/agent-ready-repo-auditor.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/wrightops-ai/agent-ready-repo-auditor?color=green) ![LastCommit](https://img.shields.io/github/last-commit/wrightops-ai/agent-ready-repo-auditor?color=green)
* **[Aixle Flow](https://github.com/AixleHQ/flow)**: Aixle Flow — orchestrate coding agents through durable, inspectable workflows. ![Stars](https://img.shields.io/github/stars/AixleHQ/flow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/AixleHQ/flow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/AixleHQ/flow?color=green)
* **[Beads](https://github.com/steveyegge/beads)**: Beads - A memory upgrade for your coding agent ![Stars](https://img.shields.io/github/stars/steveyegge/beads.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/steveyegge/beads?color=green) ![LastCommit](https://img.shields.io/github/last-commit/steveyegge/beads?color=green)
* **[Browser Use](https://github.com/browser-use/browser-use)**: Make websites accessible for AI agents. ![Stars](https://img.shields.io/github/stars/browser-use/browser-use.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/browser-use/browser-use?color=green) ![LastCommit](https://img.shields.io/github/last-commit/browser-use/browser-use?color=green)
* **[deja](https://github.com/vshulcz/deja-vu)**: Shared session memory for coding agents, read from the transcripts 25 agents already write on the machine — no writes to a memory store, no embeddings, one local Go binary. ![Stars](https://img.shields.io/github/stars/vshulcz/deja-vu.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vshulcz/deja-vu?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vshulcz/deja-vu?color=green)
* **[Graphiti](https://github.com/getzep/graphiti)**: Build Real-Time Knowledge Graphs for AI Agents. ![Stars](https://img.shields.io/github/stars/getzep/graphiti.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/getzep/graphiti?color=green) ![LastCommit](https://img.shields.io/github/last-commit/getzep/graphiti?color=green)
* **[Mem0](https://github.com/mem0ai/mem0)**: The Memory layer for AI Agents. ![Stars](https://img.shields.io/github/stars/mem0ai/mem0.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mem0ai/mem0?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mem0ai/mem0?color=green)
* **[Model Catalog](https://github.com/openviglet/model-catalog)**: A vendor-neutral, kind-aware catalog of LLM/embedding/rerank/media model ids and their capabilities (context window, modalities), served as a free, unauthenticated, versioned JSON API. ![Stars](https://img.shields.io/github/stars/openviglet/model-catalog.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openviglet/model-catalog?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openviglet/model-catalog?color=green)
* **[OpenAI CUA](https://github.com/openai/openai-cua-sample-app)**: Computer Using Agent Sample App. ![Stars](https://img.shields.io/github/stars/openai/openai-cua-sample-app.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openai/openai-cua-sample-app?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openai/openai-cua-sample-app?color=green)
* **[SidClaw](https://github.com/sidclawhq/platform)**: The approval and accountability layer for AI agents. Identity → Policy → Approval → Trace. 13 framework integrations. Free during early access. ![Stars](https://img.shields.io/github/stars/sidclawhq/platform.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/sidclawhq/platform?color=green) ![LastCommit](https://img.shields.io/github/last-commit/sidclawhq/platform?color=green)
* **[Vestige](https://github.com/samvallad33/vestige)**: Local-first cognitive memory MCP server for AI coding agents. SQLite, FSRS-6 retention with active forgetting and prediction-error gating. ![Stars](https://img.shields.io/github/stars/samvallad33/vestige.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/samvallad33/vestige?color=green) ![LastCommit](https://img.shields.io/github/last-commit/samvallad33/vestige?color=green)
* **[Webcmd](https://github.com/agentrhq/webcmd)**: Self-learning browser infrastructure for AI agents that compiles a site's navigation into deterministic per-site CLI commands. ![Stars](https://img.shields.io/github/stars/agentrhq/webcmd.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/agentrhq/webcmd?color=green) ![LastCommit](https://img.shields.io/github/last-commit/agentrhq/webcmd?color=green)
* **[WritBase](https://github.com/Writbase/writbase)**: MCP-native task management for AI agent fleets ![Stars](https://img.shields.io/github/stars/Writbase/writbase.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Writbase/writbase?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Writbase/writbase?color=green)

### Chatbot

* **[5ire](https://github.com/nanbingxyz/5ire)**: 5ire is a cross-platform desktop AI assistant, MCP client. It compatible with major service providers, supports local knowledge base and tools via model context protocol servers. ![Stars](https://img.shields.io/github/stars/nanbingxyz/5ire.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/nanbingxyz/5ire?color=green) ![LastCommit](https://img.shields.io/github/last-commit/nanbingxyz/5ire?color=green)
* **[AnythingLLM](https://github.com/Mintplex-Labs/anything-llm)**: The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility,  and more. ![Stars](https://img.shields.io/github/stars/Mintplex-Labs/anything-llm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Mintplex-Labs/anything-llm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Mintplex-Labs/anything-llm?color=green)
* **[Chat SDK](https://github.com/vercel/ai-chatbot)**: A full-featured, hackable Next.js AI chatbot built by Vercel ![Stars](https://img.shields.io/github/stars/vercel/ai-chatbot.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vercel/ai-chatbot?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vercel/ai-chatbot?color=green)
* **[Chatbot UI](https://github.com/mckaywrigley/chatbot-ui)**: AI chat for any model. ![Stars](https://img.shields.io/github/stars/mckaywrigley/chatbot-ui.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mckaywrigley/chatbot-ui?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mckaywrigley/chatbot-ui?color=green)
* **[Cherry Studio](https://github.com/CherryHQ/cherry-studio)**: 🍒 Cherry Studio is a desktop client that supports for multiple LLM providers. Support deepseek-r1. ![Stars](https://img.shields.io/github/stars/CherryHQ/cherry-studio.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/CherryHQ/cherry-studio?color=green) ![LastCommit](https://img.shields.io/github/last-commit/CherryHQ/cherry-studio?color=green)
* **[FastChat](https://github.com/lm-sys/FastChat)**: An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena. ![Stars](https://img.shields.io/github/stars/lm-sys/fastchat.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lm-sys/fastchat?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lm-sys/fastchat?color=green)
* **[Gradio](https://github.com/gradio-app/gradio)**: Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work! ![Stars](https://img.shields.io/github/stars/gradio-app/gradio.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/gradio-app/gradio?color=green) ![LastCommit](https://img.shields.io/github/last-commit/gradio-app/gradio?color=green)
* **[Jan](https://github.com/janhq/jan)**: Jan is an open source alternative to ChatGPT that runs 100% offline on your computer. ![Stars](https://img.shields.io/github/stars/janhq/jan.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/janhq/jan?color=green) ![LastCommit](https://img.shields.io/github/last-commit/janhq/jan?color=green)
* **[LLM](https://github.com/simonw/llm)**: Access large language models from the command-line ![Stars](https://img.shields.io/github/stars/simonw/llm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/simonw/llm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/simonw/llm?color=green)
* **[Lobe Chat](https://github.com/lobehub/lobe-chat)**: 🤯 Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / DeepSeek / Qwen), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Plugins/Artifacts) and Thinking. One-click FREE deployment of your private ChatGPT/ Claude / DeepSeek application. ![Stars](https://img.shields.io/github/stars/lobehub/lobe-chat.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lobehub/lobe-chat?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lobehub/lobe-chat?color=green)
* **[NextChat](https://github.com/ChatGPTNextWeb/NextChat)**: ✨ Light and Fast AI Assistant. Support: Web | iOS | MacOS | Android | Linux | Windows. ![Stars](https://img.shields.io/github/stars/chatgptnextweb/nextchat.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/chatgptnextweb/nextchat?color=green) ![LastCommit](https://img.shields.io/github/last-commit/chatgptnextweb/nextchat?color=green)
* **[opcode](https://github.com/getAsterisk/opcode)**: A powerful GUI app and Toolkit for Claude Code - Create custom agents, manage interactive Claude Code sessions, run secure background agents, and more. ![Stars](https://img.shields.io/github/stars/getAsterisk/opcode.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/getAsterisk/opcode?color=green) ![LastCommit](https://img.shields.io/github/last-commit/getAsterisk/opcode?color=green)
* **[Open WebUI](https://github.com/open-webui/open-webui)**: User-friendly AI Interface (Supports Ollama, OpenAI API, ...). ![Stars](https://img.shields.io/github/stars/open-webui/open-webui.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/open-webui/open-webui?color=green) ![LastCommit](https://img.shields.io/github/last-commit/open-webui/open-webui?color=green)
* **[PrivateGPT](https://github.com/zylon-ai/private-gpt)**: Interact with your documents using the power of GPT, 100% privately, no data leaks. ![Stars](https://img.shields.io/github/stars/zylon-ai/private-gpt.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/zylon-ai/private-gpt?color=green) ![LastCommit](https://img.shields.io/github/last-commit/zylon-ai/private-gpt?color=green)

### Evolve Agent

* **[AgentEvolver](https://github.com/modelscope/AgentEvolver)**: AgentEvolver: Towards Efficient Self-Evolving Agent System ![Stars](https://img.shields.io/github/stars/modelscope/AgentEvolver.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/modelscope/AgentEvolver?color=green) ![LastCommit](https://img.shields.io/github/last-commit/modelscope/AgentEvolver?color=green)
* **[EvoAgentX](https://github.com/EvoAgentX/EvoAgentX)**: 🚀 EvoAgentX: Building a Self-Evolving Ecosystem of AI Agents ![Stars](https://img.shields.io/github/stars/EvoAgentX/EvoAgentX.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/EvoAgentX/EvoAgentX?color=green) ![LastCommit](https://img.shields.io/github/last-commit/EvoAgentX/EvoAgentX?color=green)

### Database

* **[chroma](https://github.com/chroma-core/chroma)**: the AI-native open-source embedding database. ![Stars](https://img.shields.io/github/stars/chroma-core/chroma.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/chroma-core/chroma?color=green) ![LastCommit](https://img.shields.io/github/last-commit/chroma-core/chroma?color=green)
* **[deeplake](https://github.com/activeloopai/deeplake)**: Database for AI. Store Vectors, Images, Texts, Videos, etc. Use with LLMs/LangChain. Store, query, version, & visualize any AI data. Stream data in real-time to PyTorch/TensorFlow. ![Stars](https://img.shields.io/github/stars/activeloopai/deeplake.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/activeloopai/deeplake?color=green) ![LastCommit](https://img.shields.io/github/last-commit/activeloopai/deeplake?color=green)
* **[Faiss](https://github.com/facebookresearch/faiss)**: A library for efficient similarity search and clustering of dense vectors. ![Stars](https://img.shields.io/github/stars/facebookresearch/faiss.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/facebookresearch/faiss?color=green) ![LastCommit](https://img.shields.io/github/last-commit/facebookresearch/faiss?color=green)
* **[Hindsight](https://github.com/vectorize-io/hindsight)**: Hindsight: Agent Memory That  Learns ![Stars](https://img.shields.io/github/stars/vectorize-io/hindsight.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vectorize-io/hindsight?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vectorize-io/hindsight?color=green)
* **[milvus](https://github.com/milvus-io/milvus)**: Milvus is a high-performance, cloud-native vector database built for scalable vector ANN search. ![Stars](https://img.shields.io/github/stars/milvus-io/milvus.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/milvus-io/milvus?color=green) ![LastCommit](https://img.shields.io/github/last-commit/milvus-io/milvus?color=green)
* **[weaviate](https://github.com/weaviate/weaviate)**: Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database​. ![Stars](https://img.shields.io/github/stars/weaviate/weaviate.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/weaviate/weaviate?color=green) ![LastCommit](https://img.shields.io/github/last-commit/weaviate/weaviate?color=green)

### Sandbox

* **[BoxLite](https://github.com/boxlite-ai/boxlite)**: The micro-VM for AI agents — light enough to embed on your laptop, elastic enough to power an agentic cloud. ![Stars](https://img.shields.io/github/stars/boxlite-ai/boxlite.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/boxlite-ai/boxlite?color=green) ![LastCommit](https://img.shields.io/github/last-commit/boxlite-ai/boxlite?color=green)
* **[CubeSandbox](https://github.com/TencentCloud/CubeSandbox)**: Instant, Concurrent, Secure & Lightweight Sandbox for AI Agents. ![Stars](https://img.shields.io/github/stars/TencentCloud/CubeSandbox.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/TencentCloud/CubeSandbox?color=green) ![LastCommit](https://img.shields.io/github/last-commit/TencentCloud/CubeSandbox?color=green)
* **[Daytona](https://github.com/daytonaio/daytona)**: Daytona is a Secure and Elastic Infrastructure for Running AI-Generated Code. ![Stars](https://img.shields.io/github/stars/daytonaio/daytona.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/daytonaio/daytona?color=green) ![LastCommit](https://img.shields.io/github/last-commit/daytonaio/daytona?color=green)
* **[E2B](https://github.com/e2b-dev/E2B)**: Secure open source cloud runtime for AI apps & AI agents. ![Stars](https://img.shields.io/github/stars/e2b-dev/E2B.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/e2b-dev/E2B?color=green) ![LastCommit](https://img.shields.io/github/last-commit/e2b-dev/E2B?color=green)
* **[Lizard Sandboxes](https://github.com/lizard-build/lizard-sdk)**: Lizard SDK for TypeScript and Python. Boot Lizard Sandboxes, Firecracker microVMs for AI agents, then run commands, read and write files, open HTTPS ports, pause, resume and snapshot. Docs: https://lizard.build/docs/sandboxes ![Stars](https://img.shields.io/github/stars/lizard-build/lizard-sdk.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/lizard-build/lizard-sdk?color=green) ![LastCommit](https://img.shields.io/github/last-commit/lizard-build/lizard-sdk?color=green)
* **[microsandbox](https://github.com/superradcompany/microsandbox)**: Easy, fast, programmable and local-first microVM runtime and library. ![Stars](https://img.shields.io/github/stars/superradcompany/microsandbox.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/superradcompany/microsandbox?color=green) ![LastCommit](https://img.shields.io/github/last-commit/superradcompany/microsandbox?color=green)
* **[MXC](https://github.com/microsoft/mxc)**: Policy-driven, layered isolation and containment. ![Stars](https://img.shields.io/github/stars/microsoft/mxc.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/microsoft/mxc?color=green) ![LastCommit](https://img.shields.io/github/last-commit/microsoft/mxc?color=green)
* **[OpenSandbox](https://github.com/opensandbox-group/OpenSandbox)**: Secure, Fast, and Extensible Sandbox runtime for AI agents. ![Stars](https://img.shields.io/github/stars/opensandbox-group/OpenSandbox.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/opensandbox-group/OpenSandbox?color=green) ![LastCommit](https://img.shields.io/github/last-commit/opensandbox-group/OpenSandbox?color=green)
* **[SPACE](https://github.com/perplexityai/SPACE)**: Sandboxed Platform for Agentic Code Execution: pluggable-runtime sandboxes for running agent and user code in isolation. ![Stars](https://img.shields.io/github/stars/perplexityai/SPACE.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/perplexityai/SPACE?color=green) ![LastCommit](https://img.shields.io/github/last-commit/perplexityai/SPACE?color=green)

### Observation

* **[ClawMetry](https://github.com/vivekchand/clawmetry)**: Self-hosted observability for coding agents. Reads the session logs runtimes already write on disk, so there is no SDK and nothing in the request path. ![Stars](https://img.shields.io/github/stars/vivekchand/clawmetry.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/vivekchand/clawmetry?color=green) ![LastCommit](https://img.shields.io/github/last-commit/vivekchand/clawmetry?color=green)
* **[Failproof](https://github.com/FailproofAI/failproofai)**: Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement.  40 built-in policies, a local dashboard, no account required with a generous free cloud plan ![Stars](https://img.shields.io/github/stars/FailproofAI/failproofai.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/FailproofAI/failproofai?color=green) ![LastCommit](https://img.shields.io/github/last-commit/FailproofAI/failproofai?color=green)
* **[Gait](https://github.com/davidahmann/gait)**: OSS Go CLI for signed runpacks (verify, deterministic stub replay, diff), CI regressions, and policy-gated high-risk tool calls for agent workflows. ![Stars](https://img.shields.io/github/stars/davidahmann/gait.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/davidahmann/gait?color=green) ![LastCommit](https://img.shields.io/github/last-commit/davidahmann/gait?color=green)
* **[Latitude](https://github.com/latitude-dev/latitude-llm)**: Latitude is the open-source ai monitoring platform. ![Stars](https://img.shields.io/github/stars/latitude-dev/latitude-llm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/latitude-dev/latitude-llm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/latitude-dev/latitude-llm?color=green)
* **[LLMeter](https://github.com/amedinat/LLMeter)**: Open-source LLM cost monitoring with per-customer cost attribution. Polls provider billing APIs (OpenAI, Anthropic, DeepSeek, OpenRouter) directly — no proxy or code changes — with budget alerts before surprise bills. ![Stars](https://img.shields.io/github/stars/amedinat/LLMeter.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/amedinat/LLMeter?color=green) ![LastCommit](https://img.shields.io/github/last-commit/amedinat/LLMeter?color=green)
* **[Tracekit](https://github.com/Cygnux-Labs/Tracekit)**: Tamper-evident tracing, policy gates and cross-checks for AI agents: see what the agent did, and why. ![Stars](https://img.shields.io/github/stars/Cygnux-Labs/Tracekit.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Cygnux-Labs/Tracekit?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Cygnux-Labs/Tracekit?color=green)
* **[Vobo](https://github.com/getbeton/vobo)**: Open-source human review for AI output. Anchored comments and a hashed evidence chain. ![Stars](https://img.shields.io/github/stars/getbeton/vobo.svg?style=flat&color=green) ![LastCommit](https://img.shields.io/github/last-commit/getbeton/vobo?color=green)
* **[OpenLLMetry](https://github.com/traceloop/openllmetry)**: Open-source observability for your LLM application, based on OpenTelemetry. ![Stars](https://img.shields.io/github/stars/traceloop/openllmetry.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/traceloop/openllmetry?color=green) ![LastCommit](https://img.shields.io/github/last-commit/traceloop/openllmetry?color=green)
* **[OrcaReplay](https://github.com/Continuum-AI-Corp/OrcaReplay)**: Record, replay and fork debugger for AI agents. Records a run below the harness, replays it offline with the network off, and forks it from any checkpoint onto another model. ![Stars](https://img.shields.io/github/stars/Continuum-AI-Corp/OrcaReplay.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/Continuum-AI-Corp/OrcaReplay?color=green) ![LastCommit](https://img.shields.io/github/last-commit/Continuum-AI-Corp/OrcaReplay?color=green)
* **[Observatory](https://github.com/The-Context-Company/observatory)**: Open-source TypeScript and Python SDKs for instrumenting AI agents and analyzing production runs, traces, sessions, feedback, and recurring patterns. ![Stars](https://img.shields.io/github/stars/The-Context-Company/observatory.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/The-Context-Company/observatory?color=green) ![LastCommit](https://img.shields.io/github/last-commit/The-Context-Company/observatory?color=green)
* **[Rius](https://github.com/glassflow/rius-sdk-python)**: OpenTelemetry-native agent observability by GlassFlow: trace every LLM call, tool call and agent step, with cost per call, alerts, and Claude Code session tracing (Codex and Cursor in beta). MIT Python SDK for a managed backend. ![Stars](https://img.shields.io/github/stars/glassflow/rius-sdk-python.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/glassflow/rius-sdk-python?color=green) ![LastCommit](https://img.shields.io/github/last-commit/glassflow/rius-sdk-python?color=green)
* **[wandb](https://github.com/wandb/wandb)**: The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production. ![Stars](https://img.shields.io/github/stars/wandb/wandb.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/wandb/wandb?color=green) ![LastCommit](https://img.shields.io/github/last-commit/wandb/wandb?color=green)

## Training

### Framework

* **[AXLearn](https://github.com/apple/axlearn)**: An Extensible Deep Learning Library ![Stars](https://img.shields.io/github/stars/apple/axlearn.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/apple/axlearn?color=green) ![LastCommit](https://img.shields.io/github/last-commit/apple/axlearn?color=green)
* **[Candle](https://github.com/huggingface/candle)**: Minimalist ML framework for Rust. ![Stars](https://img.shields.io/github/stars/huggingface/candle.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/huggingface/candle?color=green) ![LastCommit](https://img.shields.io/github/last-commit/huggingface/candle?color=green)
* **[ColossalAI](https://github.com/hpcaitech/ColossalAI)**: Making large AI models cheaper, faster and more accessible. ![Stars](https://img.shields.io/github/stars/hpcaitech/ColossalAI.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/hpcaitech/ColossalAI?color=green) ![LastCommit](https://img.shields.io/github/last-commit/hpcaitech/ColossalAI?color=green)
* **[DLRover](https://github.com/intelligent-machine-learning/dlrover)**: DLRover: An Automatic Distributed Deep Learning System ![Stars](https://img.shields.io/github/stars/intelligent-machine-learning/dlrover.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/intelligent-machine-learning/dlrover?color=green) ![LastCommit](https://img.shields.io/github/last-commit/intelligent-machine-learning/dlrover?color=green)
* **[Ludwig](https://github.com/ludwig-ai/ludwig)**: Low-code framework for building custom LLMs, neural networks, and other AI models. ![Stars](https://img.shields.io/github/stars/ludwig-ai/ludwig.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ludwig-ai/ludwig?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ludwig-ai/ludwig?color=green)
* **[MaxText](https://github.com/google/maxtext)**: A simple, performant and scalable Jax LLM! ![Stars](https://img.shields.io/github/stars/google/maxtext.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/google/maxtext?color=green) ![LastCommit](https://img.shields.io/github/last-commit/google/maxtext?color=green)
* **[MLX](https://github.com/ml-explore/mlx)**: MLX: An array framework for Apple silicon. ![Stars](https://img.shields.io/github/stars/ml-explore/mlx.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ml-explore/mlx?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ml-explore/mlx?color=green)

### FineTune

* **[Axolotl](https://github.com/axolotl-ai-cloud/axolotl)**: Go ahead and axolotl questions. ![Stars](https://img.shields.io/github/stars/axolotl-ai-cloud/axolotl.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/axolotl-ai-cloud/axolotl?color=green) ![LastCommit](https://img.shields.io/github/last-commit/axolotl-ai-cloud/axolotl?color=green)
* **[EasyLM](https://github.com/young-geng/EasyLM)**: Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Flax. ![Stars](https://img.shields.io/github/stars/young-geng/EasyLM.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/young-geng/EasyLM?color=green) ![LastCommit](https://img.shields.io/github/last-commit/young-geng/EasyLM?color=green)
* **[LLaMa-Factory](https://github.com/hiyouga/LLaMA-Factory)**: Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024). ![Stars](https://img.shields.io/github/stars/hiyouga/llama-factory.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/hiyouga/llama-factory?color=green) ![LastCommit](https://img.shields.io/github/last-commit/hiyouga/llama-factory?color=green)
* **[LMFlow](https://github.com/OptimalScale/LMFlow)**: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All. ![Stars](https://img.shields.io/github/stars/optimalscale/lmflow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/optimalscale/lmflow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/optimalscale/lmflow?color=green)
* **[maestro](https://github.com/roboflow/maestro)**: streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL. ![Stars](https://img.shields.io/github/stars/roboflow/maestro.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/roboflow/maestro?color=green) ![LastCommit](https://img.shields.io/github/last-commit/roboflow/maestro?color=green)
* **[MLX-VLM](https://github.com/Blaizzy/mlx-vlm)**: MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX. ![Stars](https://img.shields.io/github/stars/blaizzy/mlx-vlm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/blaizzy/mlx-vlm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/blaizzy/mlx-vlm?color=green)
* **[Overmind](https://github.com/overmind-core/overmind)**: Turn production traces into specialised models you own: curate datasets from traces, evaluate, LoRA fine-tune open-weight models and serve them behind an OpenAI-compatible API. ![Stars](https://img.shields.io/github/stars/overmind-core/overmind.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/overmind-core/overmind?color=green) ![LastCommit](https://img.shields.io/github/last-commit/overmind-core/overmind?color=green)
* **[Swift](https://github.com/modelscope/ms-swift)**: Use PEFT or Full-parameter to finetune 450+ LLMs (Qwen2.5, InternLM3, GLM4, Llama3.3, Mistral, Yi1.5, Baichuan2, DeepSeek-R1, ...) and 150+ MLLMs (Qwen2.5-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL2, Phi3.5-Vision, GOT-OCR2, ...). ![Stars](https://img.shields.io/github/stars/modelscope/ms-swift?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/modelscope/ms-swift?color=green) ![LastCommit](https://img.shields.io/github/last-commit/modelscope/ms-swift?color=green)
* **[torchtune](https://github.com/pytorch/torchtune)**: PyTorch native post-training library. ![Stars](https://img.shields.io/github/stars/pytorch/torchtune.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/pytorch/torchtune?color=green) ![LastCommit](https://img.shields.io/github/last-commit/pytorch/torchtune?color=green)
* **[Transformer Lab](https://github.com/transformerlab/transformerlab-app)**: Open Source Application for Advanced LLM Engineering: interact, train, fine-tune, and evaluate large language models on your own computer. ![Stars](https://img.shields.io/github/stars/transformerlab/transformerlab-app.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/transformerlab/transformerlab-app?color=green) ![LastCommit](https://img.shields.io/github/last-commit/transformerlab/transformerlab-app?color=green)
* **[unsloth](https://github.com/unslothai/unsloth)**: Finetune Llama 3.3, DeepSeek-R1 & Reasoning LLMs 2x faster with 70% less memory! 🦥 ![Stars](https://img.shields.io/github/stars/unslothai/unsloth.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/unslothai/unsloth?color=green) ![LastCommit](https://img.shields.io/github/last-commit/unslothai/unsloth?color=green)

### RLHF

* **[OpenRLHF](https://github.com/OpenLLMAI/OpenRLHF)**: An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT). ![Stars](https://img.shields.io/github/stars/openllmai/openrlhf.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openllmai/openrlhf?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openllmai/openrlhf?color=green)
* **[Self-RLHF](https://github.com/PKU-Alignment/safe-rlhf)**: Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback. ![Stars](https://img.shields.io/github/stars/pku-alignment/safe-rlhf.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/pku-alignment/safe-rlhf?color=green) ![LastCommit](https://img.shields.io/github/last-commit/pku-alignment/safe-rlhf?color=green)

### Agentic RL

* **[AReaL](https://github.com/inclusionAI/AReaL)**: Lightning-Fast RL for LLM Reasoning and Agents. Made Simple & Flexible. ![Stars](https://img.shields.io/github/stars/inclusionAI/AReaL.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/inclusionAI/AReaL?color=green) ![LastCommit](https://img.shields.io/github/last-commit/inclusionAI/AReaL?color=green)
* **[rLLM](https://github.com/rllm-org/rllm)**: Democratizing Reinforcement Learning for LLMs ![Stars](https://img.shields.io/github/stars/rllm-org/rllm.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/rllm-org/rllm?color=green) ![LastCommit](https://img.shields.io/github/last-commit/rllm-org/rllm?color=green)
* **[slime](https://github.com/THUDM/slime)**: slime is an LLM post-training framework for RL Scaling. ![Stars](https://img.shields.io/github/stars/thudm/slime.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/thudm/slime?color=green) ![LastCommit](https://img.shields.io/github/last-commit/thudm/slime?color=green)
* **[verl](https://github.com/volcengine/verl)**: verl: Volcano Engine Reinforcement Learning for LLMs ![Stars](https://img.shields.io/github/stars/volcengine/verl.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/volcengine/verl?color=green) ![LastCommit](https://img.shields.io/github/last-commit/volcengine/verl?color=green)

### Benchmark

* **[AgentBench](https://github.com/THUDM/AgentBench)**: A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24). ![Stars](https://img.shields.io/github/stars/thudm/agentbench.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/thudm/agentbench?color=green) ![LastCommit](https://img.shields.io/github/last-commit/thudm/agentbench?color=green)
* **[ASQI Engineer](https://github.com/asqi-engineer/asqi-engineer)**: ASQI (AI Solutions Quality Index) Engineer - run containerised AI tests and map to score cards! ![Stars](https://img.shields.io/github/stars/asqi-engineer/asqi-engineer.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/asqi-engineer/asqi-engineer?color=green) ![LastCommit](https://img.shields.io/github/last-commit/asqi-engineer/asqi-engineer?color=green)
* **[Decision Models as Guardrails](https://github.com/raxITlabs/decision-models-as-guardrails)**: Can a decision model replace your guardrail? Open benchmark of 19 systems (Jev, Clef, Strands Deciders, the Perplexity and OpenAI Decisions APIs, Bedrock Guardrails) on the same 9,975 checks, scored on accuracy and cost. ![Stars](https://img.shields.io/github/stars/raxITlabs/decision-models-as-guardrails.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/raxITlabs/decision-models-as-guardrails?color=green) ![LastCommit](https://img.shields.io/github/last-commit/raxITlabs/decision-models-as-guardrails?color=green)
* **[LiveBench](https://github.com/livebench/livebench)**: LiveBench: A Challenging, Contamination-Free LLM Benchmark ![Stars](https://img.shields.io/github/stars/livebench/livebench.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/livebench/livebench?color=green) ![LastCommit](https://img.shields.io/github/last-commit/livebench/livebench?color=green)
* **[lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness)**: A framework for few-shot evaluation of language models. ![Stars](https://img.shields.io/github/stars/eleutherai/lm-evaluation-harness.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/eleutherai/lm-evaluation-harness?color=green) ![LastCommit](https://img.shields.io/github/last-commit/eleutherai/lm-evaluation-harness?color=green)
* **[LongBench](https://github.com/THUDM/LongBench)**: LongBench v2 and LongBench (ACL 2024). ![Stars](https://img.shields.io/github/stars/thudm/longbench.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/thudm/longbench?color=green) ![LastCommit](https://img.shields.io/github/last-commit/thudm/longbench?color=green)
* **[MLE-bench](https://github.com/openai/mle-bench/)**: MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering ![Stars](https://img.shields.io/github/stars/openai/mle-bench.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/openai/mle-bench?color=green) ![LastCommit](https://img.shields.io/github/last-commit/openai/mle-bench?color=green)
* **[OpenCompass](https://github.com/open-compass/opencompass)**: OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets. ![Stars](https://img.shields.io/github/stars/open-compass/opencompass.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/open-compass/opencompass?color=green) ![LastCommit](https://img.shields.io/github/last-commit/open-compass/opencompass?color=green)
* **[opik](https://github.com/comet-ml/opik)**: Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards. ![Stars](https://img.shields.io/github/stars/comet-ml/opik.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/comet-ml/opik?color=green) ![LastCommit](https://img.shields.io/github/last-commit/comet-ml/opik?color=green)
* **[skill-optimizer](https://github.com/fastxyz/skill-optimizer)**: Benchmark and self-optimize SDK/CLI/MCP guidance so every agent model can use your tool reliably. ![Stars](https://img.shields.io/github/stars/fastxyz/skill-optimizer.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/fastxyz/skill-optimizer?color=green) ![LastCommit](https://img.shields.io/github/last-commit/fastxyz/skill-optimizer?color=green)
* **[terminal-bench](https://github.com/laude-institute/terminal-bench)**: A benchmark for LLMs on complicated tasks in the terminal ![Stars](https://img.shields.io/github/stars/laude-institute/terminal-bench.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/laude-institute/terminal-bench?color=green) ![LastCommit](https://img.shields.io/github/last-commit/laude-institute/terminal-bench?color=green)

### Workflow

* **[Flyte](https://github.com/flyteorg/flyte)**: Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks. ![Stars](https://img.shields.io/github/stars/flyteorg/flyte.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/flyteorg/flyte?color=green) ![LastCommit](https://img.shields.io/github/last-commit/flyteorg/flyte?color=green)
* **[Kubeflow](https://github.com/kubeflow/kubeflow)**: Machine Learning Toolkit for Kubernetes. ![Stars](https://img.shields.io/github/stars/kubeflow/kubeflow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/kubeflow/kubeflow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/kubeflow/kubeflow?color=green)
* **[Metaflow](https://github.com/Netflix/metaflow)**: Build, Deploy and Manage AI/ML Systems. ![Stars](https://img.shields.io/github/stars/netflix/metaflow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/netflix/metaflow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/netflix/metaflow?color=green)
* **[MLflow](https://github.com/mlflow/mlflow)**: Open source platform for the machine learning lifecycle. ![Stars](https://img.shields.io/github/stars/mlflow/mlflow.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/mlflow/mlflow?color=green) ![LastCommit](https://img.shields.io/github/last-commit/mlflow/mlflow?color=green)
* **[Polyaxon](https://github.com/polyaxon/polyaxon)**:  MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle. ![Stars](https://img.shields.io/github/stars/polyaxon/polyaxon.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/polyaxon/polyaxon?color=green) ![LastCommit](https://img.shields.io/github/last-commit/polyaxon/polyaxon?color=green)
* **[Ray](https://github.com/ray-project/ray)**: Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads. ![Stars](https://img.shields.io/github/stars/ray-project/ray.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/ray-project/ray?color=green) ![LastCommit](https://img.shields.io/github/last-commit/ray-project/ray?color=green)
* **[Seldon-Core](https://github.com/SeldonIO/seldon-core)**: An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models. ![Stars](https://img.shields.io/github/stars/seldonio/seldon-core.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/seldonio/seldon-core?color=green) ![LastCommit](https://img.shields.io/github/last-commit/seldonio/seldon-core?color=green)
* **[ZenML](https://github.com/zenml-io/zenml)**: ZenML 🙏: The bridge between ML and Ops. https://zenml.io. ![Stars](https://img.shields.io/github/stars/zenml-io/zenml.svg?style=flat&color=green) ![Contributors](https://img.shields.io/github/contributors/zenml-io/zenml?color=green) ![LastCommit](https://img.shields.io/github/last-commit/zenml-io/zenml?color=green)