Pure-Rust AI pipeline system with multi-modal model support and graph-backed knowledge retrieval.
AI-GraphDB-Engine is a modular, high-performance AI orchestration framework that combines:
- Multi-modal AI models — Text (Llama GGUF), Audio (Whisper), Image (CLIP)
- Graph-based knowledge — Community detection, GraphRAG retrieval, JEPA world models
- Production pipelines — RAG, tool calling, structured extraction, agentic orchestration
- Native interfaces — Full-featured CLI and interactive TUI
- Cross-platform GPU support — NVIDIA CUDA, AMD ROCm, Apple Metal, Intel MKL
The system is built as a Cargo workspace with pure-Rust implementations, ensuring safety, performance, and easy deployment without Python runtime dependencies.
graph-db-engine/
├── rust/ ← Core graph engine (graphdb_rs)
│ • GraphStore, Node, Edge persistence
│ • Community detection (Louvain)
│ • GraphRAG (local/global search)
│ • JEPA world model (VICReg, latent search)
│
├── model-hub/ ← AI pipeline library (model-hub)
│ • TextModel, AudioModel, ImageModel traits
│ • Llama GGUF / Whisper / CLIP via candle
│ • RAG, tool calling, extraction pipelines
│ • Needle engine (heuristic-based routing)
│ • GraphBackend abstraction
│
├── cli/ ← Command-line interface (graphdb-cli)
│ • Subcommands: generate, embed, rag, extract, ask, transcribe, info
│ • Engine toggle: --engine llm|needle
│ • GPU device selection: --device cuda|metal|cpu
│
└── tui/ ← Terminal UI (graphdb-tui)
• Interactive modes: Chat, RAG, Extract, Info
• Live engine switching (Ctrl+E)
• Message history, status bar
The python/ directory contains the original Python implementation:
- Core graph engine —
ai_memory/with GraphStore, embeddings, JEPA-GraphRAG - Training dataset pipeline — Multi-format document loader, graph builder, export to JSONL/Alpaca/OpenAI formats
- Needle2 integration —
NeedleAgentGroup,NeedleOrchestratorfor trainable function-calling agents - Web UI — Flask dashboard with 2D/3D graph visualization (vis-network + 3d-force-graph)
See python/README.md and python/SDK.md for Python-specific documentation.
| Modality | Implementation | Backend | Status |
|---|---|---|---|
| Text | Llama (GGUF quantized) | candle-transformers |
✅ Implemented |
| Audio | Whisper (GGUF) | candle-transformers |
🔧 Stub (API ready) |
| Image | CLIP | candle-transformers |
🔧 Stub (API ready) |
All models support offline operation via stub implementations for testing without downloading weights.
- RAG — Vector retrieval → BFS neighborhood expansion → context-augmented generation
- Tool Calling — Needle-compatible schema, deterministic routing or LLM-based dispatch
- Structured Extraction — JSON schema enforcement, entity extraction, knowledge triples
- Agentic Orchestration — Multi-turn tool use, conversation memory, hybrid search
- Community Detection — Louvain algorithm (pure Rust)
- GraphRAG Retrieval — Local search (k-hop) + global search (community scoring)
- JEPA World Model — Context/target encoders, VICReg loss, latent-space retrieval
- Hybrid Search — Combines vector, graph, and learned latent modes
Switch between two inference engines:
| Engine | Use Case | Latency | Offline |
|---|---|---|---|
| LLM | Complex reasoning, generation | ~seconds | |
| Needle | Intent detection, routing | ~milliseconds | ✅ Always |
Toggle via --engine llm or --engine needle (CLI) or Ctrl+E (TUI).
| Feature Flag | Hardware | Notes |
|---|---|---|
cuda |
NVIDIA CUDA (sm_70+) | GeForce RTX 20-series+ |
cudnn |
NVIDIA cuDNN | Faster convolutions |
rocm |
AMD ROCm (HIP) | Radeon RX 5000+ |
metal |
Apple Metal | M1/M2/M3, Intel integrated |
mkl |
Intel MKL | CPU + Intel Arc partial |
flash-attn |
Flash Attention 2 | CUDA only, 2-4× faster |
- Rust 1.75+ — Install from rustup.rs
- Git — For cloning the repository
# Clone the repository
git clone https://github.com/frederico-tech6868/ai-graph-db-engine.git
cd ai-graph-db-engine
# Build the workspace (CPU-only by default)
cargo build --release --workspace
# Or enable GPU support (example: NVIDIA CUDA)
cargo build --release --workspace --features cuda# Show system info and available backends
cargo run --release --bin graphdb-cli -- info
# Generate text with a stub model (no GGUF needed)
cargo run --release --bin graphdb-cli -- generate "Explain JEPA in one sentence."
# Extract named entities using the Needle engine
cargo run --release --bin graphdb-cli -- \
--engine needle \
extract --schema entities "Apple was founded by Steve Jobs in Cupertino."
# Ask a question with tool routing (stub model)
cargo run --release --bin graphdb-cli -- \
ask "What documents are available?"With a real model:
# Download a GGUF model (example: Llama-3.2-1B)
# See https://huggingface.co/models?library=gguf for options
# Generate text with the model
cargo run --release --bin graphdb-cli -- \
--model /path/to/model.gguf \
generate "Hello, world!"
# RAG search over a knowledge graph
cargo run --release --bin graphdb-cli -- \
--model /path/to/model.gguf \
rag "What is JEPA?" --top-k 5 --hops 2# Launch the interactive terminal UI
cargo run --release --bin graphdb-tui
# Or with a model loaded
cargo run --release --bin graphdb-tui -- --model /path/to/model.ggufTUI Controls:
| Key | Action |
|---|---|
1 / 2 / 3 / 4 |
Switch mode (Chat / RAG / Extract / Info) |
Tab |
Cycle modes |
Ctrl+E |
Toggle engine (LLM ↔ Needle) |
Enter |
Submit input |
Ctrl+Q / Esc |
Quit |
# Ensure CUDA Toolkit 11.8+ is installed
# https://developer.nvidia.com/cuda-downloads
cargo build --release --workspace --features cuda
# Enable Flash Attention 2 for faster inference (RTX 30-series+)
cargo build --release --workspace --features "cuda,flash-attn"# Ensure ROCm 5.4+ is installed
# https://rocm.docs.amd.com/
cargo build --release --workspace --features rocm# Works out-of-the-box on macOS
cargo build --release --workspace --features metal# Install Intel oneAPI Math Kernel Library
# https://www.intel.com/content/www/us/en/developer/tools/oneapi/onemkl.html
cargo build --release --workspace --features mkl# 1. Prepare a model (download a GGUF file)
wget https://huggingface.co/TheBloke/Llama-2-7B-GGUF/resolve/main/llama-2-7b.Q4_K_M.gguf
# 2. Run RAG with seed retrieval + BFS expansion
cargo run --release --bin graphdb-cli -- \
--model llama-2-7b.Q4_K_M.gguf \
--device cuda \
rag "What are the main features of Rust?" \
--top-k 10 \
--hops 2# Extract entities using the LLM engine
cargo run --release --bin graphdb-cli -- \
--model llama-2-7b.Q4_K_M.gguf \
extract --schema entities \
"OpenAI released GPT-4 in March 2023. Sam Altman is the CEO."
# Output (JSON):
# {
# "entities": [
# { "name": "OpenAI", "type": "Organization" },
# { "name": "GPT-4", "type": "Product" },
# { "name": "Sam Altman", "type": "Person", "context": "CEO of OpenAI" }
# ]
# }# Ask a question — the system routes to tools automatically
cargo run --release --bin graphdb-cli -- \
--engine needle \
ask "Search for documents about machine learning"
# The Needle engine will:
# 1. Detect intent → search_knowledge_base
# 2. Extract query parameter → "machine learning"
# 3. Execute the tool
# 4. Format the response# Start the TUI with a model
cargo run --release --bin graphdb-tui -- --model llama-2-7b.Q4_K_M.gguf
# Inside the TUI:
# 1. Press '2' to switch to RAG mode
# 2. Type your query: "Explain RAG"
# 3. Press Enter
# 4. Watch the model retrieve context and generate an answer
# 5. Press Ctrl+E to switch to Needle engine for faster responsesExports: graphdb_rs
Pure-Rust graph database with PyO3 Python bindings.
Key modules:
core— Node, Edge, PropertyValuestore— GraphStore (CRUD, indexes, persistence)community— Louvain algorithmgraphrag— Local/global searchjepa— JEPA world model (encoders, VICReg, latent search)vector— Cosine similarity, top-k searchquery— BFS, DFS, path finding
Python bridge: Exposes PyGraphStore, PyCommunityDetector, PyGraphRAGRetriever, PyJEPAGraphRAG to Python.
Exports: model_hub
Pure-Rust AI orchestration framework.
Key modules:
backend— Device selection (CPU/CUDA/Metal/MKL)models— TextModel, AudioModel, ImageModel traits + Llama/Whisper/CLIPgraph— GraphBackend trait + InMemoryGraphpipeline— RAG, ToolExecutor, StructuredExtractor, Orchestratorneedle— NeedleAgent (heuristic-based routing)hub— ModelHub (registry + GGUF loader)
Dependencies: candle-core, candle-nn, candle-transformers, tokenizers, hf-hub
Binary: graphdb-cli
Full-featured CLI with subcommands for all pipeline operations.
Subcommands:
generate— Text completionembed— Embedding extractionrag— Retrieval-augmented generationextract— Structured JSON extractionask— Orchestrated tool callingtranscribe— Audio transcription (Whisper)info— System information
Global flags:
--model <PATH>— GGUF model file--device <cpu|cuda|metal|mkl>— Compute device--engine <llm|needle>— Inference engine--verbose— Debug output
Binary: graphdb-tui
Interactive ratatui-based TUI with four modes.
Features:
- Real-time message history
- Mode switching (Chat / RAG / Extract / Info)
- Engine toggle (LLM ↔ Needle)
- Keyboard-driven navigation
- Status bar with model info
Tech stack: ratatui, crossterm
- Python SDK —
python/SDK.md· PDF · DOCX - Python Setup —
python/README.md - Web UI —
python/webui/README.md - Rust API docs — Run
cargo doc --workspace --open
# Run all Rust tests (graphdb_rs + model-hub)
cargo test --workspace
# Test a specific crate
cargo test --package graphdb_rs
cargo test --package model-hub
# Run with GPU features
cargo test --workspace --features cudacd python
# Using uv (recommended)
uv pip install -e ".[dev]"
uv run pytest tests/ -v
# Using pip
pip install -e ".[dev]"
pytest tests/ -v| Device | Throughput | Latency (p50) | Notes |
|---|---|---|---|
| CPU (Intel i7-12700K) | ~8 tok/s | 64 ms | Single-threaded |
| CUDA (RTX 4090) | ~95 tok/s | 11 ms | FP16 quantized |
| Metal (M2 Max) | ~42 tok/s | 24 ms | Unified memory |
| MKL (Intel Arc A770) | ~18 tok/s | 56 ms | Partial acceleration |
Measured with --features cuda,flash-attn on CUDA. Your mileage may vary.
| Model | Size | RAM (CPU) | VRAM (GPU) |
|---|---|---|---|
| Llama-3.2-1B (Q4) | 0.8 GB | ~1.2 GB | ~1.0 GB |
| Llama-2-7B (Q4) | 3.8 GB | ~4.5 GB | ~4.2 GB |
| Llama-2-13B (Q4) | 7.4 GB | ~8.0 GB | ~7.8 GB |
- Full Whisper GGUF support — Complete audio transcription pipeline
- CLIP image encoder — Enable multi-modal retrieval
- LoRA fine-tuning — Train custom adapters on domain-specific data
- Persistent GraphBackend — SQLite/RocksDB storage for production graphs
- Web API — REST/gRPC server for CLI/TUI functionality
- Distributed graph — Sharding + federation for large-scale knowledge bases
- Agent workflows — DAG-based multi-agent orchestration
- Model hub integration — Auto-download from HuggingFace Hub
- Benchmarking suite — Standardized performance metrics
- Documentation site — mdBook-based guides + API reference
- Custom GGUF quantization — Train and export models in Rust
- Federated learning — Privacy-preserving collaborative training
- Production deployment guides — Kubernetes, Docker Compose, systemd
- Language bindings — FFI for Go, Node.js, .NET
We welcome contributions! Please see CONTRIBUTING.md for guidelines.
Areas of interest:
- GPU backend optimization (CUDA kernels, Metal Performance Shaders)
- New model implementations (Mistral, Qwen, Gemma)
- Pipeline improvements (better prompt templates, retrieval strategies)
- Documentation and examples
- Bug reports and feature requests
This project is licensed under the MIT License — see LICENSE for details.
- candle — Pure-Rust ML framework by Hugging Face
- llama.cpp — GGUF format and quantization techniques
- ratatui — Terminal UI framework
- PyO3 — Rust ↔ Python bindings
- vis-network & 3d-force-graph — Graph visualization
- GitHub: frederico-tech6868/ai-graph-db-engine
- Issues: Issue Tracker