I created this repository as an independently authored set of companion notebooks for learners taking DeepLearning.AI's Retrieval Augmented Generation (RAG) course. It follows the course's broad five-module learning progression while exploring the concepts through Azure AI Search, Azure OpenAI, Azure AI Foundry, and credential-free local implementations.
This project is not affiliated with, endorsed by, or maintained by DeepLearning.AI. It does not reproduce or replace the official lessons, videos, notebooks, or assessments. The labs and assignments here are original materials designed to give Azure users another way to practise the underlying concepts.
20 notebooks — 14 labs, 5 graded assignments, 1 setup notebook.
Every notebook runs with or without an Azure subscription. Without credentials, the labs use local implementations of the same core techniques, so you can complete all companion labs and assignments without cloud charges.
The official course teaches the concepts and provides its own examples and assessments. Use this repository alongside it when you want to implement the same categories of RAG capability with services commonly used in Azure-based organisations.
| Official course area | Emphasis in this repository |
|---|---|
| RAG overview | A small, inspectable pipeline with citations and failure diagnosis |
| Information retrieval | BM25, semantic search, hybrid retrieval, and metrics |
| Vector databases | HNSW concepts and Azure AI Search vector indexes |
| LLMs and generation | Azure OpenAI, grounded prompting, and agentic RAG |
| Production RAG | Azure AI evaluation, observability, cost, security, and scaling |
A practical rhythm is to study an official module, run the corresponding notebooks here, and compare the design choices. Topic alignment does not imply exercise-for-exercise equivalence.
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
jupyter lab setup/00_environment_setup.ipynbThat notebook checks your environment, downloads the corpus, and tells you what will run in Azure mode versus local mode. Start there.
To connect real Azure resources (optional):
./setup/provision.sh # API keys
./setup/provision.sh --keyless # Entra ID + RBAC, no keys on disk
./setup/provision.sh --subscription <id> # pick the subscription explicitlyRequires the Azure CLI and az login. It creates a resource group, an Azure AI
Search service, an Azure OpenAI account with two model deployments, and writes
your .env. It prints which subscription it is using before it creates
anything.
Already have Azure resources? Copy .env.example to .env and fill in what
you have — one service at a time is fine, and anything you leave as a
<placeholder> falls back to local mode. When you're done:
./setup/provision.sh --destroy| Module | Notebooks | Topics |
|---|---|---|
| 0 · Setup | 00_environment_setup |
Environment check, corpus download, cost expectations |
| 1 · RAG Overview | lab1_1 · lab1_2 · assignment |
The three components; diagnosing which stage failed; first pipeline on Azure AI Search |
| 2 · Information Retrieval | lab2_1 · lab2_2 · lab2_3 · assignment |
Inverted indexes, TF‑IDF, BM25 from scratch; embeddings; hybrid search + RRF; recall/precision/MRR/nDCG |
| 3 · Vector Databases | lab3_1 · lab3_2 · lab3_3 · assignment |
HNSW built from scratch; quantization & truncation; Azure vector indexes; chunking strategy; reranking & late interaction |
| 4 · LLMs and Generation | lab4_1 · lab4_2 · lab4_3 · assignment |
Temperature/top‑k/top‑p; token budgets; grounded prompting; hallucination detection; tool-calling agents; Foundry Agents |
| 5 · Production | lab5_1 · lab5_2 · lab5_3 · assignment |
Azure AI Evaluation SDK; LLM-as-judge; tracing & cost models; caching; prompt injection; access control; scaling |
Each lab ends with a Key points summary. Each assignment is auto-graded with a pass mark of 80% and has worked solutions at the bottom.
| Azure mode | Local mode | |
|---|---|---|
| Retrieval | Azure AI Search (BM25, HNSW, semantic ranker) | in-memory BM25 + exact KNN |
| Embeddings | text-embedding-3-* |
TF‑IDF + truncated SVD (LSA) |
| Generation | Azure OpenAI chat deployment (gpt-5-mini by default) |
extractive sentence selector |
| Cost | pay per token + hourly service | zero |
Local mode is not a stub — it's a working implementation of the same algorithms. What you lose is the managed service itself: real ANN indexes at scale, the trained cross-encoder reranker, and fluent generation.
Suggested path: Modules 1–2 in local mode (they're about algorithms), then provision Azure for Modules 3–5 where the managed service is the subject.
Set RAG_COURSE_FORCE_LOCAL=1 to disable Azure calls even when credentials
exist.
25 documents of real Azure AI technical documentation, downloaded from the
public MicrosoftDocs/azure-ai-docs
repository and cached in data/corpus.json, which is tracked in git.
The download is pinned to a commit, not to main. That repository changes
daily, and the corpus is an input to nearly every number these labs print — so
a floating corpus would mean your BM25 scores, your chunk counts and your
recall figures quietly disagree with the text explaining them. To move to a
newer snapshot, bump CORPUS_COMMIT in ragcourse/corpus.py, run
python -m ragcourse.corpus --refresh, and run the tests to see what moved.
The Azure AI Search articles are fetched as real Markdown. The Azure OpenAI articles in that repository are INCLUDE-transclusion stubs of a few hundred bytes, so those five documents ship bundled with this repository. If the download fails entirely, a complete offline corpus is used and nothing breaks.
Evaluation uses 15 queries with heuristic relevance labels — a chunk is relevant if it contains the query's required terms. These are "silver" labels, and the notebooks are explicit about what that costs you: Module 2 shows how to sanity-check them, Module 3 shows an experiment they distort, and Module 5 replaces them with an LLM judge and compares the two.
ragcourse/ shared package — import this from any notebook
config.py environment detection and capability flags
corpus.py corpus download, caching, chunking, gold queries
fallback_corpus.py bundled offline documents
embeddings.py AzureOpenAIEmbedder + LocalLSAEmbedder (disk-cached)
llm.py AzureOpenAIChat + LocalExtractiveLLM, cost tracking
search_local.py BM25Index, VectorIndex, HybridIndex, RRF
azure_search.py index creation, upload, and querying
rag.py RAGPipeline, prompt assembly, RAGAnswer
metrics.py recall/precision/MRR/nDCG/MAP + groundedness proxies
grading.py the assignment auto-grader
setup/
00_environment_setup.ipynb
provision.sh az CLI provisioning + teardown
build_notebooks.py headless execution check for every notebook
tests/ offline test suite (see Verification)
data/
corpus.json tracked: pins the exact corpus every learner retrieves over
embedding_cache/ generated, untracked
eval_data.jsonl generated by Module 5 Lab 1, untracked
Every claim in this section is checked automatically on each push, on Python
3.11 and 3.12, against both the declared minimum dependency versions and the
pinned set in requirements-lock.txt. Nothing in CI touches Azure.
pip install -r requirements-dev.txt
pytest # unit tests + assignment solutions
pytest -m "not slow" # skip the notebook-executing cases
python setup/build_notebooks.py --execute # all 20 notebooks, offlineAll 20 notebooks execute end to end in local mode with no errors — about 80 seconds for the full sweep.
Every assignment's worked solutions score full marks against its own
grader. This is a real test, not a note: tests/test_assignment_solutions.py
lifts the reference implementations out of each notebook's Solutions section,
splices them into the exercise cells, runs the notebook, and fails if a single
check is left unpassed.
| Assignment | Worked solutions score |
|---|---|
| Module 1 | 20.5 / 20.5 |
| Module 2 | 23 / 23 |
| Module 3 | 20.5 / 20.5 |
| Module 4 | 22.5 / 22.5 |
| Module 5 | 22.5 / 22.5 |
These are local-mode totals. They are also fixed totals: a second test executes each assignment with every answer left untouched and fails unless it is scored out of exactly the same denominator. That matters because the pass mark is a percentage — if skipping an exercise also removed its points from the total, 80% would mean something different for every attempt.
Notebooks are shipped without saved outputs — you run the cells.
Nothing above touches Azure, so none of the Azure branches — index creation,
upload, the four query modes, the embedding and chat clients — is covered by a
test that runs automatically. setup/azure_smoke_test.py covers them against
real resources:
python setup/azure_smoke_test.py --yesIt costs a few cents, builds a scratch index named <your-index>-smoke so it
cannot disturb the one the notebooks build, and deletes it afterwards. Worth
running after provisioning and after any SDK version bump.
Verified against azure-search-documents 12.0.0 (April 2026) and
11.6.0. The notebooks deliberately use only the API surface common to both,
so either line works. Note that 12.0.0 removed serialize()/deserialize()
in favour of as_dict(), and several compression model names moved — Module 3
shows how to inspect what your installed version actually exposes.
Also verified: azure-ai-projects 2.4.0, azure-ai-agents 1.1.0,
azure-ai-evaluation 1.18.3, openai 2.52.0.
Several labs produce results that contradict common advice:
- Reranking makes things worse on this corpus (Module 3, Lab 3) — and the lab explains why rather than tuning until it looks good.
- Standard retrieval metrics cannot choose a chunk size, because the labels move with the chunking. Module 3 builds a boundary-independent measure instead.
- A semantic cache scores an antonym higher than a true paraphrase (Module 5, Lab 2), so no threshold separates them safely.
These are kept as they are. The habit these companion materials try to build is measuring techniques on your own data rather than adopting them because they're well-regarded.
If you run all companion notebooks in Azure mode:
| Item | Approximate cost |
|---|---|
| Azure AI Search, Basic tier | ~$0.10/hour, billed while the service exists |
| Embedding the corpus (~350 chunks, once) | under $0.01 |
| All chat generation across the labs | a few cents with gpt-5-mini |
| Semantic ranker | free tier covers 1,000 queries/month |
The search service is the only meaningful cost and it accrues hourly whether or
not you use it. Embeddings are cached to data/embedding_cache/ keyed by a hash
of the text, so re-running a notebook doesn't re-bill you.
Run ./setup/provision.sh --destroy when you finish.
The code, labs, assignments, and bundled fallback prose were written for this repository and are released under the MIT License. The high-level module organisation was informed by DeepLearning.AI's public RAG course syllabus; no official course material is included. When online, the corpus loader downloads and transforms selected articles from MicrosoftDocs/azure-ai-docs, which are available under CC BY 4.0. See Third-Party Notices for full attribution and the non-affiliation statement.