lab-dl

Download HackTheBox Academy modules and TryHackMe rooms as clean Markdown for personal offline study.

Install

uv sync

Use

uv run python main.py htb 293                 # HTB module by id or URL
uv run python main.py thm csrfintroduction    # THM room by slug or URL
uv run python main.py htb path 419            # whole job-role path, resumable (-y skips prompts)
uv run python main.py htb 293 --dry-run       # check auth, write nothing

You can skip cookie setup if you're logged in to either site through a Firefox-based browser (Floorp, Firefox, LibreWolf, Zen, Waterfox). The scraper reads the session from your profile and caches it to cookies.txt or cookies-thm.txt, both gitignored, and re-grabs it automatically when a cached copy expires.

To supply the cookie yourself instead, pass --cookie "htb_academy_session=..." or put it in cookies.txt (see cookies.txt.example).

A path run asks twice: once before it starts downloading, and once at the end to offer a .tar.xz beside the folder. With -y it downloads without asking and skips the archive.

[23:50:32] [ACTION] Fetching path 419 metadata…
[23:50:36] [INFO] Junior Cybersecurity Analyst (20 modules)
╭────┬─────┬──────────────────────────────────────╮
│  # │ ID  │ Module                               │
├────┼─────┼──────────────────────────────────────┤
│  1 │ 293 │ Introduction to Information Security │
│  2 │ 289 │ Network Foundations                  │
╰────┴─────┴──────────────────────────────────────╯
[23:50:36] [INPUT] Download 20 module(s) into output/? [y/N] y
[23:50:36] [INFO]  1/20 █░░░░░░░░░░░ 293 Introduction to Information Security (skipped)
[23:50:41] [ACTION]  2/20 █░░░░░░░░░░░ 289 Network Foundations
[23:51:02] [SUCCESS] saved 13 file(s) to 02-289-Network-Foundations/
[23:53:18] [SUCCESS] The path saved to /home/you/lab-dl/output/419-Junior-Cybersecurity-Analyst
[23:53:18] [INPUT] Archive it to 419-Junior-Cybersecurity-Analyst.tar.xz? [y/N]

A second run skips the modules that already finished, so an interrupted one picks up where it stopped. Pass --force to download them again.

Output

output/
  293-Introduction-to-Information-Security/
    README.md              # module metadata + contents
    01-Introduction.md
    02-InfoSec-Domains.md
    assets/                # images, downloaded locally
  csrfintroduction.md      # THM rooms are one file each

HTB pages mix Markdown with embedded HTML, so converter.py converts just the fragments (alerts, cards, images, inline tags) instead of running the whole page through an HTML engine that would flatten the Markdown around them. THM pages are plain HTML, so they go straight through markdownify. Both download images and rewrite the links to local paths.

Flags

Flag	Purpose
--cookie "..."	Inline Cookie: header.
--cookie-file PATH	Read the header from a file.
--reload-cookie	Force a fresh grab from the browser.
--output DIR	Output directory (default ./output).
--timeout N	HTTP timeout in seconds (default 30).
--dry-run	List sections or tasks, write nothing.

HTB only: --debug-json dumps the raw API response, --no-walkthrough skips the "Show solution" file, and --no-jitter drops the polite inter-section sleep.

Notes

• If a module is locked (is_unlocked == false, progress == 0) the tool refuses to run, so you cannot spend a cube by accident. Unlock it in the Academy UI first.
• A few images only resolve on HTB's CDN. The downloader retries there automatically; re-run if any fail.
• CLAUDE.md documents the API layout and conventions in detail.

Disclaimer

For personal study only. HTB Academy and TryHackMe content is proprietary. Export only what you have legitimate access to, and do not upload, share, or redistribute it. The contributors are not responsible for misuse.