Lightcrawl

Quick start

1. Set an API key in docker-compose.yml:

environment:
  LIGHTCRAWL_API_KEY: your-secret-api-key

2. Start the services:

docker compose up --build -d

The API will be available at:

http://localhost:8000

Scrape a page

curl -X POST http://localhost:8000/v2/scrape \
  -H "Authorization: Bearer your-secret-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "formats": ["markdown", "html"]
  }'

Request body

Field	Type	Description
url	string	URL to scrape
formats	array	Output formats: markdown, html, or both

If formats is omitted, Markdown is returned by default.

Response

{
  "success": true,
  "data": {
    "markdown": "# Example Domain",
    "metadata": {
      "title": "Example Domain",
      "sourceURL": "https://example.com"
    }
  }
}