firecrawl-proxidize

Community integration. This is an open-source integration maintained by Proxidize, not an official Firecrawl integration or partnership. It is not affiliated with, endorsed by, or officially supported by the Firecrawl project.

Configuration, scripts, and docs for running self-hosted Firecrawl through a Proxidize residential or mobile proxy. This repo does not contain a copy of Firecrawl — it clones the official repo for you and layers proxy configuration, verification scripts, and docs on top.

What this does

• Points you at the official Firecrawl self-hosting setup (Docker Compose).
• Gives you a tested .env.example with the Proxidize variables Firecrawl actually supports.
• Provides scripts to start Firecrawl, run a scrape/crawl, and verify your traffic is actually going through Proxidize (not just assuming it).
• Documents residential, mobile, sticky-session, and geo-targeting configuration, based on testing against a real self-hosted instance.

Requirements

• Git
• Docker Desktop (includes the docker compose v2 plugin)
• curl
• jq
• A Proxidize residential or mobile proxy endpoint (host, port, username, password)

Run ./scripts/check-requirements.sh to verify all of the above.

30-second setup

git clone <this-repo-url> firecrawl-proxidize
cd firecrawl-proxidize
./scripts/check-requirements.sh
cp .env.example firecrawl/.env   # after scripts/start-firecrawl.sh has cloned firecrawl/ once — see below

In practice, run it in this order the first time:

./scripts/check-requirements.sh
./scripts/start-firecrawl.sh          # clones firecrawl/ if missing, then stops because .env is missing
cp .env.example firecrawl/.env        # now edit firecrawl/.env with your real Proxidize credentials
./scripts/start-firecrawl.sh          # builds and starts Firecrawl
./scripts/test-scrape.sh
./scripts/verify-proxy.sh

Cloning the official Firecrawl repo

scripts/start-firecrawl.sh does this for you (into ./firecrawl, gitignored by this repo). To do it manually instead:

git clone https://github.com/firecrawl/firecrawl.git
cd firecrawl

This repo intentionally does not vendor or copy Firecrawl's source — it always works against a fresh clone of the official repo, so you're never running an outdated fork.

Configuring Firecrawl's .env

Copy .env.example from this repo into the Firecrawl clone as .env:

cp .env.example firecrawl/.env

Then edit firecrawl/.env and fill in your real values:

PORT=3002
HOST=0.0.0.0
USE_DB_AUTHENTICATION=false
BULL_AUTH_KEY=change-this-value

PROXY_SERVER=http://your-proxidize-host:port
PROXY_USERNAME=your-proxidize-username
PROXY_PASSWORD=your-proxidize-password

These three PROXY_* variables are the only proxy configuration Firecrawl's official Docker Compose file recognizes. It wires them automatically into both the api and playwright-service containers — you do not need to edit docker-compose.yaml.

.env is git-ignored. Never commit it.

Starting Firecrawl

./scripts/start-firecrawl.sh

Or manually:

cd firecrawl
docker compose build
docker compose up -d
docker compose ps

Firecrawl is then available at http://localhost:3002. With USE_DB_AUTHENTICATION=false, no API key is required for local requests.

Scrape example

./scripts/test-scrape.sh https://firecrawl.dev

Or directly:

curl -X POST http://localhost:3002/v2/scrape \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://firecrawl.dev", "formats": ["markdown"]}'

Crawl example

./scripts/test-crawl.sh https://firecrawl.dev 5

Or directly:

curl -X POST http://localhost:3002/v2/crawl \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://firecrawl.dev", "limit": 5}'

Verifying your proxy is actually being used

Don't take it on faith — check the exit IP:

./scripts/verify-proxy.sh

This compares your machine's direct IP against Firecrawl's scrape-reported exit IP over several requests, reports whether they differ, and whether the exit IP rotates. It never prints your proxy credentials — only whether they're set.

Residential proxies

See docs/residential-proxies.md — configuration, confirmed rotating behavior, and geographic targeting via a -country-<code> username suffix.

Mobile proxies

See docs/mobile-proxies.md — configuration, confirmed rotating behavior, and notes on transient carrier-side errors (expected, retry-recoverable).

Sticky sessions

See docs/sticky-sessions.md — the confirmed -s-<session-id>- username syntax for a stable exit IP (client-controlled, no dashboard step required), for both residential and mobile.

Troubleshooting

See docs/troubleshooting.md for Docker issues, port conflicts, connection errors, proxy auth failures, special characters in credentials, timeouts, geo-targeting issues, and self-hosted vs. cloud differences (e.g. screenshot and actions are not available self-hosted — they require Firecrawl's cloud-only Fire-engine).

Security

• Never commit .env or real Proxidize credentials. .env is git-ignored by default in both this repo and the official Firecrawl repo.
• Use placeholder values in anything you share or commit.
• BULL_AUTH_KEY protects Firecrawl's queue admin UI — set it to a real secret if your instance is reachable from anyone besides you.
• If you're deploying beyond local testing, review the "Security considerations" section of Firecrawl's own SELF_HOST.md (Postgres credentials, network exposure, etc.).

Related links

• Proxidize
• Proxidize Help Center
• Get Mobile Proxies
• Get Residential Proxies
• Residential perGB docs
• Mobile perGB docs
• perGB Session Builder (rotation/sticky/targeting)
• How to Use Proxidize Proxies With Self-Hosted Firecrawl (Proxidize article)
• Official Firecrawl repository
• Official self-hosting documentation

Responsible use

Scraping and crawling should follow the target site's terms of service and robots.txt directives, applicable laws in your jurisdiction, and general responsible data-collection practices. Proxidize proxies provide IP routing, diversity, session control, and location selection — they do not grant permission to access, bypass restrictions on, or collect data from any site you're not otherwise authorized to access.