Self-Host fastCRW and Connect Remote Apps with Localtonet

What you are building

The workflow has two independent layers. First, Docker runs fastCRW and publishes its HTTP API on the host. Second, our Localtonet client establishes an outbound connection to a relay and forwards a public HTTPS address to that local API. A successful tunnel does not prove that fastCRW can scrape a page, and a successful local scrape does not prove that a remote client can reach the tunnel.

We therefore verify from the inside out: container status, local HTTP response, useful scrape output, and finally the same request through the public address. Install the Localtonet client on the Docker host for this walkthrough. That lets us publish Docker's port on loopback rather than making the unauthenticated API accessible across the LAN.

Scraping also requires outbound access. The fastCRW container must resolve DNS names and reach the requested websites. Localtonet addresses inbound application connectivity; it does not replace the scraper's outbound network access or grant permission to access a target website.

This tutorial establishes a reproducible evaluation workflow, not high availability, capacity guarantees, or complete compatibility with every Firecrawl client. Test the precise routes, options, and response handling your application uses before adopting a compatible API as a replacement.

Choose the deployment that supplies your required capabilities

The current fastCRW self-hosting documentation distinguishes three configuration paths: a single binary using config.default, Docker Compose using config.docker, and Compose with the stealth override. It also provides a single-container Docker quick start. That quick start is a convenient API baseline, but it is not interchangeable with the multi-service Docker stack.

Search is a stack capability, not a consequence of opening port 3000

fastCRW documents /v1/search and the crw_search MCP tool as working with the Docker stack's search sidecar. A successful scrape in the standalone container does not verify that backend. If search is essential, select and verify the documented stack instead of repeatedly restarting the minimal container.

The documented search request accepts a query, a result limit, and optional language, time-filter, and source-category fields. Those request fields do not install a backend. Test search independently once the selected deployment supplies its dependencies.

Browser rendering and extraction need their own acceptance tests

A static page may produce useful Markdown without executing JavaScript. That does not prove a JavaScript-heavy application will yield its rendered content. Compose bundles LightPanda, while the additional Chrome and stealth profiles represent different rendering arrangements. Verify the renderer your workload actually needs, including a permitted page that depends on JavaScript.

The documentation also lists /v1/extract and structured JSON extraction through /v1/scrape with formats:["json"] and jsonSchema. The supplied self-hosting details do not establish a complete standalone extraction-provider configuration. Do not infer that managed LLM extraction, hosted features, or every structured-output option is available from the bare container.

Prepare the host and test environment

Use a machine with Docker Engine or Docker Desktop installed and running, permission to run containers, internet access to retrieve the image, and an unused host port 3000. You also need curl and a shell that understands the examples below. They use POSIX shell quoting and line continuations. On Windows, use an appropriate shell such as WSL, or translate the quoting while preserving the JSON body.

docker version
docker ps
curl --version

docker version should show that the client can communicate with the Docker server. A permission error or unavailable daemon must be resolved before proceeding. Docker installation differs by operating system; these checks assume you have completed the supported installation for your host.

Port 3000 must be available. If Docker reports a binding conflict, inspect the existing service before stopping anything. You may deliberately use another host port, but then every local request and the Localtonet target must use that host-side port. The container-side port in this example remains 3000.

Use a disposable evaluation environment with no sensitive mounted directories or unnecessary access to internal systems. Have your remote test client ready before starting the tunnel, and obtain authorization for the test under your organization's network policy. There is no verified universal fastCRW memory or concurrency requirement in the supplied evidence, so observe your own workload rather than adopting an invented minimum.

The CLI is optional, not the API server prerequisite

fastCRW also documents a CLI installer for macOS and Linux:

curl -fsSL https://fastcrw.com/install | sh
crw https://example.com

A download piped into sh executes remote code, so review and trust the installation source before using it. This CLI path is optional here. A successful one-shot command does not establish that a persistent HTTP API is listening. The Docker workflow below is sufficient for this tutorial.

Start the self-hosted fastCRW API

The published quick start uses docker run -p 3000:3000 ghcr.io/fastcrw/crw, followed by a local scrape request. We preserve that start-then-test sequence, with two ordinary Docker adjustments: a container name for repeatable operations and an explicit loopback binding to limit direct network exposure.

docker run --name fastcrw-eval \
  -p 127.0.0.1:3000:3000 \
  ghcr.io/fastcrw/crw

curl -i -X POST http://localhost:3000/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown"]}'

The first start may take time while Docker retrieves image layers. The command does not configure automatic restart, authentication, or persistent application storage. Do not run it again merely to restart the same named container; use the lifecycle commands later in this guide.

The image reference has no explicit tag, so Docker interprets it as latest. That is a mutable tag, not a reproducible release identifier. Docker can also reuse a locally cached image, so an unpinned command does not necessarily retrieve the newest build on each run. Record the actual image used before comparing results or planning an upgrade.

Verify local health and useful API output

Check the HTTP status and JSON body from the scrape. A successful result should report success and include useful output from the requested page, such as Markdown. Do not compare the response byte for byte: page content and metadata can change. An HTTP response by itself is not enough if the application reports a failed scrape.

Now test the exact address our client will use. This also avoids differences in how localhost resolves on your host:

curl -i -X POST http://127.0.0.1:3000/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown"]}'

Inspect container state and available health information

docker ps -a --filter name=fastcrw-eval
docker logs --tail 100 fastcrw-eval
docker inspect --format '{{json .State}}' fastcrw-eval

These are Docker diagnostics, not fastCRW-specific management commands. Confirm that the container is running rather than exited or repeatedly restarting. Inspect startup errors before changing API settings. Logs may contain requested URLs or other sensitive details, so redact them before sharing.

fastCRW documents health support, but the supplied excerpt does not specify an exact health URL, response contract, or authentication exemption. Do not borrow a Firecrawl health path or assume that this image has a Docker HEALTHCHECK. Docker state inspection will show health information if one is configured; its absence is not proof of a healthy or unhealthy application. The real scrape remains the verified end-to-end check in this tutorial.

Understand the routes without assuming all dependencies are installed

After scraping works, an optional bounded mapping request tests another route:

curl -i -X POST http://127.0.0.1:3000/v1/map \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","limit":50}'

For a small crawl, the documented request uses {"url":"https://example.com","maxPages":5} with POST /v1/crawl. Preserve the returned identifier and poll GET /v1/crawl/{id}. Do not interpret an accepted job as completed content, or assume a job survives container replacement without a documented persistence design.

One successful scrape does not prove browser compatibility, capacity, authentication, rate limiting, or recovery. Save the known-good request and use additional acceptance tests only for capabilities your application needs.

Set the remote-exposure boundary before starting a tunnel

Current fastCRW self-hosting documentation describes optional bearer-token authentication. The default no-header quick start does not mean authentication is unsupported. However, the supplied material does not include the exact server configuration key, token-loading mechanism, or protected-route behavior. Guessing an environment variable would create a false security guarantee.

For a permitted evaluation, use a disposable host with no sensitive data, minimize access to private network services, keep the public URL out of shared logs and repositories, submit only the prepared test request, and stop the tunnel immediately afterward. Do not leave the endpoint running while developing an integration over several days.

What must change before routine remote use

• Configure native bearer authentication using instructions matching the deployed release, or put a reviewed gateway in front of the API.
• Verify that missing and incorrect credentials are rejected and valid credentials work on every application route you expose.
• If using a gateway, forward Localtonet to that protected listener rather than leaving an alternate route directly to the API.
• Apply workload limits, request-size limits, concurrency controls, and caller restrictions appropriate to your environment.
• Restrict outbound access so submitted URLs cannot unnecessarily reach private services or host metadata endpoints.
• Define credential rotation, logging, retention, and incident-response procedures.

A bearer header only has meaning when the server validates it. Merely adding Authorization: Bearer ... to a client request does not turn on authentication. Do not confuse a hosted fastCRW API key with a credential configured for your own server.

Scraping can consume CPU, memory, bandwidth, and browser resources. Authentication alone does not bound that work. Respect target-site permissions, applicable laws, terms, and relevant robots policies. Neither self-hosting nor a tunnel grants permission to evade website or organizational access controls.

Connect the verified API with Localtonet

HTTP is the appropriate tunnel family for this REST API. Our client establishes an outbound connection to a relay, so you do not need inbound router port forwarding, firewall changes, VPN setup, or a public IP address. Continue only after local verification and the security decision above.

The following uses our supported common configuration sequence. The supplied material does not include a current field-by-field HTTP documentation transcript, so it should not be read as a claim about the exact number of dashboard screens. Consult our HTTP tunnel documentation alongside the dashboard for current layout and options; no unverified commands or relay codes are required below.

Repeat the same functional request remotely

From the remote test machine, replace the placeholder below with your assigned HTTPS address, without a trailing API path. This is the explicitly unauthenticated evaluation request, not a production client example.

export CRW_PUBLIC_URL="https://your-assigned-public-address"

curl -i -X POST "$CRW_PUBLIC_URL/v1/scrape" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown"]}'

Check the HTTP status, success indicator, and returned Markdown just as you did locally. Once the request completes, stop the tunnel. If local succeeds but remote fails, investigate the selected device, tunnel state, and target address. If both fail, investigate fastCRW before changing Localtonet.

Integrate clients only after the access design is ready

fastCRW publishes Python crw and TypeScript crw-sdk packages. Its examples use CRW_API_URL to select a self-hosted address instead of the hosted default:

export CRW_API_URL=http://localhost:3000

Use the base-URL mechanism supported by your installed SDK version. A remote application would use the protected public HTTPS base URL. Do not assume that every SDK reads the same environment variables automatically. First reproduce a failing SDK call with raw HTTP, then compare URL, method, headers, and body.

Routine operation, upgrades, and recovery

Inspect status and logs

docker ps -a --filter name=fastcrw-eval
docker logs --tail 100 fastcrw-eval
docker logs --follow fastcrw-eval

Following logs does not establish application health; combine logs with the saved scrape request. Interrupting docker logs --follow stops the log viewer, not the container. By contrast, interrupting the attached docker run session can signal the container process. Use explicit lifecycle commands for predictable operation.

Stop, start, and restart the same container

docker stop fastcrw-eval
docker start fastcrw-eval
docker restart fastcrw-eval

Run these as needed, not as a required sequence. docker start starts the existing stopped container with its existing configuration. docker restart interrupts the application and starts that same container again; it does not update the image. Expect active requests to fail during interruption, and do not assume unfinished crawls resume.

This quick start has no automatic restart policy. Inspect the actual policy with:

docker inspect --format '{{json .HostConfig.RestartPolicy}}' fastcrw-eval

Automatic restart is a separate operational decision. It does not repair bad configuration, guarantee restored jobs, or start a Localtonet tunnel. Test host reboot behavior before relying on an unattended deployment.

Keep application and tunnel lifecycles separate

Stop the tunnel when the remote test ends. Delete its configuration if you no longer need it. Stopping a tunnel does not stop Docker, and stopping Docker does not delete the tunnel configuration. A public endpoint needs a connected Localtonet client, a running tunnel, and a responsive local application.

Record a reproducible image identity

docker inspect --format '{{.Image}}' fastcrw-eval
docker image inspect ghcr.io/fastcrw/crw:latest \
  --format '{{json .RepoDigests}}'

The first command identifies the image actually used by the container. The tag in the second command may have moved since that container was created, so do not assume those values always describe the same build. Record the tested image identity, configuration, test results, and any matching repository digest. For repeatable deployments, use a verified digest or deliberately selected release rather than an unpinned tag. Do not assume a GitHub release name is also a published container tag.

Evaluate upgrades without destroying the baseline

Keep the public tunnel stopped. Preserve the old container and image, then retrieve and test a candidate separately. The following generic Docker procedure uses the publisher's current default image as a candidate, not as a promise of compatibility:

docker pull ghcr.io/fastcrw/crw:latest

docker run -d --name fastcrw-candidate \
  -p 127.0.0.1:3001:3000 \
  ghcr.io/fastcrw/crw:latest

curl -i -X POST http://127.0.0.1:3001/v1/scrape \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","formats":["markdown"]}'

Inspect candidate logs and run the routes your application uses. Record its image identity before promotion. If it fails, stop the candidate and retain the original deployment. If accepted, recreate your intended deployment using the tested image identity and configuration, then verify locally before restoring authorized access.

For this disposable baseline, rollback means returning to the preserved original container and image, using docker start fastcrw-eval if it was stopped. Do not prune the old image before the rollback window closes. Image rollback alone is not a data rollback if a future deployment adds persistent stores or migrations.

Understand persistence before removing containers

The command here creates no explicit volume or bind mount. Inspect actual mounts rather than assuming the image stores nothing:

docker inspect --format '{{json .Mounts}}' fastcrw-eval

A container's writable layer generally remains across stop/start, but is removed when that container is removed. Application data can also be memory-only or live in image-defined volumes. The supplied fastCRW excerpt does not establish which jobs, caches, or results persist, or a supported backup path. Export results your application needs and do not promise durable crawl recovery.

When the evaluation is finished and its data is disposable, stop and remove its containers. Do not add volume-deletion options casually, and do not copy state directories into a new release without a supported migration procedure.

Troubleshoot from the local application outward

Docker reports an allocated port or existing container name

A port conflict requires identifying the existing listener. A name conflict usually means your earlier container still exists, even if stopped. Use docker ps -a and start the intended container instead of launching duplicates. If changing a host port, update tests and the tunnel target together.

The local request cannot connect

Check container state, startup logs, the loopback port mapping, and whether the image finished downloading. Test 127.0.0.1 explicitly. A client in another container has its own loopback address; this walkthrough assumes Localtonet runs directly on the Docker host.

The API responds, but scraping fails

Retry the example-page smoke test. If only another site fails, investigate its DNS, reachability, permissions, bot defenses, or rendering requirements. A browser-dependent target may require Compose and a configured renderer. Changing the public tunnel cannot fix unavailable local browser dependencies.

Input errors or unexpected output appear

Confirm POST, Content-Type: application/json, valid JSON, and correct shell quoting. Check both HTTP status and application-level errors. SDKs may unwrap response data, so do not assume the raw HTTP response and an SDK return value have identical shapes.

Search or extraction fails while scraping works

Check the chosen deployment and its dependencies. Search belongs to the documented Docker stack with its sidecar; returning to the bare single-container command is not a remedy for a missing search backend. For extraction, verify the release's provider and schema requirements rather than assuming hosted behavior is included.

The public URL fails while local requests succeed

Confirm our client is connected, the correct device token is selected, the tunnel is started, and its target matches the verified address. Check any authentication gateway separately. Do not disable authentication just to make a diagnostic request pass.

Another device cannot reach the loopback-bound API

That is expected. This guide intentionally binds Docker to the local host. A different Localtonet device needs a separately designed reachable listener and access controls; 127.0.0.1 on that device points to itself. Do not broaden the unauthenticated binding merely to work around the topology.

Crawls time out or disappear after interruption

Use the asynchronous start-and-poll workflow, preserve identifiers, and apply reasonable client timeouts and polling backoff. Do not blindly retry crawl creation after an ambiguous response because it may duplicate work. Persistence and restart recovery require separate verification.

Frequently asked questions