Base
Hardened build
minimal ports, security patches applied at build time
Crawl4AI on Ubuntu 24.04
Azure
Artificial Intelligence (AI)
an open source web crawler that turns any web page into clean, LLM ready markdown over a REST API, secured from first boot by cloudimg.
Overview
Crawl4AI is a fast, open source web crawler and scraper that converts web pages into clean, LLM ready markdown and structured data through a simple REST API. It drives a headless Chromium browser, so it renders JavaScript heavy pages the way a real browser sees them, and returns markdown tuned for large language model context along with screenshots, PDF capture, on page JavaScript execution and streaming or asynchronous job endpoints.
It suits teams building retrieval augmented generation pipelines, AI agents and training datasets that need a reliable page to markdown ingestion front end they run and own inside their own cloud account, rather than a hosted scraping service.
Why the cloudimg image
cloudimg ships the official Crawl4AI server as a container fronted by an nginx reverse proxy that binds the crawler to loopback and is ready for your TLS certificate. An open crawler is a server side request forgery and denial of service risk, so security is enforced from the first request: a unique API token is generated on each virtual machine's first boot and written to a root only file, so the crawl endpoint requires your token while a health probe stays open for monitoring. There is no shared or default credential in the image. The appliance recreates the container cleanly on first boot, and the paired deployment guide covers reading your token, running your first crawl, tuning extraction and enabling HTTPS, with 24/7 support on every deployment.
Common uses
- Turning web pages into clean markdown for large language model context
- A page to markdown ingestion front end for RAG pipelines and AI agents
- A self hosted crawler and scraper you run inside your own cloud account
More in Artificial Intelligence (AI)
AnythingLLM
Artificial Intelligence (AI)
This image runs AnythingLLM, the all-in-one open source application for chatting with your documents using any large lan...
Ie
Infinity Embeddings Server
Artificial Intelligence (AI)
This image runs Infinity 0.0.77, the high-throughput, low-latency server for text embedding and reranking models (michae...
Ls
Label Studio
Artificial Intelligence (AI)
This image runs Label Studio 1.23, the open source data labeling and annotation platform for machine learning - label te...
LibreChat AI Chat Platform - with
Artificial Intelligence (AI)
This image runs LibreChat, the open source AI chat platform - a self-hosted, ChatGPT-style web interface that connects t...
Lo
LiteLLM
Artificial Intelligence (AI)
This image runs LiteLLM 1.88, the open source gateway that puts a single OpenAI-compatible API in front of more than 100...
Lo
LocalAI
Artificial Intelligence (AI)
This image runs LocalAI, the free, open source, self hosted alternative to the OpenAI API.