Apache Nutch

Azure Application Infrastructure 1 variation

Apache Nutch, the open source web crawler, paired with Solr search, ready to crawl and index on first boot.

Base
Hardened build
minimal ports, security patches applied at build time
Access
Unique credentials
generated on first boot, readable only by root
Verified
Boots working
services pass a health gate before release
Support
24/7, 365 days
by email and live chat, 24 hour response SLA

Variations

This product is available in the build below. Open it for the operating system, cloud and version detail, or read its deployment guide.

ProductCloudOperating systemVersion
Apache Nutch 1.22 on Ubuntu 24.04 Azure Ubuntu 24.04 1.22 View · Guide

Overview

Apache Nutch is a scalable open source web crawler built on Apache Lucene. This appliance runs Nutch in local mode alongside a bundled Apache Solr search index, so you can crawl web content and full text search it from one node. Nutch fetches, parses and de duplicates pages as batch jobs and hands them to Solr; Solr serves faceted, full text and vector search over everything crawled.

Why the cloudimg image

cloudimg provides Nutch and Solr hardened and fully patched, wired together so the crawl and search pipeline works on first boot. Solr is protected by a per VM admin password generated at first boot and never shipped, and the same secret is injected into the crawler so it authenticates automatically. The image is supported 24/7.

Common uses

  • Web and site crawling
  • Full text search over crawled content
  • Search engine and data mining backends