Apache Nutch 1.22 on Ubuntu 24.04

Azure Application Infrastructure

Apache Nutch, the open source web crawler, paired with Solr search, ready to crawl and index on first boot.

Base
Hardened build
minimal ports, security patches applied at build time
Access
Unique credentials
generated on first boot, readable only by root
Verified
Boots working
services pass a health gate before release
Support
24/7, 365 days
by email and live chat, 24 hour response SLA

Overview

Apache Nutch is a scalable open source web crawler built on Apache Lucene. This appliance runs Nutch in local mode alongside a bundled Apache Solr search index, so you can crawl web content and full text search it from one node. Nutch fetches, parses and de duplicates pages as batch jobs and hands them to Solr; Solr serves faceted, full text and vector search over everything crawled.

Why the cloudimg image

cloudimg provides Nutch and Solr hardened and fully patched on Ubuntu 24.04, wired together so the crawl and search pipeline works on first boot. Solr is protected by a per VM admin password generated at first boot and never shipped, and the same secret is injected into the crawler so it authenticates automatically. The image is supported 24/7.

Common uses

  • Web and site crawling
  • Full text search over crawled content
  • Search engine and data mining backends

See it running

Real screenshots taken while testing this image against its deployment guide.

Apache Nutch 1.22 on Ubuntu 24.04 screenshot 1 Apache Nutch 1.22 on Ubuntu 24.04 screenshot 2 Apache Nutch 1.22 on Ubuntu 24.04 screenshot 3