Applications Azure

Datashare Document Analysis & Search on Ubuntu 24.04 on Azure User Guide

| Product: Datashare 21.17.0 on Ubuntu 24.04 LTS on Azure

Overview

This guide covers the deployment and use of Datashare on Ubuntu 24.04 on Azure using cloudimg Azure Marketplace images. Datashare is an open source, self-hosted search engine for documents, built by the International Consortium of Investigative Journalists (ICIJ) — the team behind the Panama Papers and other large cross-border investigations. Point it at a collection of files and it extracts the text (including OCR of scanned pages and images via Apache Tika and Tesseract), indexes everything into Elasticsearch, and gives you a fast, clean web interface to search the full text and explore your documents.

The image installs Datashare 21.17.0 (the official icij/datashare container, pinned by digest) running in local mode — single analyst, no external services beyond Elasticsearch — alongside Elasticsearch 8.19.8, both orchestrated by Docker Compose. Datashare uses an in-memory task queue and a SQLite metadata store, so there is no Redis and no PostgreSQL to operate. Unattended security upgrades are configured to keep the host patched on your running VM.

Locked down by default. Datashare's local mode has no login of its own, so this image never leaves a document-search interface open to the internet. The Datashare backend (127.0.0.1:8080) and Elasticsearch (127.0.0.1:9200) are bound to loopback, and nginx is the only public listener. On the first boot of every VM a one-shot service generates a unique web password and a fresh self-signed TLS certificate, writes them to /root/datashare-credentials.txt (mode 0600, root only), and only then starts nginx — so the interface is reachable only over HTTPS with that per-VM credential.

Ready to search out of the box. A small fictional sample investigation ("Zephyr Maritime Holdings") is shipped and indexed into Elasticsearch at build time, so searching for Zephyr returns real results the first time you open the app. Replace it with your own documents to get started.

What is included:

  • Datashare 21.17.0 (official icij/datashare OCI image, pinned by digest) in local mode, with Apache Tika + Tesseract OCR bundled for text extraction
  • Elasticsearch 8.19.8 as the search index, bound to loopback
  • nginx as the only public listener, serving the UI + REST API over TLS behind a per-VM HTTP Basic Auth gate, with an unauthenticated /healthz probe on port 80
  • A per-VM web password and self-signed TLS certificate generated on first boot and documented in /root/datashare-credentials.txt (0600), with the backend refusing to be served until they exist
  • A pre-indexed sample investigation so full-text search works immediately

Prerequisites

  • Active Azure subscription, SSH public key, VNet and subnet in the target region
  • The recommended VM size is Standard_B2ms (2 vCPU, 8 GiB) — Elasticsearch and Datashare are both JVM services, so 8 GiB is the sensible floor. Larger document collections benefit from more memory and a larger OS disk.
  • Inbound 443/tcp (HTTPS) open to the operators who need the search UI. Port 80 is used only for an unauthenticated health probe and to redirect to HTTPS.

Deploy the image

Create the VM from the cloudimg Datashare offer (replace the resource group, image URN and SSH details with your own):

az group create --name datashare-rg --location eastus

az vm create \
  --resource-group datashare-rg \
  --name datashare-vm \
  --image <cloudimg-datashare-offer-urn> \
  --size Standard_B2ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_rsa.pub \
  --public-ip-sku Standard

az vm open-port --resource-group datashare-rg --name datashare-vm --port 443 --priority 1001

Give the VM a minute after first boot to generate its per-VM credentials before you sign in.

Confirm the services are running

SSH to the VM and confirm the Datashare stack and nginx are active. The Datashare backend and Elasticsearch run as containers; nginx runs on the host as the only public listener.

sudo systemctl is-active datashare.service nginx.service docker.service

All three report active. The two containers — Datashare and Elasticsearch — are both up, with Elasticsearch reporting healthy:

sudo docker compose -f /opt/datashare/docker-compose.yml ps

The Datashare and Elasticsearch containers both reported Up and healthy, and the datashare, nginx and docker services all reported active

Check the search index

Elasticsearch holds the document index. Confirm the cluster is healthy and the sample project is populated:

curl -s http://127.0.0.1:9200/_cluster/health | python3 -m json.tool
curl -s "http://127.0.0.1:9200/_cat/indices?v"

A single-node cluster reports yellow (the replica shard has nowhere to go on one node — this is expected and does not affect search); the local-datashare index holds the shipped sample documents.

The Elasticsearch cluster health reported as a single node, and the local-datashare index listed with its indexed sample documents

Retrieve your per-VM credentials

Every VM writes its own unique web password on first boot. Read it (root only):

sudo cat /root/datashare-credentials.txt

The file records the URL, the username (admin) and the generated password. The password shown here is redacted — yours will be a unique 24-character value.

The datashare credentials file showing the URL, the admin username and the per-VM password, readable only by root at mode 0600

Verify the gate and run a search round-trip

nginx fronts the UI and REST API over TLS behind HTTP Basic Auth. An anonymous request is rejected with 401; the per-VM credential returns 200. The full-text search API (which proxies to Elasticsearch) returns the shipped sample documents for a query on Zephyr:

# Anonymous request is rejected by the per-VM Basic Auth gate
curl -sk -o /dev/null -w 'anonymous: HTTP %{http_code}\n' https://127.0.0.1/

# The unauthenticated health probe is available for load balancers
curl -sk -o /dev/null -w 'healthz: HTTP %{http_code}\n' https://127.0.0.1/healthz

# Authenticated full-text search returns the shipped sample hits
sudo bash -c 'PW=$(grep "^DATASHARE_PASSWORD=" /root/datashare-credentials.txt | cut -d= -f2-); curl -sk -u "admin:$PW" "https://127.0.0.1/api/index/search/local-datashare/_count?q=content:Zephyr"'

The anonymous request returns 401, the health probe returns 200, and the authenticated search reports three matching documents.

The authenticated full-text search round-trip returning a count of three documents for the Zephyr query, an anonymous request rejected with HTTP 401, and the per-VM credential accepted with HTTP 200

Sign in to the Datashare web interface

Open https://<your-server-ip>/ in a browser. Because the certificate is self-signed, accept the browser warning (or place your own domain and CA-signed certificate in front of nginx for production). When prompted, sign in with the username admin and the per-VM password from /root/datashare-credentials.txt.

Datashare opens on the projects view. The pre-indexed sample project ("Default") is listed with its document count and a Search button.

The Datashare projects view listing the Default project with its document count, and the left navigation offering Search, Documents, History and Tasks

Search your documents

Enter a query — for example Zephyr — to search the full text of every indexed document. Datashare returns each matching document with the search term highlighted in a snippet of its extracted text.

The Datashare search results for Zephyr showing three matching documents, each with the term highlighted in a snippet of the extracted document text

Click a result to open the document. Datashare shows the full extracted text with your search term highlighted, alongside tabs for the original view, the metadata, and any extracted entities.

A sample memo opened in Datashare showing its full extracted text with the Zephyr occurrences highlighted, and the Text, View, Metadata and Entities tabs

Add your own documents

To analyse your own collection, copy your files onto the VM under /var/lib/datashare/data (this directory is the Datashare data volume), then use Tasks → Documents → Add documents in the UI. Choose the folder to analyse, whether to run OCR on scanned images and PDFs, and Datashare scans, extracts and indexes them into Elasticsearch.

The Datashare Add documents workflow with options to select a project and folder, choose the extraction language, enable OCR for scans and PDFs, and skip already-indexed documents

How the appliance is secured

  • Loopback binding. The Datashare backend (127.0.0.1:8080) and Elasticsearch (127.0.0.1:9200) are published on loopback only. nginx on port 443 is the sole public listener, so neither the raw backend nor the search index is reachable from the network.
  • Per-VM credential and certificate. On first boot a one-shot service generates a unique HTTP Basic Auth password (bcrypt-hashed in /etc/nginx/.datashare.htpasswd) and regenerates the self-signed TLS certificate, so no two VMs share a secret. The plaintext password is written only to /root/datashare-credentials.txt (0600).
  • Fail-closed startup. nginx is gated on a bootstrap-ready marker that first boot writes only after the credential and certificate are in place, so there is no window in which the search UI is served without a password.
  • Production hardening. For internet-facing use, place your own domain and a CA-signed TLS certificate in front of nginx, and restrict inbound 443 to the operators who need access. Datashare's named-entity (NER) extraction can be enabled by syncing its NLP models; the core OCR and full-text search work without it.

Support

Every cloudimg deployment is backed by 24/7 expert support for the image itself. Datashare is an open source project from the ICIJ; product questions and feature requests are best raised with the upstream community.