Data Analytics Azure

OpenSearch 3.6 on Ubuntu 22.04 on Azure User Guide

| Product: OpenSearch 3.6 on Ubuntu 22.04 on Azure

Overview

This image runs OpenSearch 3.6.0 and OpenSearch Dashboards 3.6.0 side by side on a single Ubuntu 22.04 LTS virtual machine. The OpenSearch daemon binds to TCP 9200 over HTTPS, the transport layer binds to TCP 9300, and Dashboards binds to TCP 5601. Authentication is enforced on the REST and transport layers by the OpenSearch Security plugin using demo TLS certificates and an admin account whose password is generated on the first boot of every deployed virtual machine, so two virtual machines launched from the same gallery image never share credentials.

The image is intended for teams that want a search and analytics engine plus a working Dashboards UI on day one, without spending hours on tarball unpacking, JVM tuning, systemd plumbing, TLS certificate provisioning, or security plugin bootstrap. It is not a multi node production cluster, the TLS certificates shipped are the upstream OpenSearch demo certificates signed by an in image self signed certificate authority, and the image does not include the Data Prepper pipeline, Logstash, or the Security Analytics detector packs. Section 13 documents the recommended path for replacing demo certificates with certificates issued by your own certificate authority before you put production traffic through the cluster.

The brand is lowercase cloudimg throughout this guide. All cloudimg URLs in this guide use the form https://www.cloudimg.co.uk.

Prerequisites

Before you deploy this image you need:

  • A Microsoft Azure subscription where you can create resource groups, virtual networks, and virtual machines
  • Azure role permissions equivalent to Contributor on the target resource group
  • An SSH public key for first login to the build user account
  • A virtual network and subnet in the same region as the Azure Compute Gallery the image is published into, with an associated network security group
  • The Azure CLI (az version 2.50 or later) installed locally if you intend to use the CLI deployment path in Section 2
  • The cloudimg OpenSearch 3.6 offer enabled on your tenant in Azure Marketplace

Step 1: Deploy the Virtual Machine from the Azure Portal

Navigate to Marketplace in the Azure Portal, search for OpenSearch 3.6, and select the cloudimg publisher entry. Click Create to begin the wizard.

On the Basics tab choose your subscription, target resource group, and region. The region must match the region your Azure Compute Gallery exposes the image in. Set the virtual machine name. Choose a size of Standard_D2s_v5 or larger, which provides two virtual CPUs and eight gigabytes of memory. OpenSearch defaults to a one gigabyte Java heap, Dashboards uses around seven hundred megabytes of resident memory, and the operating system plus the page cache need the remainder. Choose SSH public key as the authentication type, set the username to a name of your choice, and paste your SSH public key.

On the Disks tab the recommended OS disk type is Premium SSD. Leave the OS disk size at the default. You can attach a separate Premium SSD data disk now if you intend to move the OpenSearch data directory to it later, or you can do that after the cluster is running by following Section 12.

On the Networking tab select your existing virtual network and subnet. Attach a network security group that allows inbound TCP 22 from your management IP range, inbound TCP 9200 only from the virtual network CIDR, and inbound TCP 5601 only from the virtual network CIDR or a jump host. Do not expose 9200 or 5601 to the public internet. The demo certificates ship with predictable public keys and are documented as unsuitable for production, so any exposure beyond a private network requires the certificate replacement described in Section 13.

On the Management, Monitoring, and Advanced tabs the defaults are appropriate. Click Review + create, wait for validation to pass, then click Create. Deployment takes around three minutes.

Step 2: Deploy the Virtual Machine from the Azure CLI

If you prefer the command line, use the gallery image resource identifier as the source. The exact resource identifier is published on your Partner Center plan. A representative invocation:

RG="opensearch-prod"
LOCATION="eastus"
VM_NAME="opensearch-node-1"
ADMIN_USER="opsuser"
GALLERY_IMAGE_ID="/subscriptions/<sub-id>/resourceGroups/azure-cloudimg/providers/Microsoft.Compute/galleries/cloudimgGallery/images/opensearch-3-6-ubuntu-22-04/versions/1.0.20260417"
SSH_KEY="$(cat ~/.ssh/id_rsa.pub)"

az group create --name "$RG" --location "$LOCATION"

az network vnet create \
  --resource-group "$RG" \
  --name opensearch-vnet \
  --address-prefix 10.30.0.0/16 \
  --subnet-name opensearch-subnet \
  --subnet-prefix 10.30.1.0/24

az network nsg create --resource-group "$RG" --name opensearch-nsg

az network nsg rule create --resource-group "$RG" --nsg-name opensearch-nsg \
  --name allow-ssh --priority 1000 \
  --source-address-prefixes "<your-management-cidr>" \
  --destination-port-ranges 22 --protocol Tcp --access Allow

az network nsg rule create --resource-group "$RG" --nsg-name opensearch-nsg \
  --name allow-rest --priority 1010 \
  --source-address-prefixes 10.30.0.0/16 \
  --destination-port-ranges 9200 --protocol Tcp --access Allow

az network nsg rule create --resource-group "$RG" --nsg-name opensearch-nsg \
  --name allow-dashboards --priority 1020 \
  --source-address-prefixes 10.30.0.0/16 \
  --destination-port-ranges 5601 --protocol Tcp --access Allow

az vm create \
  --resource-group "$RG" \
  --name "$VM_NAME" \
  --image "$GALLERY_IMAGE_ID" \
  --size Standard_D2s_v5 \
  --admin-username "$ADMIN_USER" \
  --ssh-key-values "$SSH_KEY" \
  --vnet-name opensearch-vnet \
  --subnet opensearch-subnet \
  --nsg opensearch-nsg \
  --public-ip-address "" \
  --os-disk-size-gb 64

The final command produces a virtual machine with no public IP, which is the recommended posture for an OpenSearch node. Connect through a bastion host, an Azure Bastion service, or a virtual private network peered into the same virtual network.

Step 3: Connect via SSH

Connect using the admin username you chose during deployment and the private key that matches the public key you pasted into the deployment form:

ssh -i ~/.ssh/id_rsa <admin-username>@<vm-private-ip>

All subsequent commands in this guide run on the virtual machine as the admin user, escalating with sudo where a command needs root. Do not log in directly as the opensearch service user; that account has /usr/sbin/nologin as its shell and is intended for the daemon only.

Step 4: Retrieve the Admin Credentials

The first boot of the virtual machine runs opensearch-firstboot.service, which generates a thirty two character admin password and writes it to /stage/scripts/opensearch-credentials.log. Read it with sudo:

sudo cat /stage/scripts/opensearch-credentials.log

The file contains four lines:

username=admin
password=<generated-password>
opensearch_url=https://<vm-private-ip>:9200
dashboards_url=http://<vm-private-ip>:5601

Copy these values somewhere safe. The password is not recoverable from the image; rotating it is documented in Section 10. Store the file off the virtual machine if your environment requires that, and then tighten the file mode or remove the file. It is owned by opensearch and mode 0600 by default.

Step 5: Server Components

The image ships the following components:

  • OpenSearch 3.6.0 — the distributed search and analytics engine. Installed under /opt/opensearch, a symlink to /opt/opensearch-3.6.0. Runs under the opensearch system user as opensearch.service.
  • OpenSearch Dashboards 3.6.0 — the web UI for search, visualisation, and cluster management. Installed under /opt/opensearch-dashboards, a symlink to /opt/opensearch-dashboards-3.6.0. Runs under the opensearch system user as opensearch-dashboards.service.
  • OpenSearch Security plugin — pre installed. TLS certificates and a minimal internal users database are written by the security demo installer during first boot.
  • Bundled OpenJDK — OpenSearch ships its own Java runtime at /opt/opensearch/jdk. The system does not have a separate JDK package installed.
  • One shot first boot unit — opensearch-firstboot.service runs once and sets the admin password. Gated by the sentinel file /var/lib/opensearch/.firstboot-done so the service is a no op on every subsequent boot.

Step 6: Filesystem Layout

Path Purpose
/opt/opensearch OpenSearch install root (symlink)
/opt/opensearch-dashboards Dashboards install root (symlink)
/opt/opensearch/config/opensearch.yml Cluster config
/opt/opensearch/config/opensearch-security/ Security plugin config (internal users, roles, mappings)
/opt/opensearch/config/esnode.pem Demo node certificate
/opt/opensearch/config/root-ca.pem Demo root CA certificate
/opt/opensearch-dashboards/config/opensearch_dashboards.yml Dashboards config
/var/lib/opensearch/data Index data (move to an attached disk for production — Section 12)
/var/log/opensearch/opensearch.log OpenSearch daemon log
/var/log/opensearch/opensearch-dashboards.log Dashboards log
/etc/systemd/system/opensearch.service Main service unit
/etc/systemd/system/opensearch-dashboards.service Dashboards service unit
/etc/systemd/system/opensearch-firstboot.service First boot credential generator
/stage/scripts/opensearch-credentials.log Admin credentials + URLs (file mode 0600)

Step 7: Start, Stop, and Check Status

The two main services are managed independently through systemd. Dashboards depends on OpenSearch being up, so start OpenSearch first when bringing the stack up from a cold stop.

sudo systemctl start opensearch.service
sudo systemctl status opensearch.service
sudo systemctl start opensearch-dashboards.service
sudo systemctl status opensearch-dashboards.service

Stopping:

sudo systemctl stop opensearch-dashboards.service
sudo systemctl stop opensearch.service

Both services are enabled at boot. If you prefer Dashboards not to start automatically, sudo systemctl disable opensearch-dashboards.service.

Log output is appended to files under /var/log/opensearch. Follow the OpenSearch daemon log with:

sudo tail -f /var/log/opensearch/opensearch.log

Step 8: Connect with curl and a REST Client

Load the admin password into a shell variable and query the cluster:

ADMIN_PW=$(sudo awk -F= '/^password=/ {print $2}' /stage/scripts/opensearch-credentials.log)

curl -sku "admin:${ADMIN_PW}" https://localhost:9200/_cluster/health?pretty
curl -sku "admin:${ADMIN_PW}" https://localhost:9200/_cat/nodes?v
curl -sku "admin:${ADMIN_PW}" https://localhost:9200

The -k flag tells curl to skip TLS verification, which is necessary as long as the image ships with the demo self signed certificate. From a remote host, replace localhost with the private IP of the virtual machine, copy the admin password into that host, and add -H 'Content-Type: application/json' when sending JSON bodies.

From the OpenSearch Python client the equivalent connection object is:

from opensearchpy import OpenSearch

client = OpenSearch(
    hosts=[{"host": "<vm-private-ip>", "port": 9200}],
    http_auth=("admin", "<generated-password>"),
    use_ssl=True,
    verify_certs=False,
    ssl_show_warn=False,
)

print(client.cluster.health())

For production code, set verify_certs=True and provide your own root CA bundle after replacing the demo certificates per Section 13.

Step 9: Index a Document and Run a Query

Create an index, index a document, run a simple search, then clean up:

curl -sku "admin:${ADMIN_PW}" -X PUT \
  https://localhost:9200/products \
  -H 'Content-Type: application/json' \
  -d '{"settings":{"number_of_shards":1,"number_of_replicas":0}}'

curl -sku "admin:${ADMIN_PW}" -X POST \
  https://localhost:9200/products/_doc?refresh=true \
  -H 'Content-Type: application/json' \
  -d '{"name":"cloudimg OpenSearch image","price":0,"tags":["azure","ubuntu"]}'

curl -sku "admin:${ADMIN_PW}" -X GET \
  https://localhost:9200/products/_search \
  -H 'Content-Type: application/json' \
  -d '{"query":{"match":{"name":"opensearch"}}}' | jq

curl -sku "admin:${ADMIN_PW}" -X DELETE https://localhost:9200/products

Step 10: Rotate the Admin Password

Generate a new password and push it into the security index using securityadmin.sh:

NEW_PW='<new-strong-password>'
HASH=$(/opt/opensearch/plugins/opensearch-security/tools/hash.sh -p "${NEW_PW}")

sudo -u opensearch sed -i \
  "s|^  hash: .*|  hash: \"${HASH}\"|" \
  /opt/opensearch/config/opensearch-security/internal_users.yml

sudo -u opensearch \
  OPENSEARCH_JAVA_HOME=/opt/opensearch/jdk \
  /opt/opensearch/plugins/opensearch-security/tools/securityadmin.sh \
  -f /opt/opensearch/config/opensearch-security/internal_users.yml \
  -icl -nhnv \
  -cacert /opt/opensearch/config/root-ca.pem \
  -cert /opt/opensearch/config/kirk.pem \
  -key /opt/opensearch/config/kirk-key.pem \
  -h localhost -p 9200

Then update /opt/opensearch-dashboards/config/opensearch_dashboards.yml to use the new password and restart Dashboards:

sudo sed -i "s|^opensearch.password:.*|opensearch.password: \"${NEW_PW}\"|" \
  /opt/opensearch-dashboards/config/opensearch_dashboards.yml
sudo systemctl restart opensearch-dashboards.service

Update /stage/scripts/opensearch-credentials.log so operators who SSH in later see the current value.

Step 11: Add Application Users and Roles

The OpenSearch Security plugin stores users, roles, and role mappings in YAML files under /opt/opensearch/config/opensearch-security. The safest workflow is to edit the files, run securityadmin.sh to upload the changes into the security index, and reload.

Add a read only user called analyst:

HASH=$(/opt/opensearch/plugins/opensearch-security/tools/hash.sh -p 'AnalystPass!234')

sudo -u opensearch tee -a /opt/opensearch/config/opensearch-security/internal_users.yml >/dev/null <<EOF

analyst:
  hash: "${HASH}"
  reserved: false
  backend_roles: []
  description: "Read only analyst account"
EOF

sudo -u opensearch tee -a /opt/opensearch/config/opensearch-security/roles_mapping.yml >/dev/null <<'EOF'

readall:
  reserved: false
  users:
    - analyst
EOF

sudo -u opensearch \
  OPENSEARCH_JAVA_HOME=/opt/opensearch/jdk \
  /opt/opensearch/plugins/opensearch-security/tools/securityadmin.sh \
  -cd /opt/opensearch/config/opensearch-security/ \
  -icl -nhnv \
  -cacert /opt/opensearch/config/root-ca.pem \
  -cert /opt/opensearch/config/kirk.pem \
  -key /opt/opensearch/config/kirk-key.pem \
  -h localhost -p 9200

The built in readall role grants cluster monitor and global read. For finer grained control, define a new role in roles.yml that names specific indices and action groups, then map your user into it in roles_mapping.yml.

Step 12: Move Data to an Attached Premium Disk

The default /var/lib/opensearch/data lives on the OS disk. For production deployments, move it to a dedicated Premium SSD data disk so index growth does not compete with operating system writes and so you can size storage independently.

Attach a new Premium SSD disk through the portal or the Azure CLI, then on the virtual machine:

sudo systemctl stop opensearch-dashboards.service
sudo systemctl stop opensearch.service

DEVICE=/dev/disk/azure/scsi1/lun0
sudo mkfs.ext4 "${DEVICE}"
sudo mkdir -p /mnt/opensearch-data
echo "${DEVICE} /mnt/opensearch-data ext4 defaults,nofail 0 2" \
  | sudo tee -a /etc/fstab
sudo mount -a

sudo rsync -aHAX /var/lib/opensearch/data/ /mnt/opensearch-data/
sudo chown -R opensearch:opensearch /mnt/opensearch-data

sudo sed -i 's|^path.data: .*|path.data: /mnt/opensearch-data|' \
  /opt/opensearch/config/opensearch.yml

sudo systemctl start opensearch.service
sudo systemctl start opensearch-dashboards.service

Check /var/log/opensearch/opensearch.log for a clean start and confirm index metadata is intact by listing indices via the REST API.

Step 13: Replace the Demo Certificates Before Production

The certificates shipped with this image are the upstream OpenSearch demo certificates signed by a demo root CA. They are suitable for evaluation and for private networks where traffic cannot be intercepted, but they are not production grade.

The replacement workflow is:

  1. Issue a node certificate and a root CA certificate from your own certificate authority, either internal or a commercial one. The node certificate must include the virtual machine's private IP and DNS name in its Subject Alternative Name extension.
  2. Copy the node certificate, its private key, the root CA certificate, and an admin certificate plus key into /opt/opensearch/config/.
  3. Update /opt/opensearch/config/opensearch.yml with the new filenames under plugins.security.ssl.transport.pemcert_filepath, plugins.security.ssl.transport.pemkey_filepath, plugins.security.ssl.transport.pemtrustedcas_filepath, and the matching plugins.security.ssl.http.* keys.
  4. Restart OpenSearch and Dashboards.
  5. Update the admin client in plugins.security.authcz.admin_dn to match the Subject DN of your new admin certificate, so securityadmin.sh continues to work.

Refer to the upstream OpenSearch Security documentation for the full list of configuration keys and for the exact Subject Alternative Name requirements.

Step 14: Troubleshooting

Cannot bind to 9200, port already in use. Run sudo ss -tlnp | grep 9200 to identify the process. If OpenSearch is already running, stop it with systemctl before starting it manually. If another daemon holds the port, stop or reconfigure that daemon.

Cluster health is red. Run curl -sku "admin:${ADMIN_PW}" https://localhost:9200/_cluster/allocation/explain?pretty. Red on a single node cluster usually means shards cannot be allocated because the disk is full; check df -h on /var/lib/opensearch/data or your attached data disk.

Dashboards shows "OpenSearch Dashboards server is not ready yet". This appears on every start until OpenSearch is healthy and the security index is reachable. Wait sixty seconds, then check tail -n 200 /var/log/opensearch/opensearch-dashboards.log for the first non zero exit condition.

Admin login fails in Dashboards. Confirm the password stored in /opt/opensearch-dashboards/config/opensearch_dashboards.yml matches the one in /stage/scripts/opensearch-credentials.log. If you rotated with securityadmin.sh and forgot to update Dashboards, Dashboards will fail to authenticate on the back channel.

install_demo_configuration.sh log references a missing OPENSEARCH_JAVA_HOME. This should only appear if you re ran the script manually without exporting the bundled JDK path. Export OPENSEARCH_JAVA_HOME=/opt/opensearch/jdk before running any OpenSearch security tool by hand.

Out of memory on a Standard_B2s. The default JVM heap is one gigabyte and Dashboards takes around seven hundred megabytes. A 2 vCPU 4 gigabyte virtual machine leaves no headroom for the page cache. Resize to Standard_D2s_v5 or stop Dashboards on that VM.

Step 15: Security Recommendations

Run with an isolated network security group that exposes TCP 22 only to your management CIDR and TCP 9200 plus 5601 only to the virtual network CIDR or an explicit client allow list. Do not place a public IP on the virtual machine unless you have already replaced the demo certificates and moved the admin account to a strong password.

Replace the demo certificates before any traffic other than evaluation traffic crosses the wire (Section 13). The demo certificates are documented upstream as publicly reproducible; anyone reading the OpenSearch source knows the demo private key.

Rotate the admin password immediately after your first Dashboards login (Section 10). Create separate user accounts for each application and each analyst, and assign least privilege roles. Avoid using the admin account as an application credential.

Keep patches current. Subscribe to the OpenSearch release notes and apply OpenSearch version upgrades within the distribution's patch windows. cloudimg publishes refreshed images at each minor release.

Enable the audit log in the Dashboards Security UI if your compliance framework requires it. The audit log writes to .opendistro_security_audit_log_* indices and captures every privileged operation.

Step 16: Support and Licensing

cloudimg provides 24/7/365 expert technical support for this image, with a guaranteed 24 hour response on all requests and a one hour average on critical issues. Contact support@cloudimg.co.uk for assistance with deployment, upgrades, certificate replacement, or cluster migration.

OpenSearch and OpenSearch Dashboards are distributed by the OpenSearch Project under the Apache License 2.0. This image is a repackaged upstream distribution provided by cloudimg. Additional charges apply for build, maintenance, and 24/7 support. The image is not affiliated with or endorsed by Amazon Web Services.

Full product documentation, versioning notes, and upgrade guides are available at https://www.cloudimg.co.uk/guides/opensearch-3-6-on-ubuntu-22-04-azure.