Apache Hive on Ubuntu 24.04 on Azure User Guide

Overview

Apache Hive is the open-source data warehouse and SQL query engine that lets you read, write and manage large datasets stored in distributed storage using familiar SQL (HiveQL). The cloudimg image installs Hive 4.0.1 on OpenJDK 11 at /opt/hive, on top of a single-node Apache Hadoop 3.3.6 HDFS cluster, with a production-grade MariaDB metastore backend (not embedded Derby). HiveServer2 exposes JDBC over Thrift on 10000 and a web UI on 10002; the Hive Metastore runs on 9083. The HiveServer2 web UI is fronted by an nginx reverse proxy on TCP 80, the HDFS data and MariaDB metastore live on a dedicated Azure data disk, and a unique Hive password is generated on the first boot of every VM. Backed by 24/7 cloudimg support.

What is included:

• Apache Hive 4.0.1 data warehouse on OpenJDK 11 at /opt/hive
• Apache Hadoop 3.3.6 single-node HDFS (NameNode + DataNode) hosting the Hive warehouse
• A production MariaDB metastore, schema initialised with schematool -initSchema -dbType mysql
• HiveServer2 JDBC/Thrift on 10000, web UI on 10002, Metastore Thrift on 9083 — all bound to loopback
• nginx reverse proxy on :80 in front of the HiveServer2 web UI, with an unauthenticated /health endpoint
• A dedicated Azure data disk at /var/lib/hive holding the HDFS data, the MariaDB metastore and logs — separate from the OS disk and re-provisioned with every VM
• Per-VM Hive password (HIVE_ADMIN_PASSWORD) generated at first boot, in a root-only file
• hadoop.service, mariadb.service, hive-metastore.service, hiveserver2.service and nginx.service as systemd units, enabled and active
• 24/7 cloudimg support

Prerequisites

An active Azure subscription, an SSH key pair, and a VNet + subnet in the target region. Standard_B4ms (4 vCPU / 16 GiB RAM) is a good starting point because the Hadoop, Metastore, HiveServer2 and MariaDB JVMs together are memory hungry; scale up for larger warehouses and concurrency. NSG inbound: allow 22/tcp from your management network and 80/tcp for the web UI (front with TLS for public exposure — see Enabling HTTPS). The JDBC port 10000 is bound to loopback by default; expose it through an SSH tunnel or a private endpoint rather than opening it to the internet.

Step 1 — Deploy from the Azure Marketplace

Sign in to the Azure Portal, choose Create a resource, search the Marketplace for Apache Hive by cloudimg, and select Create. On Basics pick your subscription, resource group, region and size; under Administrator account choose SSH public key and paste your key; under Inbound port rules allow SSH (22) and HTTP (80). Review the dedicated data disk on the Disks tab, then Review + create → Create.

Step 2 — Deploy from the Azure CLI

az vm create \
  --resource-group <your-rg> \
  --name hive \
  --image <marketplace-image-urn> \
  --size Standard_B4ms \
  --admin-username azureuser \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name <your-vnet> --subnet <your-subnet> \
  --public-ip-sku Standard

az vm open-port --resource-group <your-rg> --name hive --port 80 --priority 1010

Step 3 — Connect to your VM

ssh azureuser@<vm-public-ip>

Step 4 — Confirm the services are running

systemctl is-active hadoop.service mariadb.service hive-metastore.service hiveserver2.service nginx.service

Each service reports active. On the first boot the stack can take a couple of minutes to come fully up while HDFS leaves safe mode and HiveServer2 warms up.

Step 5 — Retrieve your Hive password

The Hive password is generated uniquely on the first boot of your VM and written to a root-only file:

sudo cat /root/hive-credentials.txt

The HIVE_ADMIN_PASSWORD value is the password; connect to HiveServer2 over JDBC as user hive with it, and sign in to the web UI as hive with the same password.

Step 6 — Run SQL with beeline

Hive ships beeline, the JDBC command-line client. Connect to HiveServer2 on the VM and run a CREATE / INSERT / SELECT round-trip against the HDFS warehouse. Replace <HIVE_ADMIN_PASSWORD> with the value from Step 5:

sudo -u hive /opt/hive/bin/beeline \
  -u 'jdbc:hive2://127.0.0.1:10000/default' \
  -n hive -p '<HIVE_ADMIN_PASSWORD>' \
  -e "CREATE TABLE IF NOT EXISTS demo(id int, v string);
INSERT INTO demo VALUES (1,'cloudimg');
SELECT v FROM demo WHERE id=1;"

beeline connects, runs each statement, and prints the row cloudimg from the SELECT. The table is stored as a managed Hive table in the HDFS warehouse under /user/hive/warehouse, with its schema recorded in the MariaDB metastore.

Step 7 — Inspect the metastore and warehouse

List databases and tables to confirm the metastore is serving schema, and look at the warehouse directory on HDFS:

export JAVA_HOME=$(dirname "$(dirname "$(readlink -f "$(command -v java)")")")
sudo -u hive JAVA_HOME="$JAVA_HOME" HADOOP_HOME=/opt/hadoop \
  /opt/hadoop/bin/hdfs dfs -ls /user/hive/warehouse

The HDFS listing shows the warehouse directory and any managed table directories you have created. The schema metadata itself lives in the MariaDB metastore database on the dedicated data disk.

Step 8 — Open the HiveServer2 web UI

Browse to http://<vm-public-ip>/ and sign in as hive with the password from Step 5. The HiveServer2 web UI shows active sessions, open and historical queries, the configuration in effect, and software attributes — useful for watching query execution and diagnosing sessions.

Step 9 — Connect from your applications

Point JDBC clients (BI tools, ETL jobs, beeline from another host) at jdbc:hive2://<vm-public-ip>:10000/default with user hive and the per-VM password. Because port 10000 is bound to loopback by default, reach it over an SSH tunnel (ssh -L 10000:127.0.0.1:10000 azureuser@<vm-public-ip>) or a private endpoint rather than opening it to the internet.

Enabling HTTPS

For production, terminate TLS at nginx with a real domain pointed at the VM's public IP. Install certbot and request a certificate (replace the domain):

sudo apt-get update && sudo apt-get install -y certbot python3-certbot-nginx
sudo certbot --nginx -d your-domain.example.com

Backup and maintenance

The HDFS data, the MariaDB metastore and Hive logs live on the dedicated data disk at /var/lib/hive. Snapshot that disk in Azure to back up your warehouse and schema, and keep the OS patched with sudo apt update && sudo apt upgrade. The stack restarts cleanly with sudo systemctl restart hadoop hive-metastore hiveserver2.

Support

This image is backed by 24/7 cloudimg support. Contact us by email and chat for help with Hive and Hadoop configuration, the metastore, HiveQL, query tuning, TLS and backups.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.