Apache Hive Data Warehouse

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

Overview

Apache Hive is the open source data warehouse software that lets you read, write and manage large datasets in distributed storage using familiar SQL (HiveQL). This image delivers Hive fully installed and configured as system services on top of a single-node Apache Hadoop HDFS cluster, with a production-grade MariaDB metastore backend rather than embedded Derby, so a working SQL-on-Hadoop engine is running within minutes of launch.

Application Stack

Apache Hive unpacked under /opt/hive and run by a dedicated unprivileged service account. A single-node Apache Hadoop cluster (HDFS NameNode plus DataNode) in pseudo-distributed mode, hosting the Hive warehouse on HDFS. A MariaDB metastore whose schema is initialised at build time with schematool, holding table and partition metadata. A headless OpenJDK 11 Java runtime. The HDFS data directories, the MariaDB metastore datadir and the Hadoop and Hive logs stored on a dedicated data disk so warehouse state and metadata are independently resizable and survive instance replacement. An nginx reverse proxy that publishes the HiveServer2 web UI on port 80.

SQL Over Big Data

HiveServer2 exposes a JDBC and Thrift interface so BI tools, ETL jobs and the bundled beeline client can run standard SQL against data in HDFS. Create managed and external tables, run joins and aggregations, and manage partitions through HiveQL, with query metadata recorded in the MariaDB metastore and table data stored on HDFS. The read-only HiveServer2 web UI shows active sessions, running and historical queries and the configuration in effect.

Secure First Boot

Apache HiveServer2 accepts unauthenticated connections by default. This image enables HiveServer2 client authentication with a per-instance credential provider and locks the web UI behind nginx HTTP Basic auth, so no unauthenticated access is possible. On the first boot of your instance a one shot service generates a fresh admin password, unique to that instance, writes it as a salted SHA-256 hash for HiveServer2 and into the nginx htpasswd, rotates the loopback-only MariaDB metastore database password, and records the admin credentials in a root only file. HiveServer2, the metastore and HDFS bind to loopback only and are never exposed without authentication. No shared or default credentials ship in the image.

Ready To Use

The HiveServer2 web UI is served on port 80 through nginx. Sign in with the generated administrator credentials to inspect sessions and queries; connect the beeline JDBC client, BI tools and ETL jobs to HiveServer2 to run SQL against your warehouse.

cloudimg Support

24/7 technical support by email and chat. Help with deployment, Hadoop and HDFS configuration, the metastore, HiveQL authoring and query tuning, JDBC and ODBC connectivity, TLS and JVM tuning.

Use Cases

A single-tenant SQL-on-Hadoop data warehouse for dev and test. Ad hoc HiveQL analytics over data in HDFS. ETL and batch processing pipelines. A metastore-backed catalog for managed and external tables. Powering BI and JDBC clients over big data.

The current release delivered is Apache Hive 4.0.1 on Apache Hadoop 3.3.6. Apache Hive and Apache Hadoop are licensed under the Apache License 2.0. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.