Apache Zeppelin on AWS User Guide

Overview

This image runs Apache Zeppelin, the open source web based notebook that brings data ingestion, exploration, visualisation, sharing and collaboration into a single interactive interface. You write and run code in many languages against the bundled interpreters, render the results as rich charts and tables, and build shareable, paragraph by paragraph notebooks for data science and analytics.

The Apache Zeppelin distribution is installed under /opt/zeppelin and runs as a dedicated unprivileged zeppelin system account under a systemd service that starts the notebook server on boot and restarts it on failure. The runtime is a headless OpenJDK 11. The notebook directory, configuration and interpreter state live on a dedicated, independently resizable EBS data volume mounted at /var/lib/zeppelin, so your saved notebooks survive instance replacement.

Apache Zeppelin allows anonymous access out of the box. This image enables Apache Shiro form based authentication with a single administrator account and requires login on every path, so the notebook UI is locked behind a sign in. The notebook server itself binds to the loopback interface only (127.0.0.1:8080) and is never exposed directly. An nginx reverse proxy publishes the notebook UI on port 80 with the WebSocket upgrade support that the live editor needs. The admin password is generated on the first boot of every deployed instance, so two instances launched from the same Amazon Machine Image never share a password. It is written to /root/apache-zeppelin-credentials.txt with mode 0600 so that only the root user can read it.

The notebook server is a JVM application and is slow to start, taking roughly 45 to 90 seconds after first boot before the UI answers.

Prerequisites

Before you deploy this image you need:

• An Amazon Web Services account where you can launch EC2 instances
• IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
• An EC2 key pair in the target Region for SSH access to the instance
• A VPC and subnet in the target Region, with a security group allowing inbound port 22 from your management network and port 80 for the notebook UI
• The AWS CLI (version 2) installed locally if you plan to deploy from the command line

Step 1: Launch the Instance from the AWS Marketplace

Sign in to the AWS Management Console, open the EC2 service, and select Launch instance. Under Application and OS Images choose AWS Marketplace AMIs and search for Apache Zeppelin. Select the cloudimg listing and choose Select, then Continue on the subscription summary.

Pick an instance type of t3.large or larger. Choose your EC2 key pair under Key pair (login). Under Network settings select your VPC and subnet, and either create or select a security group that opens port 22 from your management network and port 80 for the notebook UI. Leave the root volume at the default size or larger.

Select Launch instance. First boot initialisation generates the admin password, and the JVM notebook server then takes roughly 45 to 90 seconds to become ready after the instance state becomes Running and the status checks pass.

Step 2: Launch the Instance from the AWS CLI

The following block launches an instance from the cloudimg Apache Zeppelin Marketplace AMI into an existing subnet and security group. Replace <ami-id> with the AMI ID shown on the Marketplace listing, <key-name> with your EC2 key pair name, <subnet-id> with your subnet ID, and <security-group-id> with a security group that opens ports 22 and 80 as described above.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type t3.large \
  --key-name <key-name> \
  --subnet-id <subnet-id> \
  --security-group-ids <security-group-id> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=apache-zeppelin}]'

When the instance reaches the Running state and its status checks pass, note its public IP address or DNS name from the EC2 console or with aws ec2 describe-instances.

Step 3: Connect to Your Instance

Connect over SSH using your key pair and the login user for your operating system variant.

ssh -i <key-name>.pem ubuntu@<public-ip>

Step 4: Retrieve the Admin Password

The notebook admin password is unique to your instance and was generated on first boot. Read it as root with the command below. The file lists the notebook URL, the admin user (admin) and the generated password. Keep this password somewhere safe.

sudo cat /root/apache-zeppelin-credentials.txt

Step 5: Confirm Apache Zeppelin Is Running

Over SSH, confirm the notebook server and the nginx proxy are active. The notebook server is a JVM application and is slow to start, so allow up to 90 seconds after boot before it answers.

sudo systemctl is-active zeppelin nginx

Both services should report active. The notebook server binds to loopback only, fronted by nginx on port 80. Confirm its liveness endpoint answers locally:

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/api/version

This returns 200 once the notebook server has finished starting. The same endpoint is reachable through nginx at http://127.0.0.1/api/version.

Step 6: Sign In to the Notebook UI

The Apache Zeppelin notebook UI is served on port 80 by nginx, with Apache Shiro form based authentication. In a browser, go to:

http://<instance-public-ip>/

You are redirected to the sign in page. Sign in as admin with the password from the credentials file. After login you land on the Zeppelin home page, which lists your notebooks and offers Import Note and Create new Note.

Step 7: Create and Run a Notebook

From the home page choose Create new Note, give the note a name, pick a default interpreter and select Create. The note opens with an empty paragraph.

Type code into the paragraph and run it with the play control or with Shift+Enter. For a quick test, enter a Markdown paragraph:

%md
# Hello from cloudimg
Apache Zeppelin running through the nginx WebSocket reverse proxy.

Running the paragraph drives the live editor over the WebSocket that nginx proxies to the notebook server, and the rendered Markdown appears below the paragraph with a FINISHED status. You can mix code, queries, Markdown and visualisations across paragraphs in the same note, and Zeppelin renders tabular output as interactive charts.

Step 8: Configure Interpreters

Apache Zeppelin ships with a rich set of interpreters for data work. Open the user menu at the top right and choose Interpreter to reach the interpreter settings page. Here you can search the bundled interpreters, edit their properties, restart an interpreter, and bind or unbind interpreters from your notes.

Each interpreter has its own properties, for example the connection details for a database interpreter or the master URL for the Spark interpreter. After changing an interpreter's properties, select Save and restart the interpreter so your notebooks pick up the new configuration.

Step 9: The Data Volume

The notebook directory, configuration (shiro.ini, zeppelin-site.xml, zeppelin-env.sh), interpreter state and the local repo live on a dedicated EBS volume mounted at /var/lib/zeppelin. This keeps your notebooks off the operating system disk and lets you resize or snapshot them independently. Confirm the mount with:

df -h /var/lib/zeppelin

To grow the store, expand the EBS volume in the AWS console, then grow the filesystem on the instance with sudo resize2fs on the underlying device. Because the notebooks and configuration live on this volume, they survive instance replacement when you re-attach or restore the volume.

Step 10: Enable HTTPS

The notebook UI is served over plain HTTP on port 80 by nginx. For production use, place it behind TLS. Obtain a certificate for your domain (for example with a managed certificate on an Application Load Balancer in front of the instance, or with Certbot installed on the instance), then configure nginx to listen on 443 with your certificate and proxy to 127.0.0.1:8080 exactly as the bundled site does for port 80, keeping the WebSocket upgrade headers and the Apache Shiro login in place. Restrict the security group so ports 80 and 443 are reachable only from the networks that use the notebook.

Step 11: Backup and Maintenance

Back up your notebooks and configuration by snapshotting the /var/lib/zeppelin EBS volume, which captures the entire data tier. Apply operating system security updates with sudo apt-get update && sudo apt-get upgrade and reboot when a new kernel is installed; Apache Zeppelin and nginx start automatically on boot, and the notebook server takes roughly 45 to 90 seconds to become ready again after a reboot.

Support

This image is published and supported by cloudimg. Support covers deployment, interpreter configuration, notebook authoring, Apache Shiro authentication, data source connectivity, TLS and JVM tuning. Contact cloudimg through the support channel listed on the AWS Marketplace listing.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.