Apache SeaTunnel on AWS User Guide

Overview

This image runs Apache SeaTunnel, an open source, high performance, distributed data integration and ETL platform. SeaTunnel moves and transforms data across a very large catalogue of sources and sinks - relational databases, files, object stores, message queues and more - using its own built in distributed engine (Zeta), so you do not need a separate Apache Spark or Apache Flink cluster to run production data pipelines. Jobs are defined declaratively in configuration files and submitted through a command line client or the engine's REST API.

Apache SeaTunnel 2.3.13 is installed from the official Apache binary distribution under /opt/seatunnel and runs on OpenJDK 17. The Zeta engine runs as a standalone single node cluster, supervised by systemd as the seatunnel.service unit under a dedicated unprivileged seatunnel service account.

The engine is never exposed unauthenticated

The Zeta engine's REST API can submit and run arbitrary data jobs, which makes it high privilege, and SeaTunnel's REST API has no built in authentication. This image therefore never exposes it directly:

• The Zeta REST API (port 8080) and the Hazelcast cluster member port (5801) bind the loopback interface only. A host firewall (nftables) additionally drops any non loopback traffic to those ports, and neither port is opened in the security group.
• An nginx reverse proxy on port 443 fronts the REST API over HTTPS with HTTP Basic authentication (user admin). This authenticated HTTPS endpoint is the only way to reach the engine from off the instance.
• The only inbound ports are SSH (22) and the authenticated HTTPS endpoint (443).

Per instance credentials

The admin password, the TLS certificate and the Hazelcast cluster join secret are all generated fresh on the first boot of every deployed instance. Two instances launched from the same Amazon Machine Image never share a password, a certificate or a cluster secret, and no usable default credential ships in the image. The generated password is written to /root/seatunnel-credentials.txt with mode 0600 so that only the root user can read it.

Data volume

Checkpoints, job history, job logs and job output live on a dedicated EBS data volume mounted at /var/lib/seatunnel, separate from the operating system disk, so the data tier can be resized independently and checkpoint state survives restarts.

Prerequisites

Before you deploy this image you need:

• An Amazon Web Services account where you can launch EC2 instances
• IAM permissions to launch instances, create security groups, and subscribe to AWS Marketplace products
• An EC2 key pair in the target Region for SSH access to the instance
• A VPC and subnet in the target Region, with a security group allowing inbound port 22 from your management network and inbound port 443 from the trusted networks that will reach the REST API
• The AWS CLI (version 2) installed locally if you plan to deploy from the command line

Step 1: Launch the Instance from the AWS Marketplace

1. Open the product page in the AWS Marketplace and choose Continue to Subscribe, then Continue to Configuration.
2. Select the Region and the delivery option (the Ubuntu 24.04 AMI), then Continue to Launch.
3. Choose an instance type. Apache SeaTunnel's Zeta engine is a JVM application, so m5.large (2 vCPU, 8 GiB RAM) is a sensible starting point; scale up for heavier pipelines.
4. Select your VPC, subnet and key pair.
5. Configure the security group to allow inbound TCP 22 (SSH) and TCP 443 (HTTPS REST API) from your trusted source ranges only. Do not open these ports to 0.0.0.0/0 in production.
6. Launch the instance.

Step 2: Launch from the AWS CLI (Alternative)

If you prefer the command line, launch the AMI directly. Replace the placeholder identifiers with values for your account and Region.

aws ec2 run-instances \
  --image-id <ami-id> \
  --instance-type m5.large \
  --key-name <your-key-pair> \
  --security-group-ids <your-security-group-id> \
  --subnet-id <your-subnet-id> \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=apache-seatunnel}]'

Step 3: Connect to the Instance over SSH

Connect as the default login user for your operating system variant. The password, the cluster secret and the TLS certificate are generated on first boot, so allow a minute or two after launch before the REST API answers.

ssh -i /path/to/your-key.pem ubuntu@<public-ip>

Step 4: Retrieve Your Per Instance Credentials

The first boot service writes the generated admin password to a root only file. Retrieve it with:

sudo cat /root/seatunnel-credentials.txt

You will see the REST endpoint, the admin user and the unique password for this instance:

SEATUNNEL_REST_URL=https://<public-ip>/overview
SEATUNNEL_REST_USER=admin
SEATUNNEL_REST_PASSWORD=<generated-per-instance>
SEATUNNEL_HOME=/opt/seatunnel
SEATUNNEL_DATA=/var/lib/seatunnel

Step 5: Confirm the Engine Is Running

The Zeta engine is supervised by systemd. Confirm the engine and the HTTPS front are active:

systemctl is-active seatunnel.service nginx.service

active
active

The engine exposes a status endpoint on the loopback interface. From the instance itself you can query it directly:

curl -s http://127.0.0.1:8080/overview

{
    "projectVersion": "2.3.13",
    "gitCommitAbbrev": "a399ae8",
    "totalSlot": "0",
    "unassignedSlot": "0",
    "runningJobs": "0",
    "finishedJobs": "0",
    "failedJobs": "0",
    "pendingJobs": "0",
    "cancelledJobs": "0",
    "workers": "1"
}

Step 6: Reach the REST API over HTTPS

From off the instance the engine is reachable only through the authenticated HTTPS reverse proxy. Requests without valid credentials are refused with 401:

# No credentials -> refused
curl -sk -o /dev/null -w '%{http_code}\n' https://<public-ip>/overview
# 401

# With the per instance admin credential -> accepted
curl -sk -u admin:<password> https://<public-ip>/overview
# {"projectVersion":"2.3.13", ...}

The certificate is self signed and unique to this instance, so curl -k (or importing the certificate into your trust store) is required. For production, place your own TLS certificate at /etc/nginx/ssl/seatunnel.crt and /etc/nginx/ssl/seatunnel.key and reload nginx, and keep port 443 restricted to trusted source ranges in your security group.

Step 7: Run Your First Data Pipeline

The image ships two ready to run sample jobs so you can confirm an end to end ETL round trip immediately. Both use a FakeSource that generates deterministic rows.

Run the FakeSource to Console job (10 rows) in local mode:

sudo -u seatunnel /opt/seatunnel/bin/seatunnel.sh \
  --config /opt/seatunnel/config/cloudimg-sample.streaming.conf -m local

The engine prints a job statistic summary confirming the rows processed:

           Job Statistic Information
Start Time                : 2026-07-24 21:30:43
End Time                  : 2026-07-24 21:30:47
Total Time(s)             :                   3
Total Read Count          :                  10
Total Write Count         :                  10
Total Failed Count        :                   0

The second sample writes durable JSON output to the data volume:

sudo -u seatunnel /opt/seatunnel/bin/seatunnel.sh \
  --config /opt/seatunnel/config/cloudimg-sample.file.conf -m local
ls -l /var/lib/seatunnel/sample-output/

Step 8: Build Your Own Jobs

A SeaTunnel job is a single configuration file with three sections - env, source and sink (with optional transform steps). Create your own under /opt/seatunnel/config/ and submit it the same way:

sudo -u seatunnel /opt/seatunnel/bin/seatunnel.sh --config /opt/seatunnel/config/my-job.conf -m local

The image ships only redistributable, Apache-2.0-licensed connectors: connector-fake, connector-console, connector-jdbc (the JDBC framework) and connector-file-local. Add any other SeaTunnel connector by listing it in /opt/seatunnel/config/plugin_config and running /opt/seatunnel/bin/install-plugin.sh 2.3.13.

Adding a proprietary JDBC driver

To keep the image redistributable, no proprietary database driver (for example Oracle ojdbc or MySQL Connector/J) is bundled. When you use the JDBC connector with such a database, download the vendor driver JAR yourself and place it in /opt/seatunnel/lib, then restart the engine:

sudo cp mysql-connector-j-8.x.x.jar /opt/seatunnel/lib/
sudo chown seatunnel:seatunnel /opt/seatunnel/lib/mysql-connector-j-8.x.x.jar
sudo systemctl restart seatunnel.service

Data Volume Layout

The dedicated EBS data volume is mounted at /var/lib/seatunnel by filesystem UUID (so it survives reboots and re-launch):

/opt/seatunnel holds the binaries and configuration (config/seatunnel.yaml, config/hazelcast.yaml). To grow the data tier, expand the EBS volume in the console and run sudo resize2fs /dev/nvme1n1.

Security Hardening

• Keep the engine off the public network. Ports 8080 and 5801 bind loopback only and are dropped by the host firewall for any non loopback source; keep it that way.
• Restrict port 443. In the security group, allow inbound 443 only from the specific CIDRs that need the REST API.
• Bring your own TLS certificate for production rather than the per instance self signed certificate, and consider fronting the endpoint with an Application Load Balancer that terminates TLS.
• Rotate the admin password by re-running the first boot rotation, or update the nginx htpasswd directly: sudo htpasswd /etc/nginx/.seatunnel_htpasswd admin then sudo systemctl reload nginx.
• Restrict SSH to your management network.

Backups

Take periodic EBS snapshots of the data volume (/var/lib/seatunnel) to preserve checkpoint state and job output. Job configuration files under /opt/seatunnel/config should be kept in your own version control.

Troubleshooting

• The REST API returns 401 with the right password. Confirm you are using the admin user and the exact password from /root/seatunnel-credentials.txt, and that first boot has completed (test -f /var/lib/cloudimg/apache-seatunnel-firstboot.done).
• A job fails with a driver error. Confirm the vendor JDBC driver JAR is present in /opt/seatunnel/lib and that you restarted seatunnel.service afterwards.
• Inspect the engine logs. sudo journalctl -u seatunnel.service -n 200 --no-pager and the per job logs under /var/lib/seatunnel/logs.

Support

This Amazon Machine Image is provided by cloudimg with 24/7 technical support by email and live chat. Our engineers assist with SeaTunnel deployment, connector and job configuration, engine tuning and upgrades. Critical issues receive a one hour average response time. Contact support@cloudimg.co.uk.

Apache SeaTunnel, SeaTunnel and Apache are trademarks of the Apache Software Foundation. All other product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.