Arroyo

AWS Streaming & Messaging

run SQL over real-time data streams as stateful, fault tolerant pipelines

Base
Hardened build
minimal ports, security patches applied at build time
Access
Unique credentials
generated on first boot, readable only by root
Verified
Boots working
services pass a health gate before release
Support
24/7, 365 days
by email and live chat, 24 hour response SLA

Overview

Arroyo is a distributed stream processing engine that lets you run SQL over real-time data streams. You express your logic as a SQL query and Arroyo turns it into a stateful, fault-tolerant pipeline that processes events continuously as they arrive, with event-time windows, streaming joins, stateful aggregation and exactly-once checkpointing. It connects to Kafka, Kinesis, Redpanda, Pulsar, MQTT and more, and pipelines are authored in a browser console with a SQL editor and live query preview.

Why the cloudimg image

The cloudimg image runs Arroyo preconfigured behind an nginx reverse proxy as a self-contained single-node cluster with an embedded metadata store, so there is no external database to provision. Arroyo ships with no authentication of its own, so the image binds every internal service to loopback and adds a login prompt plus a per-instance engine key, both generated fresh on first boot and never baked in. It comes with a paired deployment guide and 24/7 cloudimg support.

Common uses

  • Turn a Kafka or Kinesis stream into a live materialized view with a single SQL query
  • Detect anomalies and raise alerts on sensor or event data in real time
  • Continuously aggregate and window high-volume event streams with exactly-once correctness

Key features

  • Run SQL over real-time data streams in minutes - Arroyo is preinstalled behind an nginx reverse proxy as a self-contained single-node cluster with an embedded metadata store, so there is no external database to provision and a working stream processing engine is running within minutes of launch.
  • Secure by default from the first boot - Arroyo ships with no authentication of its own, so this image binds every internal service to loopback and adds two layers: an nginx login prompt and a per-instance engine key the proxy presents on your behalf. The administrator password and engine key are generated uniquely on each instance and never baked into the image.
  • Production-grade streaming semantics - event-time windows, streaming joins, stateful aggregation and exactly-once processing backed by periodic checkpointing, with built-in sources and sinks for Kafka, Kinesis, Redpanda, Pulsar, MQTT and more. Backed by 24/7 cloudimg technical support via email and chat.

See it running

Real screenshots taken while testing this image against its deployment guide.

Arroyo screenshot 1 Arroyo screenshot 2 Arroyo screenshot 3 Arroyo screenshot 4

Description

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

## Why This Image Instead of Self-Installing

Standing up Arroyo yourself means fetching and verifying the release, writing a cluster configuration, deciding on a metadata store, binding every internal service so it is not exposed, adding an authentication layer that Arroyo does not provide on its own, and fronting it all with a reverse proxy. This image resolves every one of those decisions: launch an instance, read your unique credentials over SSH, and open the console in your browser. A working stream processing engine is running within minutes of launch, with no manual package installation, no dependency troubleshooting, and no proxy configuration required.

## Overview

Arroyo is a distributed stream processing engine that lets you run SQL over real-time data streams. Instead of writing and operating bespoke streaming applications, you express your logic as a SQL query and Arroyo turns it into a stateful, fault-tolerant pipeline that processes events continuously as they arrive. It supports event-time semantics with watermarks, tumbling, hopping and sliding windows, streaming joins, stateful aggregation, user-defined functions, and exactly-once processing backed by periodic checkpointing, so results stay correct even across restarts and late-arriving data.

## Connect To Your Data

Arroyo connects to the systems streaming data already lives in, with built-in sources and sinks for Kafka, Redpanda, Kinesis, Pulsar, MQTT, NATS, WebSockets, server-sent events, Delta Lake, Iceberg and plain filesystem or object storage. Pipelines are authored in a browser-based console with a SQL editor, live query preview, a visual pipeline graph and per-operator throughput and checkpoint metrics, so you can see exactly what a query is doing before and after you start it.

## Self-Contained Single Node

This image runs Arroyo as a self-contained single-node cluster using an embedded metadata database, so there is no external database to provision, and pipeline state is kept on the machine's own disk. Message sources and sinks are yours to wire up: point Arroyo at your own Kafka, Kinesis or other broker as described in the deployment guide.

## Secure By Default

Arroyo has no authentication of its own and its services listen on all network interfaces out of the box. In this image every one of those services is bound to loopback and reached only through the nginx reverse proxy, which is ready for your TLS certificate. The console is protected by a login prompt, and the engine's own interface additionally requires a key that the proxy presents on your behalf, so nothing reaches the engine unauthenticated. There is no default password to change: the administrator credential and the engine key are generated uniquely on each machine's first boot and are never baked into the image. systemd manages the service for automatic restarts and clean logging, and the credentials are written before the proxy starts, so the console never comes up without a login.

## Ready To Use

1. Launch the instance from the AWS Marketplace listing.

2. SSH in and read the generated administrator credentials from the root-only file.

3. Browse to the instance on port 80 and sign in.

4. Write a SQL query in the console, preview it live, and start it as a pipeline.

5. Connect an external source such as Kafka to stream your own data through it.

## cloudimg Support

24/7 technical support by email and chat for deployment, connecting external sources and sinks, sizing the engine, checkpointing and state, enabling user-defined functions, TLS termination, and upgrades.

Arroyo is distributed under the Apache License 2.0 or the MIT license, at your option. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Related technologies

stream processingsql streamingreal time datastreaming analyticskafkakinesisevent streamingstateful stream processingexactly oncestreaming pipelines