Conduit Data Streaming Engine

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

Conduit is an open source data streaming engine that moves data in real time between data stores and message systems. It connects sources and destinations, Apache Kafka, PostgreSQL, Amazon S3, files and many more, into declarative pipelines that stream change data continuously, with built-in connectors and processors and no JVM required. Conduit exposes an HTTP REST API and an interactive OpenAPI/Swagger web UI so you can create, start, inspect and monitor pipelines and connectors directly from the browser or from automation. This image delivers Conduit as a single Go binary, installed from the pinned upstream release with its checksum verified, run as a dedicated system user under systemd, so a working streaming engine is running within minutes of launch with no manual setup, no service wiring and no reverse proxy configuration required.

Because the Conduit API and UI have no built-in authentication, exposing them directly would be unsafe. This image is hardened for that: Conduit's HTTP and gRPC APIs are bound to the loopback interface only, and nginx fronts the API and UI on port 80 behind an HTTP Basic Auth gate whose password is generated uniquely and at random on the first boot of every instance, so no shared or default credential ever ships in the image. The password is written to a root-only file on the instance, and an unauthenticated health endpoint is provided for load-balancer probes.

The image ships a fully self-contained demo pipeline that generates sample records and logs them locally. Nothing leaves the instance, the pipeline is running and processing records on first boot, and the Swagger UI shows a live working pipeline out of the box, so you can explore a real pipeline immediately and then replace it with your own source and destination.

Security: the API and UI are reachable only through the nginx Basic Auth front on port 80 - restrict that port, and 443 if you terminate TLS, to trusted networks with your security group. Enable AWS EBS encryption on the root volume to protect pipeline configuration and state at rest. Systemd manages Conduit and nginx for automatic restarts and clean logging, the OS is fully patched at build time and continues to receive unattended security updates, and 24/7 cloudimg support is included.

Conduit is a trademark of its respective owners and is distributed under the Apache License 2.0. All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.