Sequin Postgres Change Data Capture

AWS Streaming & Messaging

Postgres change data capture, streaming every insert, update and delete to the queues and search indexes you already run

Base
Hardened build
minimal ports, security patches applied at build time
Access
Unique credentials
generated on first boot, readable only by root
Verified
Boots working
services pass a health gate before release
Support
24/7, 365 days
by email and live chat, 24 hour response SLA

Overview

Sequin is an open source change data capture platform for PostgreSQL. It reads the database write ahead log through a logical replication slot and turns every insert, update and delete into an ordered stream of change events, delivered to destinations such as Kafka, Amazon SQS, SNS and Kinesis, NATS, RabbitMQ, Redis Streams, Elasticsearch, webhooks and a pull based API. Because it reads the replication log rather than polling tables, it captures every change without adding query load to the source database and without any application code publishing events. Pipelines are built and monitored in a web console, or declared as YAML.

Why the cloudimg image

The cloudimg image ships a change data capture pipeline that is already streaming when the machine finishes booting: a source database, a publication and a pre created replication slot are provisioned on first boot, so you can write a row and read the captured event back within seconds rather than wiring connectors first. Upstream ships a well known default administrator and open self registration; both are disabled here, and every secret, including the console credential, the administrator password, the management API token and a TLS certificate carrying the machine's own address, is generated uniquely on each first boot. Capture state lives on a dedicated, independently resizable data disk. A paired deploy guide walks the whole path, and 24/7 cloudimg support stands behind it.

Common uses

  • Stream database changes into Kafka, Amazon SQS or Redis to drive event driven services without dual writes
  • Keep an Elasticsearch, Typesense or Meilisearch index continuously in step with the source database
  • Fan out audit trails and downstream data pipelines from the write ahead log with no polling and no query load

Key features

  • Sequin, the open source Postgres change data capture platform, preinstalled with a working demonstration pipeline that streams real inserts, updates and deletes from the moment your instance boots
  • Delivers change events to Kafka, Amazon SQS, SNS and Kinesis, NATS, RabbitMQ, Redis Streams, webhooks and more, built and monitored in a web console or declared as YAML, with capture state on a dedicated independently resizable data disk
  • Secure first boot disables the upstream default administrator and open registration and generates every secret uniquely per instance behind an nginx TLS front door, with 24/7 technical support from cloudimg

See it running

Real screenshots taken while testing this image against its deployment guide.

Sequin Postgres Change Data Capture screenshot 1 Sequin Postgres Change Data Capture screenshot 2 Sequin Postgres Change Data Capture screenshot 3 Sequin Postgres Change Data Capture screenshot 4

Description

This is a repackaged open source software product wherein additional charges apply for cloudimg support services.

## Capture Every Postgres Change Without Polling or Application Code

Sequin is an open source change data capture platform for PostgreSQL. It reads your database's write ahead log through a logical replication slot and turns every insert, update and delete into an ordered stream of change events, delivered exactly where you need them. Because it reads the replication log rather than polling tables, it captures every change without adding query load to your database and without requiring any application code to publish events.

## A Pipeline That Is Already Streaming On First Boot

Change data capture tools normally need connector configuration and supporting infrastructure before any data flows. This image instead ships a complete working pipeline. On first boot a source database is created with a captured table and a publication, a logical replication slot is pre created, and a sink streams every insert, update and delete into a stream. Write a row and read the captured change event back within seconds, then follow the same pattern to point Sequin at your own database.

## Supported Destinations

Change events can be delivered to Amazon SQS, Amazon SNS and Amazon Kinesis; to Kafka, NATS, RabbitMQ, Redis Streams, GCP Pub/Sub and Azure Event Hubs; to Elasticsearch, Typesense and Meilisearch; and to HTTP webhooks or a pull based API for consumers that prefer to fetch. Pipelines are built and monitored in a web console, or declared as YAML and applied at boot for infrastructure as code workflows.

## AWS Destinations

Sequin delivers change events natively to Amazon SQS, Amazon SNS and Amazon Kinesis, so you can drive event driven architectures with the AWS services you already run. The image deploys on EC2 with full control over instance sizing, storage and networking.

## Secure First Boot

Upstream Sequin ships a well known default administrator and allows open self registration. Both are disabled in this image. Instead, a one shot service generates on your instance's first boot, before anything is reachable, a database password, a cache password, an application secret key, an encryption vault key, an administrator password, a management API token, an HTTP Basic credential and a self signed TLS certificate carrying that instance's own address. The console sits behind two independent credentials and the management API behind a bearer token. An nginx front door terminates TLS and the console binds only to the loopback interface. No shared or default credential ships in the image.

## Application Stack

The Sequin server, an Elixir application built on the Phoenix framework, runs alongside a PostgreSQL instance and a Redis instance as pinned upstream containers on a private container network. Neither datastore is published to a host port.

## Dedicated Data Disk

The container image layers, the PostgreSQL data directory and write ahead log, the replication slot state and the Redis append only file all live on a separate, independently resizable data volume mounted by filesystem identifier, so capture state can grow without touching the root disk.

## Get Started

Subscribe, launch an instance, then read your unique credentials from /root/sequin-credentials.txt over SSH. The demonstration pipeline is already streaming by the time you sign in. From there, connect Sequin to your own PostgreSQL database and configure the sinks your workload needs.

## cloudimg Support

24/7 technical support by email and chat covering deployment, connecting Sequin to your own PostgreSQL database, configuring sinks and destinations, replication slot and publication management, TLS termination and upgrades.

All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Related technologies

sequinchange data capturecdcpostgrespostgresqlstreamingkafkadata integrationevent streamingreplicationwebhooksopen source