Prerequisites
Before creating a deployment in the EZ-CDC portal, you need certain cloud resources already in place. This page explains what you need before starting the deployment wizard.
The deployment wizard guides you through the IAM/Service Account setup and automatically provisions security groups (AWS) or firewall rules (GCP), worker instances, and private connectivity resources. You only need to prepare your networking and databases beforehand.
Choose Your Cloud Provider
EZ-CDC supports deployment in both AWS and GCP:
| Cloud | Compute | Connectivity Modes |
|---|---|---|
| AWS | EC2 (Auto Scaling Group) | Standard, PrivateLink |
| GCP | GCE (Managed Instance Group) | Standard, Cloud NAT |
Choose Your Connectivity Mode
| Mode | Cloud | Description | Best For |
|---|---|---|---|
| Standard | AWS & GCP | Workers connect to control-plane via public internet (HTTPS) | Most use cases, simpler setup |
| PrivateLink | AWS only | Workers connect via AWS PrivateLink (no public internet) | Enterprise, strict security requirements |
| Cloud NAT | GCP only | Workers have no public IP, egress via Cloud NAT | Private GCP deployments |
AWS Prerequisites
Standard Mode
For standard connectivity, workers connect to the EZ-CDC control-plane over the public internet using HTTPS/TLS encryption.
What You Need
1. A VPC
Any VPC in a supported region. No special configuration required.
2. At Least 2 Subnets with Internet Access
- In different Availability Zones (for high availability)
- Must have outbound internet access
Workers need outbound connectivity to the EZ-CDC control-plane on port 443 (HTTPS/gRPC).
By default, workers run in subnets with direct internet access. If you prefer workers to remain in private subnets without public IPs, you can add a NAT Gateway to provide outbound connectivity.
3. Network Access to Your Databases
From the subnets you select, workers must be able to reach:
| Database | Port |
|---|---|
| PostgreSQL (source) | 5432 |
| StarRocks (sink) | 9030, 8040 |
Standard Mode Checklist
□ VPC in a supported region
□ 2+ subnets in different AZs with outbound internet access
□ PostgreSQL reachable from subnets (port 5432)
□ StarRocks reachable from subnets (port 9030, 8040)
PrivateLink Mode
For PrivateLink connectivity, workers connect to the EZ-CDC control-plane through AWS PrivateLink. Traffic never leaves the AWS network.
The VPC Endpoint that connects workers to the EZ-CDC control-plane is created automatically during deployment. You do not need to create it yourself.
What You Need
1. A VPC with DNS Enabled
Critical: Your VPC must have these settings enabled:
enable_dns_support = true
enable_dns_hostnames = true
Without these settings, the VPC Endpoint created during deployment will not work.
Verify with AWS CLI:
aws ec2 describe-vpc-attribute --vpc-id vpc-xxx --attribute enableDnsSupport
aws ec2 describe-vpc-attribute --vpc-id vpc-xxx --attribute enableDnsHostnames
2. At Least 2 Private Subnets
- In different Availability Zones (for high availability)
No outbound internet access is needed. Workers download binaries and communicate with the control-plane entirely through the VPC Endpoint.
3. Network Access to Your Databases
From the subnets you select, workers must be able to reach:
| Database | Port |
|---|---|
| PostgreSQL (source) | 5432 |
| StarRocks (sink) | 9030, 8040 |
PrivateLink Mode Checklist
□ VPC with DNS support ENABLED
□ VPC with DNS hostnames ENABLED
□ 2+ private subnets in different AZs
□ PostgreSQL reachable from subnets (port 5432)
□ StarRocks reachable from subnets (port 9030, 8040)
GCP Prerequisites
Standard Mode
Workers connect to the EZ-CDC control-plane over the public internet. Each worker gets an ephemeral external IP.
What You Need
1. A GCP Project with Required APIs
Enable the following APIs in your GCP project:
gcloud services enable compute.googleapis.com \
logging.googleapis.com \
monitoring.googleapis.com \
--project=YOUR_PROJECT_ID
2. A VPC Network with a Subnetwork
- A VPC network (default or custom)
- A subnetwork in the target region with outbound internet access
3. Network Access to Your Databases
From the subnetwork, workers must be able to reach:
| Database | Port |
|---|---|
| PostgreSQL (source) | 5432 |
| StarRocks (sink) | 9030, 8040 |
Standard Mode Checklist
□ GCP project with Compute, Logging, and Monitoring APIs enabled
□ VPC network with a subnetwork in the target region
□ Subnetwork has outbound internet access
□ PostgreSQL reachable from subnetwork (port 5432)
□ StarRocks reachable from subnetwork (port 9030, 8040)
Cloud NAT Mode
Workers have no public IPs. All egress goes through Cloud NAT, which is created automatically during deployment.
What You Need
1. A GCP Project with Required APIs
Same as Standard mode:
gcloud services enable compute.googleapis.com \
logging.googleapis.com \
monitoring.googleapis.com \
--project=YOUR_PROJECT_ID
2. A VPC Network with a Private Subnetwork
- A VPC network (default or custom)
- A subnetwork in the target region (no external IP required — Cloud NAT handles egress)
3. Network Access to Your Databases
From the subnetwork, workers must be able to reach:
| Database | Port |
|---|---|
| PostgreSQL (source) | 5432 |
| StarRocks (sink) | 9030, 8040 |
Cloud NAT Mode Checklist
□ GCP project with Compute, Logging, and Monitoring APIs enabled
□ VPC network with a subnetwork in the target region
□ PostgreSQL reachable from subnetwork (port 5432)
□ StarRocks reachable from subnetwork (port 9030, 8040)
Cloud Router and Cloud NAT are created automatically during deployment. You do not need to set them up yourself.
Database Requirements
Both clouds require your databases to be properly configured.
PostgreSQL Source
| Requirement | Value |
|---|---|
| Version | PostgreSQL 12+ |
| WAL Level | logical |
| Max Replication Slots | At least 1 available per job |
| Max WAL Senders | At least 1 available per job |
Verify with:
SHOW wal_level; -- Must be 'logical'
SHOW max_replication_slots; -- Must have available slots
SHOW max_wal_senders; -- Must have available senders
See PostgreSQL Requirements for detailed setup.
StarRocks Sink
| Requirement | Value |
|---|---|
| Version | StarRocks 2.5+ |
| Stream Load | Enabled |
| HTTP Port | 8040 accessible |
| MySQL Port | 9030 accessible |
See StarRocks Requirements for detailed setup.
Supported Regions
AWS
| Region | Name |
|---|---|
| us-east-1 | N. Virginia |
| us-east-2 | Ohio |
| us-west-2 | Oregon |
| eu-west-1 | Ireland |
| eu-central-1 | Frankfurt |
| ap-southeast-1 | Singapore |
GCP
| Region | Name |
|---|---|
| us-central1 | Iowa |
| us-east1 | South Carolina |
| europe-west1 | Belgium |
| asia-southeast1 | Singapore |
Summary
| Requirement | AWS Standard | AWS PrivateLink | GCP Standard | GCP Cloud NAT |
|---|---|---|---|---|
| Network | VPC + 2 subnets | VPC (DNS enabled) + 2 private subnets | VPC + subnetwork | VPC + subnetwork |
| Internet access | Required | Not required | Required (external IPs) | Not required (Cloud NAT) |
| DB access | From subnets | From subnets | From subnetwork | From subnetwork |
Created Automatically During Deployment
These resources are provisioned by EZ-CDC when you create a deployment — you do not need to set them up:
| Resource | AWS | GCP |
|---|---|---|
| Identity | IAM Role (via CloudFormation) | Service Account |
| Network Rules | Security Groups | Firewall Rules |
| Compute | Auto Scaling Group | Managed Instance Group |
| Private Connectivity | VPC Endpoint (PrivateLink only) | Cloud NAT (Cloud NAT mode only) |
| Logging | CloudWatch Log Groups | Cloud Logging (automatic) |
| Instance Permissions | IAM Instance Profile | Service Account scopes |
Next Steps
Once you have all prerequisites in place:
- AWS: AWS Setup — Configure your AWS environment
- GCP: GCP Setup — Configure your GCP environment
- Create Deployment — The wizard guides you through the rest