Create Deployment

This guide walks you through creating an EZ-CDC deployment in your cloud account (AWS or GCP).

Prerequisites

Before creating a deployment, ensure you have:

• Completed AWS Setup or GCP Setup
• Noted your VPC/network details and subnet/subnetwork IDs

Create a Deployment

Step 1: Start Deployment Wizard

1. Log in to the EZ-CDC Portal
2. Navigate to Deployments → New Deployment

Step 2: Basic Configuration

Enter the basic deployment settings:

Field	Description	Example (AWS)	Example (GCP)
Name	Unique identifier for this deployment	production	production
Description	Optional description	Production CDC workers	Production CDC workers
Cloud Provider	Cloud where workers will run	AWS	GCP
Region	Region for worker instances	us-west-2	us-central1

Step 3: IAM Configuration

AWS

1. Deploy the CloudFormation template to create the EZ-CDC deployment role in your account.
2. Paste the IAM Role ARN from the CloudFormation stack output:

   arn:aws:iam::123456789012:role/ez-cdc-deployment-role

3. Click Validate Role to verify permissions.

GCP

1. Enter your GCP Project ID:

   my-project-123456

2. EZ-CDC will automatically create a Service Account in your project using Workload Identity Federation — no exported keys required.
3. Click Validate to confirm permissions.

Step 4: Network Configuration

AWS

Select your VPC and subnets:

Field	Description
VPC	The VPC where workers will run
Subnets	Private subnets for worker instances (multi-AZ recommended)

tip

Select subnets in multiple Availability Zones for high availability.

GCP

Select your VPC network and subnetwork:

Field	Description
VPC Network	The VPC network where workers will run
Subnetwork	The subnetwork for worker instances

Step 5: Connectivity Mode

Choose how workers connect to the control plane:

Standard (AWS & GCP)

Workers connect via public internet using HTTPS/gRPC:

Worker → NAT Gateway / Cloud NAT → Internet → Control Plane

• ✅ Simpler setup
• ✅ No additional cloud resources
• ⚠️ Traffic traverses internet (encrypted)

PrivateLink (AWS only)

Workers connect via AWS PrivateLink:

Worker → VPC Endpoint → PrivateLink → Control Plane

• ✅ No internet exposure
• ✅ Private IP connectivity
• ⚠️ Additional setup required
• ⚠️ PrivateLink costs apply

See PrivateLink Setup for details.

Cloud NAT (GCP only)

Workers have no public IP and reach the control plane through Cloud NAT:

Worker → Cloud NAT → Internet → Control Plane

• ✅ Workers have no public IP
• ✅ Egress via Cloud NAT + mTLS
• ⚠️ Additional Cloud NAT setup required

See Cloud NAT Setup for details.

Step 6: Worker Configuration

Configure the worker instances:

Field	Default (AWS)	Default (GCP)	Description
Instance Type	c5.2xlarge	c2-standard-8	Instance/machine type
Min Instances	1	1	Minimum workers (ASG/MIG)
Max Instances	3	3	Maximum workers (ASG/MIG)
Volume Size	20 GB	20 GB	Boot disk size

Step 7: Create

1. Review your configuration
2. Click Create Deployment

EZ-CDC will provision the infrastructure:

Creating deployment...
├─ Creating security group / firewall rules  ✓
├─ Creating IAM instance profile             ✓
├─ Creating launch template / instance template  ✓
├─ Creating Auto Scaling Group / MIG         ✓
└─ Waiting for workers                       ⏳

This typically takes 2-3 minutes.

Verify Deployment

In the Portal

Once complete, you'll see:

Deployment: production
Status: ● Active
Region: us-west-2
Workers: 1/1 healthy
Connectivity: Standard

In AWS Console

Verify the created resources:

EC2 → Instances:

Name: ez-cdc-production-worker
Instance ID: i-0abc123def456
State: running

EC2 → Auto Scaling Groups:

Name: ez-cdc-production-asg
Desired: 1
Running: 1

EC2 → Security Groups:

Name: ez-cdc-production-worker-sg
Inbound: (none)
Outbound: 443, 5432, 8040, 9030

In GCP Console

Verify the created resources:

Compute Engine → VM Instances:

Name: ez-cdc-production-worker-xxxx
Status: RUNNING
Zone: us-central1-a

Compute Engine → Instance Groups (MIG):

Name: ez-cdc-production-mig
Target size: 1
Running: 1

VPC network → Firewall Rules:

Name: ez-cdc-production-worker-egress
Direction: EGRESS
Allow: tcp:443, tcp:5432, tcp:8040, tcp:9030

Worker Lifecycle

Workers go through these states:

Provisioning → Registering → Active → (Running Jobs)
      │              │           │
      │              │           └── Healthy, accepting jobs
      │              │
      │              └── Connecting to control plane
      │
      └── Instance launching (EC2 / GCE)

Health Checks

Workers report health every 30 seconds:

Metric	Description
CPU	Current CPU utilization
Memory	Current memory usage
Disk	Available disk space
Jobs	Number of running jobs

Scaling

Manual Scaling

Adjust worker count in the portal:

1. Go to Deployments → your deployment
2. Click Settings → Scaling
3. Update Min and Max instances
4. Click Save

Auto Scaling (Coming Soon)

Automatic scaling based on job queue depth and resource utilization.

Updating a Deployment

Update Configuration

Some settings can be updated without recreating:

• Instance type (requires instance refresh)
• Min/Max instances
• Volume size (new instances only)

Update Workers

To update worker binaries:

1. Go to Deployments → your deployment
2. Click Settings → Maintenance
3. Click Upgrade Workers

Workers are upgraded one at a time (rolling update).

Delete a Deployment

danger

Deleting a deployment will:

• Stop all running jobs
• Terminate all worker instances
• Delete associated cloud resources

To delete:

1. Go to Deployments → your deployment
2. Stop all running jobs first
3. Click Settings → Danger Zone
4. Click Delete Deployment
5. Confirm by typing the deployment name

Troubleshooting

Workers not becoming healthy (AWS)

1. Check worker logs in CloudWatch: /ez-cdc/workers/{instance-id}
2. Verify security group allows outbound 443
3. Check NAT Gateway is functioning
4. Verify IAM instance profile has S3 access

Workers not becoming healthy (GCP)

1. Check worker logs in Cloud Logging: filter by resource.labels.instance_id
2. Verify firewall egress rules allow outbound 443 and 50051
3. Check Cloud NAT is configured and healthy (if using Cloud NAT mode)
4. Verify the Service Account has the required roles

Deployment stuck in "Provisioning" (AWS)

1. Check CloudFormation events for errors
2. Verify IAM role permissions
3. Check VPC has available IP addresses

MIG not creating instances (GCP)

1. Check MIG status in Compute Engine → Instance Groups
2. Review instance template for errors (machine type availability in zone)
3. Verify Service Account permissions allow instance creation
4. Check project quota for the selected machine type and region

Startup-script failures (GCP)

1. View startup script output via Serial port logs in GCP Console
2. Confirm the GCS bucket with worker binaries is accessible from the instance
3. Check that the Service Account has storage.objects.get on the binary bucket

Health check failing on port 50051 (gRPC)

1. Verify the firewall egress rule allows tcp:50051 to the control plane
2. Confirm CONTROL_PLANE_ENDPOINT is set correctly in the worker configuration
3. Check that no intermediate proxy is terminating TLS (use TCP passthrough for gRPC)

Workers terminated unexpectedly

1. Check instance state/termination reason (EC2 state reason / GCE lastAttempt error)
2. Review Auto Scaling / MIG activity history
3. Check for Spot/Preemptible instance interruptions (if using Spot/Preemptible)

Next Steps

With your deployment created:

1. Add PostgreSQL Source
2. Add StarRocks Sink
3. Create Your First Job