Transmission 036 ended with the EKS cluster on the wrong VPC and a list of manual steps queued for next session.

Today the goal was: live state backend, old cluster deleted, fixed plan applied, cluster on Kubernetes 1.32.

The constraint

Three blockers stood between the current state and a clean apply:

1. The S3 bootstrap module was written but never applied — state was still local and unversioned
2. The old EKS cluster (nexusflow, Kubernetes 1.31, old VPC) was still running — OpenTofu’s create-before-destroy replacement strategy cannot work when EKS prohibits two clusters with the same name in the same account
3. Two resources in phase1_security.tf duplicated what the EKS module already creates — a standalone OIDC provider and a standalone access entry that would both hit AlreadyExistsException on apply

All three had to clear before phase2b could run.

The proof

Installing the AWS CLI from scratch

The WSL session opened without aws. The first install attempt ran commands out of order — unzip executed before the package was installed, so the zip extraction failed silently and the installer was never reached. The fix was chaining all four steps so no step ran unless the previous one succeeded:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" && \
unzip -q awscliv2.zip && \
sudo ./aws/install && \
rm -rf aws awscliv2.zip

You can now run: /usr/local/bin/aws --version
aws-cli/2.36.45 Python/3.14.6 Linux/6.18.33.2-microsoft-standard-WSL2 exe/x86_64.ubuntu.24

Bootstrap: S3 bucket and DynamoDB lock table

The bootstrap module ran in infra/terraform/bootstrap. Five resources, 20 seconds:

Apply complete! Resources: 5 added, 0 changed, 0 destroyed.

state_bucket_name        = "nexusflow-tfstate-<ACCOUNT_ID>"
dynamodb_lock_table_name = "nexusflow-tfstate-lock"
state_bucket_arn         = "arn:aws:s3:::nexusflow-tfstate-<ACCOUNT_ID>"

tofu init in the main root picked up the new S3 backend and offered to copy the existing local state into it:

Successfully configured the backend "s3"! OpenTofu will automatically
use this backend unless the backend configuration changes.

State was no longer a file on disk. From this point, every apply writes to S3 and every lock goes to DynamoDB.

Removing the duplicate resources

phase1_security.tf had two standalone resources that would fail apply:

• aws_iam_openid_connect_provider.eks — the EKS module creates this automatically when enable_irsa = true. A second owner hits AlreadyExistsException.
• aws_eks_access_entry.terraform_caller + its policy association — the module creates these automatically when enable_cluster_creator_admin_permissions = true. Same result.

Both blocks were removed. The IRSA trust policy’s principal reference changed from the removed resource’s ARN to module.eks.oidc_provider_arn. The plan count dropped from 23 adds to 20.

First apply: cluster already exists

The plan was clean. Apply started. The NAT gateway came up after 98 seconds:

module.vpc.aws_nat_gateway.this[0]: Creation complete after 1m38s [id=nat-0617a709504919b5b]
module.vpc.aws_route.private_nat_gateway[0]: Modifications complete after 2s

Then the cluster:

Error: creating EKS Cluster (nexusflow): operation error EKS: CreateCluster,
StatusCode: 409, ResourceInUseException: Cluster already exists with name: nexusflow

The old 1.31 cluster from 036 was still running. OpenTofu’s default replacement strategy is create-before-destroy — it tried to create the new cluster before removing the old one. EKS rejected it. The NAT gateway was in state. Everything else was not.

Manual cluster deletion

Node group name first:

aws eks list-nodegroups --cluster-name nexusflow --region us-east-1

{ "nodegroups": ["nexusflow-nodes-20260913054042968900000016"] }

Node group must go before the cluster. Both deletions kicked off in sequence:

aws eks delete-nodegroup \
  --cluster-name nexusflow \
  --nodegroup-name nexusflow-nodes-20260913054042968900000016 \
  --region us-east-1

aws eks delete-cluster --name nexusflow --region us-east-1

Both returned "status": "DELETING". The node group drained first. A second list-nodegroups came back empty:

{ "nodegroups": [] }

Second apply: the computer restarted

With the cluster gone, a fresh plan ran clean and was saved as phase2b.tfplan. The apply started. OpenTofu began provisioning the EKS control plane — the slow part, routinely 8–12 minutes:

module.eks.aws_eks_cluster.this[0]: Still creating... [8m40s elapsed]

The computer restarted.

AWS does not stop when the client disconnects. EKS finished provisioning the cluster and node group in the cloud. OpenTofu never wrote the completed state back to S3. The DynamoDB lock held — it was still owned by the crashed apply process.

After the restart: stale lock, split state

The first tofu command back in WSL:

Error: Error acquiring the state lock

Lock Info:
  Operation: OperationTypeApply
  Path:      nexusflow/terraform.tfstate

The lock was from the crashed session. Cleared with tofu force-unlock. A tofu state list showed the VPC, KMS key, IAM roles, ECR repos, and the NAT gateway — but no cluster and no node group. They existed in AWS and nowhere in state.

I thought about running tofu import right then. I decided not to. Importing against a cluster that may still be in a post-boot settling state, without first confirming the exact resource IDs from AWS, is how the next round of drift starts. The cluster was not going anywhere. The right move was to stop, rest, and import from a clean session with a clear head rather than a tired one that had already watched two applies fail today.

Where this sits

Next session: confirm resource IDs from AWS, import cluster and node group into state, then plan and apply the remaining resources cleanly.