Debugging an EKS LoadBalancer That Won't Go Healthy
The symptom
A LoadBalancer-type Service on EKS (a classic ELB, provisioned by the in-tree
AWS cloud provider) was returning curl: (52) Empty reply from server, and
aws elb describe-instance-health showed every registered instance as
OutOfService.
Three unrelated problems turned out to be layered on top of each other. None of them alone would have been obvious from the symptom — the fix only became clear by checking each layer of the traffic path independently, from the Kubernetes Service outward to AWS networking.
The traffic path
Internet --> ELB (port 80) --> NodePort 31148 on a worker node --> pod (port 8080)
Every layer in that chain has its own way of silently dropping traffic. The debugging approach that worked: verify each hop independently, starting from the Kubernetes side (cheapest to check) and working outward to AWS.
Root cause 1 — Service selector didn’t match the pod’s labels
kubectl get endpoints hk-deployment
# NAME ENDPOINTS AGE
# hk-deployment <none> 17h
An empty ENDPOINTS list means the Service’s selector matched zero pods.
Kubernetes does exact key+value matching on labels — there’s no fuzzy or
partial match:
# deployment.yml — pod template label
labels:
app: hk
# service-lb.yml — selector (BEFORE)
spec:
selector:
app.kubernetes.io/name: hk # different KEY, same value "hk"
app and app.kubernetes.io/name are two different keys. Even though both
happened to equal "hk", the Service considered this zero matching pods —
same as if the value were also different. Fix: make the selector key match
the pod label key exactly (app: hk).
Gotcha: this is invisible unless you specifically check
kubectl get endpoints <service>. kubectl get svc alone looks completely
normal — Service, ClusterIP, and NodePort all show up fine even with zero
working endpoints.
Root cause 2 — the ELB was only enabled in one Availability Zone
Even after fixing the selector, most targets stayed OutOfService. The
actual reason code from AWS was specific and useful:
Instance is in the EC2 Availability Zone for which LoadBalancer
is not configured to route traffic to.
The Service’s subnet annotation only listed one subnet:
annotations:
service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-06675bda1a1539f1f
A subnet belongs to exactly one Availability Zone. A classic ELB only creates a load-balancer node in the AZ(s) of the subnets it’s given — registering an instance from a different AZ does not make the ELB route to it, even with cross-zone load balancing on. Cross-zone balancing spreads load across already-enabled AZs; it doesn’t add new ones.
Fix: list one subnet per AZ, comma-separated, in the same annotation:
service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-06675bda1a1539f1f,subnet-09ffb20c4da788637,subnet-0b4a2635964d4983e
Gotcha: this annotation’s value is a plain string, not a YAML list —
no square brackets. [subnet-a, subnet-b] would parse as an actual YAML
array and break the annotation’s expected string type. Comma-separated, no
brackets; spaces after the commas don’t seem to matter but there’s no reason
to risk it.
Root cause 3 (a red herring, sort of) — the security group work wasn’t the fix
A lot of time went into adding a security group rule allowing port 31148 (the NodePort) from the ELB’s security group, on each worker group individually. It turned out to be unnecessary: the shared EKS cluster security group (attached to every node automatically by the Terraform EKS module) already had a broad rule:
Type: All traffic, All ports
Source: <the ELB's own security group>
Security group rules are additive across every SG attached to an interface — if any attached security group allows the traffic, it’s allowed, regardless of what the others say. That one existing rule already covered the specific NodePort rule the team spent time adding elsewhere.
Lesson, not really a gotcha: when a symptom looks like a networking problem, it’s tempting to fix the security groups first because they’re the most familiar knob to turn. Worth checking all security groups attached to the actual network interface (not just the obviously-named ones) before concluding a specific rule is missing — a shared/default rule elsewhere in the stack might already cover it.
Other gotchas that came up along the way
NodePort values are randomly assigned, not fixed. A LoadBalancer or
NodePort Service without an explicit nodePort: field gets a random port
from the cluster’s NodePort range (typically 30000–32767) the first time
it’s created. If the Service is ever deleted and recreated (not just
applyd again), it can get a different port — silently breaking any
security group rule that references the old port number by value. Pin it
explicitly if you want it stable:
ports:
- port: 80
targetPort: 8080
nodePort: 31148 # otherwise this is random
git and infrastructure state are two separate systems. terraform apply and kubectl apply both act on whatever is in your local files right
now — committed or not, pushed or not. It’s entirely possible to change real
AWS/cluster state locally without a single commit existing anywhere, which is
exactly how a teammate’s local terraform apply caused confusion earlier in
this project: the change was real, but invisible to anyone looking at
GitHub.
kubectl always reads live cluster state, not local files. kubectl get pod --show-labels and kubectl get svc -o jsonpath=... ask the API server
directly — they reflect whatever was last applyd to the cluster, which can
differ from what’s currently sitting in your working directory if you’ve
edited a file since the last apply (in either direction).
A smart cd (zoxide, autojump, etc.) can silently jump you to the wrong
directory. If your shell aliases cd to a fuzzy-jump tool
(zoxide init --cmd cd), running cd /exact/path/that/no/longer/exists
doesn’t error — it silently fuzzy-matches to the closest known directory
instead, which might be a different clone of the same repo. Confirm you’re
where you think you are with pwd and git rev-parse --show-toplevel
before trusting output, especially after multiple clones of a repo have
accumulated on disk.
ELB reason codes are worth reading literally. aws elb describe-instance-health doesn’t just say “unhealthy” — the description
field gives the actual cause (Instance registration is still in progress,
Instance is in the EC2 Availability Zone for which LoadBalancer is not configured to route traffic to, Instance has failed at least the UnhealthyThreshold number of health checks). Each of these points to a
completely different fix; don’t treat “OutOfService” as one undifferentiated
problem.
Debugging order that worked
kubectl get endpoints <service>— confirms the Service has something to route to at all, before touching anything network-related.kubectl get svc <service> -o jsonpath='{.spec.selector}'vskubectl get pod <pod> --show-labels— catches selector/label mismatches directly.aws elb describe-instance-health --load-balancer-name <name>— read theDescriptionfield, not just theState.aws elb describe-load-balancers— checkAvailabilityZones/Subnetsagainst the AZs your actual instances are running in.- Only then, security groups — check every SG attached to the relevant network interface, not just the one with the obvious name.