Container Orchestration and Kubernetes

The Problem It Solves

Imagine Acebook in production: one EC2 instance, one Docker container, one database. It works. Now scale it:

50,000 users — one container isn’t enough
Zero-downtime deployments — every deploy currently causes a brief outage
3am container crash — nobody noticed for four hours; users got 502 errors
Evening traffic spikes — you need more capacity, but only at peak times

The instinctive fixes — run multiple containers, write a restart script, spin up a bigger instance — all require manual effort, custom scripts, and someone awake at 3am. Container orchestration automates all of it.

What Container Orchestration Is

Container orchestration is the automation of deploying, managing, scaling, and recovering containerised applications across multiple machines.

The four things that matter most:

1. Scheduling

You declare: “I want 3 copies of this container running.” The orchestrator decides which machines to place them on based on available CPU and memory. You don’t SSH anywhere. You don’t pick servers manually.

2. Self-Healing

A container crashes — the orchestrator notices within seconds and starts a replacement. A whole machine loses power — the containers that were on it get rescheduled onto healthy machines automatically.

3. Rolling Deployments

Deploying a new version: the orchestrator takes down one old container, starts one new container, checks it’s healthy, then moves to the next. At no point is the entire app down.

4. Service Discovery

You’re running 10 copies of your app across 5 machines, each with a different IP. Other services reach them by name — the orchestrator handles routing behind the scenes. No hardcoded IPs.

Kubernetes

Kubernetes (K8s) is the most widely used container orchestration tool. Built at Google, open-sourced in 2014. The name comes from the Greek word for helmsman — the logo is a ship’s wheel, and the containers in the name are not a coincidence.

Cluster Architecture

A cluster is the set of machines running your application. Two types of node:

Cluster
├── Control plane node        ← "make sure 3 copies of this are always running"
├── Worker node 1
│   ├── Pod (your app)
│   └── Pod (your app)
└── Worker node 2
    └── Pod (your app)

Component	Role
Control plane node	The brain. Watches cluster state, compares it to what you asked for, issues instructions to close the gap. Your app never runs here.
Worker node	The machines that actually run your containers. The control plane schedules work onto them.
Pod	The smallest deployable unit. A wrapper around one or more containers that share the same network and storage. In practice, almost always one container per pod.

Research Exercise: Answers

1. You ask for 3 pods. One worker node loses power — what does Kubernetes do?

The control plane stops receiving heartbeats from the dead node.
After a short timeout (~5 minutes by default), it marks that node NotReady.
The Deployment controller sees that it’s short of its desired replica count.
It schedules replacement pods onto the remaining healthy nodes.
The Service automatically stops routing traffic to the dead node’s pods and sends it only to the new, running ones.

You sleep through it.

2. You deploy a new version and it crashes immediately — what does Kubernetes do, and what does it not do?

What it does:

The new pods fail their startup health checks (readiness/liveness probes).
Kubernetes never marks them Ready.
The rolling update halts — no more old pods are taken down.
Traffic continues to go to the old, healthy pods.

What it does NOT do automatically:

It does not roll back. The broken new pods sit there in a crash-restart loop.
You must manually run:

kubectl rollout undo deployment/your-app

Why that matters:

Auto-rollback sounds convenient, but it could be dangerous. What if the crash is intermittent? What if the “bad” version is intentional (a database migration you still need to run)? Kubernetes surfaces the failure and stops the rollout — the decision to roll back is yours.

3. A request comes in. How does Kubernetes decide which pod handles it?

A Service object sits in front of the pods. It has:

A stable ClusterIP (doesn’t change even as pods come and go)
A stable DNS name (e.g. my-app.default.svc.cluster.local)

When traffic arrives at the Service:

kube-proxy uses iptables or IPVS rules to route it to one of the ready pods.
The default algorithm is round-robin across all healthy, ready pods.
Pods that fail their health checks are automatically removed from the rotation.

You never need to know individual pod IPs. Other services just call my-app and Kubernetes handles the rest.

Vocabulary Map

What you want to do	Kubernetes term
Run N copies of a container and keep them running	Deployment
One running instance of a container	Pod
Give pods a stable name so other things can reach them	Service
A machine in the cluster that runs pods	Node
The part that watches and manages everything	Control plane
Config values passed to containers	ConfigMap
Sensitive config (passwords, API keys)	Secret

You don’t need to memorise these — you’ll use each one in practice and they’ll stick.

Why This Matters

The mental model shift Kubernetes demands: stop thinking about servers, start thinking about desired state.

Old way: “SSH into server 3 and restart the container.”
Kubernetes way: “I want 3 healthy copies of this app running.”

You declare what you want. Kubernetes figures out how to make reality match — and keeps making it match, continuously, automatically.