By INI8 Labs · 2026-05-21 · 9 min read
Kubernetes Architecture Explained: A Visual Guide for Engineering Leaders
Kubernetes adoption is no longer a debate. According to the CNCF's 2025 Annual Survey, 82% of container users now run Kubernetes in production — up from 66% in 2023. Additionally, 66% of organizations deploying generative AI models rely on Kubernetes to manage inference workloads.
But here's the disconnect: most engineering leaders approving Kubernetes investments can't clearly explain how the system actually works. They know it "orchestrates containers" and it's "the industry standard." What they often lack is a working mental model of the architecture — the control plane, worker nodes, networking, and storage abstractions that determine whether Kubernetes serves their organization well or becomes an operational tax.
This guide provides that mental model. Not the deep-dive reference for a Kubernetes administrator, but the architectural understanding that engineering leaders need to make informed decisions about cluster design, high availability, security boundaries, and when Kubernetes is — or isn't — the right choice.
The Two-Layer Architecture
Kubernetes separates decision-making from execution. Think of it as air traffic control.
The control plane is the air traffic control tower. It stores the desired state of the entire system, makes scheduling decisions about where workloads run, and continuously drives the system toward that desired state. When something drifts — a pod crashes, a node becomes unhealthy — the control plane detects the deviation and triggers corrective action.
Worker nodes are the aircraft. They execute instructions, run your application containers, enforce networking behavior, and report their status back to the control plane. Their job is to operate predictably and reliably.
When the control plane is unavailable, no new workloads can be scheduled and no changes can be applied — but existing workloads continue running. When worker nodes fail, applications experience outages. This separation means you can design different availability strategies for each layer.
Control Plane Components
The control plane consists of four core components that work together.
API Server (kube-apiserver)
The front door to the cluster. Every interaction with Kubernetes — from kubectl commands to automated controllers — passes through the API server. It authenticates requests, validates data, and serves as the central coordination point between all other components.
Why this matters for leaders: The API server is the single most critical component. It must be highly available for production clusters. It also supports horizontal scaling — you can run multiple instances behind a load balancer. Every monitoring, security, and access control decision for your cluster starts here.
etcd
A distributed key-value store that holds the entire cluster state — every deployment, service, configuration, and secret. Only the API server interacts with etcd directly. If etcd is lost and unrecoverable, the cluster state is gone.
Why this matters for leaders: etcd is your cluster's memory. It requires robust backup strategies and, for production, should run as an odd-numbered cluster (3 or 5 instances) to maintain quorum-based leader election. etcd performance directly impacts cluster responsiveness — slow etcd means slow everything.
Scheduler (kube-scheduler)
When a new pod needs to be placed on a node, the scheduler decides where it goes. It evaluates available resources, affinity rules (which pods should run together), taints and tolerations (which pods shouldn't run on certain nodes), and constraints to find the optimal placement.
Why this matters for leaders: Scheduling decisions directly affect application performance and cost. Poorly configured scheduling leads to hotspots (overloaded nodes) or waste (underutilized capacity). Understanding scheduler behavior is key to Kubernetes cost optimization.
Controller Manager (kube-controller-manager)
A collection of control loops that watch the cluster state through the API server and make corrective actions. The ReplicaSet controller ensures the right number of pod replicas are running. The Node controller monitors node health. The Deployment controller manages rollouts and rollbacks.
Why this matters for leaders: Controllers are what make Kubernetes self-healing. They're the reason a crashed pod gets recreated automatically and a failed node's workloads get rescheduled. Understanding controllers explains why Kubernetes keeps fixing things without being told.
Worker Node Components
Every worker node runs three components that coordinate with the control plane.
Kubelet
The agent on each node that receives instructions from the control plane, ensures pods are running as specified, applies runtime and networking behavior, and reports status back. The kubelet closes the feedback loop — without it, the control plane loses visibility into what's actually happening on the node.
Key detail for 2026: As of Kubernetes v1.35, kubelet can adjust a pod's CPU and memory requests and limits while the pod is running, often without restarting the container. This in-place resource resizing capability significantly improves operational flexibility.
Kube-proxy
Handles networking on each node, routing service traffic to the correct pods. Kube-proxy maintains network rules that allow pods to communicate across nodes and external traffic to reach services inside the cluster.
Container Runtime
The software that actually runs containers. Docker Engine is no longer supported as a Kubernetes runtime. Modern deployments use containerd (which Docker itself relies on internally) or CRI-O. From Kubernetes v1.26 onward, runtimes must support the v1 CRI (Container Runtime Interface) API.
Pods: The Smallest Deployable Unit
A pod is the atomic unit in Kubernetes — a logical grouping of one or more containers that share networking and storage. In most production scenarios, a pod contains a single application container.
Why pods matter, not containers: Kubernetes doesn't manage individual containers. It manages pods. This abstraction allows sidecar containers (logging agents, service mesh proxies) to run alongside application containers, sharing the same network namespace and storage volumes.
Pods are ephemeral by design. They can be created, destroyed, and rescheduled at any time. This ephemerality is why Kubernetes uses higher-level abstractions — Deployments, StatefulSets, DaemonSets — to manage pod lifecycle rather than expecting you to manage individual pods.
Networking: The Complexity That Catches Everyone
Kubernetes networking follows a flat model: every pod gets its own IP address, and any pod can communicate with any other pod across the cluster without NAT (Network Address Translation). This simplifies application design but requires a Container Networking Interface (CNI) plugin to implement.
Popular CNI choices in 2026:
- Cilium — eBPF-based, high performance, advanced security policies. Increasingly the default for production clusters.
- Calico — policy-based networking and routing, strong for network policy enforcement.
- Flannel — simple overlay networking, good for smaller clusters and learning environments.
Services provide stable networking endpoints for pods. Since pods are ephemeral (they get new IPs when recreated), Services provide a consistent DNS name and IP address that routes to healthy pods. Service types (ClusterIP, NodePort, LoadBalancer, Ingress) determine how traffic reaches your applications from inside and outside the cluster.
Storage: Stateful Workloads in a Stateless System
Local storage on Kubernetes nodes is ephemeral — deleted when a pod shuts down. For stateful applications (databases, message queues, file storage), Kubernetes provides Persistent Volumes (PV) and Persistent Volume Claims (PVC) that decouple storage lifecycle from pod lifecycle.
Why this matters: Running stateful workloads on Kubernetes is now routine, but it requires understanding the storage abstractions. A misconfigured storage class can lead to data loss. For production stateful workloads, use managed storage (cloud provider block storage, EBS, Azure Disk) with appropriate backup strategies.
High Availability Architecture
For production, a single control plane node is a single point of failure. High availability requires:
- Multiple control plane nodes (typically 3 or 5) running API server, scheduler, and controller manager
- Replicated etcd (3 or 5 instances for quorum)
- Worker nodes across availability zones for resilience against zone failures
- Load balancing in front of API server instances
Most enterprises use managed Kubernetes (EKS, AKS, GKE) where the cloud provider handles control plane high availability. This eliminates significant operational burden but reduces control over control plane configuration.
When Kubernetes Is — and Isn't — the Right Choice
Kubernetes is the right choice when you're running multiple services that need independent scaling, deployment, and lifecycle management. It excels at microservices architectures, containerized workloads, multi-team environments, and applications that need to run consistently across multiple cloud providers.
Kubernetes is not the right choice for a single monolithic application, small teams without dedicated operations capacity, or environments where the operational complexity isn't justified by the scale. XplorX exists specifically to reduce this operational barrier — providing managed Kubernetes with built-in DevOps so teams can focus on building instead of managing infrastructure.
The architecture understanding matters because it drives these decisions. If you understand that the control plane is a distributed system requiring its own availability strategy, that networking complexity grows with cluster size, and that storage management for stateful workloads requires deliberate design — you can make informed decisions about whether the investment is justified for your use case.
FAQ
How many nodes do we need for a production Kubernetes cluster?
Start with 3 worker nodes across different availability zones for basic high availability. Scale based on workload requirements. Most mid-market production clusters run 5–20 worker nodes. For managed Kubernetes (EKS, AKS, GKE), the control plane nodes are managed by the provider. For self-managed clusters, add 3–5 control plane nodes for high availability.
What's the operational cost of running Kubernetes?
Managed Kubernetes control plane costs $75–$250/month per cluster. Compute costs depend on worker node sizing and count. A typical mid-market production cluster (3 control plane + 5 worker nodes on m5.xlarge equivalent) costs $2K–$5K/month on AWS/Azure/GCP. The larger cost is operational — the engineering time to manage clusters, security, upgrades, and troubleshooting. This is why many teams opt for managed platforms.
Should we use managed Kubernetes or self-manage?
Managed Kubernetes (EKS, AKS, GKE) is the right choice for the vast majority of organizations. It eliminates control plane operations, handles upgrades, and integrates with cloud provider services. Self-managed (kubeadm, kOps, Rancher) makes sense only for air-gapped environments, extreme customization requirements, or edge deployments where cloud managed services aren't available.
How does Kubernetes handle application updates without downtime?
Kubernetes uses rolling updates by default. When you update a Deployment, it gradually replaces old pods with new ones — maintaining the desired number of healthy pods throughout the process. Combined with readiness probes (which prevent traffic from routing to pods that aren't ready) and pod disruption budgets (which control how many pods can be unavailable simultaneously), zero-downtime deployments are achievable for most applications.
Understanding Kubernetes architecture is the first step. Building production clusters that are reliable, secure, and cost-optimized is the next. INI8 Labs helps engineering teams design and manage Kubernetes infrastructure — from initial cluster architecture to platform engineering and cost optimization.