Introduction to Container Images and Orchestration

As modern apps shift toward microservices and cloud-native architectures, containers have become the standard for packaging and deploying software. However, running containers in production requires more than just building images—it demands scalable orchestration and intelligent management. This blog introduces container images. It explains why orchestration is essential in production environments. The blog also explores Kubernetes, the industry-standard platform for container orchestration. What Are Container Images? A container image packages app code, its runtime, libraries, and all dependencies in a predefined, portable format. This enables consistent deployment across various environments. Container runtimes—such as containerd, runC, and CRI-O—use these prebuilt images to create and run one or more containers. While these runtimes are effective on a single host, they lack scalability and fault tolerance required for production environments. In production scenarios, apps must meet several critical requirements: Fault tolerance: Automatically recover from failures. Scalability: Adjust resources based on demand. Efficient resource utilization: Optimize hardware usage. Service discovery: Enable components to find each other dynamically. External accessibility: Expose services to external clients. Seamless updates and rollbacks: Deploy new versions without downtime. Managing containers manually or through scripts becomes impractical as the number of containers grows. This is where container orchestrators come into play. What Is a Container Orchestrator? A container orchestrator automates the deployment, scaling, networking, and management of containers across multiple hosts. It treats a group of systems as a single cluster, providing: High availability: Ensures services are always accessible. Distributed workloads: Balances tasks across nodes. Optimized resource allocation: Efficiently utilizes system resources. Automated health checks and restarts: Maintains application health. Common features of orchestrators include: Cluster management: Combine multiple hosts into a unified cluster. Container scheduling: Deploy containers based on resource availability. Service discovery: Enable communication across containers, regardless of the host. Storage binding: Attach persistent storage volumes to containers. Load balancing: Distribute traffic across containers. Security policies: Control access to containerized applications. Resource optimization: Automatically manage and scale resources based on demand. Popular container orchestrators and services: Kubernetes: Open-source and cloud-agnostic; the industry standard for container orchestration. Amazon ECS: A fully managed service by AWS for running Docker containers at scale. Amazon EKS: A managed Kubernetes service by AWS. Azure Kubernetes Service (AKS): Microsoft's managed Kubernetes offering. Google Kubernetes Engine (GKE): Google's managed Kubernetes service. HashiCorp Nomad: A flexible orchestrator for containers and other workloads. Container orchestrators are platform-agnostic and can be deployed on: Bare metal servers Virtual machines (VMs) On-premises infrastructure Public clouds (AWS, Azure, Google Cloud, etc.) Hybrid cloud environments For instance, Kubernetes can be deployed on a local machine, in a private data center, or across public cloud services like AWS EC2, Google Compute Engine, or OpenStack. Understanding Kubernetes What Is Kubernetes? Kubernetes (K8s) is an open-source system that automates the deployment, scaling, and management of containerized applications. It provides a robust, extensible platform for orchestrating containers across clusters of machines, simplifying the management of distributed, cloud-native systems. Key features of Kubernetes: Automated scheduling: Assigns containers to nodes based on resource requirements and constraints. Extensibility: Supports custom resources and controllers without modifying the core codebase. Self-healing: Monitors container health and replaces failed or unresponsive containers automatically. Service discovery and load balancing: Assigns stable DNS names and IP addresses to services, distributing network traffic evenly across pods. Automated rollouts and rollbacks: Manages application updates and configuration changes incrementally, with automatic rollbacks on failure. Secret and configuration management: Separates sensitive data and configuration from application code, injecting secrets securely into the runtime environment. Storage orchestration: Mounts persistent storage from various sources dynamically, based on declarative configuration. Batch and job processing: Supports batch jobs, cron jobs, and long-running tasks with automatic retries and failure handling. Managed Kubernetes-as-a-Service (KaaS) Managed Kubernetes offerings simplify setup and o

May 7, 2025 - 09:25

Introduction to Container Images and Orchestration

As modern apps shift toward microservices and cloud-native architectures, containers have become the standard for packaging and deploying software. However, running containers in production requires more than just building images—it demands scalable orchestration and intelligent management.

This blog introduces container images. It explains why orchestration is essential in production environments. The blog also explores Kubernetes, the industry-standard platform for container orchestration.

What Are Container Images?

A container image packages app code, its runtime, libraries, and all dependencies in a predefined, portable format. This enables consistent deployment across various environments.

Container runtimes—such as containerd, runC, and CRI-O—use these prebuilt images to create and run one or more containers. While these runtimes are effective on a single host, they lack scalability and fault tolerance required for production environments.

In production scenarios, apps must meet several critical requirements:

Fault tolerance: Automatically recover from failures.
Scalability: Adjust resources based on demand.
Efficient resource utilization: Optimize hardware usage.
Service discovery: Enable components to find each other dynamically.
External accessibility: Expose services to external clients.
Seamless updates and rollbacks: Deploy new versions without downtime.

Managing containers manually or through scripts becomes impractical as the number of containers grows. This is where container orchestrators come into play.

What Is a Container Orchestrator?

A container orchestrator automates the deployment, scaling, networking, and management of containers across multiple hosts. It treats a group of systems as a single cluster, providing:

High availability: Ensures services are always accessible.
Distributed workloads: Balances tasks across nodes.
Optimized resource allocation: Efficiently utilizes system resources.
Automated health checks and restarts: Maintains application health.

Common features of orchestrators include:

Cluster management: Combine multiple hosts into a unified cluster.
Container scheduling: Deploy containers based on resource availability.
Service discovery: Enable communication across containers, regardless of the host.
Storage binding: Attach persistent storage volumes to containers.
Load balancing: Distribute traffic across containers.
Security policies: Control access to containerized applications.
Resource optimization: Automatically manage and scale resources based on demand.

Popular container orchestrators and services:

Kubernetes: Open-source and cloud-agnostic; the industry standard for container orchestration.
Amazon ECS: A fully managed service by AWS for running Docker containers at scale.
Amazon EKS: A managed Kubernetes service by AWS.
Azure Kubernetes Service (AKS): Microsoft's managed Kubernetes offering.
Google Kubernetes Engine (GKE): Google's managed Kubernetes service.
HashiCorp Nomad: A flexible orchestrator for containers and other workloads.

Container orchestrators are platform-agnostic and can be deployed on:

Bare metal servers
Virtual machines (VMs)
On-premises infrastructure
Public clouds (AWS, Azure, Google Cloud, etc.)
Hybrid cloud environments

For instance, Kubernetes can be deployed on a local machine, in a private data center, or across public cloud services like AWS EC2, Google Compute Engine, or OpenStack.

Understanding Kubernetes

What Is Kubernetes?

Kubernetes (K8s) is an open-source system that automates the deployment, scaling, and management of containerized applications. It provides a robust, extensible platform for orchestrating containers across clusters of machines, simplifying the management of distributed, cloud-native systems.

Key features of Kubernetes:

Automated scheduling: Assigns containers to nodes based on resource requirements and constraints.
Extensibility: Supports custom resources and controllers without modifying the core codebase.
Self-healing: Monitors container health and replaces failed or unresponsive containers automatically.
Service discovery and load balancing: Assigns stable DNS names and IP addresses to services, distributing network traffic evenly across pods.
Automated rollouts and rollbacks: Manages application updates and configuration changes incrementally, with automatic rollbacks on failure.
Secret and configuration management: Separates sensitive data and configuration from application code, injecting secrets securely into the runtime environment.
Storage orchestration: Mounts persistent storage from various sources dynamically, based on declarative configuration.
Batch and job processing: Supports batch jobs, cron jobs, and long-running tasks with automatic retries and failure handling.

Managed Kubernetes-as-a-Service (KaaS)

Managed Kubernetes offerings simplify setup and operations, allowing you to provision production-grade clusters with minimal effort. Examples include:

Amazon EKS
Azure Kubernetes Service (AKS)
Google Kubernetes Engine (GKE)

These platforms handle cluster provisioning, scaling, patching, and security, enabling teams to focus on application development.

Kubernetes Architecture Overview

A Kubernetes cluster consists of two main node types:

Control plane nodes: Manage the cluster and maintain its desired state.
Worker nodes: Run the containerized applications.

Image credits : https://trainingportal.linuxfoundation.org

Control Plane Node

The control plane is the brain of the Kubernetes cluster, managing cluster state, responding to user requests, scheduling workloads, and ensuring the desired state matches the actual state. Users interact with the control plane using the Kubernetes API—through the CLI (kubectl), a web UI (Dashboard), or external tools.

Core components:

API Server (kube-apiserver): Exposes the Kubernetes API, validating and processing requests, and communicating with etcd.
Scheduler (kube-scheduler): Assigns pods to nodes based on resource availability and constraints.
Controller Manager (kube-controller-manager): Runs background reconciliation loops to maintain the desired cluster state.
Cloud Controller Manager: Integrates the cluster with cloud provider APIs for storage, load balancing, and node management.
etcd: Stores all configuration and state data for the cluster, using the Raft consensus algorithm for leader election and fault tolerance.

Image credits : https://trainingportal.linuxfoundation.org

For high availability(HA), replicate control plane nodes and configure them in HA mode. In HA setups, one node acts as the leader, while others remain synchronized and ready to take over if needed.

Worker Node

Pods are the smallest deployable units in Kubernetes. A pod can contain one or more containers sharing the same network and storage context.

Image credits : https://trainingportal.linuxfoundation.org

Worker nodes host the containerized applications in pods. Each worker node contains the necessary services to run and manage these pods:

Container Runtime: Executes containers. Supported runtimes include containerd, CRI-O, and Docker (via cri-dockerd).
Kubelet: Agent that communicates with the control plane and manages pods on the node.
CRI Shim: Interfaces between the Kubelet and container runtime using the Container Runtime Interface (CRI).
kube-proxy: Manages network rules and forwards traffic to the correct pods based on Kubernetes services.
Add-ons: Optional services like DNS, logging, monitoring, and dashboards.

Networking in Kubernetes

Kubernetes networking supports four main types of communication:

Container-to-container: Containers in the same pod communicate over localhost.
Pod-to-pod: Uses the "IP-per-pod" model, with each pod receiving a unique IP.
Service-to-pod: Enables load-balanced access to pods using stable service endpoints.
External-to-service: Routes external traffic into the cluster via NodePorts, Ingress, or LoadBalancers.

Container Network Interface (CNI)

Kubernetes relies on the CNI specification to configure networking. Common CNI plugins include:

Flannel
Calico
Cilium
Weave

These plugins handle IP allocation, routing, and network policies.

Kubernetes Extensibility and Ecosystem

Kubernetes has a modular, pluggable architecture, supporting the development of:

Custom resources and operators
Custom APIs and admission controllers
Custom scheduling rules and plugins

This flexibility enables you to tailor Kubernetes to your specific needs, especially in complex microservices environments.

Installing Kubernetes

You can install Kubernetes using several cluster configurations, each serving different use cases:

All-in-One Single-Node Installation: Installs both control plane and worker components on a single node. Ideal for learning, development, and testing. Not recommended for production due to lack of high availability and scalability.
Single-Control Plane and Multi-Worker Installation: Includes a single control plane node running a stacked etcd instance, managing multiple worker nodes. Suitable for small-scale environments but introduces a single point of failure.
Single-Control Plane with External etcd and Multi-Worker Installation: The control plane runs independently from an external etcd instance, improving data durability. The single control plane manages multiple worker nodes.
Multi-Control Plane and Multi-Worker Installation: High-availability setup with multiple control plane nodes, each running a stacked etcd instance forming an HA etcd cluster. Offers better fault tolerance.
Multi-Control Plane with External etcd and Multi-Worker Installation: The most robust and production-ready configuration. Each control plane node connects to a dedicated external etcd instance, all configured in a highly available cluster. Ensures maximum resilience and scalability.

As cluster complexity increases, so do the hardware and infrastructure requirements. For production environments, use a multi-node setup with high availability and redundant control planes.

When planning infrastructure, consider: