Beyond Basics: Building a More Powerful Container in Go — Network Isolation & Advanced Features

Containers Uncovered: More Than Just Lightweight Virtual Machines!” If you’re like me — always wondering how things work and eager to build them with your own mind and hands — you’re in the right place! In the first part of this article (Part 1), I attempted to build a minimal container system using only Go, relying on Linux’s unshare and namespaces. It was purely a demonstration, and I wasn’t aiming to develop a fully functional container runtime tool. I intentionally left out many critical aspects, such as security, networking, and image management. I initially thought it would be simple, but I quickly realized that even a basic container system involves thousands of concepts and implementations. However, my passion for understanding and building things kept me going. Now, after a year since my first article on Building a Minimal Container in Go, I’ve realized that both my code and my original article need a fresh perspective. So, it’s time for a revisit! System Architect Core Components User CLI Responsibilities: Parse user commands (run, exec, ps, rm) Communicate with daemon via RPC or any other way Format and display output Key Features: Command completion Output formatting (JSON/YAML) Log streaming Container Daemon Responsibilities: Manage container lifecycle Maintain container state database Coordinate between components Key Features: REST/gRPC API Event logging Resource tracking Container Runtime Components: Namespace Manager: CLONE_NEW* flags handling and more flags in real world . Cgroups Manager: Resource constraints Filesystem Setup: RootFS preparation Features: OCI runtime spec compliance User namespace remapping Seccomp/AppArmor profiles Image Service Components: Registry Client: Docker Hub integration or you own images services if you will go wiled Layer Manager: OverlayFS/BTRFS Snapshotter: Copy-on-write layers Features: Image caching Signature verification Garbage collection Network Manager Components: CNI Plugins: Bridge, MACVLAN, IPVLAN IPAM: DHCP/Static allocation Service Mesh: DNS, service discovery Features: Multi-host networking Network policies Port mapping Storage Driver Components: Volume Manager: Bind mounts Snapshot Manager: Incremental backups Quota Enforcer: Disk limits Features: Persistent storage Temporary filesystems Encryption support this schema will give you a bigger picture +---------------------+ | User CLI | | (run, exec, ps, rm) | +----------+----------+ | | (gRPC/HTTP) v +---------------------+ | Container Daemon | | (State Management) | +----------+----------+ | +------------------+------------------+ | | | +----------+----------+ +-----+--------+ +-------+---------+ | Container Runtime | | Image Service| | Network Manager | | (namespace/cgroups) | | (OCI Images) | | (CNI Plugins) | +----------+----------+ +-----+--------+ +-------+---------+ | | | +---------v---------+ +------v-------+ +--------v---------+ | Linux Kernel | | Storage Driver| | Host Networking | | - namespaces | | (OverlayFS) | | (iptables/bridge)| | - cgroups v2 | +---------------+ +------------------+ | - capabilities | +--------------------+ It has been a long journey for me to learn and think through every component. I encountered many challenges, especially with aspects like OverlayFS and networking. My biggest issue in my first implementation was networking. It was really difficult to isolate the child container and set up its own bridged network. To solve network isolation, you need to think clearly

Feb 21, 2025 - 03:19

Beyond Basics: Building a More Powerful Container in Go — Network Isolation & Advanced Features

Containers Uncovered: More Than Just Lightweight Virtual Machines!”

If you’re like me — always wondering how things work and eager to build them with your own mind and hands — you’re in the right place!
In the first part of this article (Part 1), I attempted to build a minimal container system using only Go, relying on Linux’s unshare and namespaces. It was purely a demonstration, and I wasn’t aiming to develop a fully functional container runtime tool. I intentionally left out many critical aspects, such as security, networking, and image management.
I initially thought it would be simple, but I quickly realized that even a basic container system involves thousands of concepts and implementations. However, my passion for understanding and building things kept me going.
Now, after a year since my first article on Building a Minimal Container in Go, I’ve realized that both my code and my original article need a fresh perspective. So, it’s time for a revisit!

System Architect

Core Components

User CLI
Responsibilities:
Parse user commands (run, exec, ps, rm)
Communicate with daemon via RPC or any other way
Format and display output

Key Features:

Command completion
Output formatting (JSON/YAML)
Log streaming

Container Daemon

Responsibilities:

Manage container lifecycle
Maintain container state database
Coordinate between components

Key Features:

REST/gRPC API
Event logging
Resource tracking

Container Runtime

Components:

Namespace Manager: CLONE_NEW* flags handling and more flags in real world .
Cgroups Manager: Resource constraints
Filesystem Setup: RootFS preparation

Features:

OCI runtime spec compliance
User namespace remapping
Seccomp/AppArmor profiles

Image Service

Components:

Registry Client: Docker Hub integration or you own images services if you will go wiled
Layer Manager: OverlayFS/BTRFS
Snapshotter: Copy-on-write layers

Features:

Image caching
Signature verification
Garbage collection

Network Manager

Components:

CNI Plugins: Bridge, MACVLAN, IPVLAN
IPAM: DHCP/Static allocation
Service Mesh: DNS, service discovery

Features:

Multi-host networking
Network policies
Port mapping

Storage Driver

Components:

Volume Manager: Bind mounts
Snapshot Manager: Incremental backups
Quota Enforcer: Disk limits

Features:

Persistent storage
Temporary filesystems
Encryption support

this schema will give you a bigger picture


                           +---------------------+
                           |      User CLI       |
                           | (run, exec, ps, rm) |
                           +----------+----------+
                                      |
                                      | (gRPC/HTTP)
                                      v
                           +---------------------+
                           |   Container Daemon  |
                           | (State Management)  |
                           +----------+----------+
                                      |
                   +------------------+------------------+
                   |                  |                  |
         +----------+----------+ +-----+--------+ +-------+---------+
         |   Container Runtime | | Image Service| | Network Manager |
         | (namespace/cgroups) | | (OCI Images)  | | (CNI Plugins)   |
         +----------+----------+ +-----+--------+ +-------+---------+
                   |                  |                  |
         +---------v---------+ +------v-------+ +--------v---------+
         | Linux Kernel       | | Storage Driver| | Host Networking |
         | - namespaces       | | (OverlayFS)   | | (iptables/bridge)|
         | - cgroups v2       | +---------------+ +------------------+
         | - capabilities     |
         +--------------------+

It has been a long journey for me to learn and think through every component. I encountered many challenges, especially with aspects like OverlayFS and networking.

My biggest issue in my first implementation was networking. It was really difficult to isolate the child container and set up its own bridged network.

To solve network isolation, you need to think clearly