| |

Docker 98 🐳 Building Custom OCI Compliant Runtimes or Custom Images from Scratch

ā±ļø Estimated read: 11 min | |

The OCI specification defines two independent standards that are often confused: the Runtime Specification describes how to configure and execute a container (the config.json and the lifecycle), and the Image Specification describes how to package a filesystem and its metadata. Building a custom runtime means implementing the former; building a custom image means producing the latter. The two can be done independently, and a minimal container image can be built without a Dockerfile, a daemon, or a base image.

Key point: An OCI bundle is a directory containing a rootfs/ and a config.json. The runtime reads the config, sets up the namespaces and cgroups, pivots into the rootfs, and executes the process. Building an image from scratch means populating an empty rootfs/ with a statically linked binary and the minimal configuration needed to run it.


Why these two paths exist

The runtime abstraction problem. Docker and Podman are high-level tools. Beneath them is an OCI runtime—runc or crun—that does the actual work of creating namespaces, configuring cgroups, mounting the rootfs, and executing the process. The runtime is a small binary with a narrow interface: it reads a config.json, starts a container, and reports status. Building a custom runtime means implementing that interface, which is a way to understand what the high-level tools actually do .

The image minimalism problem. A standard base image contains a shell, a package manager, and hundreds of files that the application never uses. Each of those files is an attack surface and a CVE waiting to be discovered. buildah from scratch creates an image with no base at all. The only files in the final image are the ones explicitly copied in. A statically linked Go or Rust binary produces an image of 10–20 MB with no shell to exploit .

The compliance problem. An image built from scratch has no package manager, no shell, and no unnecessary libraries. The CVE scanner finds nothing because there is nothing to scan. The CIS benchmark passes because there is no root user, no writable filesystem, and no privileged mode. The minimal image is the compliance strategy, not just the size optimization.


a. The OCI Runtime Bundle and config.json

The OCI Runtime Specification defines the configuration file and the lifecycle. A bundle is a directory with a config.json and a rootfs/ subdirectory. The config.json specifies the process to execute, the namespaces to create, the mounts to perform, and the resource limits to apply .

The runc spec command generates a default config.json for a bundle. The file is large and most of it can be left at its defaults for a minimal container. The fields that matter are process.args (the command to run), process.env (the environment variables), root.path (the rootfs path), and the linux.namespaces array (which namespaces to create) .

{
  "ociVersion": "1.0.2",
  "process": {
    "terminal": false,
    "user": { "uid": 0, "gid": 0 },
    "args": ["/app"],
    "env": ["PATH=/usr/local/bin:/usr/bin:/bin"],
    "cwd": "/"
  },
  "root": {
    "path": "rootfs",
    "readonly": true
  },
  "linux": {
    "namespaces": [
      { "type": "pid" },
      { "type": "network" },
      { "type": "ipc" },
      { "type": "uts" },
      { "type": "mount" }
    ]
  }
}

The runc run command reads the config, creates the namespaces, pivots into the rootfs, and executes the process. The container is now isolated by the kernel, not by a daemon. The runc binary does not stay running; it starts the container and exits. The container process is the child of the runtime, and when it exits, the container is gone .

A custom runtime replaces runc with a different implementation. The libcontainer package from the runc project provides the namespace, cgroup, and pivot_root primitives that a custom runtime would otherwise have to implement with raw syscall calls. The config.json is the interface; any binary that reads it and produces a running container is a valid OCI runtime .


b. Building an Image from Scratch with Buildah

buildah from scratch creates a working container with an empty rootfs. There is no base image, no shell, and no package manager. The rootfs is a directory that can be populated with buildah copy or by mounting it and copying files directly .

# Create a working container with an empty rootfs
ctr=$(buildah from scratch)

# Copy a statically linked binary into the rootfs
buildah copy "$ctr" ./myapp /app

# Set the entrypoint
buildah config --entrypoint '["/app"]' "$ctr"

# Commit the image
buildah commit "$ctr" myapp:scratch

The resulting image contains exactly one file: the myapp binary. The image size is the size of the binary plus the OCI metadata. There is no shell, no /etc/passwd, no /bin/sh. The container runs the binary directly, and when the binary exits, the container exits .

The scratch image is appropriate for statically linked binaries. A Go binary compiled with CGO_ENABLED=0 has no dynamic library dependencies, so it runs in an empty rootfs. A Rust binary compiled with the musl target has the same property. A dynamically linked binary requires the shared libraries it links against, which means the rootfs must contain those libraries and the dynamic linker. The ldd command lists the dependencies, and those files must be copied into the rootfs .

For a binary that needs TLS certificates, the ca-certificates.crt file must be copied into the rootfs. For a binary that needs timezone data, the zoneinfo directory must be copied. The minimal image is minimal because it contains only what the binary actually needs, and the process of building it is the process of discovering what those needs are.


c. Building a Custom Runtime with Go and libcontainer

A custom OCI runtime is a Go program that reads a config.json, uses the kernel’s namespace and cgroup interfaces to create the isolation, and executes the process. The libcontainer package provides the primitives, and the runc source code is the reference implementation .

The runtime’s entry point is a create command that reads the bundle and prepares the container, and a start command that executes the process. The lifecycle is defined by the OCI spec: create sets up the namespaces and mounts without executing the process, and start executes the process inside the prepared environment .

// Conceptual runtime structure
func main() {
    switch os.Args[1] {
    case "create":
        // Read config.json, create namespaces, mount rootfs
        // Write state.json with the container ID and PID
    case "start":
        // Read state.json, signal the init process to exec
    case "delete":
        // Clean up cgroups and namespaces
    }
}

The namespace setup uses unix.Clone with the CLONE_NEW* flags, or unix.Unshare to create new namespaces for the current process. The cgroup setup writes to the cgroup.procs and memory.max files in the cgroup v2 hierarchy. The rootfs setup uses pivot_root to switch the process’s root to the bundle’s rootfs, which is the same mechanism that runc uses .

The libcontainer package abstracts these operations. The libcontainer.Create function takes a config and a container ID, creates the cgroups and namespaces, and returns a container handle. The container.Run function executes the process. The custom runtime is a thin wrapper around these functions, with the OCI CLI as the interface .


Complete Example Session

# ============================================
# PART 1: CREATE A MINIMAL OCI BUNDLE
# ============================================
mkdir -p /opt/busybox-container/rootfs
cd /opt/busybox-container


# ============================================
# PART 2: POPULATE THE ROOTFS
# ============================================
# Option A: copy a statically linked binary
cp /path/to/myapp rootfs/app

# Option B: use apk to install busybox into the rootfs
apk --arch x86_64 -X https://dl-cdn.alpinelinux.org/alpine/edge/main/ \
  --root /opt/busybox-container/rootfs --initdb --no-cache \
  --allow-untrusted add busybox


# ============================================
# PART 3: GENERATE THE CONFIG
# ============================================
runc spec
# Creates a default config.json in the bundle directory


# ============================================
# PART 4: EDIT THE CONFIG
# ============================================
# Set process.args to ["/app"] or ["/bin/sh"]
# Set root.readonly to true
# Remove unnecessary namespaces if needed


# ============================================
# PART 5: RUN THE CONTAINER
# ============================================
runc run mycontainer
# The runtime creates namespaces, pivots into rootfs, executes the process


# ============================================
# PART 6: BUILD A SCRATCH IMAGE WITH BUILDAH
# ============================================
ctr=$(buildah from scratch)
buildah copy "$ctr" ./myapp /app
buildah config --entrypoint '["/app"]' "$ctr"
buildah commit "$ctr" myapp:scratch


# ============================================
# PART 7: VERIFY THE IMAGE
# ============================================
buildah images | grep myapp
# The size should be the size of the binary plus metadata


# ============================================
# PART 8: RUN THE IMAGE
# ============================================
podman run --rm myapp:scratch
# The binary runs with no shell, no package manager


# ============================================
# PART 9: INSPECT THE IMAGE
# ============================================
buildah inspect myapp:scratch
# Shows the OCI manifest and config


# ============================================
# PART 10: PUSH TO A REGISTRY
# ============================================
buildah push myapp:scratch registry.example.com/myapp:scratch

The ten parts trace the two paths: the runtime bundle with runc and the image with buildah. Part 2 shows the two ways to populate the rootfs. Part 6 shows the scratch image build.


Quick Reference

OCI Runtime Bundle

ComponentPurpose
config.jsonProcess, namespaces, mounts, cgroups
rootfs/The container’s filesystem
runc specGenerate default config
runc runCreate and start the container
runc createCreate without starting
runc startStart a created container

Buildah Scratch Image

CommandPurpose
buildah from scratchCreate empty rootfs
buildah copyCopy files into rootfs
buildah configSet entrypoint, env, port
buildah commitCreate image from working container
buildah pushPush to registry

Runtime Primitives

PrimitiveGo PackagePurpose
Namespacesunix.Clone / unix.UnshareIsolation
CgroupslibcontainerResource limits
Pivot rootlibcontainerRootfs switch
ExeclibcontainerProcess execution

Best Practices

āœ… Do This:

# Use a statically linked binary for scratch images
CGO_ENABLED=0 go build -o myapp                                   # āœ…
# Copy CA certificates if the app makes HTTPS calls
buildah copy "$ctr" /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ # āœ…
# Set rootfs read-only in config.json
"root": { "readonly": true }                                          # āœ…

āŒ Don’t Do This:

# Use a dynamically linked binary in scratch without its libraries
buildah copy "$ctr" ./myapp /app  # fails at runtime if libc is missing   # āŒ
# Assume scratch images work for all applications
# A shell script needs a shell; a Python app needs the interpreter       # āŒ
# Use runc spec defaults without reviewing them
# The default config may include unnecessary namespaces or mounts       # āŒ

Common Pitfalls

PitfallWhy It HappensFix
Scratch binary fails to runDynamically linkedUse static binary or copy libraries
TLS fails in scratchNo CA certificatesCopy ca-certificates.crt
Timezone wrongNo zoneinfoCopy /usr/share/zoneinfo
runc run permission deniedNo root, no user namespaceUse --rootless or run as root
Image larger than expectedBinary is not strippedstrip or use -ldflags="-s -w"

Real-World Examples

1. Generate Config

runc spec

2. Run Bundle

runc run mycontainer

3. Create Scratch Rootfs

buildah from scratch

4. Copy Binary

buildah copy "$ctr" ./myapp /app

5. Set Entrypoint

buildah config --entrypoint '["/app"]' "$ctr"

6. Commit Image

buildah commit "$ctr" myapp:scratch

7. Static Go Binary

CGO_ENABLED=0 go build -o myapp

8. Copy CA Certs

buildah copy "$ctr" /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/

9. Read-Only Rootfs

"root": { "readonly": true }

10. Push Image

buildah push myapp:scratch registry.example.com/myapp:scratch

Visual

The OCI Bundle

ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│  BUNDLE DIRECTORY                                            │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”ā”‚
│  │ config.json                                             ││
│  │   process.args: ["/app"]                                ││
│  │   root.path: "rootfs"                                   ││
│  │   linux.namespaces: [pid, net, ipc, uts, mount]         ││
│  │                                                         ││
│  │ rootfs/                                                 ││
│  │   └── app                                               ││
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜ā”‚
│                                                              │
│  runc run reads config, creates namespaces,                 │
│  pivots into rootfs, executes /app.                         │
│                                                              │
│  The runtime is stateless. The bundle is the container.     │
│                                                              │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜

Scratch Image Build

ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│  buildah from scratch                                        │
│         │                                                    │
│         ā–¼                                                    │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”                                        │
│  │ Empty rootfs    │  No shell, no package manager          │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜                                        │
│           │ buildah copy myapp /app                          │
│           ā–¼                                                  │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”                                        │
│  │ rootfs/app      │  The only file in the image            │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”¬ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜                                        │
│           │ buildah commit                                   │
│           ā–¼                                                  │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”                                        │
│  │ myapp:scratch   │  ~10 MB, no CVE surface                │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜                                        │
│                                                              │
│  The image contains exactly what was copied.                 │
│                                                              │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜

Runtime vs. Image

ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”
│  RUNTIME SPEC                    IMAGE SPEC                  │
│                                                              │
│  ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”            ā”Œā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”         │
│  │ config.json     │            │ OCI manifest    │         │
│  │ rootfs/         │            │ OCI config      │         │
│  │ runc run        │            │ layers          │         │
│  ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜            ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜         │
│                                                              │
│  How to execute                  How to package              │
│  a container                    a filesystem                 │
│                                                              │
│  buildah produces an image.                                 │
│  runc consumes a bundle.                                     │
│  The image contains a rootfs; the bundle is a rootfs + config.│
│                                                              │
ā””ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”€ā”˜

Summary

ItemValue
OCI Runtime Specconfig.json + rootfs/ lifecycle
OCI BundleDirectory with config and rootfs
runc specGenerate default config
runc runCreate and start container
Buildah scratchbuildah from scratch
Static binaryRequired for scratch without libs
CA certificatesNeeded for TLS in scratch
Read-only rootfs"root": { "readonly": true }

Key takeaways:

  • The OCI Runtime Spec defines the bundle, not the image. A bundle is a config.json and a rootfs/. The runtime reads the config, creates the isolation, and executes the process. The image spec is a separate standard for packaging filesystems .
  • buildah from scratch creates an image with no base. The rootfs is empty until files are copied in. A statically linked binary runs with no shell, no package manager, and no libraries. The image size is the binary size plus metadata .
  • The scratch image is the compliance strategy. No shell means no shell exploits. No package manager means no package CVEs. No root user means no privilege escalation. The CVE scanner finds nothing because there is nothing to scan.
  • The binary must be statically linked. A dynamically linked binary requires the shared libraries it depends on. The ldd command lists them, and they must be copied into the rootfs. A Go binary compiled with CGO_ENABLED=0 has no such dependencies.
  • The runtime is a small binary with a narrow interface. runc reads config.json, creates namespaces and cgroups, pivots into the rootfs, and executes the process. A custom runtime implements the same interface using libcontainer or raw syscall calls .

Remember: The OCI specification separates the execution model from the packaging model. The runtime spec defines how to run a container; the image spec defines how to ship one. Building a custom runtime means implementing the config.json lifecycle. Building a scratch image means populating an empty rootfs with a statically linked binary and the minimal configuration it needs. The two paths are independent, and both are exercises in minimalism: the smallest runtime that satisfies the spec, and the smallest image that satisfies the application.

Was this helpful?


Stop using slow, ad-bloated tool sites! 🤮

šŸ”Ž Search “KandZ Tools” on Google to use many professional utilities for free.

KandZ.me is the ultimate minimalist hub for:
āœ… Finance (Mortgage, Interest, Inflation)
āœ… Tech (Base64, JSON, Dev Suite, IP)
āœ… Health (BMI, BMR, TDEE)
āœ… Productivity (Timer, Workspace, QR)

āš”ļø Fast & Private
šŸ”’ No data leaves your device
šŸ’Ž 100% Free

šŸ”— Use it now: https://tools.kandz.me
šŸ”– Bookmark it—you’ll need it later!