Isolation, Security, Scale: Sandboxing AI Agents on Kubernetes with kagent and Agent Substrate

Spinning an AI agent up in a plain container is not sandboxing. Ingress and egress are wide open, the workload can send packets wherever it likes, and there is zero policy enforcement. Network policies and segmentation rules help, but container sandboxing is "flat" by nature — it stops at your kernel, where everything runs in the same user space. For a workload that holds model output, possibly credentials, and issues tool calls, that is not a production posture.

A post published today on the CNCF blog by Michael Levan (CNCF Ambassador) makes the case that agentic workloads need three things before they belong in production — isolation, security, and scale — and demonstrates the stack that delivers them: kagent as the runtime for declarative and bring-your-own agents (covering frameworks like LangGraph and ADK), with Agent Substrate as the sandbox environment underneath.

What proper agent sandboxing means

Beyond a plain container, the article defines five properties of a secure sandbox:

• Untrusted agent code cannot escape into the host environment or compromise underlying infrastructure.
• The blast radius of a harness is minimized.
• The agent — which holds model output, possibly credentials, and tool calls — sits inside an isolated boundary.
• Agents are snapshotted, so they know where to pick up where they left off and where their state exists.
• Agent multiplexing — routing traffic to multiple agents in a multi-tenant environment — is isolated and bounded.

Achieving this takes isolation at both the software and hardware level, plus policy enforcement at the AI gateway where agent traffic reaches its destination.

Where Agent Substrate fits in

Agent Substrate runs agents in Workers, which are Pods. The agents themselves are facilitated via Actors, managed by the ateapi — Agent Substrate's control plane. The isolation comes from two layers:

• gVisor for software-level isolation: the workload runs in user space against a dedicated user-space application kernel called Sentry, which intercepts and handles system calls instead of passing them to the host kernel.
• microVM for hardware-level isolation: the guest OS runs directly on physical CPU cores using hardware virtualization extensions.

How the client routes to the agent — and what sits in the middle facilitating that routing — is what actually counts for isolation, not the container label.

The kagent layer

Without kagent, the only real entry point to Agent Substrate is a programmatic call — something like curl against an Actor. kagent wraps the substrate in what the industry is comfortable with: a UI and declarative Kubernetes CRDs (kagent.dev/v1alpha3).

Two CRDs do the heavy lifting:

• Harness — the Actor (agent) itself. It can be a known harness such as Codex or Claude Code, or a bring-your-own agent built with a framework like LangGraph. The spec points at the workload image, references a workerPoolRef, defines a snapshotPolicy location (for example a gs:// bucket for snapshots), and constrains which AgentTemplates it may use via a label selector.
• AgentTemplate — the reusable configuration, something like a golden image: model config, description, and system prompt, shared across deployed agents.

A default Helm installation of kagent v1.0 creates one WorkerPool with two Workers (Pods) via the substrateWorkerPool.create=true setting. Onboarding requires configuring the kubectl-ate CLI (Agent Substrate's control-plane interface), installing Agent Substrate, then installing kagent — a step-by-step guide is linked from the original post. Both the Harness and AgentTemplate can also be created through the UI: the Agents pane offers a + New template button and a + New Harness flow covering namespace, name, runtime adapter, and image location.

Actors only run while a session is live

The resource story is the part worth internalizing: an Actor is only running while there is an active session. The article's walkthrough starts at 0/8 Actors running. Starting a new session spins up a fresh Actor; when the session completes, it spins down (0/9, then 0/10 for the next new session). Re-entering an existing session does not change the count — it is still the same session. So it is not "every agent use spawns a pod"; only new sessions do, which keeps hardware resource usage proportional to actual conversational load.

Why this matters

For engineering teams and enterprises alike, the ability to isolate, secure, govern, and control agents is the holding point for production agentic workloads today — the step that turns a demo into something an operator can reason about. Sandboxing is the mechanism; kagent plus Agent Substrate is a concrete, declarative way to get it on Kubernetes.