boxy - Kubernetes-native sandbox runtime

Architecture

Three tiers, each with a single responsibility. The router is the only public-facing component. All sandbox state lives in Kubernetes CRs and controller node storage - the router and operator are fully stateless.

Clients

🌐 REST Client

🤖 AI / MCP

Bearer →

Bearer ↓

boxy-router Deployment × N

1Auth - SA token via TokenReview (or static dev bypass)

2Validate body, parse request fields

3Create / read / delete Sandbox CRs

4Proxy exec to controller via mTLS

5MCP server - exposes bash tool

CRD ↔

CRD ↕

boxy-operator leader-elected

ABin-pack → assign pod

BAuto-scale StatefulSet

CTTL expiry + GC

mTLS (mutual cert auth)

↓

boxy-controller StatefulSet (auto-scaled by operator)

boxy-ctrl-0

sandbox-A
/workspace

sandbox-B
/workspace

boxy-ctrl-1

sandbox-C
/workspace

scale on demand

nsjail sandbox isolation layers

🗂️

Mount Namespace

R/O Ubuntu 24.04 rootfs
R/W /workspace
tmpfs /tmp per-exec

🔌

Network Namespace

No internet by default
opt-in: allowInternetAccess
MACVLAN / pasta modes

🔬

PID + UTS Namespace

Isolated process tree
Custom hostname
SIGKILL on timeout

⚡

cgroup + rlimits

memoryMb via cgroup
cpu / nofile / nproc
wall-clock timeout

🪝

Lifecycle Hooks

setupScript on create
teardownScript on delete
per-sandbox scriptEnv

Sandbox lifecycle

Pending

assign pod→

Creating

ctrl POST /v1/sandboxes→

Running

TTL / delete→

Deleting

GC after N s→

Terminated

TTL is a sliding window - refreshed on every exec. A sandbox pinned to a controller pod; if that pod restarts, the router detects the stale route, resets the CR to Pending, and the operator reassigns it.

Dual-track output design

One sandbox runtime, two client models. Developers talk REST; AI agents talk MCP. The same nsjail execution engine serves both - no separate code paths, no duplication.

🌐

REST API

For developers, services, CI pipelines

✓JSON over HTTP - works with any HTTP client
✓Full lifecycle: create, exec, delete sandboxes
✓SA token auth via Kubernetes TokenReview
✓Structured error codes with clear semantics

{"exitCode": 0, "stdout": "hello from nsjail\n",
 "stderr": "", "timedOut": false}

🤖

MCP Server

For AI agents - Claude, Codex, any MCP client

✓Model Context Protocol (JSON-RPC 2.0)
✓Streamable HTTP transport - stateless
✓bash tool agents call like a function
✓Optional default sandbox for stateless clients

{"content": [{"type": "text",
  "text": "hello from nsjail\n"}]}

Use cases

🤖

AI agent code execution

Give Claude, Codex, or any LLM a real bash shell backed by an isolated Ubuntu environment. Low-latency exec, stateful workspace per session, native MCP support.

🏗️

CI step isolation

Isolate build steps from each other without spinning up full containers. File isolation between jobs, reproducible Ubuntu 24.04 rootfs, no image pull on every run.

🧑‍💻

Interactive shells

Provision on-demand shell environments for users. Strong cross-sandbox isolation, TTL enforcement, per-session persistent workspace that survives multiple exec calls.

🔍

Script evaluation API

Evaluate user-supplied scripts safely. Output capture, timeout enforcement, cgroup memory caps, and blocked network namespaces prevent abuse.

🔒

Secure code execution API

Build a service that runs untrusted code. Validated inputs, read-only base OS, isolated network namespace, and rlimits bound every execution.

⚙️

Sandboxed tool execution

Bind-mount specific binaries (curl, python3, jq) into sandboxes via allowedBinaries - whitelist exactly what each sandbox can run.

Request flows

+ Create sandbox

Client
POST /v1/sandboxes {sandboxId, ttlSeconds, …}

↓

boxy-router
Validates auth + body · creates Sandbox CR (phase: Pending)

↓

boxy-operator
Watches CR · bin-packs to a controller pod · sets phase: Creating

↓

boxy-controller
POST /v1/sandboxes · mkdir /workspace · runs setupScript (if set) · nsjail ready

↓

boxy-operator
Sets phase: Running

↓

Client
← 201 Created {sandboxId, phase: "Running", ready: true}

▶ Execute command

Client
POST /v1/sessions/exec {sandboxId, sessionId, command, args, timeoutSeconds}

↓

boxy-router
Reads Session CR · resolves controllerAddress

↓

boxy-router
POST controller /v1/exec over mTLS → boxy-controller

↓

boxy-controller
Builds NsjailConfig proto · writes to temp file · spawns nsjail process

↓

nsjail sandbox
Executes command · captures stdout/stderr · exits (or SIGKILLed on timeout)

↓

Client
← 200 {exitCode, stdout, stderr, timedOut} + X-Boxy-Session-Id

Router also updates lastExecAt on the CR to slide the TTL window.

Quick start

Requires a Kubernetes cluster (kind works), kubectl, helm ≥ 3.x

1 · Install

helm upgrade --install boxy \
  oci://ghcr.io/niradler/charts/boxy \
  --version 0.0.3 \
  --namespace boxy --create-namespace \
  --set router.auth.staticToken="$(openssl rand -hex 16)"

2 · Get a token

# Kubernetes ServiceAccount token (production)
export TOKEN=$(kubectl create token  \
  -n  --duration=3600s)

# Or use the static token set during install (dev only)
export TOKEN=
export BASE=http://<router-service>:8080

3 · Create a sandbox

curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sandboxId":"demo-1","ttlSeconds":3600}' \
  $BASE/v1/sandboxes | jq .

4 · Create and exec

curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sessionId":"demo","sandboxId":"demo-1",
       "owner":"you"}' \
  $BASE/v1/sessions | jq .

curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sessionId":"demo","sandboxId":"demo-1",
       "command":"sh","args":["-c","echo hello"],
       "timeoutSeconds":30}' \
  $BASE/v1/sessions/exec | jq .

# Response:
# { "exitCode": 0, "stdout": "hello\n",
#   "stderr": "", "timedOut": false }

API reference

Method	Path	Description
GET	/healthz	Liveness check - returns `200 ok`
POST	/v1/sandboxes	Create a sandbox config. Returns `201`
GET	/v1/sandboxes	List all sandbox configs
GET	/v1/sandboxes/{sandboxId}	Get sandbox config metadata and active session count
PUT	/v1/sandboxes/{sandboxId}	Update sandbox config (same body as POST)
DELETE	/v1/sandboxes/{sandboxId}	Delete sandbox config
POST	/v1/sandboxes/{sandboxId}/evict	Evict all active sessions for a sandbox
GET	/v1/sandboxes/{sandboxId}/sessions	List sessions belonging to a sandbox
POST	/v1/sessions	Create a running session from a sandbox config. Returns `201`
GET	/v1/sessions	List sessions. Query params: `sandboxId`, `owner`, `phase`
GET	/v1/sessions/{sessionId}	Get session status
DELETE	/v1/sessions/{sessionId}	Delete (terminate) a session
POST	/v1/sessions/exec	Execute a command in a session (creates session if needed)
POST	/mcp	MCP endpoint - JSON-RPC 2.0 over Streamable HTTP

POST /v1/sandboxes - request body

Field	Type	Notes
sandboxId*	string	Required. Unique ID
ttlSeconds	int	0 = no expiry. Sliding window on sessions
env	map	Max 64 keys. KUBERNETES_* / BOXY_* blocked
allowedBinaries	string[]	Whitelist of binaries to bind-mount
vm	object	memoryMb, vcpus, workdir, shell, hostname, rlimits, scripts…
network	object	allowInternetAccess, macvlan, usePasta
volumes	object[]	{guestPath, type, hostPath, sizeMb, readonly}
patches	object[]	{type, path, content, bytes, hostPath, mode, replace}
setupScript	string	Path to executable on controller. Runs after provisioning
teardownScript	string	Path to executable on controller. Runs before cleanup
scriptEnv	map	Custom env vars injected into setup/teardown scripts

POST /v1/sessions - request body

Field	Type	Notes
sandboxId*	string	Required. Sandbox config to instantiate
sessionId	string	Auto-generated if omitted
owner	string	Caller identity for tracking

POST /v1/sessions/exec - request body

Field	Type	Notes
sandboxId*	string	Required
sessionId*	string	Required
command*	string	Required. Resolved against rootfs PATH
timeoutSeconds*	int	Required. 1–3600. Hard SIGKILL
args	string[]	Max 256 args
env	map	Per-exec env overrides

HTTP status codes

200/201	Success
400	Bad request - missing field, blocked env prefix, body >6 MB
401	Missing, expired, or invalid bearer token
404	Sandbox not found
409	Sandbox already exists
429	Concurrency limit hit (semaphore full)
502/503	Controller pod unreachable or not ready

MCP server

POST /mcp - JSON-RPC 2.0 over Streamable HTTP. Stateless, no session management. Built with the official Go MCP SDK.

Required headers

Authorization: Bearer <BOXY_ROUTER_TOKEN>
Content-Type: application/json
Accept: application/json, text/event-stream

Sandbox routing

Set X-Sandbox-Id header to target a specific sandbox. Omit to use the default sandbox (requires BOXY_DEFAULT_SANDBOX_ENABLED=true).

Available tools

bash

Runs a shell command in the target sandbox. Parameters: command (string, required), timeoutSeconds (int, default 60).

Example calls

# Initialize MCP session
curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"initialize",
       "params":{"protocolVersion":"2025-03-26",
                 "clientInfo":{"name":"cli","version":"1.0"},
                 "capabilities":{}}}' \
  $BASE/mcp | jq .

# Call bash tool
curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "X-Sandbox-Id: demo-1" \
  -d '{"jsonrpc":"2.0","id":2,"method":"tools/call",
       "params":{"name":"bash",
                 "arguments":{"command":"echo hello"}}}' \
  $BASE/mcp | jq .

Configuration

All configuration is via environment variables. The Helm chart injects sensible defaults; override only what you need.

boxy-router

Variable	Default
BOXY_ROUTER_TOKEN	-
BOXY_SANDBOX_NAMESPACE	default
BOXY_LISTEN_ADDR	:8080
BOXY_MAX_CONCURRENCY	100
BOXY_MAX_TIMEOUT_SECONDS	3600
BOXY_MAX_BODY_BYTES	6 MB
BOXY_MAX_OUTPUT_BYTES	6 MB
BOXY_MTLS_DISABLED	false
BOXY_DEFAULT_SANDBOX_ENABLED	false

boxy-operator

Variable	Default
BOXY_NAMESPACE	required
BOXY_MAX_SANDBOXES_PER_CONTROLLER	20
BOXY_MAX_CONTROLLER_REPLICAS	50
BOXY_MIN_CONTROLLER_REPLICAS	1
BOXY_SCALE_DOWN_COOLDOWN_SECONDS	300
BOXY_TERMINATED_RETENTION_SECONDS	3600
BOXY_CONTROLLER_PORT	8080
BOXY_MTLS_DISABLED	false

boxy-controller

Variable	Default
BOXY_MAX_SANDBOXES	20
BOXY_MAX_EXEC_CONCURRENCY	50
BOXY_MAX_OUTPUT_BYTES	6 MB
BOXY_CONTROLLER_TOKEN	auto (Helm)
BOXY_NSJAIL_PATH	/usr/sbin/nsjail
BOXY_NSJAIL_ROOTFS	/rootfs/ubuntu-24.04
BOXY_NSJAIL_SANDBOX_ROOT	/var/lib/boxy/sandboxes
BOXY_MTLS_DISABLED	false

Custom Resource Definitions

boxy ships three CRDs in the boxy.dev group. All state lives in these CRs; the router and operator are fully stateless. Short names work with kubectl (kubectl get sbx, kubectl get sess, kubectl get cp).

Sandbox sandboxes.boxy.dev / v1alpha1

group: boxy.dev · scope: Namespaced · shortName: sbx

spec fields

Field	Type	Notes
sandboxId*	string	Required. 1–63 chars
ttlSeconds	integer	0–604800. Sliding TTL window
env	map<string>	BOXY_* / KUBERNETES_* reserved
allowedBinaries	string[]	Bind-mount whitelist
vm.memoryMb	integer	64–65536 MB cgroup cap
vm.vcpus	integer	1–64
vm.workdir	string	Working directory inside the sandbox
vm.shell	string	Shell binary to use
vm.hostname	string	UTS hostname for the sandbox
vm.user	string	User to run as inside the sandbox
vm.maxDurationSec	integer	Hard nsjail time_limit
vm.idleTimeoutSec	integer	Idle kill timeout
vm.rlimits	object[]	{resource, soft, hard}
vm.scripts	object[]	{name, content} init scripts
vm.seccompString	string	Custom seccomp profile JSON string
vm.cloneNewTime	bool	Clone time namespace
network.enabled	bool	Enable network namespace
network.allowInternetAccess	bool	Disable network NS isolation
network.macvlan	object	{interface, ip, netmask, gateway, mac} MACVLAN config
network.usePasta	bool	Use pasta instead of MACVLAN for networking
volumes	object[]	{guestPath, type, hostPath, sizeMb, readonly}
patches	object[]	{type, path, content, bytes, hostPath, mode, replace}
setupScript	string	Controller-side executable. Runs after provisioning
teardownScript	string	Controller-side executable. Runs before cleanup
scriptEnv	map<string>	Custom env vars for setup/teardown scripts

example manifest

apiVersion: boxy.dev/v1alpha1
kind: Sandbox
metadata:
  name: demo-1
spec:
  sandboxId: demo-1
  ttlSeconds: 3600
  vm:
    memoryMb: 512
    vcpus: 2
  network:
    allowInternetAccess: true
  setupScript: /opt/boxy/scripts/setup.sh
  teardownScript: /opt/boxy/scripts/teardown.sh
  scriptEnv:
    CUSTOMER_TIER: premium

Session sessions.boxy.dev / v1alpha1

group: boxy.dev · scope: Namespaced · shortName: sess

spec fields

Field	Type	Notes
sessionId*	string	Required. 1–63 chars
sandboxId*	string	Required. References a Sandbox config
owner	string	Optional. Caller identity for tracking

status fields

Field	Type	Notes
phase	string	Pending · Creating · Running · Deleting · Terminated
controllerPool	string	Assigned ControllerPool name
controllerPod	string	Assigned controller pod name
controllerAddress	string	mTLS dial address for the router
port	int32	Listening port on controller
createdAt	date-time	When session was created
expiresAt	date-time	Computed TTL expiry (from sandbox ttlSeconds)
terminatedAt	date-time	When session terminated
lastExecAt	date-time	Updated on every exec (slides TTL)
message	string	Human-readable status detail

example manifest

apiVersion: boxy.dev/v1alpha1
kind: Session
metadata:
  name: my-session
spec:
  sessionId: my-session
  sandboxId: demo-1
  owner: alice

ControllerPool controllerpools.boxy.dev / v1alpha1

group: boxy.dev · scope: Namespaced · shortName: cp

spec fields

Field	Type	Notes
maxSandboxes*	integer	Required. Min 1. Max per controller pod
maxReplicas*	int32	Required. Min 1. StatefulSet upper bound
minReplicas*	int32	Required. Min 1. Always-warm floor
image	string	Controller container image override
preinstalledBinaries	string[]	Binaries available in all sandboxes on this pool

status fields

Field	Type	Notes
readyReplicas	int32	Controller pods ready
activeSandboxCount	int32	Running sandboxes across the pool
lastScaleTime	date-time	Last autoscale event
conditions	object[]	{type, status, reason, message, lastTransitionTime}

example manifest

apiVersion: boxy.dev/v1alpha1
kind: ControllerPool
metadata:
  name: default
spec:
  maxSandboxes: 20
  minReplicas: 1
  maxReplicas: 10

Security model

Authentication layers

Client → Router: Bearer token

SA token validated via Kubernetes TokenReview API (production) or static bypass token (dev/e2e). All authenticated callers have equal access - no per-caller RBAC.

Router/Operator → Controller: mTLS

Mutual TLS with a Helm-generated CA. Identity is CA membership. Disable with BOXY_MTLS_DISABLED=true for local dev only.

Controller pod capabilities

+ SYS_ADMIN(namespace setup)

+ SETUID, SETGID, SETPCAP

+ NET_ADMIN(network namespaces)

+ SYS_CHROOT, MKNOD

− ALL others dropped

− allowPrivilegeEscalation: false

Threat model

Threat	Mitigation
Unauthorized API access	Bearer token on every route - SA token via TokenReview (prod) or static dev bypass
Router/operator impersonation toward controller	mTLS with shared CA - both sides verify cert against same CA
Sandbox escaping to host filesystem	nsjail mount namespace + R/O rootfs; only /workspace (R/W) and /tmp (tmpfs) are writable
Sandbox reaching other sandboxes over network	Isolated network namespace per sandbox by default
Sandbox exhausting host memory	`--cgroup_mem_max` enforced by nsjail when vm.memoryMb is set
Sandbox running forever	`--time_limit` (nsjail SIGKILLs) + TTL sliding window (operator GCs the CR)
Malicious env var injection (KUBERNETES_, BOXY_)	Blocked prefixes at the router; max 64 keys, values ≤ 16 KB
⚠ No per-caller tenant isolation	All authenticated callers share equal access - single-tenant only in this release
⚠ Sandbox processes run as uid 0	Linux user namespace uid remapping is disabled (incompatible with most container runtimes + CAP_SYS_ADMIN). Security boundary is mount/PID/network NS isolation