Hardened, Sandboxed Local LLM Agent Hosting
A reference architecture, template, and test harness for hosting local inference backends and running containerized AI coding agents.
$ git clone https://codeberg.org/ubunatic/lmcoder.git
$ cd lmcoder
$ make test
Why lmcoder
Running coding agents against a local LLM sounds simple until the agent needs shell access, the model needs a memory budget it won't blow past on shared GPU/APU hardware, and every "it worked once" run needs to keep working after the next dependency bump. Most local setups skip straight to "point the agent at an OpenAI-compatible endpoint" and hope — no sandboxing, no regression signal, no way to tell a real fix from a lucky run.
lmcoder exists to make that reproducible: rootless Podman sandboxes so an agent's shell access can't touch the host, a proxy that injects sampling profiles instead of hand-tuning each session, and a canary-first harness that has to pass before a change is trusted — not after.
Architecture & Capabilities
Built for developers hosting local models on dedicated or shared GPU/APU hardware.
🛡️ Hardened Sandboxes
Runs coding agents (Prime Agent, Codex CLI, OpenCode, Pi) inside isolated, rootless Podman containers with single-writable-mount workspace restrictions.
⚡ Model Serving & MTP
Manages local llama-server instances with memory-budget validation and automated MTP (Multi-Token Prediction) draft head acceleration.
🔀 Dynamic Profile Proxy
Reverse proxy injecting sampling profiles (fast, default, quality, deep) per session over standard OpenAI-compatible APIs.
🐤 Canary-First Harness
Automated compiled Go test runners (llama-canary, agent-canaries) validating server health, vision models, and autonomous coding workflows.
Usage Overview
Run local inference backends and sandboxed agents with unified CLI commands.
# Serve a model with automatic memory overload check and MTP support $ lmcoder serve Qwen3.8-27B-UD-Q4_K_M.gguf # Run dynamic profile proxy $ lmcoder proxy --backend-host 127.0.0.1 --profile fast # Run sandboxed agent canary against isolated workspace $ go run ./scripts/agent-canaries --agent pi --task coding-hello