Voice, Local AI & Agent Tooling
lmcoder Local model and coding agent hosting
A reference architecture, template, and test harness for hosting local inference backends and running containerized AI coding agents.
go install ubunatic.com/lmcoder/cmd/lmcoder@latestWhy lmcoder
Why lmcoder
Running coding agents against a local LLM sounds simple until the agent needs shell access, the model needs a memory budget it won't blow past on shared GPU/APU hardware, and every "it worked once" run needs to keep working after the next dependency bump. Most local setups skip straight to "point the agent at an OpenAI-compatible endpoint" and hope — no sandboxing, no regression signal, no way to tell a real fix from a lucky run.
lmcoder exists to make that reproducible: rootless Podman sandboxes so an agent's shell access can't touch the host, a proxy that injects sampling profiles instead of hand-tuning each session, and a canary-first harness that has to pass before a change is trusted — not after.
Architecture & Capabilities
Architecture & Capabilities
Built for developers hosting local models on dedicated or shared GPU/APU hardware.
🛡️ Hardened Sandboxes
Runs coding agents (Prime Agent, Codex CLI, OpenCode, Pi) inside isolated, rootless Podman containers with single-writable-mount workspace restrictions.
⚡ Model Serving & MTP
Manages local llama-server instances with memory-budget validation and automated MTP (Multi-Token Prediction) draft head acceleration.
🔀 Dynamic Profile Proxy
Reverse proxy injecting sampling profiles (fast, default, quality, deep) per session over standard OpenAI-compatible APIs.
🐤 Canary-First Harness
Automated compiled Go test runners (llama-canary, agent-canaries) validating server health, vision models, and autonomous coding workflows.
Usage Overview
Usage Overview
Run local inference backends and sandboxed agents with unified CLI commands.
# Serve a model with automatic memory overload check and MTP support
$ lmcoder serve Qwen3.8-27B-UD-Q4_K_M.gguf
# Run dynamic profile proxy
$ lmcoder proxy --backend-host 127.0.0.1 --profile fast
# Run sandboxed agent canary against isolated workspace
$ go run ./scripts/agent-canaries --agent pi --task coding-hello
Recent engineering
Recent engineering
Recent work strengthened proxy lifecycle discovery, service/network safety, install and uninstall handling, and container tool cleanup with zombie reaping. The change history connects these milestones to issue 163 (2f1c6b4), issue 156 (0909235), issue 164 (8037408, af5ba4b, d5571f3), and issue 165 (5f13951).
Quick start
Build and test from source
git clone https://codeberg.org/ubunatic/lmcoder.git
cd lmcoder
make test