From Local Search to Constrained Decisions: Neus's Classifier Story

Neus began as a practical search tool: help an agent find the right passage in a large repository when the query and the code use different words. It combines ordinary keyword retrieval with vector similarity, then groups results by file. Over a short, unusually active stretch, the project expanded that local-first idea into a second capability: making small, bounded decisions from free-form text, such as mapping an informal role description to one of a few declared labels.

That sounds like a prompt-engineering problem until a program has to consume the answer. A model can surround JSON with prose, invent a label, or spend its response budget thinking aloud. A downstream CLI needs a predictable result and a clear way to decline uncertain input. The project therefore treated structured output as an engineering boundary, with tests and measured behavior rather than assuming a well-worded prompt would be enough.

The first important episode was a failed model choice. Ticket 036 explored starting a small local model only when needed. The runner worked, but tests exposed an awkward mismatch: the small Qwen3-0.6B reasoning model used much of its limited generation budget on internal reasoning and often failed to produce usable structured output. The team recorded that limitation instead of calling the runner a success just because the server started. It then shifted the fast-classifier experiment to a non-reasoning instruct model and added grammar-constrained generation in ticket 037. The GBNF grammar restricts the decoder to an allowed output shape and enum values. In the recorded benchmark, constrained outputs were structurally valid, while semantic accuracy still varied by task and model size. That distinction matters: valid JSON is not the same as a correct answer.

The implementation grew from internal helpers into a user-facing interface. The fallback package gained enum and multi-field grammars, an HTTP client for constrained completion, a local runner, and a repeatable six-case benchmark. Later, ticket 041 took classification out of the internal-only layer and added the neus classify CLI command, with input validation, deterministic exact and prefix matching, JSON output, and endpoint-backed classification. The implementation and follow-up fix are visible in commits b4b582f and dd778cc; documentation followed in 752a068. The deterministic path is a small but meaningful design choice: if the input already names a class, the command does not need to ask a model to rediscover it.

Neus is built in Go and keeps its own binary free of a bundled Python ML runtime. SQLite with FTS5 stores searchable text and metadata; Go packages handle Markdown and Go extraction, embeddings, ranking, evaluation, and CLI behavior. Model inference is provided through HTTP services or an on-demand local llama-server process. The repo's tests exercise the packages and CLI; the checked-in benchmark records both latency and decision outcomes. The project is explicit about the limits of those results. Its evaluation notes warn against synthetic query leakage, distinguish recalled files from chunks, and preserve experiments that regressed quality instead of silently promoting them.

The pace is also visible in the repository itself: 215 commits landed in the six-week window ending 2026-10-04, and the repository contains 10,988 tracked Go lines, 14 Go test files, and 124 test functions. Its issue tracker has 43 numbered tickets; nine are marked closed in the current index, while others remain open, parked, or in progress. Recent commits include a classifier feature, benchmark and grammar work, and two newly filed bugs around cloud fallback model naming and selecting a model that is not loaded (tickets 042 and 043). Those bugs are useful evidence of the project's working style: integration behavior is still being found at the boundary between Neus and the services it calls, and findings become tickets rather than being presented as solved.

The record shows a broad and fast-moving workload: search indexing and ranking, model-process lifecycle, constrained generation, benchmark design, CLI contracts, evaluations, and documentation all received changes in a few days. Agentic development is most instructive here when it makes that breadth reviewable. Tickets provide bounded goals, studies preserve measurements and rejected approaches, and tests catch a mismatch between an implementation and its intended contract. The same record also keeps claims honest: the classifier is a useful new interface, but its benchmark does not establish universal accuracy or completion-speed suitability.

Facts

Evidence

• Ticket 036: local fallback runner
• Ticket 037: grammar-constrained classifier
• Ticket 041: classify CLI
• 2026-10-02 classifier study
• 2026-10-03 CLI assessment