A V-Cycle of Models: Routing a Coding Agent Between a Local Smoother and a Frontier Corrector

The idea

Multigrid is fast because it never grinds the easy, high-frequency error with an expensive solver — a cheap smoother handles that at the fine level, and the expensive coarse-grid correction is spent only on the global, low-frequency part. Point that idea at LLM inference and you get a V-cycle of models: a cheap local model does the bulk of the work (the smoother), and a frontier model is called only for the corrections the local model can't make.

This campaign tests whether it pays off, on a scalable, objectively-graded coding problem — N buggy plugin files that must each satisfy a rule and conform to a shared interface, graded by a verify.py the models never see. Two questions: does routing between a small and a large model preserve quality while cutting the expensive tier's work — and what is the right way to decide, per task, which tier handles it?

Setup

• Weak / smoother: qwen2.5-coder:7b (Ollama, Apple M4, 16 GB) — a strong 2024 coder.
• Strong / corrector: first codellama:13b, then qwen2.5-coder:14b (a genuinely stronger same-family model; the older 13B is not reliably better than the 7B).
• Router: NVIDIA NeMo Switchyard — an Anthropic/OpenAI-compatible routing proxy — and then, finally, no router at all.
• Grading: each plugin is checked against the rule on records that exercise every condition; the residual is any file that still fails.

Everything below is all-local: no API key, no spend.

Experiment 1 — naive auto-routing has a crippling tax

Switchyard's llm_classifier (capability mode) judges each request's required capability with the local model, then routes weak or strong. It works — on n=6 it split the files 3 local / 3 strong and solved everything — but look at where the tokens went.

The llm_classifier runs a ~1,250-token judging prompt on every request, so 74% of its tokens are the router thinking, not answering — and on a 16 GB box the weak↔strong model swapping thrashes. Net: 4.2× slower than just using the big model. A finding, not a feature. (A config gotcha worth recording: base_threshold is inverted from intuition — 1.0 escalates everything, 0.0/0.5 stay weak.)

Experiment 2 — the stage router kills the tax (but needs a real agent)

Switchyard's stage_router routes on tool-result / progress signals from recent turns, with no extra model call. Swapping it in eliminated the tax and came in fastest of all — 41 s, beating even the all-13B baseline (right bar above).

The honest caveat: our driver sends stateless, single-shot fix requests, so the stage router has no signals to escalate on — it routed everything to the efficient 7B. It "won" by being 7B-only with no overhead, not by smart escalation. Its real value needs a genuine multi-turn agent session, where tool results and progress actually signal when to escalate.

Experiment 3 — route by the verifier

The cleanest routing signal isn't a classifier or conversation state — it's whether the work is correct. The residual-driven V-cycle:

1. Smooth: fix every file with the 7B.
2. Residual: run the per-file check — which files still fail?
3. Correct: re-fix only the failing files with the strong model.

No proxy, no classifier: the strong tier touches exactly the files the cheap one couldn't fix. I made the suite genuinely harder (a multi-condition rule — bonus, negative-clamp, three-way status, VIP override) and used the stronger 14B.

The 7B solved the harder suite outright — residual 0/8 — so escalation never fired. The V-cycle equals weak-only: all-strong quality (all correct) at zero expensive-tier calls and half the wall-clock of the big model.

What it means

1. Route by the verifier. Because escalation is gated on correctness, the residual-driven V-cycle is a safety net that can never underperform the strong model, and it costs nothing extra when the cheap model suffices. No classifier tax, no guessing which tier a task needs.
2. A modern 7B coder is enough for routine work. Even the "hard" suite didn't stump qwen2.5-coder:7b, so the local offload is near-total: frontier-verified correctness at local cost. The interesting question isn't "how do I route?" but "is my local model already good enough?" — and for a lot of real work, it is.
3. Difficulty from "more conditions" doesn't stump a strong 7B. Escalation earns its keep only when tasks genuinely exceed the local model — hard algorithms, real multi-step reasoning — or when the corrector is a true frontier model (Claude) on genuinely hard problems, where escalating a few files is real money saved.
4. Overheads are real and local. The classifier tax (74% of tokens) and 16 GB model-swap thrash each flipped a "hybrid" from win to loss. Measure them; routing is not free.

The framing — why any of this is a multigrid V-cycle — is developed in the companion piece Solve for A, Use A: Multigrid Across Training and Inference. The full harness, configs, data, and plots are open at codeberg.org/srinathv/vcycle-model-routing.

If your team is weighing a local-plus-frontier serving strategy — deciding what actually needs the expensive model, and what a good small model can already do — that's the work I do.