The Agent That Fits on a GPU: Distillation Is Closing the Frontier Gap

Today Meta released Muse Spark Open, its most capable open-weight model yet. It's an 816-billion-parameter mixture of experts with a 1-million-token context window, and it is big in a physical way. The full-precision checkpoint runs to about 1.65 terabytes of weights; even the 4-bit build needs a 537 GB allocation. So when Meta talks about running it on "your desktop," that means two Apple M5 Ultra Mac Studios cabled together, sharing a terabyte of unified memory.

But that's only half of this week's story, and arguably the less interesting half. In August, Meta shipped Muse Glimmer, a 30-billion-parameter model trained by distillation from that same frontier-class teacher. It fits on a single consumer GPU. And it's built to run entire agent workflows locally, no cloud round-trip required.

One lab, two ends of the spectrum

What's notable isn't that Meta released two models; it's that they're designed as two ends of one design spectrum. Muse Spark Open — 816B total, 42B active, natively multimodal, tuned for coding and long-horizon agent work — marks the ceiling on capability. Muse Glimmer marks the floor on size. At full precision it would need more than 55 GB of memory, so Meta compressed the weights to roughly 4-bit precision, shrinking the language model to under 20 GB. That leaves headroom for the key-value cache, a 1.8-billion-parameter vision encoder for images, and a speculative-decode head to run together inside a 24 or 32 GB card.

What survived the squeeze

Distillation isn't just a diet for the weights; it decides which behaviors carry over. Muse Glimmer was evaluated end-to-end on real agentic benchmarks — DeepSearch QA, MCP-Atlas, tau3-Bench, and SWE-Bench — and Meta reports state-of-the-art results for its size class against Gemma4-31B and Qwen3.6-27B on tool use, multi-step planning, and task completion.

The headline behavior is failure recovery. When a tool call comes back wrong, the model diagnoses the error and retries instead of stopping. That loop is what makes a local agent genuinely useful, and it's exactly what past "small but smart" models struggled to pull off. The model is also multilingual (more than a hundred languages) and works within agent scaffolds like OpenClaw and Hermes Agent.

A pattern, not a one-off

Zoom out and the same thesis keeps showing up: distill from a strong teacher, quantize hard, keep the runtime tight. In the sub-billion range, MobileLLM-Pro and MobileMoE chase full agentic behavior with a fraction of the parameters; MedPsy's 4B medical model reports matching a 27B baseline in its domain. The resolution of "frontier quality" is creeping downward — and under Apache 2.0 in Muse Glimmer's case, that means the weights are yours to fine-tune, not a service to rent.

The practical wins are real: data stays on-disk, latency drops to local speeds, and cost stops scaling with usage.

A moment for honest caveats

The benchmark numbers here are vendor-reported against specific peers, not the whole frontier, and this space moves monthly. Once a model acts on the world — editing files, sending mail, moving money — pushing it onto user hardware changes the safety story too. Meta itself recommends human-in-the-loop confirmation for irreversible actions.

For developers, the near-term question has shifted. It's no longer "can I run a model around here?" but "which agent can I afford to run myself?" Muse Glimmer argues the answer is settling somewhere under 20 GB — and shrinking.