1.58 Bits per Weight: When Every LLM Weight Is Just -1, 0, or +1

The default unit of an LLM is the float: a 16-bit number with room for values like 0.3729181. In February 2024, Microsoft Research published BitNet b1.58 (arXiv:2402.17764) with a stranger suggestion. What if every weight in the model could only ever be -1, 0, or +1?

Three values need log2(3) bits to store — 1.58 — which is where the name comes from. The claims that followed were the interesting part: a model whose weights live on that three-point line can match a full-precision (FP16/BF16) Transformer of the same size and training tokens on perplexity and end-task benchmarks, and the gap only narrows as models grow. The paper even framed it as a new scaling law.

The real payoff is arithmetic. Multiplying an activation by a ternary weight is not multiplication at all; it is addition, subtraction, or skipping the term. b1.58 keeps 8-bit activations, but the weight side of the matmul stops needing multipliers, and the paper estimated the arithmetic energy of matrix multiplication drops by roughly 71x versus a Llama-class baseline.

Then the software caught up. bitnet.cpp, the official open-source stack (the microsoft/BitNet repo), delivers lossless inference — outputs token-for-token identical to a full-precision kernel on a 1,000-prompt WildChat eval — while running 2.37-6.17x faster than FP16 llama.cpp on x86 CPUs and 1.37-5.07x on ARM, with energy down 71.9-82.2% and 55.4-70.0% respectively. The headline result still reads like a typo: a 100B-parameter ternary model runs on a single CPU at 5-7 tokens per second. Human reading speed. No GPU required.

The line kept moving. BitNet a4.8 (November 2024) pushed activations down to 4 bits, activates only about 55% of parameters, and compresses the KV cache to 3 bits without losing ground at equal training cost. In 2025, the BitNet b1.58 2B4T technical report described the first open native 1-bit model at the 2-billion scale, trained on 4 trillion tokens and landing in the same league as full-precision peers its size. llama.cpp ships ternary formats (TQ1_0, TQ2_0); the Falcon3 family (1B-10B) runs through bitnet.cpp. The experiment stopped being a curiosity years ago.

Now the honest part: nothing here is free, and the bill lands in training. 2B4T is two billion parameters at four trillion tokens — its pretraining cost rivals a conventional model's. That detail keeps showing up in the literature: the paper behind Tequila, a "trapping-free" ternary-quantization method presented at ICLR 2026, opens by noting that BitNet trains from scratch on 4T tokens, "a cost rivaling standard pre-training," and spends its pages trying to make ternary training cheaper — its 3B model edges out a 3.9B Spectra ternary model on average accuracy while using roughly a tenth of the training tokens. Activations, the harder half of quantization, are why a4.8 exists at all.

So where does this leave us? Nobody is claiming ternary weights replace precision everywhere; small models still lose more than big ones, and 8-bit activations are doing hidden heavy lifting. But the core idea — store and compute the weight side of an LLM in two bits, turn matmul into additions, and run a 100B model on a laptop CPU — has moved from paper provocation to shipping software. The training-cost question is the bottleneck now, and it is a far more interesting bottleneck than "will anyone build 1-bit hardware," because the software already did.