There is something quietly strange about the AI stack of 2026. The biggest models still need a data center, but the ones people actually use for day-to-day work keep getting smaller — and a growing share of them now ship straight to your browser. No server, no API key, no "uploading to the cloud." The weights download once, and thereafter the tab works offline.

That shift is real, it is verifiable, and it was not true even a year ago. This is a quick field guide to how it works and where it still breaks.

The unlock was WebGPU, not WebAssembly

WebAssembly has been able to run neural networks in the browser for years. It was just too slow — pure WASM SIMD inference typically lands 5 to 10 times slower than native code, with a 3.8B-class model crawling along at roughly 3 to 8 tokens per second on a normal laptop. Genuinely usable speed waited for WebGPU: the browser API that hands your JavaScript real access to the graphics card. With GPUs on the table, the whole equation changes. In-browser inference today runs at close to native speeds on the hardware the browser actually sees.

Three ways to run a model in a tab

The three main paths map roughly onto how much control you want.

WebLLM, from MLC AI, is the speed and simplicity leader. Its September 2026 release (v0.2.85) ships 163 prebuilt model builds, from a 360M SmolLM2 up to Llama 2 13B, all behind the same OpenAI-shaped chat.completions.create() API you might already call against a hosted vendor. Swap the endpoint for a local engine and your existing code barely changes. In the team's own measurements, WebLLM decoded Llama 3.1 8B at 41.1 tokens per second on an M3 Max MacBook Pro — about 71 percent of what native MLC-LLM managed on the same machine. That is the gap between "hosted" and "in a tab" in 2026: a faster-than-you-read speed, on your own GPU.

Transformers.js (v4.3.0, from Hugging Face) takes the opposite bet. It is WebGPU-first but keeps a WebAssembly fallback, so it works in browsers without a usable GPU. And it is not only about chat — embeddings, speech recognition, image classification all run under the same roof. If you want one library for the whole huggingface zoo in the browser, this is it.

Chrome's built-in Prompt API is the third, and the one that needs no library at all. The LanguageModel interface went stable for the open web in Chrome 148 back in May, so a plain <script> can call the browser's own interface. Barely any setup, but you are tied to whatever the browser ships.

There is also wllama (v3.6.1) for the llama.cpp purists — it wraps llama.cpp in WASM and will happily load raw GGUF files on the CPU, with the caveat that WebAssembly multithreading needs a couple of cross-origin headers your server may not be sending.

Where the edges still bite

Three honest limits. First, memory: a 4-bit Llama 3.2 1B is a 695 MB download that wants roughly 880 MB of GPU memory once running. Second, format friction: WebLLM does not load GGUF directly — models must be requantized into MLC's own format, and storage quotas and first-run downloads are a UX problem no one has fully solved. Third, the vendor churn: Google has put its MediaPipe LLM Inference API into maintenance-only mode, pointing developers at a replacement not everyone has caught up with.

The research is catching up fast too

Microsoft Research's "Llamas on the Web" project, published this spring, adds a WebGPU backend to llama.cpp itself, claiming 29 to 33 percent lower memory use across 16 devices from 8 vendors and 45 to 69 percent higher decode throughput on the four GPUs they tested head-to-head.

None of this means cloud APIs are going away. Reasoning-heavy work and huge models still need real infrastructure. But for a private, offline, zero-cost chat or agent headed straight to a static site? In 2026, that is increasingly a hardware problem, not a "should we build it" one.