Training a Frontier LLM Across the Open Internet: Inside Hermes 4.3 and the Psyche Network

For a long time, one assumption sat under almost every big model: you train in a single data center, or you don't train at all. Splitting a run across sites was a non-starter. The interconnect bandwidth that keeps billions of gradients moving just isn't there over the public internet, and a sync that lags cuts straight into GPU utilization — the most expensive meter in the building.

Nous Research's Hermes 4.3, released December 3, 2025, is a pointed counter-argument. It's a 36-billion-parameter reasoning model, and it was post-trained entirely across a peer-to-peer network that spans the open internet — with consensus state tracked by a smart contract on the Solana blockchain. It's believed to be the first frontier-adjacent production model trained this way.

What actually shipped

Hermes 4.3 is built on ByteDance's Seed-OSS-36B base and supports a context window up to 512K tokens (524,288 to be exact). Nous claims it nearly matches — and in a few cases exceeds — its own 70B Hermes 4, at about half the parameters. The GGUF quantizations sit comfortably in the VRAM of off-the-shelf consumer GPUs, which is what makes "local intelligence, globally trained" more than a slogan. Like its predecessor, the weights ship under Apache 2.0.

Why the internet was the hard part

The problem with training across data centers is communication. DisTrO, Nous's optimization framework, attacks the bandwidth ceiling directly by shrinking how much the nodes have to say to each other, so gradients can make the trip across the open internet without bottlenecking every step.

Psyche, the network it rides on, uses a dual-layer model. One layer handles control: a Solana smart contract keeps consensus state shared and auditable. The other carries the actual payload — training gradients move out-of-band over a custom mesh P2P network. DisTrO's "overlapped collective" strategy then hides that peer-to-peer traffic behind the compute itself, so the network chatter overlaps with tensor work instead of stalling it. In the released numbers, that worked: the run averaged roughly 144,000 tokens per second across 24 Psyche nodes, effectively matching the throughput of a conventional centralized setup.

Two runs, on purpose

To make the claim credible, Nous trained the model twice. One run went through Psyche (TP + DisTrO). The other used a traditional centralized path — an FSDP + AdamW recipe on a TorchTitan fork, the same approach as Hermes 4. The centralized checkpoint was released as a research artifact so anyone can compare. Notably, the 4.3 training run was roughly twice as large as Hermes 4's, driven by the extended context and a bigger corpus.

The honest caveats

"Nearly matches" is doing precise work. Independent teardowns note that 4.3 does not win every scorecard against the 70B model — SimpleQA fact recall is a notable weak spot, for example. Nor is Psyche doing fully novel pretraining from scratch; this is post-training on a powerful open base. And the centralized derivative carries some murkier licensing language than the Psyche BF16 and GGUF files.

Still, the big idea stands. Distributed training over the open internet, with blockchain-anchored consensus, means far-flung contributors can pool compute for a single run — and read the ledger to prove nothing was silently altered. If the cost of frontier-scale training drops because the data center ceiling is gone, Hermes 4.3 is the first real artifact to demonstrate it end to end. Worth a close read for anyone building (or renting) large-scale training infrastructure.