What happened

According to an AWS ML blog post, i14 built Laplace 3B as a linear attention alternative to softmax transformers and worked with AWS to run it on Trainium2 via native PyTorch support through TorchNeuron Native, preserving Gated Linear Attention and the Muon optimizer.

The write-up states Laplace 3B keeps a 29.5 MB recurrent state per sequence regardless of context length, and on one trn2.48xlarge instance it trained at about 120,000 tokens per second while serving 11,544 output tokens per second across 1,280 concurrent sequences.

Evidence

• Laplace 3B is positioned as a linear attention alternative to softmax transformers with a fixed recurrent state size.

  AWS ML · attributed

  i14 designed Laplace 3B as a linear attention alternative. Its recurrent state stays at 29.5 MB per sequence, independent of context length.

• Training and serving throughput figures on a single Trainium2 instance.

  AWS ML · attributed

  On one trn2.48xlarge instance, Laplace 3B trained at about 120,000 tokens per second. The model served 11,544 output tokens per second across 1,280 concurrent sequences.

• Softmax attention KV caches grow during inference and can affect serving economics and agent history.

  AWS ML · attributed

  During inference, their key-value (KV) cache grows with the conversation or document context. This can reduce concurrency, increase serving cost, and force agentic applications to discard useful history.

• Google open-sourced TorchTPU as a native PyTorch backend for Cloud TPUs.

  MarkTechPost · attributed

  Google has open-sourced TorchTPU , a native PyTorch backend for Cloud TPUs. It was co-developed with the PyTorch team at Meta and announced at PyTorch Conference 2026 on October 20, 2026.

• PyTorch highlights diverse frontier model architectures and production use on PyTorch.

  PyTorch · attributed

  Frontier models span language, video and audio, on architectures from dense and mixture-of-experts transformers to diffusion, state-space, world and structure-prediction models.

• Recent research explores linear-time attention via a fixed number of recurrent states.

  arXiv cs.LG · attributed

  We expand the exponential kernel of attention into a truncated Taylor series of $n$ terms and factorize each term into a tensor-power inner product, which collapses the causal history into exactly $n$ recurrent states, each updated by one rank-one outer product per token and independent of context len

Why it matters

Builders and operators considering long-context or agentic applications may find linear attention architectures attractive for reducing memory growth and serving costs, but should validate quality and throughput trade-offs against their own workloads.

Limits and uncertainties

The excerpt does not establish comparative quality, latency, cost, or performance against transformer baselines, nor training data or evaluation benchmarks.

Reported throughput is from a single trn2.48xlarge instance without comparison to transformer baselines or standard benchmarks.

The primary Laplace account is a vendor blog post that may emphasize favorable results.

Practical implications

Teams on Trainium2 can experiment with Laplace 3B through native PyTorch and TorchNeuron Native rather than rewriting the stack for a custom runtime.

Serving planners weighing long-context agents should model fixed recurrent state memory against growing KV caches, then verify task quality before committing production traffic.

What to watch

Independent benchmarks comparing Laplace 3B quality and end-to-end serving cost against softmax transformer baselines on representative workloads.

Whether additional linear-attention models report Trainium2 throughput at scale beyond a single instance configuration.