"Anti-Data Centers is a Chinese Psyop" | How Many Will Actually Get Built? | Positron AI Co-founder

Sohmers argues training is largely FLOPS/compute-bound because inputs can be massively parallelized, while inference is bottlenecked by repeatedly reading model weights and user-specific state for each next token in an autoregressive loop. That shift changes which chip architectures and system designs matter most (bandwidth, memory capacity, and data movement).

He frames the “memory wall” as a decades-old structural issue: compute performance has improved far faster than memory bandwidth (he cites ~120× GPU FLOPS gain vs ~17× bandwidth over ~2014–2024). The result is widening inefficiency for memory-bound workloads like modern LLM inference.

Sohmers claims token pricing hides major economics: cached token processing is “essentially free” relative to recompute, so providers earn very high margins on cached reads even when they offer discounted cached rates. He uses this to argue API businesses can be highly profitable if frontier training spend is moderated.

KV caching reduces per-token compute by storing attention keys/values, trading quadratic attention recomputation for linear cache growth—but the cache becomes huge at long contexts and at scale across many users. He highlights a practical tipping point where aggregate user-session caches can exceed model-weight storage (especially for long-context, large-parameter models).

He views calls to “pace the frontier” as risky because regulation may entrench a few labs and create durable monopolies—his core fear is concentration of capability, not sci‑fi extinction scenarios. He also argues the U.S. cannot effectively pace unilaterally if geopolitical competitors do not participate.