No Priors Ep. 95 | Best of 2024

Jensen Huang argues that serious AI software requires designing and validating entire data centers, then disaggregating and selling components so NVIDIA’s stack can graft into all major cloud providers while keeping CUDA ubiquitous.

Andrej Karpathy suggests that if AI becomes an extension of our cognition, people will care deeply about owning model weights or having open-source fallbacks rather than entirely ‘renting their brain’ from closed platforms.

Karpathy believes current LLMs waste capacity memorizing irrelevant details; with better data curation and distillation, performant models could shrink to around a billion parameters while using tools for external knowledge.

Bret Taylor predicts that by around 2025, interacting with a business will mean talking to its AI agent that can handle everything from support to commerce, eventually encompassing the full scope of what the company does.

The Sora team describes how training on video leads models to implicitly learn 3D structure, causality, and object interactions—potentially yielding better-than-human predictive capabilities as scale increases by simply ‘predicting data.’

Dmitri Dolgov emphasizes that while modern models can quickly produce mind-blowing self-driving prototypes, removing the driver and achieving a safety record better than humans demands solving the long-tail of rare edge cases.

Alexandre Wang contends that AGI will resemble curing cancer more than inventing a vaccine: limited cross-domain generalization means many niche capabilities and data flywheels must be solved and built over decades, enabling society to adapt gradually.