Vladimir Vapnik: Predicates, Invariants, and the Essence of Intelligence | Lex Fridman Podcast #71

Vapnik insists that engineering systems that imitate human behavior (e.g., self-driving cars) is fundamentally different from discovering the abstract structures—predicates and invariants—that constitute intelligence itself.

He posits, in a Platonic spirit, that there exists a relatively small set of abstract predicates (functions over data) that, when combined with reality, generate invariants and make learning data-efficient and interpretable.

Beyond strong (pointwise) convergence, weak convergence focuses on integral properties (inner products with predicates). Enforcing that learned functions match observed predicate averages on data sharply reduces the hypothesis space and data needed.

Vapnik challenges the community to match state-of-the-art MNIST performance using orders of magnitude fewer samples by leveraging good predicates (like degree and types of symmetry), arguing this would demonstrate genuine progress in understanding visual intelligence.

He characterizes convolution as essentially a single, human-designed predicate enforcing translation invariance, and criticizes the field for exploring huge classes of piecewise-linear functions without a clear, idea-level account of the invariants they respect.

Analogous to physics, he suggests identifying situations where existing predicates fail (where invariants don’t hold), then adding new predicates that restore invariance, iteratively refining the admissible function set.

Vapnik notes that music and literature critics routinely use a compact vocabulary of abstract descriptors; mining such language could expose high-level predicates that transfer to domains like image understanding and digit recognition.