No PriorsNo Priors Ep. 32 | With NEAR’s Illia Polosukhin
CHAPTERS
- 0:00 – 1:38
Illia’s origin story: from LSTMs to the Transformer breakthrough
Illia recounts how practical frustrations with slow, production-unfriendly RNNs/LSTMs led his team to explore attention-based alternatives. He describes early experimentation with attention for encoder-decoder models and how that work evolved into the Transformer architecture.
- •Limitations of LSTMs for real-world question answering and document-scale processing
- •Jacob’s early use of attention for query similarity and the leap to encoder-decoder attention
- •Experimenting with training stability and word order understanding using attention
- •How exploratory research snowballed into the Transformer paper’s impact
- 1:38 – 3:15
NEAR’s surprising pivot: an AI ‘machines that code’ mission becomes blockchain
NEAR started as an AI company aiming to teach machines to code, fueled by early excitement around Transformers-era progress. The company’s immediate operational need—paying a global crowd workforce for data tasks—pushed them toward blockchain as a payment and coordination layer, eventually becoming the core focus.
- •Original mission: building AI to translate language ↔ code
- •Crowdsourced dataset creation via global developer/student community
- •Payment friction across countries with capital controls and limited banking access
- •Blockchain adopted first as a solution to compensation logistics, then expanded into a broader platform bet
- 3:15 – 4:04
What a “blockchain operating system” means in practice
Illia explains NEAR’s goal of abstracting away low-level blockchain complexity so users can simply discover and use applications—similar to how mobile operating systems hide networking and hardware details. The emphasis is on a developer-friendly platform that delivers Web3 experiences to mainstream consumers.
- •“Go upstack” to focus on user experience and app discovery
- •OS analogy: abstract complexity; let developers build apps on top
- •Positioning Web3 as a consumer platform, not just infrastructure
- •Delivering an integrated framework for building and using Web3 apps
- 4:04 – 6:44
Where AI and Web3 intersect most: marketplaces and AI economic agents
The conversation shifts to overlap between AI and blockchain: resource marketplaces are the obvious starting point, but AI agents with wallets/accounts are the more transformative idea. Equipping agents with on-chain identity and funds could turn them into autonomous economic actors that coordinate people and services.
- •Marketplaces for compute, models, and data as a near-term intersection
- •Blockchain strengths: traceability, open participation, equitable market design
- •AI agents + blockchain accounts = agents that can pay for labor/services
- •AIs communicating directly with humans removes the need for “interpreter” middle layers
- 6:44 – 10:00
AI-run organizations and DAOs: the path to an ‘AI CEO’
Illia paints a picture of organizations where an AI agent functions as a CEO/project manager, tasked with KPIs and governed by community or oversight mechanisms. DAOs are positioned as a natural early adoption venue because much DAO work is repetitive onboarding/coordination and already payment-native.
- •AI agents coordinating work: tasking, context-sharing, and feedback loops
- •Example use case: AI-coordinated cancer research pipelines to reduce overhead and bias
- •DAOs as early proving ground for AI management and automation
- •Practical adoption constraints: social resistance to “AI boss” in traditional companies
- 10:00 – 11:37
Reframing alignment: ‘AI alignment’ is really ‘human alignment’
Illia argues society must address the human roots of misinformation and adversarial behavior rather than treating alignment as purely an AI problem. He connects this to Byzantine fault tolerance and emphasizes building social systems—identity, reputation, provenance—that can operate under malicious conditions at scale.
- •Alignment as a societal problem: humans create the incentives and failures
- •Byzantine fault tolerance as a historical analogy for misinformation
- •AI increases scale and personalization of misinformation, not the underlying phenomenon
- •Need for robust reputation/identity systems as core social infrastructure
- 11:37 – 14:23
Content authenticity and provenance: cryptographic signatures as the ‘green lock’ for media
They discuss how cryptographic tooling could authenticate media from capture through edits and publication, providing verifiable provenance. Illia suggests a future where content platforms display trust signals akin to SSL—verifying who signed content and how it was processed—while acknowledging identity and reputation remain crucial.
- •Secure enclaves can sign photos at capture; provenance can track subsequent processing
- •Podcast example: hosts signing final content to prove authenticity
- •On-chain identity as a hub for content, interactions, and reputation context
- •A “green lock” UX for media authenticity, analogous to HTTPS/SSL adoption
- 14:23 – 17:09
Why blockchain identity hasn’t fully arrived: usability, apps, and critical mass
Elad probes what blockchain-based identity might look like beyond ‘wallet as identity.’ Illia explains that private keys are too hard for most users, describes NEAR’s named accounts and permissioned keys, and argues adoption requires many apps using the same identity layer—similar to the long slog of SSL becoming default.
- •Wallets function as identity today, but private-key UX is a major barrier
- •NEAR model: human-readable accounts with multiple keys and granular permissions
- •Agents can be delegated keys with scoped access; keys can be revoked
- •Identity needs an ecosystem of apps to reach critical mass and become a default standard
- 17:09 – 20:59
Near-term threat landscape: elections, fraud, and law enforcement blind spots
Illia forecasts a wave of AI-driven manipulation—especially around elections—via synthetic candidates, personalized propaganda, and mass-produced media outlets. He also highlights growing criminal misuse of voice cloning and deepfakes, arguing that verification must be embedded into phones, calls, and everyday workflows.
- •AI-generated political narratives and hyper-personalized campaign pages
- •Flooding media with synthetic content absent provenance and validation frameworks
- •Consumer fraud and broader criminal exploitation via voice cloning and impersonation
- •Need for built-in authentication in phones/communications leveraging existing secure hardware
- 20:59 – 24:37
Decentralized compute for AI: why training is hard, but inference is promising
They examine whether crypto-era GPU capacity and decentralized networks can support AI workloads. Illia notes mining GPUs often aren’t ideal for modern training, and distributed training faces extreme bandwidth requirements, while decentralized inference is more feasible—especially for privacy and bursty scaling—potentially aided by MPC and zero-knowledge methods.
- •Mining-era GPUs often lag what training stacks want (A100/H100-class demand)
- •Distributed training is bottlenecked by bandwidth/latency (data-center-grade interconnects)
- •Inference needs far more aggregate compute over time than training does
- •Crypto primitives (MPC, ZK) could enable private, decentralized inference and coordination
- 24:37 – 29:13
Decentralized data labeling: incentives, escrow, and quality control mechanics
Returning to NEAR’s roots, Illia argues Web3 marketplaces can outcompete centralized labeling by widening access and adding fairer payment guarantees. He explains why quality is difficult—domain expertise and validation tooling—and describes mechanisms like buy-ins, honeypots, and economic incentives to reduce low-quality work.
- •Web3 labeling marketplaces: global workforce without local entity setup
- •Escrow and rules-based payment reduce exploitation seen in systems like Mechanical Turk
- •Domain expertise access (e.g., developers) is easier with open participation
- •Quality controls: cross-validation, honeypots, and staking/buy-in penalties for bad work
- 29:13 – 31:04
NEAR’s next bets: data marketplaces, Web2→Web3 partnerships, and ‘socialware’
Illia outlines areas he’s most excited about: spinning out a Web3 AI data marketplace and partnering with consumer apps migrating from Web2 to Web3. He uses Sweatcoin as an example of turning a narrow app economy into composable on-chain participation and hints at broader shifts in how users engage with digital products.
- •Spinning out a dedicated Web3 AI data marketplace product for go-to-market expansion
- •Partnership strategy with apps that have existing scale and want Web3 capabilities
- •Sweatcoin example: large install base transitioning to on-chain economic participation
- •Vision of a more composable, open ecosystem where users can do tasks/gigs and interact socially
- 31:04 – 34:14
The future of SaaS: user-owned databases + AI-generated workflows and interfaces
Illia predicts many SaaS products will be reshaped by combining Web3 data ownership with AI-driven interface generation. Instead of siloed databases with brittle integrations, users could own a core data layer while agents dynamically generate workflows, dashboards, and UIs tailored to each role and goal.
- •Today’s SaaS often equals ‘a database + a UI’; many tools share similar underlying structures
- •Integrations between siloed SaaS databases are costly and fragile
- •Web3 enables user-owned/shared data primitives; AI enables natural-language-defined processes
- •Hybrid future: agents generate pipelines and dynamic dashboards; views fork by role (e.g., conversion vs retention)
- 34:14 – 42:24
Transformers, ‘thinking time,’ and the hardware lock-in question
The episode closes with a discussion on whether Transformers are the long-term default architecture. Illia anticipates improvements largely within the Transformer paradigm—training models to critique, iterate, and ‘think’—while Elad and Sarah explore how GPU/CUDA optimization creates lock-in and how competing accelerators will still likely target Transformer-like workloads.
- •Research direction: training models to pause, self-critique, and allocate more compute before answering
- •Blending training-time changes with inference-time controls for “thinking”
- •Transformer lock-in from software and hardware co-optimization (GPU/CUDA flywheel)
- •Hardware landscape: many accelerators emerging, but mostly optimized for Transformers or similar structures
