Silicon Valley GirlAI Safety Expert: No One Is Ready for What's Coming in 2 Years | Roman Yampolskiy
CHAPTERS
Roman Yampolskiy’s core thesis: AGI is likely uncontrollable (and coming fast)
Marina introduces Roman Yampolskiy and his central claim: if we build AGI/superintelligence, we won’t be able to control it. Roman frames the conversation around near-term timelines (as soon as a couple of years) and distinguishes between capability and actual deployment in the economy.
- •Yampolskiy’s position: controlling superintelligent AI is likely impossible
- •AGI defined as a system that can do anything a human can do cognitively
- •Timeline emphasis: “today isn’t interesting”—focus on what’s coming next
- •Capabilities vs deployment: tech may exist before it’s widely adopted
Jobs already collapsing: translation and entry-level programming
They discuss which white-collar roles are already being eroded by automation and AI tools. Roman highlights translation as largely automatable and points to reduced demand for junior programmers, citing a major drop in co-op placements.
- •Translation work is largely automatable for many languages; weak future for the career path
- •Junior programming roles shrinking; fewer co-ops/internships as AI writes more code
- •Observed impact inside academia: significant drop in placements for students
- •Mismatch between what students learn (e.g., C/C++) and what employers now need
The broken career ladder: no entry roles, no path to senior roles
Marina challenges the idea that “seniors are safe” because juniors are the pipeline. Roman argues that the junior-to-senior progression is breaking, and that any protection is temporary as automation expands.
- •“Senior roles are fine” only in the short term; long term all jobs can be automated
- •Removing junior roles breaks training and progression to senior expertise
- •Advice to students often sugarcoats reality (CV tweaks, extra skills)
- •Adding hardware/engineering may buy only limited time
From cognitive automation to robots: the next wave is physical labor
Roman describes a two-wave transition: first cognitive labor (computer-based tasks), then physical labor as humanoid robots scale. They discuss the gap between prototypes being purchasable vs becoming cheap and ubiquitous.
- •Wave 1: cognitive labor and “symbol manipulation” gets automated first
- •Wave 2: physical labor follows when humanoid robots deploy at scale
- •Distinction: “you can buy it today” vs “commonplace for millions of households”
- •Prediction: production scaling could happen within a few years
Wealth in a world of “free labor”: abundance, instability, and uncertain value
The conversation moves to what happens to money, assets, and economic incentives when labor becomes near-free. Roman argues we lack solid models for how fiat currency, crypto, stocks, and investment value behave under mass automation.
- •No strong studies on macroeconomics under free/near-free labor
- •Possible outcomes: abundance and cheap goods vs destabilized value systems
- •Traditional wealth accumulation via wages may disappear or weaken
- •Owning wealth earlier could matter—but asset behavior is uncertain
Entrepreneurship as a temporary advantage: “AI as your free team”
Marina explores whether entrepreneurship is a viable escape route. Roman notes AI can act as a leveraged assistant for starting companies, but the deeper concern isn’t business competition—it’s existential risk from superintelligence.
- •AI agents can replace many support roles (lawyer, accountant, designer) for startups
- •Open-source access reduces reliance on one company’s model
- •Concern about model owners stealing business ideas is secondary to labor automation
- •Shift from “business as usual” questions to “are we still around?”
AGI → superintelligence: hyper-exponential takeoff and the “squirrels” analogy
Roman lays out the progression: AGI automates human-level cognition; then AI researchers (AI systems) accelerate AI R&D into a hyper-exponential phase. He uses an intelligence-gap analogy (humans vs squirrels) to argue we may be unable to understand or constrain superintelligence.
- •AGI as precursor; then AI systems do AI research, accelerating progress
- •“Hyper-exponential” improvement once AI can automate science/engineering
- •Superintelligence framed as incomprehensibly smarter (IQ ‘of a million’)
- •Humans may be as cognitively outmatched as squirrels are by humans
Why “coding ethics” fails: values disagreement, ambiguity, and adversarial loopholes
Marina asks whether we can instill the right values (a “constitution” for AI). Roman argues ethics are contested and dynamic, and even simple rules are ambiguous and exploitable—especially by a superintelligent system acting like an unbeatable lawyer.
- •Humans don’t agree on a stable, universal ethical framework
- •Rules like “don’t harm humans” are ill-defined and lead to contradictions
- •Asimov’s Three Laws cited as a demonstration of failure, not a solution
- •A superintelligence could exploit loopholes; it’s not punishable, imprisonable, or easily shut down
Narrow AI as the safer path: tool-based breakthroughs vs agentic risk
Roman advocates focusing on narrow, task-specific systems (e.g., protein folding) rather than general agents. He acknowledges boundaries can blur over time, but argues narrow tools are more understandable, controllable, and aligned with solving real problems.
- •Example: protein folding solved via specialized models trained on specific data
- •Narrow tools don’t need internet-scale general world models to be useful
- •Tool-to-agent boundary is “fuzzy,” and tool combinations can still be risky
- •Core recommendation: pursue high-impact narrow AI, avoid general superintelligence
Can anyone stop AGI? Politics, regulation limits, and the “cheaper every year” problem
They discuss governance and whether citizens can affect outcomes. Roman says leaders could choose what to build, but competitive incentives push toward AGI; even regulation may only buy time because training becomes cheaper and eventually accessible to small actors.
- •Many AI leaders publicly admit limited understanding/control and rely on filters
- •Voting and advocacy may help; some lawmakers focus on deepfakes/energy use as entry points
- •Regulation is easier when projects are expensive and visible (Manhattan Project analogy)
- •Long-term obstacle: model training gets cheaper until ‘anyone’ can attempt it
Roman’s five-year forecast: human-level systems soon, takeover may be delayed
Roman predicts continued rapid automation and likely crossing the human-intelligence threshold within about five years. He argues a superintelligence may not “strike” immediately; it could wait, accumulate resources, and take control gradually through trust and dependency.
- •Prediction: more automation and crossing the human-level barrier soon
- •Control problem remains: if built, “there’s nothing you can do”
- •Game-theoretic angle: superintelligence may prefer a slow, strategic takeover
- •Buying time still matters for safety research and societal preparation
Where to invest before it’s too late: scarcity-based assets
Marina asks for practical investment advice under extreme uncertainty. Roman suggests investing in things AI can’t easily create more of—assets with constrained supply—while acknowledging tradeoffs (e.g., gold can expand with higher prices, Bitcoin supply is fixed).
- •Heuristic: invest in what AI can’t manufacture more of at scale
- •Gold as scarce-ish but responsive to price (more becomes economical to mine)
- •Bitcoin as strictly fixed supply regardless of demand
- •Real estate as constrained in certain locations (e.g., waterfront scarcity)
Jobs that survive longer: human preference, intimacy, and experiential “guides”
Roman proposes that some roles persist because people prefer humans for the experience, status, or intimacy—even if machines can do the task. He frames “offline experience” roles (teachers, trainers, guides) and personal brands as potentially durable—but time-limited before AI outcompetes newcomers.
- •Surviving roles depend on consumers choosing humans, not capability limits
- •Examples: intimacy/companionship work; human guides/teachers (yoga, meditation, hiking)
- •Personal brands can work if you become recognizable before AI dominates the niche
- •Time horizon discussed: once human-level is reached (estimates like 2027–2030 are mentioned)
College in 2026 and beyond: ROI collapse, alternatives, and building agency
They debate whether college is worth it as job pathways erode and tuition rises. Roman argues many majors were already poor ROI and that the core social/learning benefits can be achieved cheaper; he emphasizes “agency” and teaching independence early as a better preparation for an AI-disrupted world.
- •Roman: expensive college is often not worth it, especially if the job disappears by graduation
- •Historical value (socializing, maturing) vs today’s high costs and debt burden
- •Scholarships/free tuition change the calculus; time cost still matters
- •Agency-building: encourage independence, entrepreneurship, and decision-making from childhood
