All-In PodcastHow AI agents are creating a shadow payroll problem
Kalshi and Polymarket hit Super Bowl scale; CBO debt projections stoke fear. The bigger story: AI token spend is approaching salary-equivalent costs per seat.
CHAPTERS
AI tools boost productivity—but also intensify work and hours
The episode opens with a Harvard Business Review–covered UC Berkeley field study showing that AI users don’t necessarily work less—they work faster, take on more tasks, and extend work into more hours. The group discusses why this may increase demand for knowledge workers while also raising stress and burnout risk.
- •Study findings: faster pace, broader scope, longer hours, higher perceived productivity
- •Sacks’ contrarian view: AI increases demand for knowledge workers rather than replacing them
- •Shift from task-based jobs to purpose-based work enabled by offloading menial tasks
- •Key skill becomes structuring work for yourself plus AI agents
- •Burnout/stress as a tradeoff of intensified work
The rise of the “agent manager” role and bottom-up enterprise adoption
Sacks and Jason argue that early adopters will look like they have “superpowers” at work, driving adoption from the bottom up. The discussion reframes “prompt engineering” into a broader function: deploying, supervising, and improving AI agents inside everyday workflows.
- •Early adopters gain leverage: faster decks, spreadsheets, analyses, and execution
- •New role evolving beyond prompt engineering: building/managing agents
- •Bottom-up adoption: consumerized AI tools entering companies faster than top-down initiatives
- •Parallel to SaaS adoption patterns inside enterprises
- •Career opportunity for AI-native employees (referencing viral commentary like Matt Shumer’s)
Jason’s “replicants” in a venture firm: real workflows replaced by agents
Jason details how his team is operationalizing agents as “replicants” with Slack/Notion/Docs access, steadily migrating work to them each week. He describes practical automations (clip generation, analytics, reporting) and claims substantial leverage gains across the organization.
- •Agents assigned identities and tool access (Notion, Slack, Google Docs, email)
- •Automations: clipping podcast moments, monitoring social stats, suggesting growth tactics
- •Measured migration: 5–10% of work per week shifting to agents; ~20% for investment team
- •Leverage estimate: top AI users performing 10–20x compared to others
- •Agents reduce checklist burden by executing consistently once configured
Recursive output and “Ultron”: agents managing other agents
Friedberg highlights a surprising angle: recursion may be emerging through iterative outputs and agent loops rather than only via model retraining. Jason expands on this by describing a “meta-replicant” that coordinates and audits other agents, creating an internal supervisory layer.
- •Recursive output loops can mimic the benefits people expected from recursive model training
- •“Ultron” meta-agent manages multiple specialized agents and checks their work
- •Skill-building routines: nightly improvements, deep research modules, CRM integrations
- •Trust and monitoring are prerequisites before giving agents real operational authority
- •The near-term effect: agents become reliable processes, not just chat assistants
On-prem vs cloud: confidentiality, privilege, and enterprise panic risk
Chamath raises the possibility that “on-prem is the new cloud” due to data leakage and legal exposure when using public AI endpoints. The group discusses how prompt metadata, agent traces, and legal rulings around privilege could force enterprises to rethink deployment models.
- •Public endpoints risk leaking confidential prompts/responses and agent traces to model vendors
- •Legal concern: court rulings implying diminished attorney-client privilege in cloud AI usage
- •Enterprises may have to choose between cost efficiency and control/security
- •Potential swing back to private provisioned networks or controlled deployments
- •New opportunity: enterprise-grade wrappers around agent frameworks (e.g., security-hardened tools)
Token economics shock: when AI usage costs rival employee salaries
Jason and Chamath argue that token spend is becoming a first-class budget line item, sometimes approaching or exceeding compensation for top performers. They discuss rising per-agent costs, the need for cheaper inference, and how firms may ration “token budgets” across staff.
- •Jason cites rapid API spend: up to ~$300/day per agent (~$100K/year)
- •Chamath: organizations will allocate token budgets like scarce resources
- •Superstar developers may already exceed salary-equivalent token costs
- •Hardware and compute market incentives to reduce token cost (NVIDIA, Groq, Google, AMD)
- •Cost curves won’t remove confidentiality incentives pushing private deployments
Super Bowl prediction markets hit scale: manipulation vs information edge
The conversation shifts to prediction markets (Kalshi and Polymarket) reaching billions in Super Bowl wagering. Examples of suspiciously accurate accounts raise questions about insider trading, market fairness, and how to define “material” information in these contexts.
- •Scale milestone: ~$1B on Kalshi, ~$700M on Polymarket around the Super Bowl
- •Suspicious patterns: new/anonymous accounts accurately predicting halftime/setlist details
- •Reported case: alleged classified-info betting related to Israeli military operations
- •Debate on whether staff access (e.g., rehearsals) constitutes prohibited “inside information”
- •Regulatory context: CFTC oversight, but gray areas around enforceability
Sharps vs squares: why prediction markets may mirror pre–Reg FD equities
Chamath frames prediction markets using betting archetypes: sharps exploit informational advantage while squares provide liquidity and losses. He connects this to historic securities markets before Regulation FD, arguing that asymmetry may be the defining feature—hard to regulate without killing the product.
- •Sharps vs squares dynamics likely dominate long-term platform outcomes
- •Prediction markets can become “inside information” markets by design
- •Reg FD analogy: restriction of selective disclosure changed equity alpha dynamics
- •Claim: markets ‘thrive’ on information asymmetry, which can be socially useful but exploitative
- •Society must decide whether faster truth discovery outweighs fairness concerns
All-In’s “Liquidity” summit pitch: opening closed-door capital allocation
Jason and Chamath pause for an extended promotion of a new All-In event in Yountville. Chamath describes modeling it after high-signal hedge fund and investment bank gatherings, aiming to connect LPs, GPs, and top operators in a relationship-driven setting.
- •Event details: May 31–June 3 in wine country; application-based attendance
- •Goal: replicate elite idea-sharing conferences (e.g., Ira Sohn) and closed bank events
- •Mix of public-market ideas, private-company access, and LP capital allocation
- •Emphasis on relationship building and emerging manager inclusion
- •Positioned as a “money-making” conference format—curated and off-the-record in spirit
CBO fiscal outlook: deficit trajectory, Social Security, and “debt spiral” fears
The panel reviews the latest CBO forecast showing large sustained deficits, rising debt, and earlier Social Security trust fund exhaustion. Friedberg reiterates his “debt death spiral” thesis, warning that higher rates could explode interest expense and force difficult policy choices.
- •CBO numbers: ~$1.9T deficit (2026), debt rising toward ~$56T by 2036
- •Social Security trust fund depletion projected around 2032
- •Interest-rate sensitivity: higher rates could add hundreds of billions in annual costs
- •Friedberg warns about compounding interest + deficits creating a spiral
- •Concern extends to state/local pension liabilities potentially shifting federal
Growth vs austerity: Sacks and Chamath argue we may be entering an AI-driven boom
Sacks challenges CBO assumptions by arguing growth projections are too low given recent GDP prints and AI CapEx. Chamath adds historical context: debt-to-GDP trends upward across nations, so relative positioning and owning real assets may matter more than absolute ratios.
- •Sacks: CBO growth assumptions (≈2%) are too pessimistic vs recent 4–5% GDP quarters
- •AI infrastructure CapEx (hyperscalers) as a meaningful tailwind to GDP
- •Policy idea: freeze spending growth until outlays fall back toward ~20% of GDP
- •Chamath: debt-to-GDP rises across major economies; relative competition matters
- •Hedging implication: own durable/real assets as currencies debase over time
Immigration enforcement debate: target employers vs “police state” concerns
Jason argues that reducing illegal labor demand (construction and hospitality) via audits and fines is a scalable way to reduce illegal immigration. Sacks and Chamath push back on feasibility and civil-liberty implications, leading to a heated exchange about surveillance, incentives, and enforcement priorities.
- •Jason: biggest illegal employment sectors are construction and leisure/hospitality
- •Proposal: focus enforcement on employers via evidence, pay stubs, and tax compliance
- •Criticism: broad surveillance risks and selective enforcement concerns
- •Dispute over incentives: jobs vs public benefits as primary draw for migrants
- •Separation of issues: workplace enforcement vs criminal deportation priorities
Ferrari’s first EV and the future of driving culture
The episode closes with a lighter segment on Ferrari’s upcoming all-electric vehicle and its controversial design direction. The besties discuss tactile controls vs Tesla minimalism, autonomy’s effect on car culture, and admiration for chauffeur-style luxury vans unavailable in the U.S.
- •Ferrari EV: performance specs, weight concerns, and polarizing exterior rumors
- •Interior design goes viral: tactile buttons, “jet-like” startup, Apple-esque key aesthetics
- •Sacks: interior is a good compromise; exterior mockups look non-Ferrari
- •Chamath: autonomy will shrink the population that values driving as a skill/hobby
- •Detour into luxury minivans (Lexus LM/Toyota Alphard) as ideal chauffeured vehicles
