The Next Chapter of AI Is Inside Our Software

The panel argues that many concrete safety practices (security testing, sandboxing, governance) are broadly agreeable—but branding it as “pacing” muddles the message and invites politicized interpretations. Without a clear baseline schedule or measurable velocity, “pacing” reads as PR rather than an actionable plan.

Casado and Sinofsky claim that if leading lab insiders genuinely believe there’s a material chance of human extinction, society’s rational response would be heavy government control (up to nationalization or nationalization-like regimes). If labs don’t truly believe it, then amplifying extinction rhetoric is seen as organizational/HR signaling that unnecessarily triggers regulatory backlash.

They emphasize that governments don’t “implement the industry’s preferred version” of regulation; once regulation becomes salient, it becomes an electoral issue and compromises drift toward outcomes nobody asked for. They predict 2028 could become “the AI election,” with anti-AI framing dominating because pro-AI arguments are nuanced and harder to sloganize.

A major thread is that AI changes security economics: agent swarms can attempt attacks continuously, cheaply, and at massive scale, turning many internal systems (APIs, auth flows, SaaS tools) into effective denial-of-service or credential-abuse surfaces. This pushes enterprises toward deeper telemetry, finer-grained authorization, and potentially new OS/app security primitives built for agents.

Sinofsky criticizes labs’ security incident disclosures as incomplete and untrustworthy compared to established vulnerability norms (CVE-style rigor). The gap between labs and the security community—“sloppy” execution plus selective postmortems—creates mistrust and raises the odds that regulators step in with blunt instruments.