Uncapped with Jack AltmanThe Future of AI Software Security | Ep. 39
CHAPTERS
- 0:00 – 0:40
AI “bears” and why software security is about to get harder
Daniele opens with a vivid security analogy: historically you only had to be safer than the next company, but AI changes the math by enabling attacks at massive scale. Jack frames Daniele’s background (Faire, Cash App/Square security) and sets up the conversation toward Depthfirst and AI security.
- •Security has long been a relative game (“outrun the other person”)
- •AI could create “a thousand” automated attackers at once
- •Context on Daniele’s career across marketplaces, fintech, and security
- •Teaser for Depthfirst’s mission: securing software against AI-driven threats
- 0:40 – 2:10
The founding insight behind Faire: take risk off retailers’ balance sheets
Daniele explains Faire’s contrarian bet on brick-and-mortar retail growth and the specific pain he and the founders saw in wholesale discovery and distribution. The key insight was offering retailers net terms and easy returns—absorbing discovery and inventory risk using technology.
- •Wholesale distribution was fragmented and hard to access (Amazon/Nordstrom/Walmart vs long-tail retail)
- •Max’s umbrella side-gig exposed how broken brand-to-retailer sales channels were
- •Faire’s value prop: order now, pay later (60 days) and return what doesn’t sell
- •Faire applies Square’s lesson: taking risk on behalf of customers creates value
- 2:10 – 4:28
Finding product-market fit: experiments, trade shows, and “try before you buy”
The early Faire journey wasn’t linear—teams tested multiple approaches (consignment, loyalty/points) before landing on a message customers immediately understood. A fast build-ship loop around trade shows helped crystallize the winning concept.
- •Pre-PMF involved significant meandering and experimentation
- •Consignment was capital-intensive and risky; other programs didn’t click
- •Trade-show-driven iteration enabled overnight product changes
- •“Try before you buy” framing unlocked immediate customer understanding
- •PMF feels like a discrete shift from “maybe” to obvious pull
- 4:28 – 9:00
Operating rigor in marketplaces: truth-seeking, measurement, and humility
Daniele describes why marketplace businesses demand intense operational rigor: supply/demand balance, risk management, onboarding success, and second/third-order effects. He emphasizes AB testing, data analysis, and “epistemic modesty” when complex systems surprise you.
- •Marketplace health depends on balancing brands (supply) and retailers (demand)
- •Risk and payment terms require continuous modeling and monitoring
- •Small changes can ripple through a recursive, chaotic system
- •Data rigor must be paired with intuition/vision to avoid incrementalism
- •AB tests often fail due to unanticipated second/third-order effects
- 9:00 – 10:38
Convincing the world (and yourself): PMF isn’t the end; TAM clarity takes time
Even after Faire started working, it still took years to persuade candidates and investors—and even the team—how large the market could be. Daniele argues TAM is often directional early on and becomes clear only after sustained exposure to real-world signals.
- •Post-PMF, external belief lagged by 2–3 years
- •Early TAM estimation was uncertain despite strong product signals
- •Millions of stores and trillions in wholesale took time to internalize
- •Founders often over-weight TAM precision too early
- •Eventually narratives “echo back” once the market catches up
- 10:38 – 12:00
Starting companies now vs 2017: faster cycles, higher stakes, constant uncertainty
Daniele contrasts the earlier era’s relative stability with today’s AI-driven environment where assumptions can change every few months. The intensity is amplified by the speed of growth and potentially massive rewards.
- •2017 felt like building on a steadier underlying system
- •Today, product, market, and competition can shift rapidly
- •Founders face more energy, paranoia, and urgency
- •AI creates unusually large upside for winners
- •Even the possibility of abrupt discontinuities (“singularity soon”) affects planning
- 12:00 – 16:38
Cash App’s inception at Square: big-company bets and outsized personal leverage
Daniele recounts joining Square from academia and adopting a mindset that he could “x” the value of the business through focused impact. He joins the early Cash App effort, helps replace risky email-based mechanics, and later attacks fraud losses with rules and ML to enable scale.
- •Transition from academia to industry and overcoming imposter syndrome
- •Self-fulfilling belief: act as if there’s a path to outsized impact
- •Cash App began as a Hack Week project; early implementation had security concerns
- •Daniele led fraud/risk reduction efforts and cut losses ~80%
- •Security mindset overlaps with hacker thinking: find paths others don’t see
- 16:38 – 18:08
Depthfirst’s mission: software security as a prerequisite for AI safety
Daniele explains Depthfirst as a mission-driven company: without much stronger software security, broader AI safety and control efforts are undermined. Depthfirst aims to build a flywheel between securing open-source infrastructure and delivering enterprise security products.
- •AI safety/control is mediated by software—so insecure software is foundational risk
- •Goal: align commercial success with securing the world’s software
- •Flywheel concept: secure open source + build tools that enterprises will buy
- •Target customers: companies protecting systems and customer data
- •Motivation rooted in long-term societal impact, not just market opportunity
- 18:08 – 21:27
AI security landscape: from point tools to an “AI security engineer” swarm
Daniele describes how current security products rely heavily on heuristics and rules, often producing false positives and missing deep issues. With AI reasoning, Depthfirst expects convergence across security categories into agents that understand systems end-to-end and surface real vulnerabilities.
- •Legacy tools scan narrow slices and struggle with depth and accuracy
- •AI enables better reasoning over code, infrastructure, and misconfigurations
- •Vision: a swarm of agents mapping ingress/egress and critical flows
- •Goal: detect issues previously requiring human intuition (e.g., auth bugs)
- •Security becomes more unified as AI generalizes across domains
- 21:27 – 28:08
Attackers vs defenders in an AI world: cost curves, context advantage, and PR scanning
Perfect security is impossible; instead, security is an economic balance of attack cost vs payoff and likelihood of enforcement. AI reduces the cost of attacks and increases frequency, but defenders can still win by leveraging full internal context—paired with continuous scanning (e.g., pull requests).
- •Security parallels bank vault economics: difficulty + probability of being caught
- •Online enforcement is weaker, so the system becomes cost-dominated
- •Abundant AI intelligence lowers attack cost → more frequent attacks
- •Defenders’ edge: full context and ability to spend compute mapping systems
- •Continuous PR scanning and fast review reduce security-productivity tradeoffs
- 28:08 – 31:15
Why security feels like an echo chamber: the false-positive problem and proving value
Daniele explains why security is uniquely hard to buy and sell: it’s difficult to verify claims and even harder to know what wasn’t found. Depthfirst uses AI not only to find more issues but to show its work—assumptions, verification steps, and evidence—so buyers can trust results.
- •Security buyers can’t easily validate vendor findings (false positives)
- •Even ‘no findings’ is ambiguous: good security or weak detection?
- •Information asymmetry is worse than a typical ‘market for lemons’
- •AI can increase true-positive rates and validate assumptions like a human would
- •Providing transparent reasoning and assumptions improves buyer-seller alignment
- 31:15 – 34:54
Building Depthfirst’s tech stack: infrastructure harness + RL for “superhuman hackers”
The conversation turns to Depthfirst’s core technical bets and team composition (Databricks infrastructure leadership, DeepMind RL expertise). They describe building a robust execution environment (containers/harness) and using reinforcement learning plus LLMs to discover deeper, composable vulnerabilities.
- •Founding team spans enterprise infra/security (Databricks) and frontier AI (DeepMind)
- •Security AI requires heavy infrastructure: sandboxing, containers, test execution
- •“Scaffold/harness” lets models verify hypotheses against real systems
- •RL + LLMs aim to push beyond shallow findings into complex exploit chains
- •Human security researchers still provide critical creativity and validation
- 34:54 – 39:14
Product direction, data boundaries, and human–AI collaboration in security teams
Daniele outlines the end-state product: install Depthfirst and continuously surface high-confidence vulnerabilities, including staging-environment validation and conversational workflows. He also clarifies enterprise privacy boundaries (no customer data in model weights) and frames AI as a collaborator where humans retain final contextual judgment.
- •Product goal: constant, thorough, faster vulnerability discovery with validation
- •Use staging environments to reduce uncertainty and confirm exploitability
- •Security requires deep organizational context (including historical commits)
- •Enterprise constraint: customer data does not enter model weights; improvement comes via open-source outer loop
- •Near-term operating model: AI collaborates; humans maintain final contextual authority
- 39:14 – 45:27
Platform vs pipeline lessons and decision-making cadence for building in the AI era
Daniele compares marketplace (platform) dynamics to pipeline SaaS and how that changes management—marketplaces need tighter coordination, while pipeline businesses can let more experiments run. He closes with practical operating principles: small-sample analysis to build intuition and frequent high-confidence decisions over rare near-perfect ones.
- •Platform businesses require more coordination due to second/third-order interactions
- •Pipeline businesses allow more parallel experimentation (“let flowers bloom”)
- •Use ‘30 data points’ to build intuition and make directional calls quickly
- •Avoid the trap of waiting for ‘big data’ before acting
- •Bias toward speed: multiple 90% decisions weekly over a single 99% quarterly decision
