CHAPTERS
- 1:41 – 4:22
Salesforce AI org scope: predictive, generative, and agentic systems
Jayesh describes the AI engineering organization he leads at Salesforce and the kinds of AI systems it builds and operates. He frames their mission as deeply embedding multiple “flavors” of AI into the Salesforce stack to enable customer workflows, including agentic use cases.
- •Jayesh leads Salesforce’s broader AI engineering org and sets strategy, builds, and operates AI systems
- •Work spans predictive AI, generative AI, and agentic AI embedded into the Salesforce stack
- •The team focuses on enabling customers to build agents for their own scenarios and use cases
- 4:22 – 9:34
How agentic coding and AI tools changed team structure and day-to-day work
Jayesh breaks down how the org is layered across model work, harness technologies, and the emerging agent development lifecycle. He also explains how internal workflows have shifted toward using AI coding tools and operating primarily through Slack, including closer collaboration with customers.
- •The org separates into teams for model training/tuning, harness technologies, and ADLC/agent optimization
- •Partnerships with frontier model providers are part of the model-layer workflow
- •Teams use tools like Claude and Codex in development processes
- •Slack is the primary interface for how work happens across teams
- •A major shift is learning to manage “a team of agents” rather than doing all work manually
- 9:34 – 12:56
Choosing and rolling out AI tools: choice, adoption data, and security gates
Aakash asks how executives decide when to introduce new AI tools and how rollouts should work. Jayesh emphasizes rapid tool evolution, giving teams choice after security review, and letting real usage data determine best practices over time.
- •Tools are evolving at an unprecedented pace, so bets must remain revisable
- •Teams were using one tool, then shifted as new options proved valuable in practice
- •Provide users tool choice initially, then observe adoption and outcomes
- •All tools must pass corporate security review before being made available
- •Leaders should focus on processes that track the changing “slope” of progress rather than locking to one tool indefinitely
- 12:56 – 15:25
Headless Salesforce in Claude: why the “magic” is in APIs and metadata, not UI
The conversation turns to customers working outside Salesforce’s traditional UI, such as inside Claude or Slack. Jayesh explains “headless” as exposing Salesforce’s underlying metadata, business logic, and workflows through APIs so customers can use them in whatever interface they prefer.
- •Aakash frames the shift as users living inside Claude or Slack rather than only inside Salesforce UI
- •Jayesh calls the change “liberating” and says Salesforce value was always more than the interface
- •Headless means metadata and business logic are captured in APIs
- •Customers can bring Salesforce workflows into other contexts via adapted skills
- •Claude can help create a bespoke interface while relying on deterministic backend systems
- 15:25 – 17:56
Ground truth in probabilistic AI: determinism via tool calls and Agent Script
Aakash raises the challenge of probabilistic LLM outputs when teams need deterministic answers (like pipeline numbers). Jayesh explains how Agentforce addresses this by increasing control through agent specification and tool calling, including a language that translates intent into deterministic operations.
- •Aakash highlights the need for a single ground-truth answer despite probabilistic models
- •Jayesh says Agentforce introduced more control and determinism through agent specification
- •A language can describe the agent and translate into tool calls and context lookups
- •Agent Script is designed to reduce hallucinations by relying on deterministic APIs when appropriate
- •LLMs are used to orchestrate systems, while tested APIs provide consistent results
- 17:56 – 19:34
Becoming eval-first: the cultural shift to probabilistic testing and outcome scoring
Jayesh contrasts teams that are naturally “eval native” with teams coming from deterministic software backgrounds. He describes a shift toward outcome-based evaluation, building custom scorers, and treating evals as a core capability for reliable AI systems.
- •Data science-oriented teams are described as “eval native,” expecting probabilistic failure modes
- •Engineering-oriented teams are adapting from deterministic assumptions to eval-first thinking
- •Evals focus on outcomes and include custom scorers using LLMs and other models
- •Scorers and evals must be specific to the agent or system being built, not generic
- •This represents a cultural, toolset, and mindset shift in how software is designed and validated
- 19:34 – 24:51
Why Slack becomes the AI-native work hub (and where agents plug in)
Jayesh argues Slack is the always-available asynchronous workspace where work context accumulates over time. He describes that context as valuable fuel for agents and points to built-in and extensible agent capabilities inside Slack.
- •Slack is where people are available asynchronously and where work conversations are organized
- •Longitudinal Slack context is described as highly valuable for getting work done
- •Slack can enable serendipitous discovery of expertise across channels
- •An agentic toolset can be embedded in Slack, including Slackbot
- •Third-party agents can also be brought into the Slack environment
- 24:51 – 25:54
Prototyping with customers inside Slack: speeding up the feedback loop
Jayesh explains how Slack-based collaboration changes prototyping and iteration with customers. He describes faster cycles where specs, simple UI mocks, and feedback happen directly in threads—reducing reliance on long handoffs and slow prototype workflows.
- •Developers and PMs can write a spec and create simple UI flows in Slack threads
- •PMs can mock ideas quickly with customers and ask if it matches their intent
- •This shortens the cycle versus writing PRDs and waiting for engineering builds
- •Customer feedback arrives earlier, making iteration faster and more direct
- •The overall concept-to-feedback loop is described as sped up exponentially
- 25:54 – 34:15
Future of the product trio: blurred lines, faster zero-to-one, same core unit (for now)
Aakash asks whether PM, designer, and engineer roles are converging. Jayesh says role boundaries are increasingly blurred by tools, enabling more cross-functional execution, while still valuing specialization and maintaining the Scrum-team unit structure.
- •Jayesh says role boundaries are largely mental constructs that tools now blur further
- •Engineers can put products in front of customers faster; PMs can prototype without waiting
- •The new bottleneck becomes finding and iterating with customers quickly enough
- •Salesforce’s customer trust makes it easy to recruit pilot customers for feedback
- •The Scrum team remains the smallest unit, and Jayesh hasn’t seen ratios change significantly yet
- 34:15 – 38:00
Two key AI PM skills and what to hire for: prototyping, evals, agency, and craftsmanship
Aakash proposes two defining AI PM skills—AI prototyping and AI evals—and Jayesh strongly agrees. Jayesh then shares what he looks for when hiring: personal agency, agent-native workflows, skepticism, and craftsmanship to avoid low-quality output.
- •Aakash frames AI PM skills as rapid prototyping and building evals with “golden” datasets and quality bars
- •Jayesh agrees these two skills are “absolutely key”
- •Jayesh prioritizes hiring for agency as a scarce trait
- •He looks for an agent-native mindset: do it once, then teach an agent to repeat it
- •He values skepticism and craftsmanship—avoiding “AI slop”—and translating quality bars into evals
- 38:00 – 42:18
ADLC vs SDLC: building, testing, evaluating, deploying, and observing agents
Jayesh defines ADLC as the agent development lifecycle and contrasts it with the traditional software development lifecycle. He emphasizes the need for new tooling and loops for production evaluation and improvement, including the idea of agents that can detect drift and help correct it.
- •ADLC is defined as an equivalent lifecycle for agentic systems: build, test, evaluate, deploy, observe
- •Jayesh describes a goal of simple “self-healing” by detecting drift from changed context or data
- •He distinguishes an inner loop (design/build with customers) and an outer loop (post-deploy lifecycle)
- •ADLC requires new tools like agent testing and observability centers
- •Jayesh mentions an “optimization agent” that observes agents and suggests improvements like adding context or refining prompts
- 42:18 – 46:35
Operating at AI speed: continuous release, managing tech debt, and shifting bottlenecks
Jayesh explains why the agentic era pushes teams toward continuous deployment: the underlying model substrate changes rapidly. He argues speed must be balanced with safety via tests and evals, and notes that agents can help with code reviews, security workflows, and tech-debt tasks.
- •The model substrate evolves faster than traditional software substrates like databases and SQL
- •Without evals and infrastructure, teams fear breaking things and can’t move fast
- •Continuous release should ship customer-impacting increments while maintaining system safety
- •Tech debt slows delivery, and PMs should treat it as their problem too
- •Bottlenecks shift from writing code to writing high-quality code with tests that fail before production, which agents can help accelerate
- 46:35 – 49:47
Roadmapping in an AI-native org: taste, customer feedback, and forward deployed teams
Jayesh describes roadmap formation as setting a broad directional vector guided by judgment, then refining it with rapid customer feedback and market reality from the field. He explains how forward deployed teams create a constant feedback channel that PMs and engineering leaders monitor and incorporate into planning.
- •Jayesh says taste and judgment matter more for setting a broad direction rather than a rigid plan
- •Early customer feedback is easier now because prototypes can be shown directly
- •Forward deployed teams surface real-world gaps and paper cuts from close customer work
- •Slack channels and summaries help roll up field feedback for PMs and engineering leaders
- •The organization works with customers more than before and adjusts roadmaps based on field signals
- 49:47 – 52:30
Forward deployed roles: bridging innovation speed and customer adoption
Aakash probes how forward deployed roles relate to traditional PM/engineering/design. Jayesh explains these roles exist because technology evolves faster than it diffuses to customers, and they help immerse in customer problems, integrate missing pieces, and translate custom solutions into product opportunities.
- •The role exists to close the gap between internal innovation pace and customer diffusion pace
- •Key skills include immersing in customer pain points and helping solve them with available tools
- •Forward deployed teams may plug in additional components or build custom solutions as needed
- •PM involvement helps decide whether custom work generalizes and should influence the roadmap
- •A loop emerges: build custom quickly with customers, then evaluate broader productization
- 52:30 – 57:29
Biggest mistakes in AI transformations: start simple, define ROI, minimize variables, then scale
Jayesh closes with patterns he sees in failed or slowed implementations and what works better. He advocates clear value definitions, ROI metrics tied to evals, minimizing moving parts early, and building a few successful use cases before expanding widely.
- •Jayesh emphasizes starting simple and defining value and ROI metrics early
- •He recommends tying evals and scorers directly to outcomes and ROI
- •A common mistake is over-scoping into a massive company transformation plan too early
- •He suggests minimizing variables in early implementations (models, training choices, context approach)
- •Successful teams build a few critical use cases end-to-end, then expand broadly after early wins
