No Priors Ep. 135 | With Humans& Founder Eric Zelikman
Sarah Guo (host), Eric Zelikman (guest), Elad Gil (host)
In this episode of No Priors, featuring Sarah Guo and Eric Zelikman, No Priors Ep. 135 | With Humans& Founder Eric Zelikman explores from IQ to EQ: Building Human-Centric AI That Truly Collaborates Eric Zelikman, former Stanford researcher and xAI lead, discusses his work on advancing AI reasoning through methods like STAR and Q* and his shift toward building more human-centric systems.
From IQ to EQ: Building Human-Centric AI That Truly Collaborates
Eric Zelikman, former Stanford researcher and xAI lead, discusses his work on advancing AI reasoning through methods like STAR and Q* and his shift toward building more human-centric systems.
He explains how reinforcement learning and scalable reasoning have dramatically improved model 'IQ', yet current models still lack deep understanding of human goals, context, and long-term outcomes.
Zelikman argues that the industry’s task-centric, single-turn benchmarks and automation mindset limit AI’s ability to genuinely empower people rather than replace them.
His new company, Humans&, aims to build models that understand users over time, remember their context, and act as long-horizon collaborators that expand human potential instead of just automating existing GDP slices.
Key Takeaways
Scaling reasoning via RL can continuously extend model capabilities.
STAR showed that by reinforcing successful chains of thought, models can progressively solve harder problems (e. ...
Get the full analysis with uListen AI
Model performance is highly sensitive to context and problem framing.
Today’s models do best when given rich, precise context and tasks with clearly verifiable answers; users and product designers should structure interactions to include as much relevant information and clear evaluation criteria as possible.
Get the full analysis with uListen AI
Verification and training distribution still bound what models can reliably do.
In areas like code, success depends heavily on how close a task is to training distributions and how verifiable outputs are; out-of-domain, poorly verifiable problems still reliably expose model weaknesses.
Get the full analysis with uListen AI
Single-turn, task-centric training creates shallow, brittle AI behavior.
Optimizing for one-off responses leads to models that avoid asking clarifying questions, rarely model long-term consequences, and can exhibit issues like sycophancy and harmful advice without grasping downstream impact on users’ lives.
Get the full analysis with uListen AI
Long-term, human-in-the-loop collaboration can grow the economic pie.
Instead of just automating existing work segments, models that deeply understand people's goals, constraints, and aspirations can help them pursue entirely new, out-of-distribution projects, driving net new value and innovation.
Get the full analysis with uListen AI
Memory and persistent user modeling are underexploited but crucial.
Because current paradigms treat interactions as independent tasks, there has been little pressure to build strong long-term memory; Zelikman argues future systems must continuously learn about users and use that knowledge across sessions.
Get the full analysis with uListen AI
Building EQ in AI is both a capabilities and values choice.
Labs can design scaling paths that either sideline humans (fully autonomous long-horizon agents) or keep them central (cooperative systems that model human goals and agency); choosing the latter is an explicit design and research decision.
Get the full analysis with uListen AI
Notable Quotes
“The role that [models] play in people's lives is a lot less deep, a lot less positive than it could be.”
— Eric Zelikman
“If you have a model that goes off and does its own thing for eight hours, people will probably feel less real agency over the things that they're building.”
— Eric Zelikman
“Fundamentally these models don't really understand people. They don't understand people's goals.”
— Eric Zelikman
“It's really remarkable that the field is kind of so stuck in this task‑centric regime.”
— Eric Zelikman
“We’re much more likely to solve a lot of these fundamental human problems by building models that are really good at collaborating with large groups of people.”
— Eric Zelikman
Questions Answered in This Episode
How do you practically collect and evaluate data for long-horizon, human-in-the-loop interactions without waiting months or years for outcomes?
Eric Zelikman, former Stanford researcher and xAI lead, discusses his work on advancing AI reasoning through methods like STAR and Q* and his shift toward building more human-centric systems.
Get the full analysis with uListen AI
What technical architecture or training changes are needed to give models robust, privacy-preserving memory and persistent user models?
He explains how reinforcement learning and scalable reasoning have dramatically improved model 'IQ', yet current models still lack deep understanding of human goals, context, and long-term outcomes.
Get the full analysis with uListen AI
How would you design new benchmarks that measure “life impact” or long-term user empowerment instead of single-task accuracy?
Zelikman argues that the industry’s task-centric, single-turn benchmarks and automation mindset limit AI’s ability to genuinely empower people rather than replace them.
Get the full analysis with uListen AI
In what concrete scenarios do you expect human–AI collaboration to create net new value, rather than just replacing existing jobs or workflows?
His new company, Humans&, aims to build models that understand users over time, remember their context, and act as long-horizon collaborators that expand human potential instead of just automating existing GDP slices.
Get the full analysis with uListen AI
What are the main safety, alignment, and consent challenges when building models that deeply model individuals’ goals, preferences, and weaknesses?
Get the full analysis with uListen AI
Transcript Preview
(music plays) Hi, listeners. Welcome back to No Priors. Today, we're here with Erich Zelkman, previously of Stanford and xAI. We're gonna talk about the contributions he's made to research, reasoning, and scaling up RL, as well as his new company, Humans End. Erich, thank you so much for doing this.
Thank you.
You have had an amazing impact as a researcher, including starting from just your time at Stanford. I wanna hear about that, but first, background of how you got interested in machine learning at all.
I- I guess going back, like, really far, I- I've- I've been motivated by this question of, like, you have, you know, all of these people out there who have, like, all of these things that they're really talented in, all of these things that people are really passionate about, that you have, like, so much, like... You know, there- there's just so much talent out there and I've always been, like, a little bit disappointed that, like, you know, like, so much of that talent doesn't get used just because everyone has, like, circumstances and, like, has, like, these, you know, situations where, you know, they can't actually pursue those things. And so for me, AI has-
All of humanity's not living up to their full potential.
I mean-
And so then you got into AI. (laughs)
(laughs) I mean, it's a... The- the thing I've always been excited about is, like, how do you actually build this technology that frees people up to kind of do the things that they are passionate about?
Mm-hmm.
Like, how do you basically, you know, s- a- yeah, allow people to actually focus on those things? You know, originally, I thought of automation as kind of, like, the most natural way of doing that. Like, you- you automate away the parts that, like, people kind of don't want to do, and that-
Mm-hmm.
... you know, frees up people to do the things that they do want to do. But I guess I realized, like, increasingly that that's, like, it's actually, like, pretty complex. You actually have to understand, if you want to empower people to do what they want to do, you have to really understand what people actually want to do. Um, and building systems that understand kind of people's goals and outcomes is actually really hard.
Hmm.
Um, yeah.
Did you have, like, um, this human-centric perspective when you were choosing research problems to work on originally?
I- I guess, like, at the very beginning. I was just in- like, when I was choosing research problems, I was just interested in, like, how do you actually make these things half decent?
Okay.
Like-
So it's more increased capability ev- first.
Yeah.
Yeah.
I think, I think for me, like, you know, when I looked at, like, AI, like, or, you know, language models back in, like, 2021 or whatever, you know, I was like, "Th- these things aren't very smart. They can't do that much." And- and there- there was some, like, early work around there, like, um, that showed that, like for example, you could use, like, chain of thought to, like, you know-
Install uListen to search the full transcript and get AI-powered insights
Get Full TranscriptGet more from every podcast
AI summaries, searchable transcripts, and fact-checking. Free forever.
Add to Chrome