No PriorsAI for Atoms: How Periodic Labs is Revolutionizing Materials Engineering with Co-Founder Liam Fedus
EVERY SPOKEN WORD
30 min read · 5,845 words- 0:00 – 0:05
Cold Open
- EGElad Gil
[upbeat music]
- 0:05 – 0:39
Liam Fedus Introduction
- EGElad Gil
Today on No Priors, we're talking with Liam Fedus. Liam is one of the co-creators of ChatGPT, which I think almost everybody uses at this point. He was the VP of post-training at OpenAI, and before that was at Google Brain, where he worked on a variety of really early AI innovations. Liam will be telling us a bit about Periodic Labs, his company, which is focused on building an AI foundation lab for atoms. In other words, how do we impact the physical world, material sciences, chemistry, et cetera, using AI? Very exciting topic and excited to be talking with him today.
- SPSpeaker
Great. Great.
- EGElad Gil
Liam, thank you so much for joining us today on No Priors.
- SPSpeaker
Yeah, thank you so much for having me. It's great to see you.
- 0:39 – 5:14
Liam’s Background at Google Brain, OpenAI
- EGElad Gil
Yeah. So, uh, maybe what we can do, I, I, I think you're doing incredibly interesting things in terms of alternative types of models, specifically for material sciences, for the physical world. Effectively, what you're building is, um, an AI foundation lab for atoms, which I think is fascinating.
- SPSpeaker
That's right.
- EGElad Gil
But maybe what we can start with is a little bit more of your background. You know, I think you were, uh, VP at OpenAI. You worked on one of the first trillion-parameter models ever, et cetera. Could you tell us a little bit more about just, like, what got you here and...
- SPSpeaker
Yeah. Um, so even further back, I was a physics major, um, in undergrad. Um, spent some time doing dark matter research. Um, research-- We had a apparatus that was directionally sensitive to dark matter's direction.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, so it was very interesting.
- EGElad Gil
Why, why are those... Sorry to interrupt, but I'd love to come back to this, but why are there so many physicists in AI right now? So you look at Dario Amodei, who runs, um, Anthropic.
- SPSpeaker
Of course, yeah.
- EGElad Gil
Uh, you look at Adam Brown at Google, you look at a variety of people, and they all kinda have these physics backgrounds.
- SPSpeaker
Yeah, my old manager, Zsa Zsa-
- EGElad Gil
Mm-hmm
- SPSpeaker
...also physics-
- EGElad Gil
Yeah
- SPSpeaker
...and now at Anthropic.
- EGElad Gil
Yeah. Why, why do you think that is?
- SPSpeaker
I think it's a great way to think about the world. It's, like, very principled, um, very, like, hard-nosed scientists, um, very careful, and I don't know. I think it's just, it's such a incredible field. You have such high leverage in computer science in AI.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And so I think a lot of physicists were seeing that.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, particularly in, like, high-energy physics. Um, after the discovery of the Higgs, um, I think a lot of high-energy physicists were sort of looking for what's next.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, ultimately it becomes bottlenecked on the new, um, apparatus for, you know, pushing the next energy frontier, and I think a lot of physicists were looking at their skill set and looking at the progress elsewhere and, and saying like, "Hey, I think I could be a huge contributor elsewhere."
- EGElad Gil
Mm-hmm. This has been fascinating to see, like, string theorists-
- SPSpeaker
Mm-hmm
- EGElad Gil
...and people working on-
- SPSpeaker
Yes
- EGElad Gil
...black holes and all sorts of effects, like, kind of moving into AI.
- SPSpeaker
Absolutely. Yeah.
- EGElad Gil
It's, it's almost-- It almost feels like we're recreating the Manhattan Project or something, except now what we're seeking is, you know, different forms of intelligence, so.
- SPSpeaker
Yeah, that's right. That's right.
- 5:14 – 6:34
From ChatGPT to Materials and Atoms
- SPSpeaker
Yes.
- EGElad Gil
How did that lead you to materials and atoms and, you know, the physical world again? I know that was sort of your starting point in terms of-
- SPSpeaker
Right
- EGElad Gil
...academics, but what brought you back given how much is being transformed right now through language?
- SPSpeaker
I think just the inevitability of connecting these systems to the physical world. The opinion that I and others held as part of Periodic was you're not going to see the same kind of acceleration in science and technology unless you start connecting these things to the physical world. Science ultimately isn't sitting in a room thinking really hard. Um, you have to conduct experiments, you have to learn from them, you have to interface with reality. And the creation of ChatGPT in late 2022, um, was a, you know-Important technology, but it was still far too weak. Like, we couldn't have done Periodic on technology of that era.
- EGElad Gil
Mm-hmm.
- SPSpeaker
I think over the next few years past that, we saw ever im-improving models, um, we saw reasoning. I think, like, test time inference became really important. Uh, that led to more reliable error correction, more reliable tool use, and we see, like, the rise of coding agents and other agents.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And I think those were foundational technologies necessary to then connect these systems to the physical world. Like, I... It was just not impo-not possible with, like, the AI technology of 2022.
- 6:34 – 9:52
Training Data in the Physical World
- EGElad Gil
I guess the other thing that's missing from the physical world is data, or at least data that's-
- SPSpeaker
Absolutely
- EGElad Gil
... easily accessible.
- SPSpeaker
Yes.
- EGElad Gil
So you look at something like, um, the big foundation models on the language side, and they're basically trained on the internet-
- SPSpeaker
That's right
- EGElad Gil
... as a major corpus. It's augmented in all sorts of ways with other data sources.
- SPSpeaker
Right.
- EGElad Gil
How do you think about that for what you're doing, where you're trying to model atoms in the physical world and how all that stuff kind of works?
- SPSpeaker
Yeah. So experiment, I mean, so we have simulation, physics simulations, and we have experiment. And, you know, I think exactly as you're pointing out, ML systems are good on the data you've trained them on, on the tasks you've trained them to do. Um, I think sometimes there's, like, this mythology of AGI, ASI, RSI, and I think we, we see increasingly powerful systems, but they do become limited if they don't have access to the, the raw data to actually make informed decisions.
- EGElad Gil
How mu-how much data do you need? And so I know that, um, there's some data scale related-
- SPSpeaker
Yes
- EGElad Gil
... uh, research and other things in terms of, um, how you kind of hill climb towards, like, a really good model.
- SPSpeaker
Yep.
- EGElad Gil
Uh, how many experiments do you need to run, or how many data points do you need, or how do you think about the diversity of data points you need to generate? I'm a little bit curious, like, what does that actually look like tangibly?
- SPSpeaker
Um, so there is some generalization from the existing models, so we don't need to reproduce a system that can, um, understand and write English-
- EGElad Gil
Mm-hmm
- SPSpeaker
... or write code. So we're, we're kind of, like, leveraging that.
- EGElad Gil
And are you using open source for that or closed source models or some mixture?
- SPSpeaker
We, we use a combination.
- EGElad Gil
Uh-huh.
- SPSpeaker
Yeah. So for example, like, Periodic spends zero effort on improving coding models.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, we're, you know, incredibly impressed by Codex, Cloud Code, and so that's been a huge accelerator for the company. Um, but focused our machine learning efforts where, um, you know, the existing frontiers is not sufficiently good for us. I think going back to the data question, we're leveraging, call it, order tens of trillions of tokens that went into open source models.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And that's given us, like, very, like, foundational understanding. But once we start moving into specific, um, discovery areas, chemical spaces, um, we can see, um, a very high level of sample efficiency. So the system isn't starting as, like, a randomly initialized neural net. It has a strong prior on the world. And-
- EGElad Gil
So where does that prior come from? What data-
- SPSpeaker
Oh, just like-
- EGElad Gil
... do you think that informs that? Just general-
- SPSpeaker
Just, just like, you know, papers-
- 9:52 – 11:31
Generalization Across Domains
- EGElad Gil
I see. And then, um, how do you think about diversity of data? So I look at something like, um, AlphaFold or some of the protein folding-
- SPSpeaker
Yep
- EGElad Gil
... uh, related, um, models, which are amazing, right? If you think about it, I used to work as a biologist, and we would... You know, a crystal structure would take years if it happened at all 'cause you weren't necessarily certain if you could crystallize the specific protein under certain reagent conditions in a way that would be performant for actual x-ray, uh, you know, crystal crystallography and everything, or NMR or whatever approach you took for structure. And then sort of AlphaFold comes out, and you can just arbitrarily model anything-
- SPSpeaker
Right
- EGElad Gil
... on the protein world, which was, you know, amazing as a breakthrough. Um, but it was a very specific data set that already existed that had lots and lots and lots of structures-
- SPSpeaker
Over decades
- EGElad Gil
... over decades-
- SPSpeaker
Yes
- EGElad Gil
... of work. How hard... Do you have to bootstrap that for every single materials domain, or do you choose specific ones that you think can then generalize?
- SPSpeaker
We have seen internally the greatest advances where we have an abundance of data in some space, and that, that has led to the highest rate of acceleration internally. Um, but I think you can think of, um, different levels of generalization, and for systems that are strongly governed by quantum mechanical effects, there is some generalization there.
- EGElad Gil
I see.
- SPSpeaker
Um, but, like, if you produce a system that has modeled, um, quantum mechanical objects really accurately-
- EGElad Gil
Mm-hmm
- SPSpeaker
... it's not really helping much on, like, you know, fluid dynamics or, you know-
- EGElad Gil
Uh-huh
- SPSpeaker
... like, another kind of, like, level of abstraction. And so the generalization we're seeing is quite good, um, but there's almost, like, a first principles you can-
- EGElad Gil
Oh, that's so interesting. So you could do, like, here are the basic steps of chemical synthesis. Here's quantum mechanics. Here's different aspects of how atoms interact in general or-
- SPSpeaker
Yes
- EGElad Gil
... van der Waals forces or things like that.
- SPSpeaker
Absolutely, yes.
- EGElad Gil
Oh, so interesting.
- SPSpeaker
Yeah.
- 11:31 – 12:48
Models as an Orchestration Layer
- EGElad Gil
That's cool.
- SPSpeaker
Right.
- EGElad Gil
And then from a architecture perspective, is there anything unique that you're doing or interesting, or can you talk a little bit about how you're actually constructing some of these models on top?
- SPSpeaker
Yeah. So, uh, language models are incredibly powerful. It's a very natural interface, uh, and so we continue to use these.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, but we think about them almost as, like, an orchestration layer, so that's sort of a, a co-pilot assistant, but also, like, a system that can direct, um, experiments. And-It's almost, it's orchestrating other specialized models as well. So we do construct neural nets that, um, are specially designed for atomic systems where there's like some symmetry awareness. Um, and those have much lower latency, and they've been like fine-tuned for that. And so basically, you kind of think of this like orchestrating layer that can ingest literature, it can go through our experimental data, it can go through different, uh, modalities, but they can also use specialized neural nets-
- EGElad Gil
Mm-hmm
- SPSpeaker
... as tools, as reward functions. So it's, it's like an overall system.
- EGElad Gil
Okay. Yeah, that makes a lot of sense. Yeah, I've seen a lot of people architect those sorts of approaches even for things like customer support or other-
- SPSpeaker
Right
- EGElad Gil
... areas. Like, it seems like it's the common architecture that's emerging as you're doing these different use cases of these models.
- SPSpeaker
That's right.
- EGElad Gil
Yeah.
- SPSpeaker
Yeah. But transformers have been very powerful.
- EGElad Gil
Yeah. Yeah.
- 12:48 – 16:10
Commercialization and Business Model
- EGElad Gil
That's really cool. So if I look at the language world, one of the things that was pretty unique about it, and it's the reason I think these companies like OpenAI, Anthropic, and others are growing so fast, is it just plugged into a very big domain of human existence, which is all language. And all language means enterprise software and enterprise interactions, and it means consumer behavior. It's basically how we interact with the world.
- SPSpeaker
Yes.
- EGElad Gil
Um, it seems like there's a little bit more of a leap for other areas. So, for example, in robotics, there's really interesting things, different types of robots that exist in the world, but the footprint of that is quite limited relative to language.
- SPSpeaker
Mm.
- EGElad Gil
And the same seems to be true for material sciences. So how do you think about where you're gonna commercialize this first or who you're gonna work with or-
- SPSpeaker
Right
- EGElad Gil
... are there specific domains of products that you're working on first?
- SPSpeaker
So we've begun working very closely with scientists. Um, we've treated Periodic as our customer zero and seeing how can we transform how this field of science is done.
- EGElad Gil
Mm-hmm.
- SPSpeaker
But there's huge opportunities across all of these industries, all these enterprises that are interfacing with the physical world. People who are bottlenecked by materials engineering, process engineering.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And again, those are kind of this like the same natural interfaces where engineers are asking questions about their data. They're trying to find aberrations. They're trying to debug machinery. They're trying to get to a better formulation.
- EGElad Gil
Mm-hmm.
- SPSpeaker
It's actually a, a quite universal thing as well. And so we've kind of created our little testing ground internally, and now we're sufficiently excited about the tech we've been building and to see this acceleration for advanced manufacturing more broadly.
- EGElad Gil
And is your model gonna be, um, uh, developing materials for other third parties? Is it developing your own materials that you then sell on the market? Like, uh, because it almost reminds me a little bit of a biotech model.
- SPSpeaker
Yeah.
- EGElad Gil
Where in biotech you can either partner with a big pharma and then effectively help them create a drug and take a royalty on it, or you can build your own drugs. How do you think about that in the context of what you're doing?
- SPSpeaker
We're thinking about us, ourselves as an intelligence layer-
- EGElad Gil
Mm-hmm
- SPSpeaker
... for, for these companies. So you can think about system of record, control plane for different, um, experiments and getting to solutions. Um, but like you're saying, there is, um, a very interesting aspect of some breakthroughs here could have, you know, really high value, and it might be more akin to a discovery model like we've seen in biotech and elsewhere. But starting thinking about our-- just as a software business.
- EGElad Gil
Mm-hmm. Have you ever read The Diamond Age?
- SPSpeaker
And that's, that's very fast.
- EGElad Gil
Yeah.
- SPSpeaker
Yeah.
- EGElad Gil
Have you read The Diamond Age?
- SPSpeaker
No, I haven't actually.
- EGElad Gil
It's the, uh, Neal Stephenson book. It's basically this book about... It was written in the '90s.
- SPSpeaker
Okay.
- EGElad Gil
And there's two key concepts in it. One key concept is, um, there's effectively an AI tutor that's unleashed on the world, and it kind of, um, teaches huge numbers of young girls all sorts of skills. And it's a, this is a very interesting thing about AI education. And then in parallel-
- SPSpeaker
Why young girls in particular?
- 16:10 – 17:45
How Periodic’s Success May Shape the Future
- EGElad Gil
Um, what is your vision or conception of what our world looks like in 10 years, assuming Periodic is successful?
- SPSpeaker
Well, I mean, I think as you're pointing out, you're going from systems that aren't just writing essays, not just writing software, but to literally generating matter.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And I think it's a, has pretty profound implications to semiconductors, aerospace, energy, and I think it's, it's incredibly important for can we increase like the pace of just like the physical development of the world? I mean, we see how quickly the digital realm is changing. Um, software engineering now looks wildly different than even six months ago.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, but I think we see like, you know, similar opportunities in the physical world. Of course, like atoms are hard, and so you will have, um, some limits of physics.
- EGElad Gil
Mm-hmm.
- SPSpeaker
But just because atoms are hard doesn't mean there's not an order of magnitude or two to speed up, um, just making sense of huge amounts of data and getting to solutions more quickly. Um, yeah, so I think what we're trying to do is give humanity this agency for atomic rearrangement, um, synthesis, and we think it's gonna just be a huge accelerator.
- EGElad Gil
Mm.
- SPSpeaker
So I mean, if our physical world could keep up at some fraction to our digital world-
- EGElad Gil
Mm-hmm
- SPSpeaker
... I think life will just feel dramatically different.
- EGElad Gil
Yeah. That's kind of the revolution that-
- SPSpeaker
Exactly
- EGElad Gil
... could really come. Yeah. It kind of reminds me of almost the materials equivalent of the agricultural revolution.
- SPSpeaker
Yeah.
- EGElad Gil
We suddenly had a massive spike in productivity of output.
- SPSpeaker
Exactly.
- EGElad Gil
And it seems like there's been all sorts of bottlenecks that have constrained us until now that you folks are trying to address.
- SPSpeaker
That's right.
- EGElad Gil
Yeah.
- 17:45 – 19:41
Multidisciplinary Scaling
- EGElad Gil
What, um, what aspect of the work that you're doing are you most excited about?
- SPSpeaker
The iteration with our, between these groups of people. I mean, it's like this is just irreducibly a multidisciplinary problem.
- EGElad Gil
Mm-hmm.
- SPSpeaker
We have physicists and chemists working really closely with some of the top AI researchers in the world, working closely with some of the best engineers in the world, and this multidisciplinary, like, really close collaboration is just absolutely incredible because-Seeing firsthand how a field can fundamentally change. People who have been doing research for, in some cases, decades in a field and now seeing like, oh, under these systems, under intelligent systems, it could look this very different, uh, different way. And I mean, I use like an analog to machine learning a lot, going back to the early Google Brain days where the frontier was pushed forward by the, by a few GPUs and a few people. Now you look at this era where it's really, like, industrialized and there's dozens, hundreds of researchers working together-
- EGElad Gil
Mm-hmm
- SPSpeaker
... with hundreds of thousands, millions of GPUs dictated and driven by scaling laws.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Everything is about scaling. It's given that predictability. It's allowed us to put huge amounts of capital into this field. And I think the physical sciences, physical engineering will have a very similar property where we establish these scaling properties and, um, bring that mindset. And so Periodic in this field is really thinking about how do we bring much larger scale sets of experiments to bear on this? And intelligent systems have enabled us, automation has enabled us, and you really need both, um, an improvement to automation where you bec- can soon become, uh, create bottlenecks in intelligence. And I mean, the scientists very much feel this, where they're not used to working at that level of throughput, and they just can't simply make sense of so much data.
- EGElad Gil
So interesting. Yeah.
- 19:41 – 21:12
Capital and Compute
- EGElad Gil
So I guess in terms of, um, scale here, one of the real benef- one of the things that's really benefited the fi- the fi- the frontier labs on the LLM side is just scale of capital and therefore scale of GPU and scale of data.
- SPSpeaker
Of course.
- EGElad Gil
Um, is this similarly a capital-intensive area in your mind?
- SPSpeaker
Yeah, we will require more capital.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, GPUs are so extraordinarily expensive. Um, and what's interesting is just the compute cost relative to physical infrastructure is actually surprising where, you know, so much money is spent on the compute, uh, that the physical infrastructure sometimes is actually lower but, you know, has very large lead times and there's intrinsic difficulty of having these well-calibrated, well-functioning physical systems. Um, but from a capital perspective, it's primarily a, a compute cost.
- EGElad Gil
Yeah, it's really interesting. If you look up, um, the cost of a Stanford postdoc, for example-
- SPSpeaker
Mm
- EGElad Gil
... relative to a machine learning engineer, it's, like, such a big difference.
- SPSpeaker
Absolutely.
- EGElad Gil
And you, you've really... You know, my takeaway is that, um, many people working in science, particularly in academic center, set- setting, are very undercompensated relative to sort of their societal value.
- SPSpeaker
Absolutely.
- EGElad Gil
And so I always like it when companies kind of help bring people into the, into the fold in terms of both human impact, but also, you know, that, um, that ability to do things at real scale and, you know, really do things a different way. So it must be very exciting for the people on your team.
- SPSpeaker
Yeah.
- EGElad Gil
Um-
- SPSpeaker
I mean, it's like, I mean, some of the scientists who've joined us are, you know, among the best in the world-
- EGElad Gil
Mm-hmm
- SPSpeaker
... and it's been absolutely incredible working with them.
- EGElad Gil
Yeah, I, I mean, it sounds like you've built such an amazing inter- interdisciplinary
- 21:12 – 21:44
Hiring at Periodic
- EGElad Gil
team. Are there specific roles that you're actively looking for right now or key things that you really wanna hire up?
- SPSpeaker
Absolutely. So on our site, we have decomposed the world into bits and atoms. Um, you know, it's a, a loose taxonomy, but on bit side, we're really thinking about, um, mid-training, pre-training roles from the AI side, always more infrastructure roles. And on atom side, like control engineering, system engineering, uh, but also now thinking too about, you know, spanning that with like product engineering. So, um-
- EGElad Gil
Yeah.
- SPSpeaker
Yeah.
- EGElad Gil
A lot of work.
- SPSpeaker
A lot of active roles.
- EGElad Gil
Yeah, that's exciting. Yeah, that's really cool.
- 21:44 – 23:30
Thoughts on AGI and ASI
- EGElad Gil
So I, I think one of the things that everybody's really thinking deeply about or is excited about right now is AGI, ASI, sort of these-
- SPSpeaker
Yes
- EGElad Gil
... advanced systems that are as good as humans or better than humans at different things-
- SPSpeaker
Right
- EGElad Gil
... or are very generalizable in terms of their abilities to do a broad swath of things. How do you think about that but in the context of what's happening over the overall foundation model curve?
- SPSpeaker
Mm-hmm.
- EGElad Gil
'Cause obviously you were very integral in terms of the development of so many systems. And then how do you think about that applied specifically to some of the areas you're working in?
- SPSpeaker
I think one fallacy is thinking about intelligence as a scalar. We've consistently seen these systems have, uh, a very odd spikiness.
- EGElad Gil
Hmm.
- SPSpeaker
And it's actually possible to architect a system that is world-class on some math domain, but then you could do some perturbations to the questions and actually degrade it sub- substantially so it's like a bad high school student.
- EGElad Gil
Hmm.
- SPSpeaker
And so there's this like odd spikiness to these systems.
- EGElad Gil
So basically you can make a system that's like a genius at one thing and not very good at a bunch of other stuff?
- SPSpeaker
And I guess the point I was making is those fields can actually be quite adjacent.
- EGElad Gil
Hmm.
- SPSpeaker
Um, so like sometimes the generalization can be non-intuitive.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, but one way I think about, you know, re- recursive self-improvement is really kind of akin to neural architecture search from, you know, roughly 10 years ago, and I think there's a very clear path for software engineering. So these systems have become so incredibly impressive on this, on this domain as a result of huge amounts of data, really cheap verifiable environments. Like, you know, you can check unit tests go from failing to passing with just a few CPUs. It's basically instantaneous. There's no domain expertise gap between an AI researcher or software engineer.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, and obviously this will become and is becoming a larger contributor to the next generation of
- 23:30 – 25:39
Timeline for Machine-Directed Self-Improvement
- SPSpeaker
the system.
- EGElad Gil
When do you think it just flips into we just, uh, everything is machine self-improvement versus human-directed or h- or needs a lot of human intervention? So do you think-
- SPSpeaker
Right
- EGElad Gil
... that's two years away? Do you think that's five years away? Do you think that's 10 years away?
- SPSpeaker
Well, I guess, like, building on what I was saying is I think there's a domain caveat to that.
- EGElad Gil
Sure.
- SPSpeaker
So rolling forward that software engineering self-improvement-
- EGElad Gil
Mm-hmm
- SPSpeaker
... I think you're gonna have a system that can write, um, complete repositories, identify bugs, refactor code-
- EGElad Gil
Mm-hmm
- SPSpeaker
... but it doesn't suddenly understand biology.
- EGElad Gil
Sure.
- SPSpeaker
Right? It's just like there's a domain gap there in knowledge.
- EGElad Gil
Yeah.
- SPSpeaker
But even beyond that, there's, um, sets of strategies done in, um, software engineering that differ from scientific or engineering strategies. So it's, you're not-Operating under... It's not like decision-making under uncertainty to the same degree. It's, like, very verifiable, and that's driven so much of our work.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, so in that domain, I think it's happening now-ish.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And, you know, so I, I think we'll see the same thing too for AI research.
- EGElad Gil
Uh-huh.
- SPSpeaker
That's a slower outer loop because now the experiment isn't just checking some unit test passing, but it's checking what was the scaling property, um, did this model converge, um, what's the generalization of the system. That requires GPUs. That requires, you know, many hours of experiments. But I think that will also will, um-
- EGElad Gil
And those are all evals that people use today as they're looking at existing models-
- SPSpeaker
Absolutely
- EGElad Gil
... and so they do have that utility function, that feedback loop-
- SPSpeaker
That's right
- EGElad Gil
... that can be just driven by self-learning.
- SPSpeaker
That's right. That's right. But again, like, the connection of these things to the physical world is going to be so critical because both of those systems are being trained in a closed loop against that domain. So it's a closed loop for doing software engineering, a closed loop for doing AI research, and that's the premise of Periodic. Like, we need to have these closed loops of actually doing science, of actually doing engineering.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And these two, I mean, these two domains are how I think the rest of the world will go with some delay, and this is, again, like, the foundational technology
- 25:39 – 27:59
Automation and Data Generation
- SPSpeaker
that we're building out.
- EGElad Gil
Super interesting. Do you think you need, um, sufficiently good robotic systems in order to have that closed loop for what you're doing?
- SPSpeaker
Um-
- EGElad Gil
In other words, do you need something like PI or Skelos or something else to work in order for, uh, Periodic to hit that escape velocity in terms of a closed loop system?
- SPSpeaker
No, but it's a huge accelerator.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, the goal for Periodic is to generate high quantity, high quality data, diverse data, and automation is assistance to that. So right now we employ people as well, and we have autonomous parts that are just, you know, very reliable. If you had a dexterous humanoid who could wander into an unstructured lab and make sense and follow instructions reliably, that would be a huge accelerator. Right now, the automation of physical systems is... requires a very careful design, and it's slow, but I think with improvements in robotics, it's just going to accelerate this. But already the reliability of the sort of, like, hybrid systems is sufficient to produce huge amounts of, um, reliable data, but it's just gonna accelerate us further.
- EGElad Gil
Yeah. The, the... One of the reasons I ask is, um, I used to run this company, uh, Color, um, and we built our own liquid handling-
- SPSpeaker
Mm-hmm
- EGElad Gil
... robotic systems, right? We'd buy liquid handling robots-
- SPSpeaker
Yes
- EGElad Gil
... but then we'd have to adjust them dramatically. We had, like, cameras that would use ML to monitor the system-
- SPSpeaker
Right. Yes
- EGElad Gil
... and sort of make adjustments. We had to 3D print parts to decrease vibrations on the platform-
- SPSpeaker
Yes
- EGElad Gil
... because we were dealing with such small, uh, volumes of liquid.
- SPSpeaker
Right.
- EGElad Gil
And so there was enormous amounts of customization versus just having... And the firmware for it was awful, and writing against that was painful-
- SPSpeaker
Yep
- EGElad Gil
... versus just having, like, a robotic system that would work like a modern system-
- SPSpeaker
Right
- EGElad Gil
... in all the ways that you'd conceive that.
- SPSpeaker
Right.
- EGElad Gil
And so the reason that I was asking is if you really wanna do high throughput experiments, you need these underlying systems to be able to do all the liquid handling and to do-
- SPSpeaker
Of course
- EGElad Gil
... you know, all the titration of stuff and all the rest of it. So.
- SPSpeaker
Yeah. That's right. I mean, I think it's... Look, right now we're using almost, like, more, like, off-the-shelf robotics. It's, like, very simple, very commoditized.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, not doing, like, a huge amount of innovation on, on that front. But again, like, as these, um, more general robotic systems come to-
- EGElad Gil
Mm-hmm
- 27:59 – 29:25
Why Liam is Excited About the Future of Robotics
- SPSpeaker
as well.
- EGElad Gil
Yeah. You've seen such a wide range of different things happen in the AI world since-
- SPSpeaker
Indeed. Yes. Right
- EGElad Gil
... beginning your work at Google, I guess at this point about a decade ago. Um, and so you were there during the birth of the transformer model. You were there, um, for the birth of ChatGPT. Um, what are you most excited about outside of Periodic over the next few years in terms of what's happening with AI?
- SPSpeaker
I mean, of course robotics.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Again, I'm like, I'm just so excited about the interface of AI systems with the physical world.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And we're approaching one angle of that, which is science engineering.
- EGElad Gil
Mm-hmm.
- SPSpeaker
And we need that data in order to make those advances. But simply just agency and control of the physical world, um, via robotics is going to be transformative.
- EGElad Gil
Mm-hmm.
- SPSpeaker
Um, so I'm, I'm very excited about these interface layers. I think that's gonna be such a massive opportunity 'cause, I mean, you know, how many software engineers are there in the world versus people who-
- EGElad Gil
Mm-hmm
- SPSpeaker
... deal with, like, the physical world?
- EGElad Gil
Mm-hmm.
- SPSpeaker
And there's just labor shortages everywhere. So yeah, I think it's gonna be a, a very interesting decade.
- EGElad Gil
Oh, amazing. Well, thank you so much for joining us today.
- SPSpeaker
Yeah. Well, thanks so much.
- EGElad Gil
Yeah.
- SPSpeaker
It was really, really good chatting today. Yeah.
- SPSpeaker
[upbeat music] Find us on Twitter at NoPriorsPod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.
Episode duration: 29:25
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode Oru2Jxr1xHU
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome