EVERY SPOKEN WORD
55 min read · 10,692 words- 0:00 – 0:35
Introduction to Accelerating Science
- PHPatrick Hsu
I want to make science faster. Our moonshot is really to make virtual cells at Arc and simulate human biology with foundation models. Why are we so worried about modeling entire bodies over time when we can't do it for an individual cell?
- JCJorge Conde
If we can figure out how to model the fundamental unit of biology, the cell, then from that we should be able to build.
- PHPatrick Hsu
My goal is to really try to figure out ways that we can improve the human experience in our lifetime. There are a few things that if we get them right in our lifetime, will fundamentally change the world.
- JCJorge Conde
[instrumental music]
- 0:35 – 0:45
Welcome to the Podcast
- JCJorge Conde
Patrick, welcome to the podcast. Thanks for joining.
- PHPatrick Hsu
Thanks for having me on.
- JCJorge Conde
I've been trying to have you on for years, but finally I could get your time.
- PHPatrick Hsu
[laughs] Here I am. I'm excited to do it. It's gonna be great.
- 0:45 – 1:57
The Moonshot: Virtual Cells and Human Biology
- JCJorge Conde
For some of the audience who aren't familiar with you and your work at, at, at Arc and beyond, how do you describe... what's your moonshot? What, what, what is what, what you're trying to do?
- PHPatrick Hsu
I want to make science faster, right? You know, we can frame this in high-level philosophical goals like accelerating scientific progress. Maybe that's not so tangible for people. I think the most important thing is science happens in the real world. If it's not AI research, which moves as quickly as you can iterate on GPUs, right? You have to actually move things around, atoms, clear liquids from tube to tube to actually make life-changing medicines, and these are things that take place in real time. You have to actually grow cells, tissues in animals, and I think the promise of what we're doing today with machine learning and biology is that we could actually accelerate and massively, uh, massively parallelize this. And so our moonshot is really to make virtual cells at Arc and simulate human biology with foundation models. And, you know, we'd like to figure out something that feels useful for experimentalists, people who are skeptical about technology, you know, they just wanna see the data and see the results, that it's actually the default tool that they go to use when they want to do something with cell biology.
- JCJorge Conde
Okay.
- 1:57 – 2:58
Challenges in Scientific Progress
- JCJorge Conde
Well, hold on. Let's back up.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
Why is science so slow in the first place? Like, whose fault is that?
- PHPatrick Hsu
Whose fault is that? Now, that is a, that is a long one. We should get into it. We should get into it. It's really multifactorial.
- JCJorge Conde
Okay.
- PHPatrick Hsu
Right? It's this weird Gordian knot that ultimately comes down to incentives, right? Comes down to, you know... People talk a lot about science funding and how science funding can be better, but it's, it's also about how, you know, the training system works, right? How we incentivize long-term career growth, how we, you know, try to separate, you know, basic science work from, you know, commercially viable work, and generally the space of problems that people are able to work on today. Um, I think things are increasingly multidisciplinary. It's very hard for individual research groups or individual companies to be good at more than two things, right? You might be able to do, you know, computational biology and genomics, right? Or, you know, like, chemical biology and molecular glues. But, you know, how do you do five things at once is, is increasingly
- 2:58 – 5:11
Interdisciplinary Collaboration at Arc Institute
- PHPatrick Hsu
hard. And we really built Arc as an organizational experiment to try to see what happens when you bring together neuroscience and immunology and machine learning and chemical biology and genomics all under one physical roof, right? If you increase the collision frequency across these five distinct domains, there would hopefully be a huge space of problems that you could work on that you wouldn't be able to. Now, obviously in any university or any kind of geographical region, you have all of these individual fields represented at large, right, across these different campuses, but, you know, people are distributed, and you want everyone together. Yeah.
- JCJorge Conde
Okay, but, uh, uh, if I may.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
So a univer- I would have thought-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... a university was an attempt to bring in multiple disciplines under one roof. You're saying it's not. It's too diffuse to serve that purpose.
- PHPatrick Hsu
It's across an entire campus, right?
- JCJorge Conde
Okay. So the physic-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... like, literally the physical distance, uh, creates inefficiency.
- PHPatrick Hsu
That's part of it, and I think the other part is folks have their own incentive structures, right? They need to publish their own papers. They need to do their own thing and, you know, make their own discovery. And you're not really incentivized to work together-
- JCJorge Conde
Mm-hmm
- PHPatrick Hsu
... I think, in many ways in the current academic system. And a lot of what we've done is to try to have people work on bigger flagship projects that require much more than any individual person or group or idea. Yeah.
- JCJorge Conde
That's cool. So, like, sort of the original hypothesis for the Arc Institute is if you can bring, um, multiple disciplines together to in- in- increase the collision frequency, as you said, and if, if one could remove some of the, the cross-incentives that may exist in sort of traditional structures, the combination of those two things will make science faster.
- PHPatrick Hsu
Yeah. These are, these are absolutely part of it, right? We have two flagship projects, one trying to find Alzheimer's disease drug targets, the other to make these virtual cells. And the... I think it's not just the people and the infrastructure, but also the models will hopefully literally make science faster-
- JCJorge Conde
Mm-hmm
- PHPatrick Hsu
... that you could, you know, do experiments at the speed of forward passes of a neural network if these models could become accurate and useful.
- JCJorge Conde
Mm-hmm. Yeah. So that, that will be one thing that solves the length of discovery is you compress the time discovery takes naturally by just, uh, throwing technology at the problem, at the risk of oversimplifying.
- PHPatrick Hsu
Well, we're, we're tech and optimists here, no?
- JCJorge Conde
We are.
- PHPatrick Hsu
Yeah.
- 5:11 – 10:18
The Role of AI in Biology
- JCJorge Conde
Why has AI progressed so much faster in, um, sort of image generation and language models th- than, than biology and, and if we could wave a wand, like, where, where are we excited to, to speed certain things up?
- PHPatrick Hsu
To be honest, it's a lot easier.
- JCJorge Conde
Yeah.
- PHPatrick Hsu
Right? Maybe that's a hot take, right?
- JCJorge Conde
[laughs]
- PHPatrick Hsu
But, uh-
- JCJorge Conde
You mean technology's easier than biology.
- PHPatrick Hsu
Uh, natural language and video modeling is easier than modeling biology.
- JCJorge Conde
Correct.
- PHPatrick Hsu
Right? And to some degree, like, if you understand and learn machine learning, right, and how to train these models, you have already learned how to speak. You already know how to look at pictures, and so your ability to evaluate the generations or predictions of these models are very native, right? We, we don't speak the language of biology, right? You know, you know, at, at very best with an incredibly thick accent, right? So when you-
- JCJorge Conde
[laughs]
- PHPatrick Hsu
... you're training these DNA foundation models, I don't speak DNA natively, so I only have a sense of the types of tokens that I'm feeding into the model and what's actually coming out, right? Similarly, these virtual cell models, you know, I think a lot of theGoal is to figure out ways that you can actually interpret the weird fuzzy outputs that the model is giving you. And I think that's what slows down the iteration cycle, is you have to do these lab in the loop things where you have to run actual experiments to actually test with experimental ground truth. And, you know, I think increasing the speed and dimensionality of that is gonna be really important. Yeah.
- JCJorge Conde
How much of this is the fact that like, you know, you talk about, you know, we speak biology poorly or with a very thick accent, how much of this is like, if you're training on an image, we can see the image, and so we can see how, you know, how good the output is.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
What about all the things in biology that we can't see or don't even know exist yet? Like, how, how can we create a virtual cell, and maybe we should come back to the, what a virtual cell model is, by the way, for the lay audience. But like, how can we create a virtual cell model when we're not even sure if we understand all of the components that are in a cell and how they function?
- PHPatrick Hsu
People talked a lot about this in NLP as well. There's this long academic tradition in natural language processing, right? And then it was just weird and non-intuitive and intensely controversial that you could just feed all this unstructured data into a transformer, and it would just work. Now, we're not saying this will just work in all the other domains, including in biology, but I think there is this, you know, controversy around what does it mean to be an accurate biological simulator? What does it mean to be a virtual cell? It's true, we can't measure everything, right? We can't measure, I think things like metabolites and really high throughput with spatial resolution. And there are gonna be different phases of capability where initially they model individual cells, then they model pairs of cells, then they model cells in a tissue, and then in a broader, physiologically intact animal environment. And those are so length scales and, uh, kind of layers of complexity that will aggregate and, you know, improve upon over time. And I think the other kind of non-intuitive thing in many ways are the scaling laws that you get in data and in modeling. I'll give you an example, right? There's a lot of discussion in molecular biology about how, you know, RNAs, you know, don't reflect protein and protein function, right? Um, and so while we don't have, you know, proteomic measurement technologies that are nearly as scalable as transcriptomic measurement technologies today, like at the single cell resolution certainly, but we're getting there, and you can layer on certain nodes of protein information that you can add on top of the RNA information. But in many ways, the RNA representation is a mirror, right? Uh, it might be a lower resolution mirror for what's happening at the protein layer, but eventually what is happening in protein signaling will get reflected in a transcriptional state, right? And so for an individual cell, this may not be very accurate, but when you imagine the massive data scale that we're generating in genomics and functional genomics, right? You start to gather tremendous amounts of RNA data that will read in kind of like what's happening at the protein level at some, at some sort of mirror echo, right? And then that can, you know, be the case for metabolic, uh, metabolic information as well and so on. Yeah.
- JCJorge Conde
So it's a low pixel image, but if we can get sort of zoomed out far enough, we'll get a sense of what's going on.
- PHPatrick Hsu
You have to bet on what you can scale today, right? We're able to, you know, scale single cell and transcriptional information today. We're able to add on, you know, protein-level information over time. We'll need spatial information, spatial tokens, and we'll need temporal dynamics as well. And we'll, you know, I kind of, uh, bucket things into three tiers. There's invention, engineering, and scaling. And there are certain things today biotechnologically that are scale ready.
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
And then there are things that we still need to invent, right? And that's part of why we felt like we needed a research institute to be able to tackle these types of problems, that we weren't just going to be an engineering shop that's just trying to scale single cell perturbation screens, right? That, you know, would be interesting, but in three years would feel very dated, I think, right? And so there's a lot of novel technology investment that we're making that we think will bear fruit over time. Yeah.
- 10:18 – 22:13
Understanding Virtual Cells
- JCJorge Conde
Can we flesh out the virtual cell concept, why that's the ambition we, we, we've landed on and what it's gonna take to get there, or what are the bottlenecks?
- PHPatrick Hsu
I, I would say the most kind of famous success of ML in biology is AlphaFold, right? And this solved the protein folding problem of, you know, when you take a sequence of any amino acid, what does the protein look like, right? And you know, it's pretty good. It's not perfect. It certainly doesn't simulate the biophysics and the molecular dynamics, but it gives you a sense of what the end state is with ninety percent plus accuracy, right?
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
And that's the AlphaFold moment that people talk about, right? Where any time you want to, you know, work with a protein, you're-- if you don't have an experimentally solved structure, you're just gonna fold it with this, uh, uh, with this algorithm. And we kind of want to get to that point with virtual cells as well, and the way that at Arc we're operationalizing this is to do, uh, perturbation prediction, right?
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
Where the idea is you have some manifold of cell types and cell states, right? Um, that can be a heart cell, a blood cell, a lung cell, and so on. And you know that you can kind of move cells across this manifold, right? Sometimes they become inflamed, sometimes they become apoptotic, sometimes they become cell cycle arrested, they become stressed, they're metabolically starved, they're hungry in some way. And so if you have this re- uh, sort of this representation of universal sort of cell space, right? Can you figure out what are the perturbations that you need to move cells-
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
Around this manifold? And this is fundamentally what we do in making drugs, right? Whether we have small molecules, which started out as natural products from, you know, boiling leaves or antibodies when we injected proteins into cows and rabbits and sheep and took their blood to, to get those antibodies. We are, we are basically trying to get to more and more specific probes, right? And we had experimental ways to kind of cook these up. Now we have computational ways to zero shot these binders. But ultimately, what you're trying to do with these binders is to inhibit something-And then by doing so, kind of click and drag it from a kind of toxic gain-of-function, disease-causing state to a more quiescent homeostatic healthy one, right? And the thing that is very clear in complex diseases, right, where you don't have a single cause of that disease, is there are some complex set of changes. There's a combination of perturbations, if you will-
- JCJorge Conde
Mm-hmm
- PHPatrick Hsu
... that you would want to make to be able to move things around. Now, you know, the people talk about this classically as things like polypharmacology, right? But, you know, I think we're moving from a, "Oh, this thing happens to have, you know, a whole bunch of different targets, um, kind of by accident," uh, to we have the ability to manipulate these things commentorially in a purposeful way, right? That to go from cell state A to cell state B, there are these three changes I need to make first, then these two changes, and then these six changes, right, over time, right? And we kind of want models to be able to suggest this. And the reason why we scoped virtual cell this way is because we felt it was just experimentally very practical. You want something that's gonna be a co-pilot for a wet lab biologist to decide, "What am I gonna do in the lab," right? We're not trying to do something that's like a theory paper that's really interesting to read, where, you know, the numbers go up on a-- the ML benchmark. But, you know, you practically can decide what are the 12 things that you're gonna do in the lab in 12 different conditions, right, and actually just test them, right? And then that's how we kind of enter the, the kind of the lab and the loop aspect of model predictions to experimental measurements to, you know, you know, kind of improved or RL'd or whatever model kind of predictions again. And the goal is to be able to do in silico target ID, where you can basically figure out new drug targets, figure out then the compositions, the drug compositions you would need to actually make those changes.
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
I think if we could do that, we could make a new AI, like vertically integrated AI-enabled pharma company, right-
- JCJorge Conde
Mm-hmm
- PHPatrick Hsu
... which, um, you know, I think is obviously a very exciting idea today. But I think in many ways, the kind of pitch and the framing of these companies precedes the fundamental research capability breakthroughs.
- JCJorge Conde
Right.
- PHPatrick Hsu
And that's what we're really invested in at Arc, is kind of just making that happen along with many other amazing colleagues in the field to just make this possible for, you know, the community.
- JCJorge Conde
So if the goal is, I'm gonna oversimplify it for you, like if we wanted to get to the AlphaFold, the moment where, you know, it kind of gives you a, a useful structure, folded structure ninety percent of the time, to use your, your data point. We wanted to take that, uh, comparison in the, in the virtual cell model, and we said, "Okay, ninety percent of the time, if I ask the model I wanna shift the cell from cell, from s-cell state A to cell state B, and it's gonna give me a list of perturbations."
- PHPatrick Hsu
Mm-hmm.
- JCJorge Conde
And let's say that at ninety percent of the time, those perturbations, in fact, result in the shifting experimentally, in the shifting from cell state A to cell state B.
- PHPatrick Hsu
Mm-hmm.
- JCJorge Conde
How far away are we from that AlphaFold moment for virtual cells?
- PHPatrick Hsu
I find it helpful to frame these in terms of like GPT one, two, three, four, five-
- JCJorge Conde
Okay
- PHPatrick Hsu
... capabilities, right? And I think most people would agree we're somewhere between GPT one and two, right? A lot of the excitement was that we could achieve GPT one in the first place, that you could see a path with scaling laws of some kind to kind of make successive generations where capabilities would improve. But, you know, these are s- you know, with like our Evo kind of, uh, DNA foundation models that we developed at Arc with, uh, Brian Kee, right, one, one of the things that we've seen is that, you know, these are really kind of-- these genome generations are like quote, unquote "blurry pictures of life," right? We don't think if you synthesize these novel genomes, they would be alive.
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
But, you know, we, we don't think that's actually also impossibly far away. We'll just have to kind of follow these capabilities. We're generating, uh, we're taking a very integrated approach to attack this problem-
- JCJorge Conde
Mm-hmm
- PHPatrick Hsu
... right, where you need to curate public data, you need to generate massive amounts of internal private data, build the benchmarks, and train the new, uh, train new models and building, uh, sort of architectures and kind of doing these things full stack. And we'll just kind of attack this, um, hill climb over time.
- JCJorge Conde
What's the GPT, I'll say GPT three moment gonna look like? And by that I mean sort of a public release that alters the public's conception of just what's possible here from a capabilities perspective, and also inspires a whole new generation of talent to like rush into, into, into, into biology.
- PHPatrick Hsu
Well, the good thing with biology is we have a lot of ground truth, right? There are entire textbooks, right, that describe cell signaling and cell biology and how these things work. And so, you know, even without a virtual cell model at all, right, if you went into ChatGPT or Claude and you basically, you know, you asked just some question about, you know, like receptor tyrosine kinase signaling, it would have an opinion on how that works, right? And so I think you would want the model to be able to predict perturbations that are kind of famous canonical examples of biological discovery. So I'll give you an example. If you loaded into the model an iPSC, a s- a, a kind of an induced pluripotent stem cell state or human embryonic stem cell state, and a fibroblast cell state, right, could it predict that the four Yamanaka factors would reprogram the fibroblast into a stem-like state, right? And essentially rediscover from the model something that won the Nobel Prize in two thousand and nine, right? That would, that would be sort of one really kind of classic example. And then you could go do the inverse. If you have a stem cell, can it discover neurogenin 2, ASCL1, MyoD? Can it find differentiation factors that will turn that into a neuron or into a muscle cell or, or so on? And you know, these are kind of classic examples in developmental biology, but you could also use this to try to discover or kind of recapitulate the mechanism of action of FDA-approved drugs, right? And so you could say, for example-You know, if you kind of inhibit HER2 in, you know, breast cancer you know cell states, right, it would become-- you know, you would get this type of response or it could predict the, you know, certain clones that, you know, will be able to kind of be more metastatic or, you know, they'll be more resistance and they'll lead to minimal residual disease. There, I think lots of kind of biological evals that you can kind of add onto these models over time that are really tangible textbook examples as opposed to, I think, what the kind of early generation of models do today, which is, you know, very quantitative things like mean absolute error over like, you know, the differential expressed genes and stuff like that, you know.
- 22:13 – 27:39
Biotech and Pharma Industry Insights
- ETErik Torenberg
It, it, it seems that biotech and pharma has, has been a shrinking, um, in, interest as certain as the, the rate of growth. What, what's it gonna take for these, um, innovations in, in, in the science to reflect themselves in, in business models and in, in g-growth for the industry?
- PHPatrick Hsu
A lot of these biotech startups would try to initially sell software to pharma companies, and then they would kind of realize, "Oh wow, we're like competing for SaaS budgets," um, which aren't very large. And then, you know, now they're realizing, "Oh, we have to compete for R&D budgets," right? And I think, and you know, there is this narrative from the current generation of these companies that, "Oh, our biological agents will compete for R&D budgets and replace headcount," or something like that, right? Just like we're seeing in, you know, agents across different verticals, right? Whether or not that will, I think, pan out, I think depends on just whether or not these things meaningfully allow us to, you know, build drugs more effectively in the pharma context, right? And I think that's just sort of the most important thing, um, in, in, in this industry. And so I think we believe in virtual cells, not just because we think it will be a fountain of fundamental mechanistic insights for discovery, but also because if, in the case of success, it could be industrially really useful, right? But, you know, we'll, we'll, we'll, we'll have to see over time, right? If we have ninety percent of drugs failing clinical trials, right, that kind of means two things, and you're not sure what percent of which, right? One is we're targeting the wrong target in the first place. The second is the composition, the drug matter that we're using doesn't do the job, right? And it's not clear for each individual failure which one it is or if it's both or what proportion of each. And, you know, we'll have to kind of sort that out over time. Like, you can imagine, even in the case of success, when we had ninety percent accurate virtual cells, you'll probably end up with suggestions like, "Okay, now you need to target, you know, this GPCR only in heart."But not in literally any other tissue, right? We don't have the drug matter that can do that today. And so that's also why, again, you probably need research to figure out novel chemical biology matter that allows you to drug pleiotropic, you know, targets, um, in a tissue or cell type-specific way, right? And so, you know, I think, you know, part of why biology is slow is because there's just this Russian nesting doll of complexity, um, in terms of understanding, in terms of perturbation, in terms of safety. And you know, the, the, the crazy thing is the progress in just the short time that I've been doing this is insane, right? Like, I did my, you know, PhD at the Broad Institute in the heyday of developing single cell genomics, human genetics, CRISPR gene editing, um, you know, and, uh, you know, so many other things. And I think the kind of early 2010s papers on single cell sequencing would have like 20 cells or 40 cells, right? And, and at Arc, in the next, you know, kind of n, like, I don't know, relatively short amount of time, we're gonna generate a billion perturbed single cells, right? That's like, I mean, how's, how's that for a Moore's Law?
- ETErik Torenberg
Yeah, that's remarkable.
- PHPatrick Hsu
Yeah.
- ETErik Torenberg
Yeah. Jorge, I, I wanna hear your, your answers to a couple of these questions too as, as the lead of, of our bio practice, both on the GPT-3 moment, what that, what that could look like, a-and also, um, like I'm curious if you think it's GLP-1s or sort of building off that, or if it's gonna be some-something different. And also, what's it gonna take for the, for the science to kind of reflect itself in, in the business for the industry to grow?
- JCJorge Conde
Yeah. So I'll, I'll take the, the, the second one first, if I could. So I think, you know, in terms of, of where the industry is right now, I think one of the big challenges we have is, as Patrick describes very nicely, like, you know, discovery's hard and it takes time. And, you know, the fail modes are exactly as you describe. Oftentimes when drugs fail, which they do 90% of the time in clinical trials, it's because we're going after the wrong thing or we made the wrong thing to go after the right thing, right? Like, those are the two fail modes, and that happens, uh, all too often. And so I think a lot of the stuff that Patrick is describing is gonna basically improve our hit rate or our batting average on figuring out what to go after and then making the right thing to go after said thing. Um, the challenge we have, I think, in the industry is that the bottlenecks still are the bottlenecks. And the biggest bottleneck we have, which is, you know, a necessary one, is we have to prove that whatever we make, that we have the right thing to go after the right thing, so to speak, and that we, when we have it, that it's going to be as, you know, de-risked as possible before you put it into humans.
- PHPatrick Hsu
And we have to be good at making them in the first place too.
- JCJorge Conde
And we gotta make them too. Yeah, exactly. And so that bottleneck is a necessarily important one. We-- that bottleneck should exist. I'm not suggesting we better remove it. But are there ways to reduce the cost and time associated with getting through the bottleneck of human clinical trials? Um, and you know, it's, it's interesting because, you know, we talk about, you know, uh, all of the various stakeholders when you're making a, a drug. Uh, there are the companies, there's of course the science that supported, um, the, the company that's trying to commercialize a, a product, and there are the regulatory agencies. You know, and everyone is trying to ensure again, that, uh, what's, you know, first and foremost is the ability to, um, uh, discover and commercialize drugs that are safe and effective for humans. That middle part, part of actually getting through that bottleneck is hard to speed up
- 27:39 – 28:13
Challenges in Clinical Trials
- JCJorge Conde
in a very obvious way. Like, you can increase the rate, the way you enroll clinical trials, you can use better technology to change the way we design these clinical trials, so maybe they can be faster or shorter, et cetera. But some of them just have a natural timeline you have to go through. Like if you want to demonstrate that a cancer drug, uh, promotes survival, guess what? You're gonna have-- It's gonna take some time to demonstrate a survival benefit. Or if, you know, [chuckles] you wanna do a longevity drug, that by definition is a lifetime, you know, of, of a, of a trial in terms of length. So there's a lot of these bottlenecks that are really hard to get through. So what helps the industry?
- 28:13 – 29:02
Capital Intensity and Technological Advancements
- JCJorge Conde
I think there are a couple of things that, uh, help the industry. One is, um, capital intensity, uh, will hopefully at some point go down over time as technology gets better. Capital intensity is something that our industry faces. In some ways, it looks a little bit like AI now, right, in terms of the cost of training these models, but the capital intensity is very, very high. That has not come down. So, um, we gotta get the success rates up to impact capital intensity to get it down. The second thing is can-- where can we compress time? So good models can help us compress early discovery time. We still haven't seen, uh, and I think it's coming, but it hasn't happened yet, we haven't seen artificial intelligence or other technologies massively compress the amount of time it takes us to do the clinical development, the clinical trials, the enrollment of patients, all of those things. We're seeing some interesting things coming. We haven't seen sort of the, the payoff there yet.
- 29:02 – 30:12
Improving Drug Development
- JCJorge Conde
Um, and the third thing is if we can make better drugs, going after better things, y- the effect size should be higher, so therefore the answer should be obvious sooner. If we can get those three things right, reduce capital intensity, compress timelines, and effectively increase effect size in, in some very tough, um, sort of, uh, intractable diseases, that is, I-- what I think fixes the industry. And from where we sit at the early stage, um, uh, at the early stage in terms of being early stage investors, the reason why that helps us is if the capital intensity goes down and the value creation goes up, it becomes easier to invest in these companies in the early days 'cause you get rewarded for coming in early. The problem we have right now is that most companies [chuckles] aren't-- you're not seeing, uh, rewards happening when there's value inflection. So you come in early, you bear the brunt of the capital intensity, and even if a company's successful, that success isn't reflected in the valuation. So we're not seeing the step-ups that you see in other parts of the industry, and that's just really, really hard from a, from an investing standpoint. So I think we need to see those various factors addressed for this space to really get, you know, fixed, to use your word.
- PHPatrick Hsu
Yeah, that was great. I, I have a, I have a lot to add onto
- 30:12 – 33:19
Market Impact and Industry Trends
- PHPatrick Hsu
this.
- JCJorge Conde
Please add away.
- PHPatrick Hsu
You know, just, you know, one-- a, a few simple observations, right? The, the first is the amount of market cap-Added to Lilly and Novo, um, based on the, you know, development of GLP-1s, um, is like over a trillion dollars is, is more, you know-- I mean, Novo's stock has decreased a lot. So, um, you know, a trillion dollars, let's say, is more than the market cap of all biotech companies combined over the last forty years have been started, right? And I think that, you know, one, one of the kind of interesting kind of corollaries of this is that, you know, when we have a ten percent kind of clinical trial success rate for kind of preclinical drug matter, right, you tend to circle the wagons a bit and try to manage your risk, right? And so the way that you do this is you try to go after really well-established disease mechanisms, where if I developed new drugs that go after well-understood biology, it should work the way that I hope it will in the trial, um, which is, you know, s- really, really expensive and costs a lot more in many ways than the, the preclinical research, right? Um, the problem with this is you go after very well-validated disease mechanisms but with really small patient populations, right? So then the expected value of this actually is relatively low. One of the kind of things that we've seen with, uh, GLP-1s is the, just the kind of value that you can create when you go after really large patient populations. And I think that has culturally really net increased the ambition of the industry, both from the investor and from the drug developer side. And I think, you know, that's something that we should keep our foot on the gas for. Yeah.
- JCJorge Conde
Yeah. And look, I think the trend on that is posi-- I would argue the trend on that is positive.
- PHPatrick Hsu
Yeah, yeah.
- JCJorge Conde
You're absolutely right. Like, the, the demonstration of, uh, the value that has been created, uh, with the, the, the use, the increasing use of GLP-1s and the value transfer that's gone to companies like Lilly and Novo, I would argue, is like very merited, right? Because they've cracked an endemic social problem, um, in terms of managing diabetes, diabetes and eventually helping manage obesity. And so I think that's remarkable, and there's a lot of value that goes to that because they tackled, they cracked a very, very, um, challenging problem-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... for, for society beyond just science. So that's great. And I agree with you. Like, the, the, the, the prize, the juice needs to be worth, worth the squeeze, right? You're right. A lot of biotech has been around, like, go after the low-hanging fruit because it's low risk, and we got to eat today, right? So you go get it, you know, and you just have to, you push off the big, the big ambitious indication, the large population or the, the really, uh, tough to crack disease. But, you know, I do think we're seeing more and more of that. And by the way, like, we can get into some of these genetic medicines, but some of these genetic medicines are going after some of the hardest problems, the things that you quite literally couldn't address but for editing, you know, DNA. And, you know, I think that's incredibly, you know, remarkable and laudable and, and, and quite frankly inspiring.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
But the fundamental, um, elements of the industry have to work, so the capital formation is there to support those kinds of things. And right now it's hard, right, because of the, the issues we talked about before.
- 33:19 – 38:15
Future Technological Breakthroughs
- ETErik Torenberg
Fifteen years from now, we're back in this room. We've barely escaped being part of the permanent underclass.
- PHPatrick Hsu
[laughing]
- JCJorge Conde
[laughing]
- ETErik Torenberg
And we're, we're reflecting on the, on sort of the GPT-3 moment or, or maybe w- the legacy of GLP-1s, um, sort of beyond wh-where, where they are now. Um, what, what do you think it could be? Or, or I'm curious to get your take on what do you think is going to be the technological breakthrough that we're going to point back to and say, "Oh, this is really what, what said it all." That, or do you think it's going to be sort of, you know, multifactor combination?
- JCJorge Conde
Yeah, look, I think, um, it's, it's going to go back to, uh, sort of where we started, uh, this combination, uh, conversation, excuse me. Uh, GLP-1s, uh, as a drug are, you know, what, four decades in the making or something like that. You know, these are, these are not overnight successes. Um, uh, but I do think what we are going to see more of and, and our hope is that when you combine the fact that we're getting better at, uh, understanding what to target, getting better at designing medicines to hit those targets, by the way, in a whole array of new creative ways. So we have small molecules, the natural products that we got from boiling leaves, as you said earlier, like, those have gotten... You know, we're getting really good at designing smarter and better, smaller molecule- small molecules that do new things, that function in ways that they didn't before. Um, we've gotten quite good at designing, uh, uh, biologics or proteins with a lot of help from things like AlphaFold that help us understand how proteins fold. We're going to get a lot better at designing some of the more complex, uh, modalities like the gene therapies of the world or the gene editors of the world. And when you can do that and combine that with our ability to hopefully use things like virtual cell models to really understand what to go after, like, we're going to have drugs, we-- I would hope and I would expect that the industry will continue to bring forward drugs that have very large effect size for very difficult diseases that hopefully affect a lot of patients. If that's true, then we'll start to see some of these really, really difficult, um, diseases that affect all of society get tackled, hopefully, you know, one by one by one by one. And so we have obesity, we have metabolic disorder, we're dealing with cardiometabolic disease. We're starting to see interesting, promising things happening in like, neurodegenerative diseases. Um, you know, if we can, you know, tackle cancer or at least, you know, several cancers that now have begun to be treated more like a chronic condition than a death sentence that they were in the past. The more we see of that, like, I think that value to society will accrete over time, and I think this should be an industry that is extraordinarily, uh, valued by society and candidly by the markets. We have to deliver.
- PHPatrick Hsu
If we play this out, right, and let's say these AI models work, right, and you can make a trillion binders in silico that will, you know, be exquisite drug matter, right? We still need to make these things physically and test them in-Animals in hopefully co- predictive models, and then actually in people, right? And I think, you know, that will increasingly be the, the, the, the, the bottleneck in many ways, right? And you know, my, my friend, uh, Dan Wang recently, um, released a book called Breakneck which talks about, um, you know, kind of like the US and China and the difference between the two countries and their philosophy, the way they approach markets, and-
- JCJorge Conde
We're, we're the s- a country of lawyers, they're a country of engineers-
- PHPatrick Hsu
Exactly
- JCJorge Conde
... sort of the political climate.
- PHPatrick Hsu
That's right. Right. China is an engineering state, right? Its, uh, kind of, uh, politburo is, you know, folks who have engineering degrees. You know, you need to build bridges and roads and buildings, and these are the ways that we solve our problems, whereas I think from, you know, the first 13 American presidents, 10 of them practiced law. From 1980 to 2020, all Democratic presidential candidates, uh, uh, y- uh, both vice- VP and president, uh, went to law school, right? And so you kind of see the echoes of that in the FDA and the regulatory regime and, you know, all, all the kind of the, the, the bottlenecks that people talk about developing drugs stateside. And increasingly you see folks thinking about how we can run phase ones overseas, right?
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
Build data packages that we can, you know, bring back domestically for phase two efficacy trials. I think that's interesting directionally, but it's not enough, right? And, you know, I think we need to kind of figure out these two bottlenecks, the making and the testing, you know. Even if we can solve the designing part.
- JCJorge Conde
Oh, I agree.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
Yeah, yeah, that, that's the bottleneck.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
You know, we, we joke about it and you have to do, is you have to get a molecule that can go, you know, first in mice and then in mutts and then in monkeys and then in man. Like, there's, you know, that takes, takes a long time, and it's just so hard to compress that. Um, and so when you do, you, you should make the journey worth, you know, uh, make the journey worth it, right?
- PHPatrick Hsu
Yeah.
- JCJorge Conde
So when you fail on the other end of that, like, that's obviously horrible.
- PHPatrick Hsu
Mm-hmm.
- JCJorge Conde
And so finding ways to make sure that when you, when you walk that path, that it'll be a successful journey as often as possible is what this industry desperately needs.
- PHPatrick Hsu
Mm-hmm.
- 38:15 – 45:50
AI in Drug Discovery
- PHPatrick Hsu
Mm-hmm.
- JCJorge Conde
Uh, AlphaFold solved a pr- pr- protein folding problem, but wh- why didn't it solve drug, drug discovery? Or more broadly, what would it take to, to get AI drug discovery? What, what is sort of the, the, the bottleneck on the, on the, on the tech side at least?
- PHPatrick Hsu
On the tech side? Um-
- JCJorge Conde
Or maybe another way to ask the question-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... is that w- 'cause I always ask the founders-
- PHPatrick Hsu
Sure, yeah
- JCJorge Conde
... a version of this question-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... like the AI ones-
- PHPatrick Hsu
Sure
- JCJorge Conde
... um, that are like, "Oh, we're gonna do, uh, AI for, like, for drug discovery."
- PHPatrick Hsu
Mm-hmm.
- JCJorge Conde
So my, my question that I always like to ask founders is give me examples of where you think AI is hyped-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... potentially overly hyped, um, where there's real hope, like the sort of what do we expect-
- PHPatrick Hsu
Mm-hmm
- JCJorge Conde
... what's next, and where we already see real heft.
- PHPatrick Hsu
Yeah.
- JCJorge Conde
So, like, if I asked you, like, in AI, you know, where is there hype? Where is there hope, and where are we seeing heft today?
- PHPatrick Hsu
I would say there's hype in toxicity prediction models.
- JCJorge Conde
Okay.
- PHPatrick Hsu
Heft-
- JCJorge Conde
So that's the idea that-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... we will say, um-
- PHPatrick Hsu
Yeah
- JCJorge Conde
... I'm gonna show you a molecule, and you're gonna tell me, the model's gonna tell me if it's gonna be toxic or not.
- PHPatrick Hsu
That's right.
- JCJorge Conde
Okay.
- 45:50 – 54:02
Investment Focus and Future Prospects
- PHPatrick Hsu
important.
- JCJorge Conde
Shifting gears a little bit. We've been talking about science and, and, and biotech, but you're-- in addition, you're an elite AI in-in-investor more broadly. So, so I wanna talk about, um, h-how you're-- I wanna talk about where your investment focus is right, right now, just as it relates to AI more broadly. Wh-where are you excited? Where are you spending time? Wh-where are you, you know, looking forward to?
- PHPatrick Hsu
Oh, yeah, my, my goal is to really try to figure out ways that we can improve the human experience in our lifetime. I kind of think of-- like, if I think about the future that we're gonna leave to our children, right, there are a few things that if we get them right in our lifetime, will fundamentally change the world, right? And, you know, how we live in it. I think synthetic biology, um, is obviously one, right? You know, think, you know, GLP-1s, right? Things that improve, uh, sleep, right? Things that can, you know, improve longevity, right?
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
These are, these are all things that are kind of, you know, easy to get excited about. I do-- I, I think, um, uh, brain computer interfaces, um, is another area where, um, we're gonna see really important breakthroughs over the decades to come. And then I think the third is in, uh, in, is in robotics, both industrial and consumer robotics, right?
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
Um, that allow us to basically, like, scale physical, like, uh, labor, right, in, in, in interesting ways. And, you know, you can kind of see how each of these three things, even in the sort of medium cases of success, really kind of change the world. And so I'm very interested in helping make these kinds of things possible, right? And so there are sort of, you know, in the kind of techno-optimist sort of vision of the world, right, there's a few different types of scarcity, right? There's as-- you know, it-it's very easy when you do research to come up with important ideas. The hard thing is to tackle them in the right timeframe, right? It's like, you know, writing futuristic sci-fi things is not that hard. Being able to actually execute on it in the next five years or eight years, um, much, much harder, right? And I would say, you know, academic discovery is littered with plenty of ideas that are interesting and important, but, you know, kind of long before their time. And in many ways, the story of technology development is, you know, trying to use new technologies to solve old tricks, right?
- JCJorge Conde
Mm-hmm.
- PHPatrick Hsu
Like, most of our tools are, you know, for productivity, right, um, in many ways, um, whether that's the Industrial Revolution or the computing revolution or the current AI revolution. We're trying to kind of do the same stuff. And, you know, a-and so there, you know, I think there's a relatively small set of very powerful ideas, new technologiesGive us new opportunities to attack them, and there's a set of people in teams that are gonna be positioned to be able to do that. They need to have technical innovation and then an intuition about product and business in a way that, you know, you know, you kind of in the RPG dice roll of the skills that you get in these three domains, people start at different base levels, right?
- ETErik Torenberg
Mm-hmm.
- PHPatrick Hsu
And, you know, you might have an incredibly technical founder who doesn't know how to think commercially, or someone who's just natively a very commercial thinker who, you know, doesn't have very strong product sense, right? Um, even though they could sell the crap out of it, right? And so I think these sort of-- these sort of three broad categories of capabilities you need to kind of bring together in a way that you can allocate capital to in the right times in order to make these ideas possible in a really differentiated way. Like, this thing literally wouldn't happen if we didn't get these people together and fund it at the right time in the right way, right? And that's the-- that's really what, what motivates me, and these are the kinds of things that I've been excited about, you know, backing, you know, longevity companies like Neolamin, right? BCI companies like Nudge, right? Robotics companies like The Bot Company, right? Um, you know, these are some of the examples of kind of, you know, things that I think must happen in the world, um, and therefore should happen and, you know, how do we actually find the right people and the right time to actually kind of go on the Fellowship of the Ring hunt, right?
- ETErik Torenberg
Yeah. [chuckles]
- PHPatrick Hsu
Yeah.
- ETErik Torenberg
If not too difficult, I wanna ask, uh, Jorge a question adopted to these s-- uh, additional spaces, um, robotics, um, sort of BCIs and, and longevity if, if appropriate in terms of-- and the three questions I believe were, what's overhyped? What-- Uh, where do you see a, a, a opportunity or path, and what's got heft already?
- PHPatrick Hsu
I think the cool thing about agents, um, generally is that they do real work, right? Um, compared to, like, SaaS companies that came before, agents replace real productivity, right? And I think, you know, they have a lot of errors today, and I would say the computer use agents will probably trail the coding agents by maybe a year, right? But, but it's coming, and we'll follow the trajectory as these go from doing, you know, minutes of work without error to hours to days, right? And I think, you know, you're gonna get a completely different product shape as we march through that across legal, BPO, you know, medicine, healthcare, whatever, right?
- ETErik Torenberg
Mm-hmm.
- PHPatrick Hsu
And we'll kind of follow that as an industry, and that's, that's gonna be really exciting. And I think that's where we're gonna see real heft is because most of the economy is services spent. It's not software spent. And, you know, the reason why we're all excited about this stuff is that it can attack, you know, the, the, the services economy. And I would say, like, you know, where, where is there hype? There's a tremendous amount.
- ETErik Torenberg
[chuckles]
- PHPatrick Hsu
Right? That's, that's, that's no doubt. The hype is in the model capabilities, right? And, you know, it's-- we, we, we're working with an architecture that, you know, dates back to twenty seventeen, right? And if you look at the history of deep learning, it's like kinda every eight years there's something really different, right? Um, and we-- it feels like in twenty twenty-five we're really overdue for some net new architecture. And I think there are lots of really interesting research ideas that are bubbling up that could, um, do that thing. And in many ways, there's a set of really interesting academic ideas, especially in the golden age of machine learning research from, I don't know, like two thousand and nine to twenty fifteen, right? There's so many interesting ideas, little archive papers that have, like, thirty citations or less. And as the marginal cost of compute goes down year on year, I think you're gonna be able to take all of these ideas and actually scale them up, right? Where you don't see the scaling laws when you're training them at a hundred million or six fifty million parameters like back then. But if you can scale them up to one B, seven B, thirty-five B, seventy B, right, you start to see, you know, whether or not these ideas will pop, right? And I think that's very exciting because, you know, there's just gonna be a lot of opportunity for new super intelligence labs to do things, um, you know, beyond what the kind of, you know, established foundation model companies are doing today, right? As they kind of, you know-- In addition to these research teams, right, you know, these are in many ways becoming applied AI companies, right? They need to build product shape and, you know, all kinds of different enterprises and do RL for businesses and make money, right? And I think, uh, or, or, or build coding agents and make API revenue, and that's important and I think, you know, a timely race to survive today. But I'm just, you know, a, a very bullish on the research of, say, like a Sakana AI, right? Which was founded by one of the authors of, uh, Attention Is All You Need, right? Ian Jones. And they're doing incredibly interesting stuff on model merging and how you can have kind of, um, sort of like evolutionary selection of, you know, kind of, uh, different, um, uh, kind of, um, you know, models in MoE. And I think the, there are sort of opportunities here, um, in the long run to move beyond just, like, RL gems, for example. Um, also to kind of figure out new ways to learn and, and find, like, kind of reward signal is, is gonna be really exciting.
- 54:02 – 55:29
Closing Remarks and Upcoming Initiatives
- ETErik Torenberg
This is a great place to wrap. G-gearing towards, um, towards closing, anything, uh, upcoming for Arc that you'd like us to know of? Anything you wanna tease? Anything-- For people who wanna learn more, w-which should they know about?
- PHPatrick Hsu
So AlphaFold, uh, in many ways came out of a protein folding competition called CASP, right? Uh, Critical Assessment of the Structure of Proteins. And, um, you know, we created our own virtual cell challenge, um, at virtualcellchallenge.org, where we have, you know, hundred thousand dollar prizes sponsored by NVIDIA and 10X Genomics and Ultima and others. And it's an open competition that anyone can enter, where you can train perturbation prediction models, and we can openly and transparently assess these model capabilities, both today and in subsequent years, follow them to get to that ChatGPT moment, right? And so I'm extremely excited about this. Um, you know, um, we, we'd like more people to, you know, train models and apply both bioML experts and engineers in any other domain. And, you know, I'm, you know-- I, I, I just-- I want this thing to exist in the world. You know, hopefully, we're important parts of making that happen, but I'd just be happy that someone does it.
- ETErik Torenberg
Yeah.
- PHPatrick Hsu
Yeah.
- ETErik Torenberg
That's an inspiring note to, to wrap on. Patrick, Jorge, thanks so much for the conversation.
- PHPatrick Hsu
Thanks so much, guys. Appreciate it.
- ETErik Torenberg
Thanks for having me. [outro jingle]
Episode duration: 55:38
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode eAODQUKqDiU
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome