Faster Science, Better Drugs

Patrick Hsu 0:00 I want to make science faster. Our moonshot is really to make virtual cells at Arc and simulate human biology with foundation models. Why are we so worried about modeling entire bodies over time when we can't do it for an individual cell?

Jorge Conde 0:14 If we can figure out how to model the fundamental unit of biology, the cell, then from that we should be able to build.

Patrick Hsu 0:21 My goal is to really try to figure out ways that we can improve the human experience in our lifetime. There are a few things that if we get them right in our lifetime, will fundamentally change the world.

Jorge Conde 0:32 [instrumental music]

Jorge Conde 0:36 Patrick, welcome to the podcast. Thanks for joining.

Patrick Hsu 0:38 Thanks for having me on.

Jorge Conde 0:39 I've been trying to have you on for years, but finally I could get your time.

Patrick Hsu 0:42 [laughs] Here I am. I'm excited to do it. It's gonna be great.

Jorge Conde 0:45 For some of the audience who aren't familiar with you and your work at, at, at Arc and beyond, how do you describe... what's your moonshot? What, what, what is what, what you're trying to do?

Patrick Hsu 0:54 I want to make science faster, right? You know, we can frame this in high-level philosophical goals like accelerating scientific progress. Maybe that's not so tangible for people. I think the most important thing is science happens in the real world. If it's not AI research, which moves as quickly as you can iterate on GPUs, right? You have to actually move things around, atoms, clear liquids from tube to tube to actually make life-changing medicines, and these are things that take place in real time. You have to actually grow cells, tissues in animals, and I think the promise of what we're doing today with machine learning and biology is that we could actually accelerate and massively, uh, massively parallelize this. And so our moonshot is really to make virtual cells at Arc and simulate human biology with foundation models. And, you know, we'd like to figure out something that feels useful for experimentalists, people who are skeptical about technology, you know, they just wanna see the data and see the results, that it's actually the default tool that they go to use when they want to do something with cell biology.

Jorge Conde 1:56 Okay.

Jorge Conde 1:57 Well, hold on. Let's back up.

Patrick Hsu 1:58 Yeah.

Jorge Conde 1:58 Why is science so slow in the first place? Like, whose fault is that?

Patrick Hsu 2:00 Whose fault is that? Now, that is a, that is a long one. We should get into it. We should get into it. It's really multifactorial.

Jorge Conde 2:06 Okay.

Patrick Hsu 2:06 Right? It's this weird Gordian knot that ultimately comes down to incentives, right? Comes down to, you know... People talk a lot about science funding and how science funding can be better, but it's, it's also about how, you know, the training system works, right? How we incentivize long-term career growth, how we, you know, try to separate, you know, basic science work from, you know, commercially viable work, and generally the space of problems that people are able to work on today. Um, I think things are increasingly multidisciplinary. It's very hard for individual research groups or individual companies to be good at more than two things, right? You might be able to do, you know, computational biology and genomics, right? Or, you know, like, chemical biology and molecular glues. But, you know, how do you do five things at once is, is increasingly

Patrick Hsu 2:58 hard. And we really built Arc as an organizational experiment to try to see what happens when you bring together neuroscience and immunology and machine learning and chemical biology and genomics all under one physical roof, right? If you increase the collision frequency across these five distinct domains, there would hopefully be a huge space of problems that you could work on that you wouldn't be able to. Now, obviously in any university or any kind of geographical region, you have all of these individual fields represented at large, right, across these different campuses, but, you know, people are distributed, and you want everyone together. Yeah.

Jorge Conde 3:35 Okay, but, uh, uh, if I may.

Patrick Hsu 3:36 Yeah.

Jorge Conde 3:36 So a univer- I would have thought-

Patrick Hsu 3:38 Yeah

Jorge Conde 3:38 ... a university was an attempt to bring in multiple disciplines under one roof. You're saying it's not. It's too diffuse to serve that purpose.

Patrick Hsu 3:44 It's across an entire campus, right?

Jorge Conde 3:46 Okay. So the physic-

Patrick Hsu 3:47 Yeah

Jorge Conde 3:47 ... like, literally the physical distance, uh, creates inefficiency.

Patrick Hsu 3:50 That's part of it, and I think the other part is folks have their own incentive structures, right? They need to publish their own papers. They need to do their own thing and, you know, make their own discovery. And you're not really incentivized to work together-

Jorge Conde 4:02 Mm-hmm

Patrick Hsu 4:02 ... I think, in many ways in the current academic system. And a lot of what we've done is to try to have people work on bigger flagship projects that require much more than any individual person or group or idea. Yeah.

Jorge Conde 4:14 That's cool. So, like, sort of the original hypothesis for the Arc Institute is if you can bring, um, multiple disciplines together to in- in- increase the collision frequency, as you said, and if, if one could remove some of the, the cross-incentives that may exist in sort of traditional structures, the combination of those two things will make science faster.

Patrick Hsu 4:33 Yeah. These are, these are absolutely part of it, right? We have two flagship projects, one trying to find Alzheimer's disease drug targets, the other to make these virtual cells. And the... I think it's not just the people and the infrastructure, but also the models will hopefully literally make science faster-

Jorge Conde 4:49 Mm-hmm

Patrick Hsu 4:49 ... that you could, you know, do experiments at the speed of forward passes of a neural network if these models could become accurate and useful.

Jorge Conde 4:56 Mm-hmm. Yeah. So that, that will be one thing that solves the length of discovery is you compress the time discovery takes naturally by just, uh, throwing technology at the problem, at the risk of oversimplifying.

Patrick Hsu 5:07 Well, we're, we're tech and optimists here, no?

Jorge Conde 5:10 We are.

Patrick Hsu 5:10 Yeah.

Jorge Conde 5:11 Why has AI progressed so much faster in, um, sort of image generation and language models th- than, than biology and, and if we could wave a wand, like, where, where are we excited to, to speed certain things up?

Patrick Hsu 5:21 To be honest, it's a lot easier.

Jorge Conde 5:23 Yeah.

Patrick Hsu 5:23 Right? Maybe that's a hot take, right?

Jorge Conde 5:25 [laughs]

Patrick Hsu 5:26 But, uh-

Jorge Conde 5:26 You mean technology's easier than biology.

Patrick Hsu 5:28 Uh, natural language and video modeling is easier than modeling biology.

Jorge Conde 5:31 Correct.

Patrick Hsu 5:32 Right? And to some degree, like, if you understand and learn machine learning, right, and how to train these models, you have already learned how to speak. You already know how to look at pictures, and so your ability to evaluate the generations or predictions of these models are very native, right? We, we don't speak the language of biology, right? You know, you know, at, at very best with an incredibly thick accent, right? So when you-

Jorge Conde 5:59 [laughs]

Patrick Hsu 5:59 ... you're training these DNA foundation models, I don't speak DNA natively, so I only have a sense of the types of tokens that I'm feeding into the model and what's actually coming out, right? Similarly, these virtual cell models, you know, I think a lot of theGoal is to figure out ways that you can actually interpret the weird fuzzy outputs that the model is giving you. And I think that's what slows down the iteration cycle, is you have to do these lab in the loop things where you have to run actual experiments to actually test with experimental ground truth. And, you know, I think increasing the speed and dimensionality of that is gonna be really important. Yeah.

Jorge Conde 6:36 How much of this is the fact that like, you know, you talk about, you know, we speak biology poorly or with a very thick accent, how much of this is like, if you're training on an image, we can see the image, and so we can see how, you know, how good the output is.

Patrick Hsu 6:52 Yeah.

Jorge Conde 6:53 What about all the things in biology that we can't see or don't even know exist yet? Like, how, how can we create a virtual cell, and maybe we should come back to the, what a virtual cell model is, by the way, for the lay audience. But like, how can we create a virtual cell model when we're not even sure if we understand all of the components that are in a cell and how they function?

Patrick Hsu 7:13 People talked a lot about this in NLP as well. There's this long academic tradition in natural language processing, right? And then it was just weird and non-intuitive and intensely controversial that you could just feed all this unstructured data into a transformer, and it would just work. Now, we're not saying this will just work in all the other domains, including in biology, but I think there is this, you know, controversy around what does it mean to be an accurate biological simulator? What does it mean to be a virtual cell? It's true, we can't measure everything, right? We can't measure, I think things like metabolites and really high throughput with spatial resolution. And there are gonna be different phases of capability where initially they model individual cells, then they model pairs of cells, then they model cells in a tissue, and then in a broader, physiologically intact animal environment. And those are so length scales and, uh, kind of layers of complexity that will aggregate and, you know, improve upon over time. And I think the other kind of non-intuitive thing in many ways are the scaling laws that you get in data and in modeling. I'll give you an example, right? There's a lot of discussion in molecular biology about how, you know, RNAs, you know, don't reflect protein and protein function, right? Um, and so while we don't have, you know, proteomic measurement technologies that are nearly as scalable as transcriptomic measurement technologies today, like at the single cell resolution certainly, but we're getting there, and you can layer on certain nodes of protein information that you can add on top of the RNA information. But in many ways, the RNA representation is a mirror, right? Uh, it might be a lower resolution mirror for what's happening at the protein layer, but eventually what is happening in protein signaling will get reflected in a transcriptional state, right? And so for an individual cell, this may not be very accurate, but when you imagine the massive data scale that we're generating in genomics and functional genomics, right? You start to gather tremendous amounts of RNA data that will read in kind of like what's happening at the protein level at some, at some sort of mirror echo, right? And then that can, you know, be the case for metabolic, uh, metabolic information as well and so on. Yeah.

Jorge Conde 9:23 So it's a low pixel image, but if we can get sort of zoomed out far enough, we'll get a sense of what's going on.

Patrick Hsu 9:28 You have to bet on what you can scale today, right? We're able to, you know, scale single cell and transcriptional information today. We're able to add on, you know, protein-level information over time. We'll need spatial information, spatial tokens, and we'll need temporal dynamics as well. And we'll, you know, I kind of, uh, bucket things into three tiers. There's invention, engineering, and scaling. And there are certain things today biotechnologically that are scale ready.

Jorge Conde 9:52 Mm-hmm.

Patrick Hsu 9:52 And then there are things that we still need to invent, right? And that's part of why we felt like we needed a research institute to be able to tackle these types of problems, that we weren't just going to be an engineering shop that's just trying to scale single cell perturbation screens, right? That, you know, would be interesting, but in three years would feel very dated, I think, right? And so there's a lot of novel technology investment that we're making that we think will bear fruit over time. Yeah.

Jorge Conde 10:18 Can we flesh out the virtual cell concept, why that's the ambition we, we, we've landed on and what it's gonna take to get there, or what are the bottlenecks?

Patrick Hsu 10:23 I, I would say the most kind of famous success of ML in biology is AlphaFold, right? And this solved the protein folding problem of, you know, when you take a sequence of any amino acid, what does the protein look like, right? And you know, it's pretty good. It's not perfect. It certainly doesn't simulate the biophysics and the molecular dynamics, but it gives you a sense of what the end state is with ninety percent plus accuracy, right?

Jorge Conde 10:49 Mm-hmm.

Patrick Hsu 10:49 And that's the AlphaFold moment that people talk about, right? Where any time you want to, you know, work with a protein, you're-- if you don't have an experimentally solved structure, you're just gonna fold it with this, uh, uh, with this algorithm. And we kind of want to get to that point with virtual cells as well, and the way that at Arc we're operationalizing this is to do, uh, perturbation prediction, right?

Jorge Conde 11:10 Mm-hmm.

Patrick Hsu 11:11 Where the idea is you have some manifold of cell types and cell states, right? Um, that can be a heart cell, a blood cell, a lung cell, and so on. And you know that you can kind of move cells across this manifold, right? Sometimes they become inflamed, sometimes they become apoptotic, sometimes they become cell cycle arrested, they become stressed, they're metabolically starved, they're hungry in some way. And so if you have this re- uh, sort of this representation of universal sort of cell space, right? Can you figure out what are the perturbations that you need to move cells-

Jorge Conde 11:46 Mm-hmm.

Patrick Hsu 11:46 Around this manifold? And this is fundamentally what we do in making drugs, right? Whether we have small molecules, which started out as natural products from, you know, boiling leaves or antibodies when we injected proteins into cows and rabbits and sheep and took their blood to, to get those antibodies. We are, we are basically trying to get to more and more specific probes, right? And we had experimental ways to kind of cook these up. Now we have computational ways to zero shot these binders. But ultimately, what you're trying to do with these binders is to inhibit something-And then by doing so, kind of click and drag it from a kind of toxic gain-of-function, disease-causing state to a more quiescent homeostatic healthy one, right? And the thing that is very clear in complex diseases, right, where you don't have a single cause of that disease, is there are some complex set of changes. There's a combination of perturbations, if you will-

Jorge Conde 12:43 Mm-hmm

Patrick Hsu 12:43 ... that you would want to make to be able to move things around. Now, you know, the people talk about this classically as things like polypharmacology, right? But, you know, I think we're moving from a, "Oh, this thing happens to have, you know, a whole bunch of different targets, um, kind of by accident," uh, to we have the ability to manipulate these things commentorially in a purposeful way, right? That to go from cell state A to cell state B, there are these three changes I need to make first, then these two changes, and then these six changes, right, over time, right? And we kind of want models to be able to suggest this. And the reason why we scoped virtual cell this way is because we felt it was just experimentally very practical. You want something that's gonna be a co-pilot for a wet lab biologist to decide, "What am I gonna do in the lab," right? We're not trying to do something that's like a theory paper that's really interesting to read, where, you know, the numbers go up on a-- the ML benchmark. But, you know, you practically can decide what are the 12 things that you're gonna do in the lab in 12 different conditions, right, and actually just test them, right? And then that's how we kind of enter the, the kind of the lab and the loop aspect of model predictions to experimental measurements to, you know, you know, kind of improved or RL'd or whatever model kind of predictions again. And the goal is to be able to do in silico target ID, where you can basically figure out new drug targets, figure out then the compositions, the drug compositions you would need to actually make those changes.

Jorge Conde 14:14 Mm-hmm.

Patrick Hsu 14:14 I think if we could do that, we could make a new AI, like vertically integrated AI-enabled pharma company, right-

Jorge Conde 14:20 Mm-hmm

Patrick Hsu 14:20 ... which, um, you know, I think is obviously a very exciting idea today. But I think in many ways, the kind of pitch and the framing of these companies precedes the fundamental research capability breakthroughs.

Jorge Conde 14:34 Right.

Patrick Hsu 14:34 And that's what we're really invested in at Arc, is kind of just making that happen along with many other amazing colleagues in the field to just make this possible for, you know, the community.

Jorge Conde 14:45 So if the goal is, I'm gonna oversimplify it for you, like if we wanted to get to the AlphaFold, the moment where, you know, it kind of gives you a, a useful structure, folded structure ninety percent of the time, to use your, your data point. We wanted to take that, uh, comparison in the, in the virtual cell model, and we said, "Okay, ninety percent of the time, if I ask the model I wanna shift the cell from cell, from s-cell state A to cell state B, and it's gonna give me a list of perturbations."

Patrick Hsu 15:13 Mm-hmm.

Jorge Conde 15:13 And let's say that at ninety percent of the time, those perturbations, in fact, result in the shifting experimentally, in the shifting from cell state A to cell state B.

Patrick Hsu 15:21 Mm-hmm.

Jorge Conde 15:22 How far away are we from that AlphaFold moment for virtual cells?

Patrick Hsu 15:27 I find it helpful to frame these in terms of like GPT one, two, three, four, five-

Jorge Conde 15:31 Okay

Patrick Hsu 15:31 ... capabilities, right? And I think most people would agree we're somewhere between GPT one and two, right? A lot of the excitement was that we could achieve GPT one in the first place, that you could see a path with scaling laws of some kind to kind of make successive generations where capabilities would improve. But, you know, these are s- you know, with like our Evo kind of, uh, DNA foundation models that we developed at Arc with, uh, Brian Kee, right, one, one of the things that we've seen is that, you know, these are really kind of-- these genome generations are like quote, unquote "blurry pictures of life," right? We don't think if you synthesize these novel genomes, they would be alive.

Jorge Conde 16:11 Mm-hmm.

Patrick Hsu 16:11 But, you know, we, we don't think that's actually also impossibly far away. We'll just have to kind of follow these capabilities. We're generating, uh, we're taking a very integrated approach to attack this problem-

Jorge Conde 16:21 Mm-hmm

Patrick Hsu 16:21 ... right, where you need to curate public data, you need to generate massive amounts of internal private data, build the benchmarks, and train the new, uh, train new models and building, uh, sort of architectures and kind of doing these things full stack. And we'll just kind of attack this, um, hill climb over time.

Jorge Conde 16:39 What's the GPT, I'll say GPT three moment gonna look like? And by that I mean sort of a public release that alters the public's conception of just what's possible here from a capabilities perspective, and also inspires a whole new generation of talent to like rush into, into, into, into biology.

Patrick Hsu 16:54 Well, the good thing with biology is we have a lot of ground truth, right? There are entire textbooks, right, that describe cell signaling and cell biology and how these things work. And so, you know, even without a virtual cell model at all, right, if you went into ChatGPT or Claude and you basically, you know, you asked just some question about, you know, like receptor tyrosine kinase signaling, it would have an opinion on how that works, right? And so I think you would want the model to be able to predict perturbations that are kind of famous canonical examples of biological discovery. So I'll give you an example. If you loaded into the model an iPSC, a s- a, a kind of an induced pluripotent stem cell state or human embryonic stem cell state, and a fibroblast cell state, right, could it predict that the four Yamanaka factors would reprogram the fibroblast into a stem-like state, right? And essentially rediscover from the model something that won the Nobel Prize in two thousand and nine, right? That would, that would be sort of one really kind of classic example. And then you could go do the inverse. If you have a stem cell, can it discover neurogenin 2, ASCL1, MyoD? Can it find differentiation factors that will turn that into a neuron or into a muscle cell or, or so on? And you know, these are kind of classic examples in developmental biology, but you could also use this to try to discover or kind of recapitulate the mechanism of action of FDA-approved drugs, right? And so you could say, for example-You know, if you kind of inhibit HER2 in, you know, breast cancer you know cell states, right, it would become-- you know, you would get this type of response or it could predict the, you know, certain clones that, you know, will be able to kind of be more metastatic or, you know, they'll be more resistance and they'll lead to minimal residual disease. There, I think lots of kind of biological evals that you can kind of add onto these models over time that are really tangible textbook examples as opposed to, I think, what the kind of early generation of models do today, which is, you know, very quantitative things like mean absolute error over like, you know, the differential expressed genes and stuff like that, you know.

Erik Torenberg 19:06 Mm-hmm.

Patrick Hsu 19:06 Um, that's-- those are ML benchmarks.

Erik Torenberg 19:08 Mm-hmm.

Patrick Hsu 19:09 And we want to increase the sophistication into something that you could explain to an old professor w-who has, you know, never touched a terminal in their life.

Erik Torenberg 19:20 By the way, you talk about textbooks as ground truth. Do you think we're gonna find that a lot of the textbooks are wrong?

Patrick Hsu 19:27 I would say textbooks are compressed, right? So for example, when you, you look at these kind of classic cell signaling diagrams of A signals to B, which inhibits C, right? That's a very kind of two-dimensional representation-

Erik Torenberg 19:41 Representation of our understanding of a complex system.

Patrick Hsu 19:43 Right. Right, right.

Erik Torenberg 19:44 Right?

Patrick Hsu 19:44 I mean, yes, textbooks are what they are. They represent the corpus of reliable knowledge, but everyone knows that there are an incredible number of exceptions, and part of what discovery is, is to find new exceptions, right?

Erik Torenberg 19:56 Why don't you talk about the difference between the simulation of biology and, and the actual understanding, and, and what would it, would it take to actually be able to model the extremely complex human body?

Patrick Hsu 20:04 You know, some people don't like the phrase virtual cells because it sounds too media-friendly. It's not rigorous enough, right? But I, I've always, um, found it funny that, you know, but, you know, many people are okay with like digital twins and digital avatars, which, you know, talks about modeling biology at a way higher [chuckles] level of abstraction. You know, I think virtual cells, if anything, is actually way more scoped and rigorous, uh, than modeling a digital twin or avatar. But, you know, I think these are useful words because they describe the goal and the ambition, right? That no, in the long run, we don't care about predicting the, uh, you know, kind of perturbation responses of an individual cell at all, actually, right? Obviously, we want to be able to predict drug toxicity. We wanna be able to predict aging. We wanna be able to predict why a liver cell becomes cirrhotic when you repeatedly challenge it with ethanol molecules or whatever, right? And, you know, the, these sort of chemical or environmental perturbations should be predictable. I think you just kind of have to layer on the complexity, right? Like, why are we so worried about modeling entire bodies over time when we can't do it for an individual cell, right? Where we sort of, you know, accept or broadly believe that this is a kind of, you know, fundamental unit of biological, you know, computation, if you will, right? And let's just kind of start there, right? Just like you kind of have to start with, you know, things like math and code and language modeling, right? And things that are just sort of easier to check. You can build to super intelligence over time. Yeah.

Erik Torenberg 21:45 Yeah, I think that makes sense, right? That, uh, that's, that's a very sort of laudable, ambitious goal. If we can figure out how to model the fundamental unit of biology, the cell, then from that, we should be able to build.

Patrick Hsu 21:57 Like in early AI, we just started with like language translation, just, you know, basic NLP tasks, right? This was long before, you know, the, the tremendous ambitious scope that we have today, and I think we, we hopefully can mirror that type of trajectory, if we're lucky.

Erik Torenberg 22:13 It, it, it seems that biotech and pharma has, has been a shrinking, um, in, interest as certain as the, the rate of growth. What, what's it gonna take for these, um, innovations in, in, in the science to reflect themselves in, in business models and in, in g-growth for the industry?

Patrick Hsu 22:27 A lot of these biotech startups would try to initially sell software to pharma companies, and then they would kind of realize, "Oh wow, we're like competing for SaaS budgets," um, which aren't very large. And then, you know, now they're realizing, "Oh, we have to compete for R&D budgets," right? And I think, and you know, there is this narrative from the current generation of these companies that, "Oh, our biological agents will compete for R&D budgets and replace headcount," or something like that, right? Just like we're seeing in, you know, agents across different verticals, right? Whether or not that will, I think, pan out, I think depends on just whether or not these things meaningfully allow us to, you know, build drugs more effectively in the pharma context, right? And I think that's just sort of the most important thing, um, in, in, in this industry. And so I think we believe in virtual cells, not just because we think it will be a fountain of fundamental mechanistic insights for discovery, but also because if, in the case of success, it could be industrially really useful, right? But, you know, we'll, we'll, we'll, we'll have to see over time, right? If we have ninety percent of drugs failing clinical trials, right, that kind of means two things, and you're not sure what percent of which, right? One is we're targeting the wrong target in the first place. The second is the composition, the drug matter that we're using doesn't do the job, right? And it's not clear for each individual failure which one it is or if it's both or what proportion of each. And, you know, we'll have to kind of sort that out over time. Like, you can imagine, even in the case of success, when we had ninety percent accurate virtual cells, you'll probably end up with suggestions like, "Okay, now you need to target, you know, this GPCR only in heart."But not in literally any other tissue, right? We don't have the drug matter that can do that today. And so that's also why, again, you probably need research to figure out novel chemical biology matter that allows you to drug pleiotropic, you know, targets, um, in a tissue or cell type-specific way, right? And so, you know, I think, you know, part of why biology is slow is because there's just this Russian nesting doll of complexity, um, in terms of understanding, in terms of perturbation, in terms of safety. And you know, the, the, the crazy thing is the progress in just the short time that I've been doing this is insane, right? Like, I did my, you know, PhD at the Broad Institute in the heyday of developing single cell genomics, human genetics, CRISPR gene editing, um, you know, and, uh, you know, so many other things. And I think the kind of early 2010s papers on single cell sequencing would have like 20 cells or 40 cells, right? And, and at Arc, in the next, you know, kind of n, like, I don't know, relatively short amount of time, we're gonna generate a billion perturbed single cells, right? That's like, I mean, how's, how's that for a Moore's Law?

Erik Torenberg 25:29 Yeah, that's remarkable.

Patrick Hsu 25:30 Yeah.

Erik Torenberg 25:31 Yeah. Jorge, I, I wanna hear your, your answers to a couple of these questions too as, as the lead of, of our bio practice, both on the GPT-3 moment, what that, what that could look like, a-and also, um, like I'm curious if you think it's GLP-1s or sort of building off that, or if it's gonna be some-something different. And also, what's it gonna take for the, for the science to kind of reflect itself in, in the business for the industry to grow?

Jorge Conde 25:49 Yeah. So I'll, I'll take the, the, the second one first, if I could. So I think, you know, in terms of, of where the industry is right now, I think one of the big challenges we have is, as Patrick describes very nicely, like, you know, discovery's hard and it takes time. And, you know, the fail modes are exactly as you describe. Oftentimes when drugs fail, which they do 90% of the time in clinical trials, it's because we're going after the wrong thing or we made the wrong thing to go after the right thing, right? Like, those are the two fail modes, and that happens, uh, all too often. And so I think a lot of the stuff that Patrick is describing is gonna basically improve our hit rate or our batting average on figuring out what to go after and then making the right thing to go after said thing. Um, the challenge we have, I think, in the industry is that the bottlenecks still are the bottlenecks. And the biggest bottleneck we have, which is, you know, a necessary one, is we have to prove that whatever we make, that we have the right thing to go after the right thing, so to speak, and that we, when we have it, that it's going to be as, you know, de-risked as possible before you put it into humans.

Patrick Hsu 26:50 And we have to be good at making them in the first place too.

Jorge Conde 26:52 And we gotta make them too. Yeah, exactly. And so that bottleneck is a necessarily important one. We-- that bottleneck should exist. I'm not suggesting we better remove it. But are there ways to reduce the cost and time associated with getting through the bottleneck of human clinical trials? Um, and you know, it's, it's interesting because, you know, we talk about, you know, uh, all of the various stakeholders when you're making a, a drug. Uh, there are the companies, there's of course the science that supported, um, the, the company that's trying to commercialize a, a product, and there are the regulatory agencies. You know, and everyone is trying to ensure again, that, uh, what's, you know, first and foremost is the ability to, um, uh, discover and commercialize drugs that are safe and effective for humans. That middle part, part of actually getting through that bottleneck is hard to speed up

Jorge Conde 27:39 in a very obvious way. Like, you can increase the rate, the way you enroll clinical trials, you can use better technology to change the way we design these clinical trials, so maybe they can be faster or shorter, et cetera. But some of them just have a natural timeline you have to go through. Like if you want to demonstrate that a cancer drug, uh, promotes survival, guess what? You're gonna have-- It's gonna take some time to demonstrate a survival benefit. Or if, you know, [chuckles] you wanna do a longevity drug, that by definition is a lifetime, you know, of, of a, of a trial in terms of length. So there's a lot of these bottlenecks that are really hard to get through. So what helps the industry?

Jorge Conde 28:13 I think there are a couple of things that, uh, help the industry. One is, um, capital intensity, uh, will hopefully at some point go down over time as technology gets better. Capital intensity is something that our industry faces. In some ways, it looks a little bit like AI now, right, in terms of the cost of training these models, but the capital intensity is very, very high. That has not come down. So, um, we gotta get the success rates up to impact capital intensity to get it down. The second thing is can-- where can we compress time? So good models can help us compress early discovery time. We still haven't seen, uh, and I think it's coming, but it hasn't happened yet, we haven't seen artificial intelligence or other technologies massively compress the amount of time it takes us to do the clinical development, the clinical trials, the enrollment of patients, all of those things. We're seeing some interesting things coming. We haven't seen sort of the, the payoff there yet.

Jorge Conde 29:02 Um, and the third thing is if we can make better drugs, going after better things, y- the effect size should be higher, so therefore the answer should be obvious sooner. If we can get those three things right, reduce capital intensity, compress timelines, and effectively increase effect size in, in some very tough, um, sort of, uh, intractable diseases, that is, I-- what I think fixes the industry. And from where we sit at the early stage, um, uh, at the early stage in terms of being early stage investors, the reason why that helps us is if the capital intensity goes down and the value creation goes up, it becomes easier to invest in these companies in the early days 'cause you get rewarded for coming in early. The problem we have right now is that most companies [chuckles] aren't-- you're not seeing, uh, rewards happening when there's value inflection. So you come in early, you bear the brunt of the capital intensity, and even if a company's successful, that success isn't reflected in the valuation. So we're not seeing the step-ups that you see in other parts of the industry, and that's just really, really hard from a, from an investing standpoint. So I think we need to see those various factors addressed for this space to really get, you know, fixed, to use your word.

Patrick Hsu 30:09 Yeah, that was great. I, I have a, I have a lot to add onto

Patrick Hsu 30:12 this.

Jorge Conde 30:12 Please add away.

Patrick Hsu 30:13 You know, just, you know, one-- a, a few simple observations, right? The, the first is the amount of market cap-Added to Lilly and Novo, um, based on the, you know, development of GLP-1s, um, is like over a trillion dollars is, is more, you know-- I mean, Novo's stock has decreased a lot. So, um, you know, a trillion dollars, let's say, is more than the market cap of all biotech companies combined over the last forty years have been started, right? And I think that, you know, one, one of the kind of interesting kind of corollaries of this is that, you know, when we have a ten percent kind of clinical trial success rate for kind of preclinical drug matter, right, you tend to circle the wagons a bit and try to manage your risk, right? And so the way that you do this is you try to go after really well-established disease mechanisms, where if I developed new drugs that go after well-understood biology, it should work the way that I hope it will in the trial, um, which is, you know, s- really, really expensive and costs a lot more in many ways than the, the preclinical research, right? Um, the problem with this is you go after very well-validated disease mechanisms but with really small patient populations, right? So then the expected value of this actually is relatively low. One of the kind of things that we've seen with, uh, GLP-1s is the, just the kind of value that you can create when you go after really large patient populations. And I think that has culturally really net increased the ambition of the industry, both from the investor and from the drug developer side. And I think, you know, that's something that we should keep our foot on the gas for. Yeah.

Jorge Conde 31:55 Yeah. And look, I think the trend on that is posi-- I would argue the trend on that is positive.

Patrick Hsu 31:58 Yeah, yeah.

Jorge Conde 31:58 You're absolutely right. Like, the, the demonstration of, uh, the value that has been created, uh, with the, the, the use, the increasing use of GLP-1s and the value transfer that's gone to companies like Lilly and Novo, I would argue, is like very merited, right? Because they've cracked an endemic social problem, um, in terms of managing diabetes, diabetes and eventually helping manage obesity. And so I think that's remarkable, and there's a lot of value that goes to that because they tackled, they cracked a very, very, um, challenging problem-

Patrick Hsu 32:29 Yeah

Jorge Conde 32:29 ... for, for society beyond just science. So that's great. And I agree with you. Like, the, the, the, the prize, the juice needs to be worth, worth the squeeze, right? You're right. A lot of biotech has been around, like, go after the low-hanging fruit because it's low risk, and we got to eat today, right? So you go get it, you know, and you just have to, you push off the big, the big ambitious indication, the large population or the, the really, uh, tough to crack disease. But, you know, I do think we're seeing more and more of that. And by the way, like, we can get into some of these genetic medicines, but some of these genetic medicines are going after some of the hardest problems, the things that you quite literally couldn't address but for editing, you know, DNA. And, you know, I think that's incredibly, you know, remarkable and laudable and, and, and quite frankly inspiring.

Patrick Hsu 33:08 Yeah.

Jorge Conde 33:09 But the fundamental, um, elements of the industry have to work, so the capital formation is there to support those kinds of things. And right now it's hard, right, because of the, the issues we talked about before.

Erik Torenberg 33:19 Fifteen years from now, we're back in this room. We've barely escaped being part of the permanent underclass.

Patrick Hsu 33:25 [laughing]

Jorge Conde 33:26 [laughing]

Erik Torenberg 33:26 And we're, we're reflecting on the, on sort of the GPT-3 moment or, or maybe w- the legacy of GLP-1s, um, sort of beyond wh-where, where they are now. Um, what, what do you think it could be? Or, or I'm curious to get your take on what do you think is going to be the technological breakthrough that we're going to point back to and say, "Oh, this is really what, what said it all." That, or do you think it's going to be sort of, you know, multifactor combination?

Jorge Conde 33:46 Yeah, look, I think, um, it's, it's going to go back to, uh, sort of where we started, uh, this combination, uh, conversation, excuse me. Uh, GLP-1s, uh, as a drug are, you know, what, four decades in the making or something like that. You know, these are, these are not overnight successes. Um, uh, but I do think what we are going to see more of and, and our hope is that when you combine the fact that we're getting better at, uh, understanding what to target, getting better at designing medicines to hit those targets, by the way, in a whole array of new creative ways. So we have small molecules, the natural products that we got from boiling leaves, as you said earlier, like, those have gotten... You know, we're getting really good at designing smarter and better, smaller molecule- small molecules that do new things, that function in ways that they didn't before. Um, we've gotten quite good at designing, uh, uh, biologics or proteins with a lot of help from things like AlphaFold that help us understand how proteins fold. We're going to get a lot better at designing some of the more complex, uh, modalities like the gene therapies of the world or the gene editors of the world. And when you can do that and combine that with our ability to hopefully use things like virtual cell models to really understand what to go after, like, we're going to have drugs, we-- I would hope and I would expect that the industry will continue to bring forward drugs that have very large effect size for very difficult diseases that hopefully affect a lot of patients. If that's true, then we'll start to see some of these really, really difficult, um, diseases that affect all of society get tackled, hopefully, you know, one by one by one by one. And so we have obesity, we have metabolic disorder, we're dealing with cardiometabolic disease. We're starting to see interesting, promising things happening in like, neurodegenerative diseases. Um, you know, if we can, you know, tackle cancer or at least, you know, several cancers that now have begun to be treated more like a chronic condition than a death sentence that they were in the past. The more we see of that, like, I think that value to society will accrete over time, and I think this should be an industry that is extraordinarily, uh, valued by society and candidly by the markets. We have to deliver.

Patrick Hsu 35:49 If we play this out, right, and let's say these AI models work, right, and you can make a trillion binders in silico that will, you know, be exquisite drug matter, right? We still need to make these things physically and test them in-Animals in hopefully co- predictive models, and then actually in people, right? And I think, you know, that will increasingly be the, the, the, the, the bottleneck in many ways, right? And you know, my, my friend, uh, Dan Wang recently, um, released a book called Breakneck which talks about, um, you know, kind of like the US and China and the difference between the two countries and their philosophy, the way they approach markets, and-

Jorge Conde 36:31 We're, we're the s- a country of lawyers, they're a country of engineers-

Patrick Hsu 36:33 Exactly

Jorge Conde 36:34 ... sort of the political climate.

Patrick Hsu 36:34 That's right. Right. China is an engineering state, right? Its, uh, kind of, uh, politburo is, you know, folks who have engineering degrees. You know, you need to build bridges and roads and buildings, and these are the ways that we solve our problems, whereas I think from, you know, the first 13 American presidents, 10 of them practiced law. From 1980 to 2020, all Democratic presidential candidates, uh, uh, y- uh, both vice- VP and president, uh, went to law school, right? And so you kind of see the echoes of that in the FDA and the regulatory regime and, you know, all, all the kind of the, the, the bottlenecks that people talk about developing drugs stateside. And increasingly you see folks thinking about how we can run phase ones overseas, right?

Jorge Conde 37:21 Mm-hmm.

Patrick Hsu 37:21 Build data packages that we can, you know, bring back domestically for phase two efficacy trials. I think that's interesting directionally, but it's not enough, right? And, you know, I think we need to kind of figure out these two bottlenecks, the making and the testing, you know. Even if we can solve the designing part.

Jorge Conde 37:40 Oh, I agree.

Patrick Hsu 37:40 Yeah.

Jorge Conde 37:40 Yeah, yeah, that, that's the bottleneck.

Patrick Hsu 37:42 Yeah.

Jorge Conde 37:42 You know, we, we joke about it and you have to do, is you have to get a molecule that can go, you know, first in mice and then in mutts and then in monkeys and then in man. Like, there's, you know, that takes, takes a long time, and it's just so hard to compress that. Um, and so when you do, you, you should make the journey worth, you know, uh, make the journey worth it, right?

Patrick Hsu 38:02 Yeah.

Jorge Conde 38:02 So when you fail on the other end of that, like, that's obviously horrible.

Patrick Hsu 38:06 Mm-hmm.

Jorge Conde 38:06 And so finding ways to make sure that when you, when you walk that path, that it'll be a successful journey as often as possible is what this industry desperately needs.

Patrick Hsu 38:14 Mm-hmm.

Patrick Hsu 38:15 Mm-hmm.

Jorge Conde 38:15 Uh, AlphaFold solved a pr- pr- protein folding problem, but wh- why didn't it solve drug, drug discovery? Or more broadly, what would it take to, to get AI drug discovery? What, what is sort of the, the, the bottleneck on the, on the, on the tech side at least?

Patrick Hsu 38:28 On the tech side? Um-

Jorge Conde 38:30 Or maybe another way to ask the question-

Patrick Hsu 38:31 Yeah

Jorge Conde 38:31 ... is that w- 'cause I always ask the founders-

Patrick Hsu 38:33 Sure, yeah

Jorge Conde 38:33 ... a version of this question-

Patrick Hsu 38:34 Yeah

Jorge Conde 38:34 ... like the AI ones-

Patrick Hsu 38:35 Sure

Jorge Conde 38:36 ... um, that are like, "Oh, we're gonna do, uh, AI for, like, for drug discovery."

Patrick Hsu 38:39 Mm-hmm.

Jorge Conde 38:39 So my, my question that I always like to ask founders is give me examples of where you think AI is hyped-

Patrick Hsu 38:46 Yeah

Jorge Conde 38:47 ... potentially overly hyped, um, where there's real hope, like the sort of what do we expect-

Patrick Hsu 38:51 Mm-hmm

Jorge Conde 38:51 ... what's next, and where we already see real heft.

Patrick Hsu 38:55 Yeah.

Jorge Conde 38:55 So, like, if I asked you, like, in AI, you know, where is there hype? Where is there hope, and where are we seeing heft today?

Patrick Hsu 39:04 I would say there's hype in toxicity prediction models.

Jorge Conde 39:07 Okay.

Patrick Hsu 39:08 Heft-

Jorge Conde 39:08 So that's the idea that-

Patrick Hsu 39:09 Yeah

Jorge Conde 39:09 ... we will say, um-

Patrick Hsu 39:11 Yeah

Jorge Conde 39:11 ... I'm gonna show you a molecule, and you're gonna tell me, the model's gonna tell me if it's gonna be toxic or not.

Patrick Hsu 39:14 That's right.

Jorge Conde 39:15 Okay.

Patrick Hsu 39:15 Right?

Jorge Conde 39:16 All right.

Patrick Hsu 39:16 There's heft in anything to do with proteins, right? Obviously protein binding, but increasingly in protein design, right? I think there's real heft there. And then, you know, where there's hype is, um, in multimodal biological models, whatever that means, right? And I think, you know, pick your favorite layers. It could be, you know, molecular layers. It could be spatial layers. It could be, you know... I, I, I mean, actually I would say there's also heft in the pathology AI prediction models, you know? Like, you know, uh, automating the work of pathologists and radiologists, that's, that's-

Jorge Conde 39:53 Yeah, right

Patrick Hsu 39:54 ... interesting.

Jorge Conde 39:54 That's a powerful use case-

Patrick Hsu 39:55 Yeah

Jorge Conde 39:55 ... for sure.

Patrick Hsu 39:55 Yeah. And there's a lot of stuff where you don't have to train, you know, weird biology foundation models, and you can write, you know, regulatory filings and reports and things like that. Um, that's impactful and important, you know.

Jorge Conde 40:10 So now go back to Eric's question is why don't, why hasn't AI churned out drugs yet? I think that was your question, right?

Patrick Hsu 40:17 Yeah. You know, AI for drugs is one of these weird things where everyone who works in the industry is trying to claim that their drug is, like, the first AI-designed molecule, right? Um, I feel like in, you know, I mean, increasingly in just a few years, this will just be a native part of the stack, right? Just like we use, you know, the internet, and we use phones, we're gonna have AI in all parts of the stack, right? And so it's just going to become a native part of everything that we do, and so, you know, like, why hasn't it worked yet is this long, multifactorial process that we've been talking about today. There's designing, there's the making, there's the testing, there's the approvals side of it, and, you know, I think the, I, I do think safety and efficacy as the kind of two pillars in the industry are the two things that we need to get right, right? We need to be able to figure out faster ways that we can predict whether or not a molecule will work and if it's going to be safe or not. You know what I mean? There are, like, ways that AI can operationalize this. If you designed a small molecule, right, you can now computationally dock it to every protein in the proteome-

Jorge Conde 41:29 Mm-hmm

Patrick Hsu 41:29 ... and see if it's likely to bind to off-target molecules. You can use this to tune binding selectivity and affinity that might be ways to predict, you know, safety and efficacy, right? And, you know, how well will that work? Well, that's a feedback loop that we'll have to actually test in the lab, and that's part of what's slow is the testing, um, you know, takes real hours, days, months, right, years. And, you know, that, that, and that's really why we've picked at Arc the virtual cell models as our initial wedge because we think it can integrate a lot of these different pieces.

Jorge Conde 42:04 In Dario Amodei's essay, Machines of Love and Grace, he predicts, among other things, the prevention of, of, uh, of many infectious diseases and the doubling of lifespans perhaps a- as soon as the, the next decade. What, what's your reaction to his, his e- essay, his bullishness in some of his predictions?

Patrick Hsu 42:19 I think the coreIntuition that Dario had was the idea that sign-- like important scientific discoveries are independent, right? Or they're largely independent. And if they are, you know, statistically independent, then it would stand to reason that we could multi-parallelize. And so we had models that were sufficiently predictive and useful. You could have not just a hundred of them, but millions, billions of these discovery agents or processes running at a time, which should compress the timeline to new discoveries, um, and turn it into a computation problem, right? I think that is a very futuristic framing for something that is actually very tangible today, right? Um, and if we can have virtual cell models at work, for example, that can start to do these kinds of things that we've been talking about, help us, you know, we can have, you know, molecular design models, we can have docking models, we can then have, you know, when you bind to this thing in this cell versus all the other off-target proteins, will a cell kind of be corrected in the right way, right? These kind of layers of abstraction and complexity start to get to things that feel very tangible to drug discovery. If you could actually traverse these steps reliably and in sequence, you could start to see how you can get the compression, right? And, and so I think in the long run of time, this should be possible.

Jorge Conde 43:51 One of the core suppositions in building a good virtual cell model is that we are feeding it all the relevant data.

Patrick Hsu 44:00 The right data, yeah.

Jorge Conde 44:01 The right data.

Patrick Hsu 44:01 Mm-hmm.

Jorge Conde 44:02 Um, and so we'll work to, you know, it's gene expression data or it's DNA data or, you know, any, any number of factors.

Patrick Hsu 44:09 Mm-hmm.

Jorge Conde 44:09 Protein and protein interactions, all the things you described.

Patrick Hsu 44:11 Sure.

Jorge Conde 44:12 Um, what if we're missing a core element? Like, what if we just haven't discovered the quirk or whatever? Like, we just don't know what we don't know, and therefore the-- what we're feeding the model is fundamentally or importantly incomplete.

Patrick Hsu 44:26 I think that's almost certainly true, right? Like, it's seems almost obvious that we're not measuring many of the most important things in biology, right? And you can of course find many important exceptions for any of these measurement technologies. Like in biology, we ultimately have two ways to study it in high throughput. It's imaging and sequencing, right? But there are so many other types of things that you would care about that those things aren't necessarily going to do at scale, right? And that's really why I think the stuff that we're talking about of the RNA layer as a mirror for other layers of biology is one that we spent a lot of time thinking about. Um, and there's a difference between a mechanistic model and a meteorological simulation type of model. So for example, if you want to predict the weather, right, you can build AI models that will predict whether or not it will rain next Tuesday. It won't explain physically or geologically or whatever why and how that happens, but as long as it knows if it's gonna rain next Tuesday, you're probably happy, right? And I, I would say similarly with a virtual cell model, it may not tell me literally why, just like a AlphaFold doesn't tell me literally why did the protein fold this way and how-

Jorge Conde 45:44 Mm-hmm

Patrick Hsu 45:44 ... but it just told me the end state, and it was reasonably accurate. I think that would already be very

Patrick Hsu 45:50 important.

Jorge Conde 45:50 Shifting gears a little bit. We've been talking about science and, and, and biotech, but you're-- in addition, you're an elite AI in-in-investor more broadly. So, so I wanna talk about, um, h-how you're-- I wanna talk about where your investment focus is right, right now, just as it relates to AI more broadly. Wh-where are you excited? Where are you spending time? Wh-where are you, you know, looking forward to?

Patrick Hsu 46:08 Oh, yeah, my, my goal is to really try to figure out ways that we can improve the human experience in our lifetime. I kind of think of-- like, if I think about the future that we're gonna leave to our children, right, there are a few things that if we get them right in our lifetime, will fundamentally change the world, right? And, you know, how we live in it. I think synthetic biology, um, is obviously one, right? You know, think, you know, GLP-1s, right? Things that improve, uh, sleep, right? Things that can, you know, improve longevity, right?

Jorge Conde 46:42 Mm-hmm.

Patrick Hsu 46:42 These are, these are all things that are kind of, you know, easy to get excited about. I do-- I, I think, um, uh, brain computer interfaces, um, is another area where, um, we're gonna see really important breakthroughs over the decades to come. And then I think the third is in, uh, in, is in robotics, both industrial and consumer robotics, right?

Jorge Conde 47:02 Mm-hmm.

Patrick Hsu 47:03 Um, that allow us to basically, like, scale physical, like, uh, labor, right, in, in, in interesting ways. And, you know, you can kind of see how each of these three things, even in the sort of medium cases of success, really kind of change the world. And so I'm very interested in helping make these kinds of things possible, right? And so there are sort of, you know, in the kind of techno-optimist sort of vision of the world, right, there's a few different types of scarcity, right? There's as-- you know, it-it's very easy when you do research to come up with important ideas. The hard thing is to tackle them in the right timeframe, right? It's like, you know, writing futuristic sci-fi things is not that hard. Being able to actually execute on it in the next five years or eight years, um, much, much harder, right? And I would say, you know, academic discovery is littered with plenty of ideas that are interesting and important, but, you know, kind of long before their time. And in many ways, the story of technology development is, you know, trying to use new technologies to solve old tricks, right?

Jorge Conde 48:09 Mm-hmm.

Patrick Hsu 48:09 Like, most of our tools are, you know, for productivity, right, um, in many ways, um, whether that's the Industrial Revolution or the computing revolution or the current AI revolution. We're trying to kind of do the same stuff. And, you know, a-and so there, you know, I think there's a relatively small set of very powerful ideas, new technologiesGive us new opportunities to attack them, and there's a set of people in teams that are gonna be positioned to be able to do that. They need to have technical innovation and then an intuition about product and business in a way that, you know, you know, you kind of in the RPG dice roll of the skills that you get in these three domains, people start at different base levels, right?

Erik Torenberg 48:51 Mm-hmm.

Patrick Hsu 48:52 And, you know, you might have an incredibly technical founder who doesn't know how to think commercially, or someone who's just natively a very commercial thinker who, you know, doesn't have very strong product sense, right? Um, even though they could sell the crap out of it, right? And so I think these sort of-- these sort of three broad categories of capabilities you need to kind of bring together in a way that you can allocate capital to in the right times in order to make these ideas possible in a really differentiated way. Like, this thing literally wouldn't happen if we didn't get these people together and fund it at the right time in the right way, right? And that's the-- that's really what, what motivates me, and these are the kinds of things that I've been excited about, you know, backing, you know, longevity companies like Neolamin, right? BCI companies like Nudge, right? Robotics companies like The Bot Company, right? Um, you know, these are some of the examples of kind of, you know, things that I think must happen in the world, um, and therefore should happen and, you know, how do we actually find the right people and the right time to actually kind of go on the Fellowship of the Ring hunt, right?

Erik Torenberg 49:58 Yeah. [chuckles]

Patrick Hsu 49:59 Yeah.

Erik Torenberg 49:59 If not too difficult, I wanna ask, uh, Jorge a question adopted to these s-- uh, additional spaces, um, robotics, um, sort of BCIs and, and longevity if, if appropriate in terms of-- and the three questions I believe were, what's overhyped? What-- Uh, where do you see a, a, a opportunity or path, and what's got heft already?

Patrick Hsu 50:20 I think the cool thing about agents, um, generally is that they do real work, right? Um, compared to, like, SaaS companies that came before, agents replace real productivity, right? And I think, you know, they have a lot of errors today, and I would say the computer use agents will probably trail the coding agents by maybe a year, right? But, but it's coming, and we'll follow the trajectory as these go from doing, you know, minutes of work without error to hours to days, right? And I think, you know, you're gonna get a completely different product shape as we march through that across legal, BPO, you know, medicine, healthcare, whatever, right?

Erik Torenberg 51:04 Mm-hmm.

Patrick Hsu 51:04 And we'll kind of follow that as an industry, and that's, that's gonna be really exciting. And I think that's where we're gonna see real heft is because most of the economy is services spent. It's not software spent. And, you know, the reason why we're all excited about this stuff is that it can attack, you know, the, the, the services economy. And I would say, like, you know, where, where is there hype? There's a tremendous amount.

Erik Torenberg 51:26 [chuckles]

Patrick Hsu 51:27 Right? That's, that's, that's no doubt. The hype is in the model capabilities, right? And, you know, it's-- we, we, we're working with an architecture that, you know, dates back to twenty seventeen, right? And if you look at the history of deep learning, it's like kinda every eight years there's something really different, right? Um, and we-- it feels like in twenty twenty-five we're really overdue for some net new architecture. And I think there are lots of really interesting research ideas that are bubbling up that could, um, do that thing. And in many ways, there's a set of really interesting academic ideas, especially in the golden age of machine learning research from, I don't know, like two thousand and nine to twenty fifteen, right? There's so many interesting ideas, little archive papers that have, like, thirty citations or less. And as the marginal cost of compute goes down year on year, I think you're gonna be able to take all of these ideas and actually scale them up, right? Where you don't see the scaling laws when you're training them at a hundred million or six fifty million parameters like back then. But if you can scale them up to one B, seven B, thirty-five B, seventy B, right, you start to see, you know, whether or not these ideas will pop, right? And I think that's very exciting because, you know, there's just gonna be a lot of opportunity for new super intelligence labs to do things, um, you know, beyond what the kind of, you know, established foundation model companies are doing today, right? As they kind of, you know-- In addition to these research teams, right, you know, these are in many ways becoming applied AI companies, right? They need to build product shape and, you know, all kinds of different enterprises and do RL for businesses and make money, right? And I think, uh, or, or, or build coding agents and make API revenue, and that's important and I think, you know, a timely race to survive today. But I'm just, you know, a, a very bullish on the research of, say, like a Sakana AI, right? Which was founded by one of the authors of, uh, Attention Is All You Need, right? Ian Jones. And they're doing incredibly interesting stuff on model merging and how you can have kind of, um, sort of like evolutionary selection of, you know, kind of, uh, different, um, uh, kind of, um, you know, models in MoE. And I think the, there are sort of opportunities here, um, in the long run to move beyond just, like, RL gems, for example. Um, also to kind of figure out new ways to learn and, and find, like, kind of reward signal is, is gonna be really exciting.

Erik Torenberg 54:02 This is a great place to wrap. G-gearing towards, um, towards closing, anything, uh, upcoming for Arc that you'd like us to know of? Anything you wanna tease? Anything-- For people who wanna learn more, w-which should they know about?

Patrick Hsu 54:11 So AlphaFold, uh, in many ways came out of a protein folding competition called CASP, right? Uh, Critical Assessment of the Structure of Proteins. And, um, you know, we created our own virtual cell challenge, um, at virtualcellchallenge.org, where we have, you know, hundred thousand dollar prizes sponsored by NVIDIA and 10X Genomics and Ultima and others. And it's an open competition that anyone can enter, where you can train perturbation prediction models, and we can openly and transparently assess these model capabilities, both today and in subsequent years, follow them to get to that ChatGPT moment, right? And so I'm extremely excited about this. Um, you know, um, we, we'd like more people to, you know, train models and apply both bioML experts and engineers in any other domain. And, you know, I'm, you know-- I, I, I just-- I want this thing to exist in the world. You know, hopefully, we're important parts of making that happen, but I'd just be happy that someone does it.

Erik Torenberg 55:11 Yeah.

Patrick Hsu 55:11 Yeah.

Erik Torenberg 55:12 That's an inspiring note to, to wrap on. Patrick, Jorge, thanks so much for the conversation.

Patrick Hsu 55:16 Thanks so much, guys. Appreciate it.

Erik Torenberg 55:17 Thanks for having me. [outro jingle]