Adam Marblestone on Dwarkesh Patel: Why AI Uses Blunt Losses

1. 0:00 – 22:20

   The brain’s secret sauce is the reward functions, not the architecture

   1. DPDwarkesh Patel0:00

      The big million-dollar question that I have that, um, I've been trying to get the answer to through all these interviews with AI researchers, how does the brain do it, right? Like, we're throwing way more data at these LLMs, and they still have a small fraction of the total capabilities that a human does. So what's going on?

   2. AMAdam Marblestone0:15

      Yeah. I mean, this might be the quadrillion-dollar question-

   3. DPDwarkesh Patel0:17

      (laughs)

   4. AMAdam Marblestone0:17

      ... or something like that. It's, it's, it's arguably, you could make an argument this is the most important, you know, question in science. I don't claim to know the answer. I, I also don't really think that the answer will necessarily come even from a lot of smart people thinking about it as much as they are. I, my, my overall, like, meta-level take is that we have to empower the field of neuroscience to just make neuroscience a, a more powerful, uh, field, technologically and other otherwise, to actually be able to crack a question like this. But maybe the, the way that we would think about this now with, like, modern AI, neural nets, deep learning, is that there are sort of these, these cer- certain key components of that. There's the architecture. Um, there's maybe hyperparameters of the architecture. How many layers do you have or sort of properties of that architecture? There is the learning algorithm itself. How do you train it? You know, backprop, gradient descent, um, is it something else? There is how is it initialized, okay? So if we take the learning part of the system, it still may have some initialization of, of the weights. Um, and then there are also cost functions. There's like, what is it being trained to do?

   5. DPDwarkesh Patel1:28

      Yeah.

   6. AMAdam Marblestone1:28

      What's the reward signal? What are the loss functions? Supervision signals. My personal hunch within that framework is that the, the field has neglected, uh, the role of this very specific loss functions, very specific cost functions. Uh, machine learning tends to like mathematically simple loss functions, right? Predict the next token. Um, you know, cross-entropy. The, you know-

   7. DPDwarkesh Patel1:53

      Right.

   8. AMAdam Marblestone1:53

      ... these, these, these, these, uh, s- simple kind of computer scientist loss functions. I think evolution may have built a lot of complexity into the loss functions. Actually, many different loss functions for different areas, turned on at different stages of development. A lot of Python code, basically, uh, generating, uh, a specific curriculum for what different parts of the brain need to learn. 'Cause evolution has seen many times what was successful and unsuccessful, and e- evolution could encode the knowledge of, of the learning curriculum. So, so in the, in the machine learning framework, maybe we can come back and we can talk about, yeah, where do the loss functions of the brain come from. Can that, can loss fun- different loss functions lead to different efficiency of learning?

   9. DPDwarkesh Patel2:31

      You know people say, like, the cortex has got the universal human learning algorithm, the special sauce-

   10. AMAdam Marblestone2:35

       Right, right.

   11. DPDwarkesh Patel2:36

       ... that humans have. What's up with that?

   12. AMAdam Marblestone2:37

       Well, this is a huge question. Uh, and we don't know. I've seen models where what the cortex, uh, you know, the cortex has typically this, like, six layered structure.

   13. DPDwarkesh Patel2:45

       Yeah.

   14. AMAdam Marblestone2:45

       Layers in a slightly different sense than layers of a neural net.

   15. DPDwarkesh Patel2:48

       Yeah.

   16. AMAdam Marblestone2:48

       It's like any one location in the cortex has six physical layers of tissue as you go in layers of the sheet, and then those areas then connect to each other, and that's more like the layers of a network. Um, I've seen versions of that where what you're trying to explain is actually just how does it approximate backprop.

   17. DPDwarkesh Patel3:07

       Yeah.

   18. AMAdam Marblestone3:07

       And what is the cost function for that? What is the network being asked to do? If you were sort of, are trying to say it's something like backprop, is it doing backprop on next token prediction or is it doing backprop on-

   19. DPDwarkesh Patel3:16

       Right, exactly.

   20. AMAdam Marblestone3:17

       ... uh, classifying images or, or what is it doing? Um, and, uh, no one, no one knows. (laughs) Um, but I think, I think one, one thought about it, one possibility about it is that, um, it's just this incredibly general, um, prediction engine. So, so any one area of cortex is just trying to predict any... basically, can it learn to predict any subset of all the variables it sees from any other subset? So, like, omnidirectional inference, um, or omnidirectional prediction. Um, whereas an LLM is just you see everything in the context window, and then it, it computes a very particular-

   21. DPDwarkesh Patel3:56

       Yeah.

   22. AMAdam Marblestone3:56

       ... conditional probability, which is given all the last thousands of things, what is the very probabilities for all the, all the, the next token?

   23. DPDwarkesh Patel4:03

       Yeah.

   24. AMAdam Marblestone4:03

       Um, but it would be weird for a large language model to say, you know, um, you know, the quick brown fox, blank, blank, the lazy dog, um, and fill in the middle.

   25. DPDwarkesh Patel4:15

       Yeah.

   26. AMAdam Marblestone4:15

       Um, uh, versus do the next token. It, it, it, if it's, if it's doing just forward, it can learn how to do that stuff in this emergent level of in-context learning, but natively, it's just predicting the next token.

   27. DPDwarkesh Patel4:27

       Yeah.

   28. AMAdam Marblestone4:27

       What if the cortex is just natively made so that it can, you know, any area of cortex can predict any pattern in any subset of its inputs given any other missing subset? Um, that is a little bit more like, quote-unquote, probabilistic AI. Um, I think a lot of things I'm saying, by the way, are extremely similar to, like, what Yann LeCun would say.

   29. DPDwarkesh Patel4:48

       Yeah.

   30. AMAdam Marblestone4:48

       Um, he's really interested in these energy-based models. Um, and something like that is like the joint distribution of all the variables. What is the, what is the likelihood or unlikelihood of just any combination of variables? And if I, if I clamp some of them, I say, "Well, definitely these variables are in these states," then I can compute with probabilistic sampling, for example, I can compute, okay, conditioned on these being set in this state, what are... And these could be any arbitrary subset of, of, of variables in the model. Uh, can I predict what any other subset is gonna do and sample from any other subset given clamping this subset? And I could choose a totally different subset and sample from that subset. Um, so it's omnidirectional inference. And so, you know, i- that could be, there are some parts of er- of cortex that might be like association areas of cortex that may, you know, predict vision from audition.
2. 22:20 – 42:42

   What the genome actually encodes

   1. DPDwarkesh Patel22:20

      I want to talk about this idea that you just, uh, g- glanced off of, which was amortized inference. Um, and maybe I should try to explain what I think it means, 'cause I think it's probably wrong, and y- this- this will help you correct, or...

   2. AMAdam Marblestone22:36

      It's been a few years for me too, so. (laughs)

   3. DPDwarkesh Patel22:37

      Okay. Um, right now the way the models work is you have an input, it maps it to an output. And this is amortizing a process that... th- the real process, which we think is, like, what intelligence is, which is, like, you have some prior over how the world could be. Like, what are the causes that make the wor- world the way that it is? And then, the way- when you see some observation, you should be like, "Okay, here's all the ways the world could be. Um, this cause explains what's happening best." Now, the- like, doing this calculation over every possible cause is computationally intractable, so then you would just have to sample, like, "Oh, here's a po- potential cause. Does this explain this o- observation? Uh, no. Forget it, let's- let's keep sampling."

   4. AMAdam Marblestone23:25

      Right.

   5. DPDwarkesh Patel23:25

      And then eventually you get the cause. The cause then- the cause explains the observation, and then this becomes your posterior.

   6. AMAdam Marblestone23:33

      That's actually pretty good, I think, of sort of- yeah. So- yeah, this- Bayesian inference, like, in general is, like, of this very intractable thing.

   7. DPDwarkesh Patel23:39

      Right.

   8. AMAdam Marblestone23:40

      It- the algorithms that we have for doing that tend to require a- taking a lot of samples, Monte Carlo methods, taking a lot of samples.

   9. DPDwarkesh Patel23:46

      Yeah.

   10. AMAdam Marblestone23:46

       And taking the samples takes time. I mean, this is like the original, like, Boltzmann machines and stuff were using-

   11. DPDwarkesh Patel23:52

       Yeah.

   12. AMAdam Marblestone23:52

       ... techniques like this. Um, and still it's used with probabilistic programming, other types of methods often. And so- uh, yeah. So the Bayesian inference problem, which is, like, basically the problem of, like, perception, like given some model of the world and given some data, like, how should I update my-

   13. DPDwarkesh Patel24:07

       Right.

   14. AMAdam Marblestone24:07

       ... or how sh- uh, what- what- what are the, like, the- the variables, you know, missing variables in my- in my internal model?

   15. DPDwarkesh Patel24:13

       And I guess the-

   16. AMAdam Marblestone24:14

       Yeah.

   17. DPDwarkesh Patel24:14

       ... the idea is that neural networks are hopefully...Um, obviously, the n- there's mechanistically, the neural network is not starting with like, "Here is my model of the world, and I'm gonna try to explain this data." But the hope is that instead of starting with, um, "Hey, d- does this cause explain this observation? No. Here, does this cause explain this explanation?"

   18. AMAdam Marblestone24:34

       Right.

   19. DPDwarkesh Patel24:34

       Yes. What you do is just like observation-

   20. AMAdam Marblestone24:36

       What's the most, what's the cause that we, the neural net thinks is, is the best one, yeah.

   21. DPDwarkesh Patel24:40

       Yeah. Observation to cause, so the feedforward like goes observation to cause.

   22. AMAdam Marblestone24:43

       Right. Observation to cause.

   23. DPDwarkesh Patel24:44

       To then output that like...

   24. AMAdam Marblestone24:45

       Um, yes. You don't have to, you don't have to evaluate all these energy values or whatever-

   25. DPDwarkesh Patel24:48

       Right, right. Yeah.

   26. AMAdam Marblestone24:48

       ... and, and sample around to make them higher and lower. Um, you just say, um, approximately that process would result in this being the top one or something like that.

   27. DPDwarkesh Patel24:58

       Exactly, yeah. One way to think about it might be that test time compute, inference time compute-

   28. AMAdam Marblestone25:03

       Right.

   29. DPDwarkesh Patel25:03

       ... is actually doing this sampling again because it... you literally read its chain of thought. It's like actually doing this toy example we were talking about where it's like, "Oh, can I solve this problem by doing X? Nah."

   30. AMAdam Marblestone25:13

       Yeah.
3. 42:42 – 50:31

   What kind of RL is the brain doing?

   1. AMAdam Marblestone42:42

   2. DPDwarkesh Patel42:42

      Right.

   3. AMAdam Marblestone42:42

      Yeah.

   4. DPDwarkesh Patel42:42

      Okay. Uh, I want to ask you about RL. So, um...Currently, the way these LLMs are trained, you know, they are, um... If, if they solve the unit test or solve a math problem, that whole trajectory, every token in that trajectory is upweighted. And what's going on with humans? Is there... Are there different types of model-based versus model-free that are happening in different parts of the brain?

   5. AMAdam Marblestone43:02

      Yeah. I mean, this is, this is another one of these things. I, I mean, again, all my answers to these questions, a- and any specific thing I say is all just kind of like... Directionally, this is... We can kind of explore around this. I find this interesting. Maybe the li- I feel like the literature points in these directions in some very broad way. What I actually want to do is, like, go and map the entire mouse brain and, like, figure this out comprehensively and, like, make neuroscience the ground truth science. So I don't know, basically. (laughs) Um, but, uh, but yeah, I mean, there... So first of all, I mean, I think with Ilya on the podcast, I mean, he was like, "It's weird that we don't use value functions," right?

   6. DPDwarkesh Patel43:33

      Right.

   7. AMAdam Marblestone43:33

      We use like the most dumbest form of RL based-

   8. DPDwarkesh Patel43:36

      Yeah.

   9. AMAdam Marblestone43:36

      And of course, there are... These people are incredibly smart, and they're optimizing for how to do it on GPUs, and it's really incredible what they're achieving. But, like, conceptually, it's a really dumb form of RL, even compared to, like, what was being done in, like, 10 years ago, right? Like, even, uh, you know, the Atari game-playing stuff, right, was using, like, Q-learning, which is basically, like... It's a kind of temporal difference learning, right?

   10. DPDwarkesh Patel43:56

       Yep.

   11. AMAdam Marblestone43:56

       And the temporal difference learning basically means you have some kind of a value function of, like, g- what action I choose now doesn't just tell me literally what happens immediately after this. It tells me, like, what is the long-run consequence of that for my expected, you know, total reward or something like that. Um, and so you would have value functions like... The fact that we don't have, like, value functions at all is like, in the LLMs, is like... It's crazy. It, I mean, I think it, I think because Ilya said it, I, I can say it, give him... I know, you know, one, 1/100th of what he does about AI, but, like, it's kind of crazy that this is working. (laughs)

   12. DPDwarkesh Patel44:32

       Yeah. (laughs)

   13. AMAdam Marblestone44:33

       Um, but, uh, yeah, I mean, in terms of the brain, um, well, so I think there are some parts of the brain that are thought to do something that's very much like model-free RL, that sort of parts of the basal ganglia, um, sort of striatum and basal ganglia. They have, like, a, a certain finite, like... It is thought that they have a certain, like, finite relatively small action space, and the types of actions they could take, first of all, might be like, tell the spinal cord, or tell the brainstem and spinal cord to do this motor action: yes/no. Um, or it might be more complicated cognitive-type actions, like tell the thalamus to allow this part of the cortex to talk to this other part, or release the memory of this in the hippocampus and start a new one or something, right? There, there's... But there's some s- finite set of actions that kind of come out of the basal ganglia, and that is just a very simple RL. So, there are probably parts of other brains and our brain that are just like doing very simple, naïve-type RL algorithms. Um, layer one thing on top of that is that some of the major work in neuroscience, like Peter Dayan's work and a bunch, bunch of work that is part of why I think DeepMind did the temporal difference learning stuff in the first place, um, is they were very interested in neuroscience. Um, and there's a lot of neuroscience evidence that the dopamine is giving this reward prediction error signal, um, rather than just reward yes/no-

   14. DPDwarkesh Patel45:57

       Right.

   15. AMAdam Marblestone45:57

       ... you know, a gazillion time steps in the future. It's a prediction error. Um, and that's consistent with, like, learning these value functions. Um, so there's that, and then there's maybe, like, higher order stuff. So, we have these cortex making this world model. Well, one of the things the cortex world model can contain is a model of when you do and don't get rewards, right? Again, it's predicting what the steering subsystem will do. It could be predicting what the basal ganglia will do. And so you have a model in your cortex that has more generalization and more concepts and all this stuff that says, "Okay, these types of plans, these types of actions will lead, in these types of circumstances, to rewards." So, I have a model of my reward. Um, some people also think that you can go the other way. And so this is part of the inference picture. There's this idea of RL as inference. Um, you could say, "Well, conditional on my having a high reward, sample a plan that I would have had to get there." That's inference of-

   16. DPDwarkesh Patel46:54

       Right.

   17. AMAdam Marblestone46:54

       ... the plan part from the reward part. I'm clamping the reward as high and inferring-

   18. DPDwarkesh Patel46:58

       Yeah.

   19. AMAdam Marblestone46:59

       ... the plan sampling from plans that could lead to that. Um, and so if you have this very general cortical thing, it can just do... If you have this, like, general, very general model-based system and the model, among other things, includes plans and rewards, then you just get it for free, basically. Yeah.

   20. DPDwarkesh Patel47:14

       I see. So, like, uh, in neural net- network l- uh, parlance, there's a value head associated to the, the, the omni-directional inference that's happening in the brain.

   21. AMAdam Marblestone47:23

       Yes, yeah.

   22. DPDwarkesh Patel47:24

       I can see.

   23. AMAdam Marblestone47:24

       Or there's a value input. Um, yeah.

   24. DPDwarkesh Patel47:26

       Oh, okay. Interesting.

   25. AMAdam Marblestone47:27

       Yeah. And it can, and it can predict... One of, one of the, one of the almost sensory variables it can predict is, is what rewards it's going to get.

   26. DPDwarkesh Patel47:34

       Yeah. By, by the way, speaking of this thing about amortizing things, um, yeah, obviously value is like amortized rollouts-

   27. AMAdam Marblestone47:44

       Right. Yeah.

   28. DPDwarkesh Patel47:44

       ... of looking up reward.

   29. AMAdam Marblestone47:46

       Yeah. Something like that. Yeah.

   30. DPDwarkesh Patel47:48

       Yeah.
4. 50:31 – 1:03:59

   Is biological hardware a limitation or an advantage?

   1. DPDwarkesh Patel50:31

      Stepping back, how, um... is it a disadvantage or an advantage for humans that we get to use biological hardware, in comparison to computers as they exist now? So let, uh, by, what I mean by this question is like, if there's the algorithm, would the algorithm just qualitatively perform much worse or much better if, um, inscribed in the hardware of today? And the reason to think it might... like, here's what I mean, like, you know, obviously the brain has had to make a t- bunch of trade-offs which are not relevant to computing hardware. It has to be much more energetically efficient. Maybe as a result, it has to learn, uh, run on slower speeds, so that there can be-

   2. AMAdam Marblestone51:09

      Right.

   3. DPDwarkesh Patel51:09

      ... a smaller voltage gap, and so the brain just runs at 200 hertz.

   4. AMAdam Marblestone51:13

      Right.

   5. DPDwarkesh Patel51:13

      Um, and has to like run on 20 watts. On the other hand, may- you know, with like robotics, we have clearly experienced that fingers are way more nimble than we can make motors so far.

   6. AMAdam Marblestone51:23

      Yeah.

   7. DPDwarkesh Patel51:23

      And so maybe there's something in the brain that is the equivalent of like cognitive, uh, dexterity-

   8. AMAdam Marblestone51:27

      Right.

   9. DPDwarkesh Patel51:27

      ... which is like, maybe due to the fact that we can do unstructured sparsity, we can co-locate the memory and the compute.

   10. AMAdam Marblestone51:32

       Yes.

   11. DPDwarkesh Patel51:33

       Where does this all net out? Are you like, "Fuck, we would be so much-"

   12. AMAdam Marblestone51:36

       (laughs)

   13. DPDwarkesh Patel51:36

       "... smarter if we didn't have to deal with these brains," or are you like, "Oh..."?

   14. AMAdam Marblestone51:38

       I mean, I think in the end we will get the best of both worlds-

   15. DPDwarkesh Patel51:41

       Right.

   16. AMAdam Marblestone51:41

       ... somehow, right? I think th- I think an obvious downside of the brain is it cannot be copied. (laughs)

   17. DPDwarkesh Patel51:45

       Yeah.

   18. AMAdam Marblestone51:45

       You don't have, you know, external read-write access to every neuron and synapse.

   19. DPDwarkesh Patel51:50

       Yeah.

   20. AMAdam Marblestone51:50

       Whereas you do, I can just edit something in the weight matrix-

   21. DPDwarkesh Patel51:52

       Right.

   22. AMAdam Marblestone51:53

       ... you know, in Python or whatever, (laughs) uh, it, you know, and- and load that up and copy that, um, in principle, right? Um, so the fact that it can't be copied and kind of random accessed is like very annoying. But otherwise, maybe these are, it like has a lot of advantages. So, or, and it also tells you that you want to like somehow do the co-design of the algorithm and the... it maybe that that even doesn't change it that much from all of what we discussed, but you want to somehow do this co-design. So, um, yeah, how do you do it with really slow, low-voltage switches? That's going to be really important for the energy consumption, the co-locating memory and compute. So like I- I think that probably just like hardware companies will try to co-locate memory and compute. They will try to use lower voltages, allow some stochastic stuff. There are some people that think that this like, all this probabilistic stuff that we were talking about, oh, oh, it's actually energy-based models and so on, is doing lo- it is doing lots of sampling. It's not just amortizing everything. That the neurons are also very natural for that, because they're naturally stochastic, and so you don't have to do a random number generator-

   23. DPDwarkesh Patel52:58

       Right.

   24. AMAdam Marblestone52:58

       ... and a bunch of Python code basically to generate a sample. The neuron just generates samples, and it can tune what the different probabilities are.

   25. DPDwarkesh Patel53:06

       Yeah.

   26. AMAdam Marblestone53:06

       And so, and like learn- learn those tunings.

   27. DPDwarkesh Patel53:10

       Yeah.

   28. AMAdam Marblestone53:10

       And so it could be that it's very co-designed with like some kind of inference method or something, yeah.

   29. DPDwarkesh Patel53:16

       I- it'd be hilarious, I mean the- the (...) these interviews like, you know all these people that folks make fun of on Twitter, you know, Yann LeCun, Yann LeCun and Beth Jezos and whatever-

   30. AMAdam Marblestone53:25

       They kind of...
5. 1:03:59 – 1:23:28

   Why we need to map the human brain

   1. AMAdam Marblestone1:03:59

   2. DPDwarkesh Patel1:03:59

      So I- l- let me ask you about this. Uh, you know, you- you guys are f- finding different groups that are trying to-

   3. AMAdam Marblestone1:04:05

      Yeah.

   4. DPDwarkesh Patel1:04:06

      ... figure out what's up in the brain. If we had a perfect representation, however you define it, of the brain, why think it would actually f- let us figure out the answer to these questions? We have neural networks which are way more interpretable, not just because we understand what's in the weight matrices, but because there are weight matrices. There are these boxes with numbers in them.

   5. AMAdam Marblestone1:04:26

      Right.

   6. DPDwarkesh Patel1:04:27

      And even then, we can tell very basic things. We can kind of see circuits for-

   7. AMAdam Marblestone1:04:33

      Yeah.

   8. DPDwarkesh Patel1:04:33

      ... uh, very basic pattern matching or following one token with another.

   9. AMAdam Marblestone1:04:37

      Right.

   10. DPDwarkesh Patel1:04:38

       I- I- I- I feel like we ha- we don't really have an explanation of why LLMs are intelligent, just because they're-

   11. AMAdam Marblestone1:04:42

       Yeah. Well, I would- I would somewhat-

   12. DPDwarkesh Patel1:04:43

       ... y- yeah, they're interpretable.

   13. AMAdam Marblestone1:04:43

       ... I would somewhat dispute it. I think we have some architectural... We have some description of what the LLM is, like, fundamentally doing. And what that's doing is that I have an architecture and I have a learning rule and I have hyperparameters and I have initialization and I have training data.

   14. DPDwarkesh Patel1:04:57

       But tha- th- those are things we learn from-

   15. AMAdam Marblestone1:04:59

       Yeah.

   16. DPDwarkesh Patel1:04:59

       ... because we built them, not because we interpreted them-

   17. AMAdam Marblestone1:05:01

       Yes, yes, yes, yes.

   18. DPDwarkesh Patel1:05:01

       ... from seeing the weights.

   19. AMAdam Marblestone1:05:02

       We built them.

   20. DPDwarkesh Patel1:05:03

       Which is the ano- the analogous thing to connectome, is like seeing the weights.

   21. AMAdam Marblestone1:05:05

       What I think we should do is we should describe the brain more in that language, of things like architectures, learning rules, initializations, rather than trying to find the Golden Gate Bridge circuit and saying exactly how does this neuron-

   22. DPDwarkesh Patel1:05:15

       Hmm.

   23. AMAdam Marblestone1:05:15

       ... actually, you know... That's gonna be some incredibly complicated learned pattern. Um, yeah, Konrad Kording and Tim Lillicrap have this paper from f- while ago, maybe five years ago, called, "What Does It Mean To Understand A Neural Network?" Or, "What Would It Mean To Understand A Neural Network?" Um, and what they say is, yeah, basically that. Like, you could imagine you train a neural network to like, compute the digits of pi or something. Or like some crazy... You know, it's like, it's like this crazy pattern. And then you also train that thing to, like, predict the most complicated thing you find, predict stock prices, equi- basically predict the really complex systems, right? Computational, you know, computationally complete systems. I could predict... I could train a neural network to d- do cellular automata or whatever crazy thing. And it's like we're never gonna be able to fully capture that with interpretability, I think. It's just gonna just be doing really complicated computations internally. But we can still say that the way it got that way is that it had an architecture and we gave it this training data (laughs) and it had this loss function. And so I want to describe the brain in the same way. And I think that this framework that I've been kind of laying out is like, we under- need to understand the cortex and how it embodies a learning algorithm. I don't need to understand how it computes Golden Gate Bridge, right?

   24. DPDwarkesh Patel1:06:19

       But if you, if you can see all the neurons, if you have the connectome-

   25. AMAdam Marblestone1:06:21

       Yeah.

   26. DPDwarkesh Patel1:06:21

       ... why does that teach you what the learning algorithm is?

   27. AMAdam Marblestone1:06:25

       Well, I guess there are a couple different views of it. So it depends on these different parts of this portfolio. So on the totally bottom-up, we have to simulate everything portfolio, it kinda just doesn't. You have to just, like, see what are the... You have to make a simulation of the zebrafish brain or something.

   28. DPDwarkesh Patel1:06:38

       Yeah.

   29. AMAdam Marblestone1:06:38

       And then you, like, see what are the, like, emergent dynamics in this, and you come up with new names and new concepts and all that. That's like, that's like the most extreme bottom-up neuroscience view. Um, but even there, the connectome is, like, really important for doing that botta- biophysical or bottom-up simulation. Um, but then on the other hand, you can say, "Well, what if we can actually apply some ideas from AI?" We basically need to figure out, is it an energy-based model or is it, you know, an amortized, you know, VAE-type model? You know, is it doing backprop or is it doing something else? Um, are the learning rules local or global? I mean-If we have some repertoire of possible ideas about this, can we... Just think of the connectome as a huge number of additional constraints that will help to refine to ultimately have a consistent picture of that. I think about this for the, the steering subsystem stuff too, just very basic things about it. How many different types of dopamine signal, or of steering subsystem signal, or thought assessor, or so on? How many different types of what broad categories are there? Like even this very basic information that there's more cell types in the hypothalamus than there are in the cortex, like that's new information, right?

   30. DPDwarkesh Patel1:07:45

       Mm-hmm.

Episode duration: 1:49:53