Modern WisdomWhy Is Behavioural Genetics A Hated Science? - Dr Stuart Ritchie
EVERY SPOKEN WORD
150 min read · 30,063 words- 0:00 – 0:25
Intro
- SRStuart Ritchie
Candidate gene research. This particular gene is related to depression. This particular gene is related to memory skills and so it makes you smarter. Endless research on that. Loads and loads and loads of papers published across all the top journals. Millions of dollars of research funding. People basing their entire careers, writing their PhD dissertations, doing job talks, getting employed at top universities on the basis of this candidate gene research. And it was all nonsense.
- 0:25 – 11:10
Why People Distrust Behavioural Genetics
- SRStuart Ritchie
(air whooshing)
- CWChris Williamson
Given your current academic background at the moment, can you explain to people why you think it is that behavioral genetics has so much distaste, distrust, dislike, generally?
- SRStuart Ritchie
There's a lot of different reasons. I think, uh, the main reason is some kind of a, a misconception about what it actually means to say that behavioral traits are related to genetics. So, uh, I think when people hear that, especially when the trait itself is controversial, like you can get into a whole debate about intelligence or personality without even mentioning genetics. You know, pe- people, people g- get upset by just, you know, mentioning those, those traits. But when you say they're linked to genetics, people make lots of assumptions. People think they know your politics. They think they know, like, what you, what you're trying to say, what you're trying to, like, slip under the, you know, uh, uh, under people's, uh, under, under the radar, under people's notice. Um, and those assumptions are things like, well, if it's related to genetics, it must be completely unchangeable. Uh, if it's unchangeable, then we don't need to do anything about our political situation, uh, uh, and we don't need to, you know, help people out. Uh, and people are just stuck like a tram on a tram line. They can't, they can't turn off or change or, or anything like that. And, uh, uh, you, you... If, if you're interested in behavior genetics, you must be using it to justify our current political situation. So, I think, I think that, that, like, that's one of the major assumptions, like immutability, the, the, the fear of immutability. Um, uh, uh, and that's of course not what behavior genetics says at all. Behavior genetics is about trying to understand, you know, how things are right now, not necessarily how things might be if we change things in the future, um, uh, or indeed they, i- if they may have been different in, in, in the past. We have lots of interesting studies of how, like, the genetic contribution to things, uh, uh, differs across different times and different places and different political regimes even. There's some interesting research on communist regimes and how that might have affected the, uh, uh, the, the, the heritability of traits. We can, we can talk-
- CWChris Williamson
How so? Yeah, t- tell us about that.
- SRStuart Ritchie
Well, there's some research, uh, in, uh, Estonia, which obviously used to be a, a communist, uh, uh, country. And, um, uh... So yeah, this is a, this is a research using a polygenic score, which, uh, your viewers may be familiar with from, uh, hearing your interview with Robert Plomin, uh, one of my, one of my colleagues, uh, here at King's College London. Um, uh, and so the idea is that you look at the genetic contribution to various traits and educational attainment being the, being the main one, um, before and after communism in Estonia. So that was, that was... It's just this incredibly cool paper that, that was done. Um, would be nice if it was replicated and stuff, but this is, you know, it's an interesting, it's an interesting, uh, uh, uh-
- CWChris Williamson
But you need more communist regimes to come and go-
- SRStuart Ritchie
Well, yeah.
- CWChris Williamson
... in order for that to happen. It's gonna be difficult.
- SRStuart Ritchie
It's, it's... Well, you need a few, you need... You certainly, you need to do that same study in lots of other, like, Warsaw Pact countries, uh, which are now no longer communism. So, but the idea is that in people who were born, uh, a- after, uh, communism was gone, genetics explains more of the variation in, uh, people's educational attainment, for instance. So like, and the, the idea is like the broad interpretation or one interpretation you can draw from that is that a freer political system, one that doesn't, you know, uh, oppress people in the same way that communism does, um, allows them to kind of, uh, uh, reach their genetic potential is one, is one way you might want to put it, um, uh, more effectively than one which kind of where the environment really was kind of suppressing people's ability to kind of be who they, w- who they really were. So, that's, that's one interesting, uh, uh, I think. But that... But, you know, in, in just saying that, you can see how the environment makes a difference to how genetics operates. Uh, and that's, you know, research that's done by, you know, behavior geneticists who are not, you know, criticizing polygenic scores. They're saying, "Look, we can use polygenic scores to illustrate interesting things about our society and how society changes," and so on. So, so I think it's a real misconception to say that, um, to say that, uh, uh, you know, genetics means that we're i- immutable in, in, in, i- i- in some way. I mean, the classic thing people talk about, the, the kind of, um, the kind of almost boring, cliched, uh, thing that people talk about is, is glasses, right? Is, is, is, um, uh, uh, eyesight is really heritable, so myopia, shortsightedness, really, really heritable, um, uh, runs in families, all that sort of stuff. But, you know, we can, we can cure it instantly or, or effectively cure it instantly by just like putting on a pair of glasses with the right prescription. So things in the environment can, uh, um, completely alter the way that genetics has, has its effect. Now, there's a slight cop out to say that because we don't have the environmental equivalent of glasses for, uh, other traits like personality. Like we don't know the stuff that will massively instantly change your personality. Um, we don't know the things that will massively instantly change your intelligence or your educational attainment and all that yet. Maybe we will at some point in the future, but we don't have that right now. Um, maybe it's a good thing that we don't know ways to like massively alter people's personalities using the environment because of course that might be used by, uh, somewhat nefarious, uh, political regimes or cults or whatever else. I mean, cults are already pretty good at like controlling people's behavior and so on. And if they had ways of, uh, changing people's personalities more effectively, then that might be, uh, rather scary. So, um, uh, yeah, but that's, but that's, you know, one example that's very often used. Um, another one is height. Like height is really heritable. We know that tall parents tend to have tall kids. Uh, short parents tend to have shorter kids on average. It's how it goes. And yet, if you don't feed a kid while they're growing up-... uh, they'll be stunted. People in North Korea are like six inches shorter than people in, or, uh, four inches, whatever it is, shorter than people in South Korea because of all the mal- malnutrition.
- CWChris Williamson
No one's criticizing the fact that behavioral genetics suggests... That, that's the mechanism by which tall parents have tall kids, right? Well, not behavioral genetics, I guess, but just heritability generally, right?
- SRStuart Ritchie
Yeah.
- CWChris Williamson
That height is heritable. What is it about other elements that causes people... A shoe size or eye color-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... wha- wha- what is it about that that causes it to be such a different, uh, ballpark?
- SRStuart Ritchie
Well, you've put your finger on the, the, the thing here, which is the double standard, which is people are very, very happy to talk about, um, uh, you know, genetic influences on stuff like height or eye color and, and things like that, that don't have a political valence. But as soon as it, you know, you use those exact same methods, and that's the important thing, like, you use the same methods, studying twins, studying families, studying molecular, you know, DNA differences between people, you use those exact same traits to study stuff like intelligence, personality, whatever, and people flip out. And that's, um, uh, uh, I think a weird double standard. I mean, what, what would, what would one of, what would one of them say in response to that? They would say, "Well, those traits are a lot more complicated. They're, they're, they're socially influenced." Things like educational attainment is influenced by all sorts of stuff, the quality of the school you went to, um, uh, uh, the socioeconomic situation that you find yourself growing up in.
- CWChris Williamson
Just to jump in there-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... didn't Plomin find that when you, uh, equate for everything else, schools on their own have less than 5% of an impact on someone's educational outcomes?
- SRStuart Ritchie
That's the... Yeah, he used the Ofsted ratings of-
- CWChris Williamson
Yes.
- SRStuart Ritchie
... of, of schools to show that. Yeah, I think, I think probably in the UK that's probably true. I think in other countries that might, that might differ slightly, like I think in countries where the schools really, really dramatically differ-
- CWChris Williamson
Mm-hmm.
- SRStuart Ritchie
... uh, uh, uh, we- we've done a pretty good job of like equalizing because of things like Ofsted inspections of fair- you know, fairly, uh-
- CWChris Williamson
For the American people that are listening, Ofsted is a external accreditation board that comes in and makes sure that the school's doing everything right and that-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... the students are getting the appropriate... And we also have a much more, uh, homogenous teaching, uh, curriculum, right? Everybody across the entire, maybe UK, but certainly England, has one set of exams at this age, at this age, at this age, th- unless you're in some weird, like, international business school-y type thing.
- SRStuart Ritchie
Yeah. Yeah.
- CWChris Williamson
Pretty much everybody has the same stuff. So you have this
- SRStuart Ritchie
It is a bit different in... It is a bit different in Scotland where I'm from. They have the... In Scotland they have the Curriculum for Excellence, which I believe is not all that excellent.
- CWChris Williamson
(laughs)
- 11:10 – 19:35
Creating an Uneven Playing Field
- SRStuart Ritchie
- CWChris Williamson
What, what... So tell me about where this is coming from, because it seems like... Uh, here's my, here's the bro science explanation, right? That coming into talking about behavioral genetics, group differences and individual differences got lumped in together. Group differences got used by people that have some pretty nasty ideas about better and worse types of individuals and that has caused downstream for behavioral genetics to kind of be lumped in with quasi-racists. That's one reason. Another reason is we live in a meritocracy. And in a meritocracy if you are your successes and you own your failures, someone that is told that you have a predisposition toward being more successful ahead of arriving into the world, given that success and your achievements in life are one of the fundamental things that you take your status and your wellbeing and your sense of-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... everything from, that also feels very unfair, you know. In a world that's trying to give people equality of opportunity, realizing that people start, like literally start the race at very, very varying levels-... is kind of a hard thing to bring in here. What do you want us to do? Do you want us to try and normalize for genetic predisposition before people get to school?
- SRStuart Ritchie
(laughs)
- CWChris Williamson
That, that doesn't seem very fair. Okay, what happens if we completely flatten the environment so that everyone gets the same? Okay, so what you're saying then is that the only differences in outcome are going to be exclusively genetic. Well, God-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... that doesn't feel very fair either.
- SRStuart Ritchie
(laughs) Yeah, yeah, absolutely. Um, uh, and the, the, the funny thing is that I think people, you know, this is something which the average person understands very, very well. There's that paper that showed that, you know, uh, uh, the people who get, who are, who are the most accurate at predicting, uh, the heritability of different traits, so, like, how heritable things are, obesity, height, intelligence, whatever, um, the people who are most accurate, uh, uh, uh, at saying, "Yeah, genetics are probably involved in that," are mothers with more than one child, right? So, there's this idea that if you've got one child, then, you know, that's interesting, but if you've got another one, you can see, oh, right, I'm not parenting this kid particularly differently. I'm not doing anything massively different and yet this child is totally different. And to be honest, if you know anyone who, if you know brothers or sisters or, you know, whatever, siblings of any, of any kind, like, you know that they, they differ dramatically in their personality. And it's probably not that they were, like, expressly parented in a particularly different way. There are genetics that influence that. So, we all, we all realize that. Some, some kids are, you know, starting school with slightly higher aptitude for, you know, sitting down and concentrating and, and some are, some are not. But the thing is, the mistake is, is to, is to assume that there's only one political outcome from that, there's only one political interpretation of that. Because the kind of more liberal interpretation of that is, well, if people start off in different things, then you have to think in terms of John Rawls, like the, um, the veil of ignorance. And this is what Paige Harden talked about in her book, the veil of ignorance, which is, you know, if you didn't know what traits you were gonna have, uh, uh, you could be any, you could be entered into the genetic lottery, as, as Paige puts it, and, uh, and, and, and start your life a- as one of many people. How should we set up the world such that you're, you know, uh, that the world is as fair as, as, as, as it possibly could be to whatever you might be like? Um, and so that implies a lot of leveling off. That implies a lot of, you know, uh, extra resources for kids who are struggling, uh, making sure that things are, are as equal as possible, making sure the opportunities are, are, are given to kids who maybe wouldn't otherwise get them. So, you could draw a very, you know, progressive, liberal conclusion from, from that just as much as you could draw the conclusion of, oh, well, we shouldn't bother helping people because, you know, uh, genetics is, is, is having an effect.
- CWChris Williamson
Yeah, it does seem like, uh, an incredibly illiberal way to view the situation that that evidence over there that could assist us in helping people to get themselves to the kind of life and the kind of world that they want to have, we, we, we shouldn't know about that. Well, hang on. It, whether you decide to believe in it or not doesn't mitigate the impact of it on those people's lives.
- SRStuart Ritchie
Yeah.
- CWChris Williamson
And I think, I, I had a great conversation with Paige, and I appreciated the fact that she engaged with, given that she's very from the left, an incredibly difficult circle to square, right? It is very hard to work out how you combine left-leaning politics with a, a deep understanding of behavioral genetics and the impact that they seem to have on people's outcomes in life.
- SRStuart Ritchie
Yeah, totally. And, and, and, uh, that's, you know, one, one, you know, we must admit as behavior geneticists that one reason that people have drawn the kind of more right-leaning conclusion and are, and, and fear the right-leaning conclusion is that there are lots of figures in history who, who have drawn that conclusion, right? But if you go right to the very start of when people were kind of inventing intelligence tests and, and so on, um, this was very much the idea. So, so, uh, Godfrey Thompson, who was this kind of very well-known intelligence researcher at the start of the 20th century, uh, at University of Edinburgh, there's a, um, there's a, a, a building named after him in the University of Edinburgh Education School, where, by the way, they probably don't teach that much about, like, IQ and, and so on anymore, but it's somewhat ironic that it's in, it's in a building named after him, but fair enough. Um, uh, he was of the opinion that, and he didn't do so much genetic stuff, but, um, he was of the opinion that, you know, intelligence differences that, that, that kids have imply that we should spend exactly the same on every single child. So, so like, um, uh, uh, uh, you know, no matter whether they're struggling or doing really well, every single child should get the same amount of resources from, from, from the state. He wanted things to be as equal as possible. And yet he was one of the pioneering intelligence researchers and made major contributions to our understanding of, of, uh, of exactly what intelligence is and what the G factor is, the general factor of intelligence, all this kind of stuff. Um, and, and, you know, the, the, the general, you know, uh, influence that he and other people around him at the time had on the education system was that, you know, there are all these people out there who, um, uh, are, you know, would not otherwise be noticed by the good schools because of course at that point, getting into a good school meant knowing people, um, your parents being rich and all that sort of stuff. Um, and so we should try and have an objective way of doing that, and they, that's why they influenced the Butler Education Act in the 1940s and, and we, that's how we got the grammar school system that we had in the UK for a long time. Now, the grammar school system turned out to be n- not particularly effective because the people who went to grammar school had a great time, but the people who didn't pass the 11 plus test, which was the, the test that you, the IQ test that you did, um, ended up in secondary moderns, which were very, very poorly resourced. Um, and there's all these like hellish stories of, you know, the latter half of the 20th century that people went to secretary moderns and had a horrible time. And I can totally understand these crumbling buildings and all this kind of stuff. Um, but, but that was not the intention. Like, the intention of the psychologist who set this up was a kind of, was a kind of noble one, which was that, you know, we, we were gonna try and find...... smart kids from poor backgrounds and give them, uh, uh, you know, a, a, an academic curriculum that was, that is appropriate to them. Not, not give them more resources or anything, just give them an academic curriculum that includes stuff that's more appropriate to them, and give, uh, uh, an academic curriculum to people who are, are, you know, are, are not as academically inclined. So, um, that was the original int- intention, um, you know, subsequent to that, there came lots of people who did in fact believe in, you know, eugenic theories and, and, and a lot of all the, the, the stuff that we now hear IQ tests, you know, associated with. But, you know, the original IQ tests were, you know, uh, uh, made for noble reasons. The very, very, very first one, Alfred Binet in the 1910s, uh, was invented to help kids who were struggling in school. Uh, you know, kids who maybe, um, had particular, you know, what we would now call, you know, special educational needs, um, w- was invited to have an objective way of diagnosing them and, and giving them extra, uh, attention and help. So, like, the, the kind of, um, if you believe this stuff, therefore you must believe in, like, predestination and im- immutability of traits, and you must be, you know, have these, these, these right-wing, uh, uh, views on things, it's just completely ... It's, it's ahistorical, and it's, uh, uh, and it doesn't follow from the science.
- CWChris Williamson
Am I right in
- 19:35 – 22:09
Evidence of IQ Heritability
- CWChris Williamson
saying that the best evidence we have at the moment is that IQ correlates .8 by your 60s or something like that with your parents, about 80, 80%? Is that right?
- SRStuart Ritchie
Uh, well, the, the heritability ... So, there's this weird, there's this weird, uh, result where the heritability, uh, uh, goes up over time.
- CWChris Williamson
Yes.
- SRStuart Ritchie
So, like, environment seems to have more of an influence at the start, uh, of your life, and then, uh, it gets less important in, in, in measuring, uh, uh, you know, the, the, its impact on intelligence. And then yeah, it goes up to ... The heritability, that is, the percentage of the, the, uh, variance and the trait that is associated with genetic differences, yeah, it's, it's 60, 70, 80% by the time you do it. And older folks, you know, there's studies like, um, the VETS, uh, the Vietnam Experience Twin Study, where you've got all these twins who were in ... They happened to be in Vietnam, but that's not the most important, uh, part of it. But you've got all these twins who, uh, were there, and they did an IQ test, and, uh, uh, uh, and so on. So, um, they've got a younger IQ test and an older IQ test, um, one that they did when they, you know, when they went to Vietnam and they were like 20 years old, and one that they did when they were later in life. And you can see that the heritability, uh, increases over, over, over time.
- CWChris Williamson
Yeah, well, I mean, that, that's fascinating. The fact that we have so much of ourselves that is a part of our parents is, it's kind of something that's beautiful. But yeah, I think the, the fact that Plomin said, uh, "It does not predetermine, but it does predispose," was, was how he put it to me.
- SRStuart Ritchie
Yeah, I ... (laughs) Robert has some of these phrases like, like that, and like, um, "It makes a, it, uh, it makes a differe- It, it matters, but it doesn't make a difference," or something. It's often like that, his phr- (laughs) It's just like-
- CWChris Williamson
Yeah. Well, he, d- d- so-
- SRStuart Ritchie
Is that the best way to express that? I don't know.
- CWChris Williamson
I'm not sure, but there was a tweet that I put out that I got a lot of pushback for when I first had Plomin on-
- SRStuart Ritchie
Right.
- CWChris Williamson
... and it was a quote from him, from the episode.
- SRStuart Ritchie
Yeah.
- CWChris Williamson
And he said something along the lines of, "The single most important thing that you can do for your child's future happiness, educational outcomes, and income, is the partner that you choose to have them with."
- SRStuart Ritchie
Yeah. (laughs) Yeah. It's, uh, uh, I can see why people would be upset by, by, by that. But, like, that's one of the things that we know will have, will make a difference. Um, uh, uh, this- other stuff in the environment, like interventions, are, we're much less certain about. Um, y- you know, just, ch- you know, for instance, things like, uh, breastfeeding. Does that im- does that improve people's intelligence? Like, that's a big, big thing. Uh, does that improve-
- CWChris Williamson
Natal nutrition.
- SRStuart Ritchie
Yeah, e- exactly. Um, all sorts of things like that. Like, we just don't know. The studies are not that good on, on that. Whereas the studies are pretty reliable in knowing that, you know, genetics has an effect on, on people's
- 22:09 – 31:33
Impact of Replication Crisis on Behavioural Genetics
- SRStuart Ritchie
outcomes.
- CWChris Williamson
That's a good question. There has been this big replication crisis everywhere at the moment-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... and it feels like studies that were often used either bro-scientifically or really scientifically are being thrown out. How much has behavioral genetics been axed by this recent replication crisis?
- SRStuart Ritchie
Well, um, I think behavior genetics ... I, I wrote about this in my most recent book, Science Fictions, which is about the replication crisis exactly, and, um, I, uh, I think behavior genetics was one of the very first fields to be completely smashed by the replication crisis in one of its respects. So, in behavior genetics, you've got lots of different, you've got lots of different types of research. So you've got research with twins, you know, do, uh, the difference between, you know, uh, uh, identical twins and fraternal twins and, like, inferring things from th- from that about how genetic particular traits are, so that's one thing. But then you've got the molecular stuff. And when I was a PhD student, so like 10 years ago, um, all the way through, um, that we had candidate gene research. So it was like, "This particular gene that we have a theory about, um, is related to depression," for instance. Or sometimes more complicated, like, "This gene, and if you get abused as a child, it's gonna relate to depression." Um, "This particular gene is related to memory skills, and so it makes you smarter if you, if you, if you have this particular variant of, of this gene." Um, and endless research on that, loads and loads and loads of papers published across all the top journals, uh, and many of the non-top journals. Uh, millions of dollars of, of research funding, people in- buil- basing their entire careers, writing their PhD dissertations, you know, um, doing job talks, getting employed at top universities on the basis of this candidate gene research. And it was all, asterisk, not all, but like 99%, eh, eh, nonsense. Uh, it was all unreplicable, um, almost all unreplicable. There, there's like one example I can think of that, uh, that didn't fail when people s- thought, "Wait a second, should we try, like, replicating these candidate gene results in big studies?" So not just like 100 people, but like several thousand people to see if this particular genetic variant is linked to, you know, variation in memory and so on. And these were big effects. Like, there were, there were studies published in some of the top journals that were like-... 20% of the variation in people's memory abilities are explained by this one, this one, uh, genetic variant. Um, looking back, it seems ridiculous, but at the time, that was very much, you know, the, the, the kind of standard study that you would do. Um, it all fell to bits. Um, uh, people tried to replicate it. They couldn't find that those genes were in fact related to the, uh, the, the disorders and the traits that were, uh, uh, related to them. Um, and there was a massive replication crisis. And now we've moved on. Instead of doing these candidate gene studies, we've moved on to genome-wide association studies, which in- instead of looking at, like, one genetic variant, you look at literally hundreds of thousands of genetic variants. And it turns out that for things that are complicated, even something like height, but also, you know, things like educational attainment and so on, it's not that there's one gene that, like, has a big effect, it's that there's tens of thousands of, of genes that each have this little infinitesimal effect and it all builds up. So, you know, you might have a difference here and I might have a difference here and, and there's so many, many, you know, many, many differences that we have that make me slightly taller or slightly shorter than you and/or, or whatever it is. Um, and, uh, uh, that's now how we think about things. And, and, and in those genome-wide association studies, we are start finding, like, replicable evidence now. But, like, behavior genetics was, was completely, you know, trashed by, by the replication crisis. Um, you know, in the early 2000s, kind of before we started even talking about the replication crisis in psychology, this was, this was sort of happening. And, you know, the one that I can think of that, that, that hasn't, you know, f- failed entirely is the ApoE gene, the apolipoprotein, uh, E allele, which is if you've got one allele of this, I think you've got, like, uh, uh, substantially higher, like, twice the risk of getting Alzheimer's disease. If you've got two alleles of this, something like ten times the risk. And that comes out every, every time for Alzheimer's disease. That's, like, the gene that we know about-
- CWChris Williamson
Pretty robust.
- SRStuart Ritchie
... that is related to Alzheimer's disease. Yeah. I've got one of them. I did my 23andMe. I've got one of them, which is, you know, enjoy me while you can-
- CWChris Williamson
(laughs)
- SRStuart Ritchie
... because, uh, it's all my-
- CWChris Williamson
Right.
- SRStuart Ritchie
It's all my, it's all my go. But, but, but, you know, n- we... Behavior genetics has made substantial reforms and we don't do that sort of research anymore, or mostly. I mean, you see the odd paper coming up. Like I get asked to review the odd candidate gene study and, like, the reviews can, can, can just be like, "This will not replicate."
- CWChris Williamson
Bollocks.
- SRStuart Ritchie
"There's no way this will replicate." It's just, it has to be bollocks. And, um, uh, so yeah, as I say, the stuff we've moved onto, which is the kind of world of genome-wide association studies, polygenic scores, all that kind of stuff, is much more reliable in its basic associations. However, there's a whole bunch of other things, uh, other, like, concerns that are around that sort of research. So, um, and I don't mean ethical concerns and political concerns. I mean scientific concerns, so, like, how much does assortative mating, i.e. people having kids with people who are like them, uh, uh, bias our, uh, uh, estimates? How much does, um, uh, uh, not having diverse samples mean that we're not learning enough about actual, you know, humanity in general? So, like, the vast majority of this research has been done on people of, from a European background. Uh, you know, white American people or white people from Europe basically. Um, and so, you know, and when you try and use those polygenic scores, they don't predict as well when you look at people of, from other backgrounds. So, we need to do, we need to do a lot more research on, on, on, on that to really understand, you know, how these traits are, are made up. And, um, yeah. So, there's... And there's all sorts of other, you know, statistical and technical and all sorts of other concerns, uh, with that. But we are getting more basic, you know, replications done, whereas in the candidate gene research, we just went way ahead of, of where the science actually was and started saying, "Well, we found the gene for intelligence or the gene for memory, the gene for depression," all this sort of stuff, and it was completely wrong. And I think we have to constantly remember and have it at the back of our minds, or maybe even the front of our minds at all times, like, for decades, for maybe a decade, we were going completely on the wrong track. And e- and, and everyone was, you know... Maybe not everyone. There were, there were some people who were saying, "Well..." For instance, there were some people saying, "Didn't Ronald Fisher, the geneticist and statistician, write a paper in 1918 that said where we should expect really tiny effects of genes and not really big effects? Is- isn't this research kind of going against that?" Some people did point that out at the time, but nobody listened to them and we had a whole field spending endless money, taxpayers' money in many, in many, uh, cases, just wasting it on statistically invalid studies. So we should, we, we should always remember that. Um, and I think even areas that are now a bit more rigorous, like behavior genetics, um, need to, need to bear that in mind. Um, it's like the... I don't know if this is ahistorical, but you know when a, when a, um, Roman general came back and did a triumph in, in Rome, there was always, like, a slave standing on, on the back of the chariot saying like, "Remember, you will die." You know, uh, uh, in, in, in their ear. I don't... I mean, I think that might not be-
- CWChris Williamson
No, I brought, I brought that up about Marcus Aurelius the other day.
- SRStuart Ritchie
Oh, yeah.
- CWChris Williamson
That as he wa- as he walked through the streets of Rome and everyone-
- SRStuart Ritchie
Right.
- CWChris Williamson
... was hailing him as this philosopher god king and he was the, this benevolent leader, he would have someone behind him the whole time-
- SRStuart Ritchie
Yeah. Right.
- CWChris Williamson
... that would say, "You are only a man."
- SRStuart Ritchie
Right. Precisely. So, like, I think we need to constantly have that. Like, there was a replication crisis in your field, like, just a few years ago. Yeah. You know? So-
- CWChris Williamson
Okay, so that's, that's the genetic side of things or the, um, looking at the individual genes itself.
- SRStuart Ritchie
Yeah.
- CWChris Williamson
In terms of the behavioral genetics environment versus, uh, uh, gene side, heritability side, how robust has that been in terms of replicating?
- SRStuart Ritchie
I think the... You know, if you look back at the twin, the twin stuff, uh, and the family stuff, uh, so, you know, not using... Not actually trying to go in and, like, measure DNA differences but looking at family, uh, uh, differences, that stuff is much more robust, uh, much more rigorous. It's... Those studies are relatively straightforward to do in the sense that they were doing them, you know, early in the 20th century and, and so on. Interesting potential fraud case with, with one of the very famous studies on, on, uh, intelligence and, and, uh, and inheritability, um, Cyril Burt, if anyone wants to look up. There's some interesting case there. I'm actually, I'm actually halfway through writing an article about this and, and whether, you know, uh, Cyril Burt was a fraud, uh...... or, or, or not 'cause there are people that have, have questions, uh, questioned that. I think probably he was a fraud, but, um, but anyway, his results have been confirmed by subsequent research which is that, yeah, if you ask, you know, to what extent are, uh, traits like intelligence related to genetic differences between people, you'll robustly get, uh, uh, the answer. And then if you take a step back and just, you know, forget the genetics and ask things like, you know, is there a general factor of intelligence, like, i- i- i.e. are people who are good at one type of intelligence test, you know, do they tend to be good at every type of intelligence test? That's one of the most replicable results in psychology. Like, you get that every single time. Tests correlate positively together.
- CWChris Williamson
And yet, a lot of people would be very upset to hear that, unfortunately.
- SRStuart Ritchie
It's funny, isn't it? It's quite ex- quite ironic that people are, are upset by one of the most rigorous, uh, replicated findings. But it, but it's, it's because it, uh, you know, it's this thing I mentioned right at the start. People assume that when you say that, you must mean all this other stuff. It's like, no, no, no, when I say that, all I mean is that, that, that people who are good at one type of test tend to, tend to be
- 31:33 – 38:48
Is EQ Real?
- SRStuart Ritchie
good at all the other ones.
- CWChris Williamson
Are there multiple types of intelligences?
- SRStuart Ritchie
Hmm. Well, it depends on what you mean. So-
- CWChris Williamson
EQ. Is EQ a thing?
- SRStuart Ritchie
Well, there's, so there's a few things to say. Um, when you do an IQ test, you will notice that if you're doing an IQ test or whatever, you will notice that there are lots of different types of tests, right? So, you'll do a memory test, a vocabulary test, a speed test, a, um, uh, all sorts of, you know, spatial rotation tests, shape rotators, everyone was talking about that on Twitter by the way.
- CWChris Williamson
Word sells.
- SRStuart Ritchie
Um, yeah, w- word sells versus shape rotators.
- CWChris Williamson
Are you a word seller or shape rotator?
- SRStuart Ritchie
I think I might be a, I think I, well, I don't know. Uh, if I look at, like, people in the physics department, I definitely feel like a wordsal.
- CWChris Williamson
(laughs)
- SRStuart Ritchie
Um, but, but, um, but I, I, uh, the, you know, those types of tests could all be described as, like, you know, specific skills, right? If you look back at Charles Spearman in the 19- uh, early 1900s, 1910s who was the first person to kind of talk about general intelligence, we, we remember him very much for talking about general intelligence, but he also talked about specific skills too. So, um, the general factor of intelligence explains, like, 40% to 50% of the variation across all different tests. Like, that's one, that's the rigorous finding. That's what comes out. Um, but there are specific skills too. So, the, and what that means is, they can vary independently of the general intelligence. I mean, you know, one silly extreme example just to illustrate it would be, like, if you sat down every single day and, like, memorized the dictionary every single day, then your vocabulary would, you know, your specific skill of your vocabulary would improve, whereas that's not gonna help your, um, your, you know, uh, shape rotation or, or, or whatever. And in fact, there's a whole literature on, on, like, working memory training, you know, like, those brain training games that were quite a thing for a while? Uh, people did research on that for a long time because they, there was this initial finding published in a big journal, uh, uh, Proceedings of the National Academy of Sciences, that said that if you do that stuff, it improves your, your general ability. Um, and ev-
- CWChris Williamson
Turned out to be bollocks, didn't it?
- SRStuart Ritchie
Everyone was like, "Oh, my God, you know, we gotta, gotta try and replicate this." And then it didn't replicate. It felt, it felt a bit when people tried to do studies, they had better controls and all that sort of stuff. Um, but that was the idea, you know, that, that training at one task can transfer to other ones. That, that seems to, at least for the case of working memory, that seems not to be the case. Um, but yeah, so that's one thing to say is that there are all these different specific skills. They tend to correlate together, but they don't correlate together at, like, one, so they are different to some, to some degree. So, if you mean are there multiple intelligences in that respect, there are specific skills. That's the most obvious one. Then there's, is there this thing called multiple intelligences theory that was made up by Howard Gardner at Harvard University. Um, and when I say made up, I actually, you know, use that word advisedly because he didn't have any data. He just decided that there were these different types of intelligence, you know. Um, it's not, it's not the same as verbal, auditory, kinesthetic learning, which is another thing which, you know, schools love, but it's that kind of thing that there are, there are these multiple different types of intelligence, arithmetic, uh, arithmetical intelligence and, uh, and so on. And over the years, he's added new intelligences. So, he's decided that there are new ones. There's existential intelligence now, um, interpersonal, intrapersonal intelligence, um, uh, naturalistic intelligence, which is about, like, how much people know about plants in the garden and stuff. And he's decided that that is an intelligence. Now, those are skills, those are real things that we, that we, that we care about. It's in- it's cool to meet someone who knows the difference between plants, you know, when you're out in the woods or whatever. That's, that's great. But is that what we would call an intelligence in the same way that we would talk about, like, you know, verbal and spatial abilities? Probably not. Um, uh, does it mean that there's no such thing as the general factor of intelligence? No, because there's actually no empirical content to the, to the multiple intelligences theory whatsoever. It's just one guy's opinion. Um, which is remarkable. Like, Harvard psychologist had a massive impact on the world and the way they think about how the human mind works having done no research. Isn't that amazing?
- CWChris Williamson
So you're saying, you're saying that there's still potential for my academic career to take off then?
- SRStuart Ritchie
(laughs) Yeah. You don't need to do any research. Just come up with an idea that people like and they will spread it round every school.
- CWChris Williamson
Dude, I'm brilliant at doing that.
- SRStuart Ritchie
(laughs) Well, exactly, exactly. So, uh, that's a good thing. Then you mentioned, uh, emotional intelligence which is-
- CWChris Williamson
EQ eats IQ for breakfast. That's the-
- SRStuart Ritchie
Which is like another, which is like another thing. And my understanding of that is a few years ago, there was a meta-analysis done on all the research predicting job performance from, uh, EQ, IQ, and personality, right? And what they found was if you just look at EQ on its own, it predicts job performance. People who have higher emotional intelligence do better at, at work, right? So, there you go. That correlation is there. If you just look at the bi-variate correlation between those two things. Totally, totally, uh, uh, uh, robust there. However, if you put intelligence and personality, like the big five personality factors, we had Christian Jarrett on the show recently talking about that sort of thing, um, into the same equation, then EQ no longer has an effect. And that's wh- uh, uh, in, in predicting, in predicting, uh, job outcomes. And that's because...... EQ is just a re-description of stuff that we already knew about, right? So, so it's just a re-description of being, you know, caring about what people's, uh, feelings and emotions are, right? So that's one, that's one part of it and some of that comes across in some personality, uh, uh, factors. And being smart enough to kind of operate in your mind, okay, that person must be thinking this way, you know, uh, I can, I can kind of, if I say this to them, it's gonna have this effect. Like, it takes quite a smart person to be able to kind of, I mean, the slightly derogatory way of putting it would be, like, manipulate people in, in, in that way. Um, so in predicting job performance, it's just a re-description. So I can't remember which way round it is. There's the like, people talk about the jingle and jangle fallacy, right? And one of them is the jingle fallacy is describing the same thing with a different name and therefore thinking it's something different. And then the jangle fallacy is calling different things the same name and thinking that they must be the same. Maybe it's the opposite way around. I can't remember, uh, which way, which is jingle and which is jangle. But psychology, psychologists do an awful lot of that, like re-describing things with a new name and- and thinking it's something different. One example is grit, right? So, um, there was this big book in like 20-
- CWChris Williamson
Angela Duckworth?
- SRStuart Ritchie
Yes, Angela Duckworth book. Um, 2016, something like that that came out that was, that was great. Man, it's in every single school in the country. Everyone loves it, like the power of persistence and passion and all that sort of stuff. If you look at the studies, like it's literally, you know, it's correlated at 0.9 on a scale of minus one to one, it's correlated at like 0.9-
- CWChris Williamson
Consciousness.
- SRStuart Ritchie
... ish. With conscientiousness. It's just the same thing.
- CWChris Williamson
(laughs) I didn't even, didn't even know that and I picked it up.
- SRStuart Ritchie
Yeah, well there you go. There you go. And, and, and, you know, there's meta-analyses that have come out saying that it really is just a re-description. So it's not wrong, it's not that grit is a load of nonsense and it's also not that emotional intelligence is a load of nonsense. It's just like-
- CWChris Williamson
It's a jangle.
- SRStuart Ritchie
It's just a new way of talking about something that we already knew about. And so, you know, the contribution there is... I'm not clear what the contribution is apart from like, you know, maybe popularizing old stuff.
- CWChris Williamson
Uh, that's what I would say. I would say that given the way that pop psych works at the moment, that create- a, a branding problem, a catchy branding problem is 90% of the battle.
- SRStuart Ritchie
Yeah. (laughs)
- CWChris Williamson
I know that, um, growth and fixed mindsets
- 38:48 – 45:39
Dying Psychological Concepts
- CWChris Williamson
came under the ax. Actually, why don't we do this?
- SRStuart Ritchie
Hmm.
- CWChris Williamson
Give, give me some of your favorite eviscerated psychological concepts-
- SRStuart Ritchie
(laughs)
- CWChris Williamson
... from the replication crisis. What is, what's dying, bleeding out on the cutting room floor?
- SRStuart Ritchie
Well, I mean, the classic one, I mean, w- I'm very happy to talk about growth mindsets and we can come to that, but the classic one that kind of was there that kicked off the replication crisis in many ways was priming, like social priming, which is that there are all these things in the environment, uh, words, phrases, uh, ideas, and that when you see them, it changes the whole way that you act. So the classic study, which I remember reading as an undergrad and I was being, I was taught it in lectures, and I remember I read the study, I was like, wow, this is amazing. It was in the textbook. Incredible. People, you sit people down in a lab with a computer, um, they're doing like some task where they have to like tell whether a sentence is a real sentence or, or a word is a real word. Like there's words and non-words, you know. Um, and some of the words that come up, uh, in one of the conditions, you know, for half the participants are to do with old age. So they say things like Zimmer frame or, um-
- CWChris Williamson
Fragile or frail or something.
- SRStuart Ritchie
Yeah, exactly. Fragile, frail. One of the words that I remember that was used in the study was Florida because apparently people in the US, uh, associate that with old people.
- CWChris Williamson
Fantastic.
- SRStuart Ritchie
It seems a bit tenuous, but that's what they, that's what they said. Um, and then what they found was that the, the, and, you know, for the other half of the participants it was just random words that didn't have any particular theme. And f- for, you know, the, the half that saw the words that are to do with old people, they walked more slowly out of the lab, right? So they, they, they... When they were leaving the lab, there were like people measuring with stopwatches and they walked more slowly. That was apparently the finding. I remember reading that in social psychology literature and going, wow, that's incredible. That's amazing to believe that. Um, and there was a whole range of studies like this, um, showing people the American flag makes them much more likely to vote Republican at the next election. Even years later. You show them the American flag once and they'll vote Republican much later. People, you know, serious scientists published that in scientific journals. Serious science journalists wrote about that in, you know, whatever, you know, whatever magazines that they were writing about science in and didn't think, "That sounds a bit, that sounds a bit implausible." Um, uh, uh, another classic example, which I don't think has ever been... I don't think anyone's ever attempted to replicate it, but it was, um, that you, you go into a room and there's a box in the middle of the room, like a big cardboard box, and half the participants, uh, sit in the box and they do like a creativity task. Like how many uses of a, for a brick can you think of? Like that sort of creativity task. And half the participants sit just next to the box, right? And what they found was that the participants who were sitting next to the box had a higher creativity score and they thought it was because they had the idea in their head activated of thinking outside the box, right? I mean, lit- literally, this is how, uh, absurd, you know, some of the social psychology, uh, was.
- CWChris Williamson
(laughs) Come on.
- SRStuart Ritchie
(laughs) I know, I know, .......................... That was published in Psychological Science, one of the top journals of the field in 2012, I think. Um, and so that idea of priming or social priming, uh, behavioral priming has really fallen by the wayside. Linguistic priming, uh, I think is pretty solid. So like if I say an active sentence, then you're more likely to say an active sentence back rather than a passive sentence. You know, that sort of, that sort of thing. Like that slightly more boring stuff is, is, is totally, is totally real. But the kind of Derren Brown thing where you like just show someone one phrase and it totally changes their, their behavior. And I think Derren Brown actually like picked up on a lot of this stuff and put it into his magic act, you know. Um, I don't think he was doing priming either. I think he was doing magic tricks, which is totally fine, but like he claimed that he was doing, you know, this kind of putting an idea in your head sort of thing. That stuff has completely fallen by the wayside. I think that's, that's... But I, I, you know, you don't see new studies on that kind of thing anymore, uh, very much anymore because there were some very prominent attempts to replicate that, including the one with the people walking more slowly out the lab. They actually got, um, instead of people with stopwatches, who by the way knew what the hypothesis of the experiment was, um, they put laser, like, like infrared, uh, uh, things on the corridor.
- CWChris Williamson
Like 100 meters.
- SRStuart Ritchie
Yeah, exactly. So that they would break them as they walked through and there was no difference. Uh, when you, when you do it objectively like that, there's n- there's no difference, um, in the people who'd seen the old words versus not.
- CWChris Williamson
Growth and fixed mindset.
- SRStuart Ritchie
Yes. Well, there's another one, which I think the story of growth and fixed mindset is a bit different in that it's not...... complete bollocks in the sense that the, the, the priming stuff was, but it was massively overplayed. And I think, um, the over- the, the, the real, um, you know, if you look at the original discussions of growth mindset early 2000s, uh, maybe late '90s, early 2000s, this was, this is gonna change your life, change the way you talk to your partner, change the way you- your kids do at school, change the way you think about your life. It's gonna be massive. It's gonna have an en- enormous effect. They published a paper in Science, so like what's meant to be the second-best journal in the world, saying that growth mindset could solve the, uh, Israel-Palestine peace process. Like I'm not kidding. That's a paper that exists and published in Science. And it's all, like, very borderline statistic results, statistical results and that sort of stuff. I actually, having now remembered that, I feel like I should go back and do a little, um, do a little debunking of that because I've not seen, uh, any- anyone talk about that. But, um, you know, and, and over time, what you now find is that people who are doing research on growth mindset claim things like growth mindset is a cheap intervention that can have a, a, a small to modest effect on kids', uh, educational attainment. That is, if you go into classrooms, you teach kids, um, working hard is good, you can change the way that you, uh, you know, you can change your level of ability, you can change your skills, you know, uh, then that, that does seem to have a s- a, a small effect on, on kids' behavior, as, as I think I would predict it would, right? It like... That makes sense. If you tell kids, "You're completely stuck, there's nothing you can do," then, you know, I can totally imagine that many of them will, uh, uh, uh, um, you know, will, will take that to heart and, and, uh, and it'll at least have a small effect on them. There seem to be slightly bigger effects on kids who are from very low income backgrounds, for instance, so that's, that makes, that makes sense too. So like now the, now the claims, they used to be these absolutely dramatic claims about how it would change your life. Now it's like, this should be part of the toolkit in education. And that's good, and that's because people have come along and done meta-analysis of all the research and found that the effects are, are really, really small on average. Like the effects are, are not earth, earth-shattering effects. They're like just about there sort of thing. Um, and, and that's much better, but I haven't seen anyone say, "Oh, sorry, we misled the entire world for decades about, about, you know, how much of an effect these things will have." I feel like this kind of thing might be being used in a few schools now if they had talked about it on the level all the way through. But it's now in a... It's in every school in, in, in the country because the claims being made for it were, were revolutionary, when in fact the data just didn't back that up at all.
- 45:39 – 51:54
The Placebo Effect in Psychology
- CWChris Williamson
I had David Robson on to talk about the expectation effect. Have you seen that?
- SRStuart Ritchie
Mm. Uh, I do... uh, I mean, I'm aware of the, the, the sort of effect, but I don't know. Is, is there, is there a book or...
- CWChris Williamson
Yes. Yeah, yeah, yeah. So he wrote a book, science writer, very good, really, really good, and I would highly recommend you go and check it out.
- SRStuart Ritchie
Hmm.
- CWChris Williamson
Uh, and y- so he's talking about basically the placebo effect, but across multiple different sorts of domains, right? The expectation that you have about your outcomes can very heavily influence the outcomes. My two favorite studies from that, one of them was talking about, um, gluten intolerance has 10Xed from 3% to 30% over the last 20 years.
- SRStuart Ritchie
Mm.
- CWChris Williamson
Uh, so they brought people into a, uh, a food hall, and they put them under study settings. Some people did and did not have self-reported gluten intolerances.
- SRStuart Ritchie
Yeah.
- CWChris Williamson
They told everyone that they were gonna give them a meal which had gluten in, had no gluten in.
- SRStuart Ritchie
Right. (laughs)
- CWChris Williamson
People were breaking out in hives, running to the toilet with diarrhea. They had inflammation, all of this stuff. The other one was a study about VO2 max tests. It turns out that, um, a particular type of genetic variation allows you to blow off CO₂ and upregulate oxygen more efficiently. People were brought in. They were split randomly into two groups that were mixed with do and do not have the genetic variant.
- SRStuart Ritchie
Okay.
- CWChris Williamson
One group was told you do, one group was told you don't. You should do really well. You should do really badly. Surprise, surprise, the group that was told that they should do badly did worse overall.
- SRStuart Ritchie
Interesting.
- CWChris Williamson
However, what they found was that people in the group that were told they would do worse, but did-
- SRStuart Ritchie
Right.
- CWChris Williamson
... have the correct, uh, genetic variant to blow off CO₂ didn't do as well as the people that didn't have the genetic variation, but were told that they should. And David's synopsis for this was, your expectations are more powerful than your genes.
- SRStuart Ritchie
Right.
- CWChris Williamson
Now he's doing it as a little bit of a tongue-in-cheek sort of play. My point being, you have the opportunity with the expectation effect to create a s- a placebo sense of what may be the outcome. Is that justified in jazzing up some sort of-
- SRStuart Ritchie
Mm-hmm.
- CWChris Williamson
... effect? And is there maybe an argument to be made that the more sexy that you make it, the more outlandish that you make it sounding, it does end up becoming a self-fulfilling prophecy? You do end up getting people who believe in, uh, ego depletion. I'm sure that you've seen this. People that believe that willpower is a limited resource have limited willpower.
- SRStuart Ritchie
Yeah. Yeah.
- CWChris Williamson
People that don't believe that that's the case seem not to. I'm not sure how replicable that is, but...
- SRStuart Ritchie
Well, I think, yeah, I think there's been some controversy over that replication. But yeah, no, I get the, I get the, the, the, the idea.
- CWChris Williamson
Is there a justification for people jazzing up any kind of psychological-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... uh, tool that you may be able to use because by doing that, you increase the expectation effect, which does-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... genuinely increase the effect. The only issue is that it's not happening on the mechanism that it's claimed on.
- SRStuart Ritchie
Yeah, I mean, well, it's not just psychological stuff. I mean, if you look at, like, the whole term of bedside manner that doctors use. It, you know, it... Doctors can have a big effect on how their patients do by just being nice to them or, or acting in a way that, acting in a way that, um, that's not like, you know, really harsh or, or accusatory or whatever. Um, I think any doctor will tell you that, that that's the case. However, I think we really, like... The, the most important thing, uh, as scientists that we, that we have... Now, when you're doing an intervention in a school and you're a teacher, I think jazzing it up, whatever, you know, c- c- carry on. I- i- if it helps, that's completely fine. But as scientists, our number one thing is to try and get to the truth. And, uh, and, you know, we need to complete like control as rigorously as possible for these kind of effects. And that's really hard in a behavioral context. It's really hard to...... make the control group feel like they're doing something similar to the, the, um, the kind of active treatment group. It's really, really difficult. Um, they're maybe not having exactly the same kind of intervention, they're, uh, if it, if, if it's one of those video game studies that I mentioned before, there may be, you know, there were loads of studies published where the control group was just people who didn't do anything, uh, versus people who were playing the video game. And what you want is someone to be playing, like, a different video game which you don't think has the active ingredient of, like, the brain training or whatever it is, um, but that's hard. And, um, it's so hard that when you look across studies, you find that tons and tons and tons of studies, even, you know, randomized medical, uh, trials, don't really have very good controls, um, whether it's like a placebo pill or whether it's just, you know, um, uh, tr- treatment as usual or whatever, um, people don't put as much effort as they should into these controls. And I think, I think, um, uh, you're absolutely right, that there are these, there are these big expectation effects. You've gotta, you've gotta try your best to rule them out because it's a big source of bias in these, in these kind of studies. Just reading a, a, a meta-analysis on homeopathy right now that claims homeopathy helps people's, uh, ADHD, I'm just writing a, a, a blog about it, which is a very interesting a- case of a meta-analysis by the way, which does everything right. It's pre-registered, it had a bias check, it had a publication bias check, it did... Or like it was all, everything was all done, you know, uh, uh, uh, by the book, and yet obviously it's bollocks anyway because it's homeopathy. So, so it... So it, it, like, there might-
- 51:54 – 1:07:55
Treating Depression with SSRIs
- CWChris Williamson
think. All right, what, the other one that I saw, which I haven't spoken to anybody about yet, and you're the guy, what happened with the recent SSRI-
- SRStuart Ritchie
Mm-hmm.
- CWChris Williamson
... and depression stuff? Because that demonstrates, I think quite nicely, how important it is to get this right because you're potentially going to commit m- m- tens of millions, hundreds of millions of people perhaps-
- SRStuart Ritchie
Mm-hmm.
- CWChris Williamson
... over several decades to a particular course of treatment that may be based on something that isn't legit. Have you looked at this?
- SRStuart Ritchie
Yeah. Um, it's funny 'cause it relates back to the homeopathy thing that I just mentioned because in that homeopathy meta-analysis they say, "Well, look, we know that there can't be any actual, you know, molecules of the substance left in the homeopathic remedy because it's been diluted so many times that there, there aren't any molecules of, of whatever, you know, uh, uh, uh, active ingredient left. But the randomized control trials show that there is overall an effect and therefore we don't need to know the mechanism, we just know it has an effect." Right? And, and I think that they're wrong about that, but that it was interesting that they said that because under most circumstances I would actually, that argument sounds quite good to me, just not in the case of homeopathy which obviously is, is, is, is literally impossible for it to be having an effect, um, given the laws of physics as we, as we, as we know them. But the SSRI thing is interesting because they're making a similar argument there, they're saying, so this paper that came out recently from, um, I think it was University of College London, um, said when we look at... You know, the, so the theory underlying SSRI antidepressants is that they s- they allow, you know, for differences in the amount of serotonin that people have in their brains and that might affect their, uh, their, um, their, their depression levels, they mean that there's more serotonin just going around in there. And it was always a bit of a vague theory, and I think most researchers would say that that kind of chemical imbalance theory, uh, is a, is a very, very high level, like, it's a thing that, that maybe doctors will say to their patients, but it's not actually that justified by the science. And indeed this study kind of confirmed that which is that there's no obvious differences in the amount of serotonin in the brain of people who are depressed versus people who aren't. Um, now, uh, uh, if you think back to the homeopathy thing then, it's like well, okay, the mechanism isn't there, the, the, maybe the mechanism that we think, uh, uh, uh, you know, has an effect isn't there. But if you look at, you know, SSRIs, they're not like homeopathy in that, like, they have big side effects. People who have SSRIs have an au- a huge range of different side effects, so there is something active going on, uh, uh, in, in, in, in, in there. And it could be that they're having an effect even if not via the serotonin. So if you look at the studies on antidepressants, um, my friend Saloni Dhatani did a great, uh, series on, um, you know the website Our World In Data? It's a great series-
- CWChris Williamson
Oh, yes I do. Yes, I do.
- SRStuart Ritchie
Yeah, yeah, you've probably, like, everyone's seen the graphs from that website and stuff.
- CWChris Williamson
Yeah.
- SRStuart Ritchie
But if you, d- d- they, they publish a, a whole lot more interesting stuff. She did a whole series on, uh, antidepressants and the research, and her overall conclusion was, and this is broadly my conclusion as well when I looked at the literature, is yes the effects are exaggerated in the literature. So, um, if you look at the way that antidepressant studies are published-... um, there was actually a really terrifying study done on the- on antidepressants. Uh, and I think this applies to almost all areas of science, which is they took, uh, registered trials. So when you do a medical trial, you got to register it with the government, right? And that's just a kind of, you know, legally you have to do it. Otherwise, there's no way you'll ever get it published. So there's registries and they have, like, all the trials that have ever been done on- on- on- on- on them. And in this particular selection of trials, they found that it was about 50/50. So about half of the trials, the antidepressant didn't work, and half the trials, it did. But by the time it got to the studies that actually got published in papers, almost all of the negative studies, just no one ever sent them for publication. Uh, but almost all of the positive studies, people did send for publication. Then if you look at those po- uh, the- the- the- the few negative studies that actually got published, they kind of shifted the outcome a little bit. They said, "Well, we were going to measure things on this depression measure, but actually, uh, we didn't find a result on that, so we'll- we'll just- we'll- we'll talk about a different depression measure which we did find a result on." So this is like dredging through the data to find any other thing. Then if you look even closer, you find that even some of the negative ones were kind of written up in a slightly spin sort of way, like a positive sort of way saying like, "Well, this is very promising," all this other stuff, when actually it was just a null trial. And so the literature that we see at the end of the kind of- this kind of process, and it's almost like a- like a l- laundering process of- of- of- of literature, doesn't bear much relation to the studies that actually were done, which is really scary, and I think that happens across all areas. Uh, they actually found in that same paper that it applies to therapy trials as well, not just antidepressant trials. So, I mean, that is like a really, really terrifying, uh, discovery. Having said that, I still think that if you adjust for that kind of thing, the publication bias and so on, it is the case that antidepressants do seem to have a- a- a very, you know, like a- like a small effect on- on people's levels of- of- of depression. I don't think the mechanism matters in that case. I think, um, uh, even if it's not to do with serotonin, they- they seem to be doing something and the randomized controlled trials do- do seem to show that. But then there's, like, massive interpretational differences in- um, in- in how you look at the numbers. So, uh, Irving Kirsch, the famous critic of, um, antidepressants, published a book called The Emperor's New Drugs, I think it was called, uh, about antidepressants, and he said, "An effect size of 0.2, uh, is the average that you get out of, uh- uh, studies on antidepressants." So, like, 0.2 standard deviation difference in the depression score. What a tiny, pathetic effect. You've got to give up these drugs. They don't have an ill- they don't have an effect. That's- that's not gonna, you know, work at all. You look at the- the, you know, most recent meta-analyses that are published that account for things like I was talking about, the publication bias, and they say, "Effect size of 0.2, effective medical treatment." This is really good. You- you- you know, uh, even though it's not a massive effect, this is still going to have a big, uh, impact across the population and all sort of stuff. So people can look at exactly the same, uh, uh, data and draw massively different interpretations. My interpretation is we've got to do a bit better on this. We've w- w- like, we've got to do proper research and really understand, uh, this, and- and- and get better at treating depression, both from the therapy angle and the- the drug angle. Um, but at the moment, it does look like on average, for- for each person who uses them, like one type of- of antidepressant will- will, uh- uh, have some kind of mild effect, at least on their- on their depression. Now, I think one of the big things that hampers depression research is that we're not very good at defining what depression actually is. Um, so there's loads of research on whether... You know, so we've- we've already talked about it, there's this thing called intelligence, right? Um, and intelligence doesn't exist in the sense that there's not like a thing we can measure with a ruler in the brain or anything like that that is intelligence. We infer the existence of intelligence from the fact that there are all these different tests that you give people and they correlate positively together and there's this thing that comes out, this latent variable that comes out, and it's called intelligence that explains, you know, half the variation in the tests and all that sort of stuff. And so the question is, you know, is there something called depression that's the same- that's the same thing, that we can infer from all the symptoms? Insomnia, low mood, just crying sometimes for no reason, uh, you know, all- all the different things that come with- with depression. Is there this thing called depression? And the argument from quite a lot of, uh, people, uh, Eiko Fried is one who's a- a- a- a- a- um, someone that I've, um, uh, known on- on- uh, for a long time, uh, a researcher who's done really good research on this. His idea- his- his idea is that there isn't necessarily this thing called depression, and what we should be focusing on is the symptoms. We should be focusing on this kind of network of symptoms which sometimes bump into each other and cause each other. So like the insomnia causes low mood, and the low mood causes you to be angry at your spouse, and that causes you to... And- and so there's all this kind of like network, and you can try, there's various statistical approaches you can try, to understand this kind of network of effects. And that's a different way of looking at things than saying that there is this thing called depression that causes all the symptoms. Um, uh, this latent variable which we'll never be able to measure that's- that- that cause those symptoms. I'm kind of undecided on this debate. I think, um, uh, it- it seems quite a- quite a strong thing to say that there's no such thing as depression because, like, a lot of people seem to have very similar symptoms over- over time, and- and the general medical approach is to try and treat the underlying cause rather than the symptoms, and- and so this would be quite a, uh, this would be quite a departure from that. But I think the- the- the, um, there's a lot of mileage in this kind of research of, like, examining the symptoms rather than saying, um, you know, there's this one thing which we ha- we have to try and treat. So, um, you know, that might be one of the reasons why the trials on antidepressants are- are kind of all over the place, is that they all measure depressions slightly differently.
- NANarrator
Mm-hmm.
- SRStuart Ritchie
People experience depression slightly differently, uh, people are at different phases of depression, and- and the symptoms might be different, and if one symptom causes another then in- in different ways in different people. So i- it becomes like a- like a moving target rather than, you know... I don't know, if you were doing a drug of- if you're doing a- a- a test of, like-... do you have the COVID v- virus in your system? Which is, you know, yes, there's difficulties in, in some ways of measuring that, but like, it's a much more objective thing than, you know, are you depressed? I mean, there's many different ways you can, you can answer that question.
- CWChris Williamson
That seems very much like a behavioral geneticist's answer. I spoke to Plomin about a year ago and he was telling me, he says he has a, a predisposition for being fat.
- SRStuart Ritchie
Mm-hmm.
- CWChris Williamson
Um, and he said, uh, i- it was such a great example and it really helped to identify how this collection of genes contributes to the outcomes that we get in life. So he said that whenever he walks past a bakery, if he smells fresh bread, he will be going in and he will be buying it and he will be eating most of it.
- SRStuart Ritchie
(laughs) Yeah.
- CWChris Williamson
But-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... there are many ways to get fat. You may be fat because you have an aversion to exercise. You may be fat because you have downregulated ghrelin release, so that the hormone that tells your stomach that it's empty that-
- SRStuart Ritchie
Mm-hmm.
- CWChris Williamson
... signals to you to eat, maybe that's o- up-regulated or some... You know, there's a million different ways that you could do that or-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... for you to be fit. Maybe you don't sleep so much so you're always up early, and because you're up early, it means that you go and exercise 'cause you've got nothing-
- SRStuart Ritchie
Right.
- CWChris Williamson
... else to do, right? There are tons and tons and tons of different things. And the same thing goes for what you're talking about here. What you talk about when you're referring to depression is a particular milieu of a bunch of different things that people t- seem to link together.
- SRStuart Ritchie
Yeah.
- CWChris Williamson
Where do they come from? Is everyone's depression the same? You're never going to actually get to experience somebody else's brain, so you use these words to describe it, but we all know how, uh, m- culturally influenced a lot of the things that we consider-
- SRStuart Ritchie
Sure.
- CWChris Williamson
... r, right? We use the language that other people have d- It's precisely why juries aren't allowed to or they're told not to watch the news while they're doing-
- SRStuart Ritchie
Right. (laughs) Yeah, yeah.
- 1:07:55 – 1:18:14
How to be Effectively Sceptical of Science
- CWChris Williamson
- SRStuart Ritchie
Yeah.
- CWChris Williamson
So given, given the fact that you've got this thicket as you put it, which is a complete mess, even for people who like you can read scientific papers, who like you understand how effect sizes and p-hacking and all of this stuff can be done, and coming out the back of two years where everybody had to be a closet-
- SRStuart Ritchie
Yeah.
- CWChris Williamson
... epidemiologist-
- SRStuart Ritchie
Agree.
- CWChris Williamson
... virologist and understand you've got, uh, the media perverse incentives, clickbait, you've got, uh, academic issues with regards to the only stuff that gets published is the stuff that is se- all of this stuff all put together. First off, it's not a surprise that people are losing faith in the powers that be, in the ones that they should have done. The bigger question I think is how people can be effectively skeptical without losing faith in everything and becoming confused and nihilistic and easy to manipulate.
- SRStuart Ritchie
I've just written, uh, the start of a book proposal on exactly that question of, like, how do you question the consensus without, like, losing your mind and becoming a conspiracy theorist? And, uh, uh, I, I, I, I think this is a really difficult task because the, the incentives are so strong. Once you see that, you know, the scientific establishment is really screwed up, uh, on something like the replication crisis or, you know, it, you, you, uh, you mentioned, you know, during the pandemic there were all those cases of studies that came out that were fraudulent, uh, papers having to be retracted from the Lancet because they were, they were based on entirely fraudulent data. Like, there's a study on hydroxychloroquine that had to be, uh, retracted, and The Lancet were like, "Well, from now on we're gonna..." Th- this, literally this is what happened. They said, "From now on, we're going to make sure that at least one of the reviewers, uh, of the paper, one of the peer reviewers has expertise in the topic, uh, uh, that they're reviewing." It's like, "Whoa, wait." (laughs) "You didn't do that before." (laughs)
- CWChris Williamson
So, so, you know, I, I can t- I... Like, I can c- I understand why some people are like, "Holy shit." But I think part of it is that we need to raise our standards in general. Like, um, we need to... First of all, we need to have some standards rather than just accepting stuff that someone on our side says, uh, which is a massive temptation for all of us. There's someone we trust, they said something, so we're like, "Oh, yeah, I'm sure that's, I'm sure that's true." Um, and that's kind of my argument in, in the previous book in Science For actions is that, like, if you raise your standards, then you won't become a homeopath or a conspiracy theorist or whatever, because, because the, the, the standards that you have are, you know... Will, will, will just kind of, you know... All that stuff will get chucked out as well. It's just that you'll also chuck out lots of really crap, you know, mainstream science too. Um, uh, so I think there's probably, like, a good checklist that can be written for, like, how you should read papers, and I, I t- I, I kind of sketch one of them out in, in the previous book, but I think I'm going to do a book length treatment of that, like what to look at when you're reading scientific evidence. Um, but even then, like, you can't be an expert in every single area, and it takes, it takes, uh, expertise in, in, in each area.
Episode duration: 1:18:51
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode o9whfr05zLQ
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome