Edward Gibson: Human Language, Psycholinguistics, Syntax, Grammar & LLMs | Lex Fridman Podcast #426

1. 0:00 – 1:13

   Introduction

   1. EGEdward (Ted) Gibson0:00

      Naively, I certainly thought that all humans would have words for exact counting.

   2. LFLex Fridman0:06

      Mm-hmm.

   3. EGEdward (Ted) Gibson0:06

      Uh, and the Piraha don't, okay?

   4. LFLex Fridman0:08

      Oh, wow.

   5. EGEdward (Ted) Gibson0:08

      So they don't have any words for even one. There's not a word for one in their language. And so there's certainly not a word for two, three or four, and so that kind of blows people's minds often. (laughs)

   6. LFLex Fridman0:18

      Yeah, that's blowing my mind.

   7. EGEdward (Ted) Gibson0:20

      (laughs) That's pretty weird, isn't it?

   8. LFLex Fridman0:20

      How are you, how are you gonna ask, "I want two of those?"

   9. EGEdward (Ted) Gibson0:23

      You just don't. And so that's just not-

   10. LFLex Fridman0:25

       (laughs)

   11. EGEdward (Ted) Gibson0:25

       ... a thing you can possibly ask in the Piraha. It's not possible. That is, there is no words for that.

   12. NANarrator0:31

       (logo whooshing)

   13. LFLex Fridman0:32

       The following is a conversation with Edward Gibson, or Ted, as everybody calls him. He is a psycholinguistics professor at MIT. He heads the MIT Language Lab that investigates why human languages look the way they do, the relationship between culture and language, and how people represent, process and learn language. Also, he should have a book titled Syntax: A Cognitive Approach published by MIT Press coming out this fall. So look out for that. This is the Lex Fridman Podcast. To support it, please check out our sponsors in the description. And now, dear friends, here's to Edward Gibson.

2. 1:13 – 5:19

   Human language

   1. LFLex Fridman1:13

      When did you first become fascinated with human language?

   2. EGEdward (Ted) Gibson1:17

      As a kid in school when we had to structure sentences in English grammar. I, I, I found that process interesting. I found it confusing (laughs) as to what it was I was told to do. I didn't, didn't understand what the theory was behind it, but I found it very interesting.

   3. LFLex Fridman1:34

      So when you look at grammar, you were almost thinking about it like a puzzle, like almost like a mathematical puzzle?

   4. EGEdward (Ted) Gibson1:39

      Yeah. I think that's right. I didn't know I was gonna work on this at all at that point. I was really just ... I was kind of a math geek person-

   5. LFLex Fridman1:46

      Mm-hmm.

   6. EGEdward (Ted) Gibson1:46

      ... a computer scientist. (laughs) I really liked computer science, and then I found language is a, a neat puzzle to work on from, uh, an engineering perspective, actually. That's what I ... It ... And as a ... I, I sort of accidentally (laughs) fi- ... I decided after I finished my undergraduate degree, which was computer science and math in Canada, in Queen's University, I decided to go to grad school. It's like, that's what I always thought I would do, and I went to, to Cambridge, where they had a master's in, a master's program in computational linguistics. And I hadn't taken a single language class before. All I'd taken was CS, computer science, math classes, pretty much, mostly, as an undergrad, and I just thought, well, this was an interesting thing to do for a year, (laughs) 'cause it was a single-year program. And, um, then I ended up spending my whole life (laughs) doing it.

   7. LFLex Fridman2:35

      So fundamentally, your journey through life was one of a mathematician, a computer scientist, and then you kinda discovered the puzzle, the problem of language and approached it from that angle, uh, to try to understand it from that angle, almost like a mathematician or maybe even an engineer.

   8. EGEdward (Ted) Gibson2:53

      As an engineer, I'd say, I mean, to be frank, I had taken an AI class, I guess it was '83 or '84, '85, somewhere, '84, in there, a long time ago.

   9. LFLex Fridman3:01

      Mm-hmm.

   10. EGEdward (Ted) Gibson3:02

       And there was a natural language section in there, and it didn't impress me. I thought, (laughs) "There must be more interesting things we can do." Didn't, it didn't seem very ... It seemed just a bunch of, uh, hacks to me. It didn't seem like a real theory of things in any way, and so I just thought this was, this seemed like an interesting area where there wasn't enough good work.

   11. LFLex Fridman3:23

       Did you ever come across, like the, the philosophy angle of logic? So if you think about the '80s with AI, the expert systems where you try to-

   12. EGEdward (Ted) Gibson3:31

       Yeah.

   13. LFLex Fridman3:31

       ... kinda, uh, maybe sidestep the, the poetry of language, and, uh, the, some of the s- the syntax and the grammar and all that kind of stuff, and go to the underlying meaning the language is trying to communicate and try to somehow compress that in a computer-representable way? Did you ever come across that in your studies?

   14. EGEdward (Ted) Gibson3:50

       I mean, I probably did, but I wasn't as interested in it. I was, I was trying to do the easier problems first, the ones I could-

   15. LFLex Fridman3:57

       Yeah.

   16. EGEdward (Ted) Gibson3:57

       ... thought maybe were handleable, which is, seems like the syntax is easier, like which is just the forms as opposed to the meaning. Like you're talking ab- ... When you started talking about the meaning, that's very hard problem. I s- ... And it still is a really, really hard problem. (laughs)

   17. LFLex Fridman4:09

       Mm-hmm.

   18. EGEdward (Ted) Gibson4:09

       But the forms is, is easier, and so I thought at least figuring out the forms of human language, uh, which sounds really hard but is actually maybe more tractable.

   19. LFLex Fridman4:19

       So it's interesting. You think there is a big divide, there's a gap, there's a distance between form and meaning. Because that's a question you, you have discussed a lot with LLMs.

   20. EGEdward (Ted) Gibson4:31

       Mm-hmm.

   21. LFLex Fridman4:31

       Because they're damn good at form.

   22. EGEdward (Ted) Gibson4:33

       Yeah. I think that's what they're good at, is form.

   23. LFLex Fridman4:35

       Yeah.

   24. EGEdward (Ted) Gibson4:35

       Exactly. And that's, that's why they're good, 'cause they can do form. Meaning's hard. (laughs)

   25. LFLex Fridman4:39

       Do you think there's ... Oh, wow. And I mean, it's an open question, right?

   26. EGEdward (Ted) Gibson4:42

       Yeah.

   27. LFLex Fridman4:43

       How close form and meaning are.

   28. EGEdward (Ted) Gibson4:44

       Yeah.

   29. LFLex Fridman4:44

       We'll discuss it, but-

   30. EGEdward (Ted) Gibson4:45

       Yeah.
3. 5:19 – 11:06

   Generalizations in language

   1. LFLex Fridman5:19

   2. EGEdward (Ted) Gibson5:19

      Yeah.

   3. LFLex Fridman5:20

      What do you find most beautiful about human language, maybe the form of human language, the expression of human language?

   4. EGEdward (Ted) Gibson5:27

      What I find beautiful about human language is the, uh, some of the generalizations that, um, happen across the human languages, within and across a language. So let me give you an example of something which I find kind of remarkable. That is if, like a language, if it has, um, a, a word order such that the verbs tend to come before their objects, and so that's like English does that. So we have the, the first ... The subject comes first in a, in a simple sentence, so I say, uh, you know, the, uh-... the dog chased the cat, or Mary kicked the ball. So the subject's first, the, and then after the subject there's the verb, and then we have objects. Yeah, all these things come after in English. And so it's a, it's generally a verb, and m- most of the stuff that we wanna say comes after the subject. It comes, it's the, it's the objects. There's a lot of things we wanna say, they come after. And, and, and there's a lot of languages like that. About 40% of the languages of the world are, look like that. They're, um, sub, uh, subject-verb-object languages. And then, um, th- these languages tend to have, um, prepositions, (laughs) these little markers on the nouns that, that connect nouns to other nouns, or nouns to verbs. So I, when I s- say a verb like, uh, sorry, a preposition like in, or on, or of, or about. I say, I talk about something, the something is the object of that preposition that we have. These little markers come also, just like verbs, they come before their, their nouns. Okay? And then, so now we look at other languages that, like Japanese or, or Hindi or some H- these are, these are so-called verb final languages. Those, it's about f- m- maybe a little more than 40%, maybe 45% of the world's languages are, or more, I mean, 50% of the world's languages are v- verb final. Those tend to be, um, postpositions. Those markers, the sa- we have the s- they have the same kinds of markers, uh, as we do in English, but they put 'em after. So, uh, it's, uh, sorry. They, they put 'em, uh, first. The markers come first. So you say instead of, um, you know, talk about a book, you say a, a book about. The opposite order there in Japanese or in Hindi, you do the opposite. And the, and the talk comes at the end. So the verb will come at the end as well. So instead of, um, Mary kicked the ball, it's Mary, uh, ball kicked. And then, uh-

   5. LFLex Fridman7:43

      Okay.

   6. EGEdward (Ted) Gibson7:43

      ... if, if we says Mary kicked the ball to John, it's John to. The to, the little, the marker there, uh, the preposition co- it's a post position in these languages. And so the interesting thing to, a fascinating thing to me is that within a language, this order, uh, aligns. It's harmonic.

   7. LFLex Fridman8:01

      (laughs)

   8. EGEdward (Ted) Gibson8:01

      And, uh, and so yeah, it's one or the other. It's either verb initial or verb final, but then you'll, then you'll have, uh, prepositions, prepositions or postpositions. And so that, and that's across the w- languages that we, we can look at.

   9. LFLex Fridman8:14

      Mm-hmm.

   10. EGEdward (Ted) Gibson8:14

       We've got around a thousand languages for, for, there's around 7,000 languages around on the, on the Earth right now. Uh, but we have information about, say, word order on around a thousand of those, uh, for a pretty decent amount of information. And for those thousand which we know about, um, about 95% fit that pattern. So they will have either verb in- it's about, it's about half and half of, half are verb initial, like English, and half are verb final, like, um, like Japanese or something.

   11. LFLex Fridman8:41

       So just to clarify, verb initial is subject-verb-object.

   12. EGEdward (Ted) Gibson8:45

       That's correct.

   13. LFLex Fridman8:46

       Verb final is still subject-object-verb.

   14. EGEdward (Ted) Gibson8:50

       That's correct. Yeah. The subject is generally first.

   15. LFLex Fridman8:52

       That's so fascinating. I ate an apple, or I apple ate.

   16. EGEdward (Ted) Gibson8:57

       Yes.

   17. LFLex Fridman8:57

       Okay. And it's fascinating that there's a pretty even division in the world amongst those 40-

   18. EGEdward (Ted) Gibson9:02

       Yeah.

   19. LFLex Fridman9:02

       ... 45%.

   20. EGEdward (Ted) Gibson9:03

       Yeah, it's pretty, it's pretty even. And, and those two are the most common by far. Those two words are, the subject tends to be first. There's so many interesting things, but these things are, the thing I find so fascinating is there are these generalizations within and across a language. And, and not only those are the, the, and there's actually a simple explanation, I think, for a lot of that. And that is, um, you, you're trying to like, minimize dependencies between words. That's basically the story, I think, (laughs) behind a lot of why word order looks the way it is, is you, we're always connecting, uh, what is it, what is the thing I'm telling you? I'm ta- I'm talking to you in sentences. You're talking to me in sentences. These are sequences of words which are connected, and the connections are dependencies between the words. A- and it turns out that what we, what we're trying to do in a language is actually minimize those dependency links. It's easier for me to say things if the words that are connecting for their meaning are close together. It's easier for you in understanding if that's also true.

   21. LFLex Fridman10:03

       Mm-hmm.

   22. EGEdward (Ted) Gibson10:03

       If they're far away, it's, it's hard as, uh, to produce, produce that, and it's hard for you to understand. And the languages of the world, within a language and across languages, you know, fit that generalization, which is, you know. So I, uh, you know, it turns out that having verbs initial and then having prepositions ends up making dependencies shorter. And, and having verbs final and having postpositions ends up making dependencies shorter than if you cross them. If you cross them, it ends up, you just end up, it's possible you can do it, it just-

   23. LFLex Fridman10:33

       You mean within a language?

   24. EGEdward (Ted) Gibson10:34

       Within a language you can do it, it just ends up with longer dependencies than if you didn't. (laughs) And so languages tend to go that way. They tend to minim- they, this is, they s- they call it harmonic. So it was observed a long time ago by, uh, without the explanation by a, a guy called Joseph Greenberg, who's a, um, famous typologist from Stanford. He observed a lot of generalizations about how word order works. And these are some of the harmonic generalizations that he observed.

   25. LFLex Fridman10:59

       Harmonic generalizations about word, word order. There's so many things I wanna ask you.

   26. EGEdward (Ted) Gibson11:04

       Okay, (laughs) good.

   27. LFLex Fridman11:05

       Uh, let me, uh,

4. 11:06 – 21:05

   Dependency grammar

   1. LFLex Fridman11:06

      just sometimes basics.

   2. EGEdward (Ted) Gibson11:07

      Okay. Yeah, yeah.

   3. LFLex Fridman11:07

      You s- you mentioned dependencies a few times.

   4. EGEdward (Ted) Gibson11:09

      Yeah, yeah.

   5. LFLex Fridman11:10

      What do you mean by dependencies?

   6. EGEdward (Ted) Gibson11:12

      Well, what I mean is in, um, in language, there's kind of three structures to, three components to the structure of language. One is the sounds. So cat is K-ah-t in English. I'm not talking about that part, I'm talking... Then there's two meaning parts, and those are the words. And, and, uh, you were talking about meaning earlier. So words have a form and they have a meaning associated with them. And so cat is a full form in English, and it has a meaning associated with whatever a cat is. And then the combinations of words, uh, that's what I'll call grammar or syntax. And, uh, that's like when I have a, uh, a combination like the cat or two cats, okay? So, uh, where I t- take, uh, two different words there and put 'em together, and I get a compositional meaning from putting those two different words together. And, and so that's the syntax. And in any...... sentence or utterance, whatever I'm talking to you, you're talking to me, we have a bunch of words, and we're putting together in a sequence. They... It turns out they are connected, so that every word is connected to just one other word in that, in that sentence.

   7. LFLex Fridman12:16

      Mm-hmm.

   8. EGEdward (Ted) Gibson12:16

      And so you end up with what's, what's called a, technically a tree. It's a tree structure. So there... Where there's a root of that, of that utterance, of that sentence, and then there's a bunch of dependents, like branches from that root that go down to the words. The words are the leaves in this metaphor for a tree.

   9. LFLex Fridman12:34

      So a tree is also sort of a mathematical construct?

   10. EGEdward (Ted) Gibson12:37

       Yeah, yeah, it's a graph theoretical thing.

   11. LFLex Fridman12:38

       A graph theory thing.

   12. EGEdward (Ted) Gibson12:39

       Yeah, yeah.

   13. LFLex Fridman12:39

       Uh, so, and, and the... It's fascinating that you can break down a sentence into a tree, and then one, every word is hanging on to another, is depending on it.

   14. EGEdward (Ted) Gibson12:47

       That's right. And, and everyone agrees on that. So all linguists will agree with that. No one, no one-

   15. LFLex Fridman12:51

       Oh, so this is not a controversial thing?

   16. EGEdward (Ted) Gibson12:52

       That is not controversial.

   17. LFLex Fridman12:53

       There's nobody sitting here listening-

   18. EGEdward (Ted) Gibson12:54

       I do not think so.

   19. LFLex Fridman12:55

       ... mad at you?

   20. EGEdward (Ted) Gibson12:55

       I don't think so.

   21. LFLex Fridman12:56

       Okay.

   22. EGEdward (Ted) Gibson12:57

       I think-

   23. LFLex Fridman12:57

       No linguist sitting there mad at this?

   24. EGEdward (Ted) Gibson12:58

       No. No. I think in every language, I think everyone agrees that all sentences are trees at some level.

   25. LFLex Fridman13:05

       Can I pause on that?

   26. EGEdward (Ted) Gibson13:06

       Sure.

   27. LFLex Fridman13:06

       Because it, it's to me, just as a layman, it, uh, it's surprising-

   28. EGEdward (Ted) Gibson13:11

       Yeah.

   29. LFLex Fridman13:11

       ... that you can break down sentences in many, mostly all languages-

   30. EGEdward (Ted) Gibson13:16

       All languages, I think.
5. 21:05 – 29:40

   Morphology

   1. LFLex Fridman21:05

      So you said that, uh, morphemes ... Lots of questions.

   2. EGEdward (Ted) Gibson21:07

      Yeah. Yeah.

   3. LFLex Fridman21:07

      So mor- morphology is what? The study of morphemes?

   4. EGEdward (Ted) Gibson21:10

      Morphology is the, is the connections between the morphemes onto the roots. The roots ... So in English, we mostly have suffixes. We have end- endings on the words, not very much, but a little bit, and, uh ... As opposed to prefixes. Some words, depending on your language, can have, you know, mostly prefixes, mostly suffixes, or mostly ... Or, or both, and then even languages, several languages have things called infixes, where you have some kind of a, uh, general, uh, form for the, for the root, and you put stuff in the middle. (laughs) You change the vowels. Stuff like that.

   5. LFLex Fridman21:43

      That's fascinating.

   6. EGEdward (Ted) Gibson21:43

      Yeah, yeah.

   7. LFLex Fridman21:44

      That's fascinating. So wait. So in general, there's what? Two morphemes per word? Usually one or two or three or-

   8. EGEdward (Ted) Gibson21:51

      Uh, uh, well, in English, it's, it's, it's one or two.

   9. LFLex Fridman21:53

      Okay.

   10. EGEdward (Ted) Gibson21:53

       In English, it tends to be one or two. There can be more. You know, in, in other languages, you know, a lang- language like, uh, like Finnish, which has el- very, uh, elaborate morphology, there may be 10 morphemes on the end of a root.

   11. LFLex Fridman22:05

       Boy.

   12. EGEdward (Ted) Gibson22:05

       Okay? And so there may be mill- there'll be millions of forms of a given word, okay?

   13. LFLex Fridman22:10

       Okay. The ... I, I will ask the same question over and over.

   14. EGEdward (Ted) Gibson22:13

       Okay. Yeah.

   15. LFLex Fridman22:13

       But, uh, (laughs) how does the j- just the ... Sometimes to understand-

   16. EGEdward (Ted) Gibson22:19

       Mm-hmm.

   17. LFLex Fridman22:20

       ... things like morphemes, it's nice to just ask the question, how does these kinds of things evolve? So you, uh, have a great book studying sort of the ... How, h- how the cognitive processing, how language used for communication, so the, the mathematical notion of how effective language is for communication, what role that plays in the evolution of language. But just high level, like, how do we ... How does a language evolve with ... Where English has two morphemes, or one or two morphemes per word and then Finnish has infinity per word?

   18. EGEdward (Ted) Gibson22:54

       (laughs)

   19. LFLex Fridman22:54

       So what ... How does that, how does that happen? Is it just pe- people?

   20. EGEdward (Ted) Gibson22:58

       That's a really good question.

   21. LFLex Fridman22:59

       Yeah.

   22. EGEdward (Ted) Gibson23:00

       That's a very good question. It's like, why do languages have more morphology versus less morphology?

   23. LFLex Fridman23:06

       Yeah.

   24. EGEdward (Ted) Gibson23:06

       And, and I don't think we know the answer to this. I don't ... I think there's just, like, a lot of good solutions to the problem of communication. So I, like, I believe, as you hinted, that language is an invented system by humans for communicating their ideas. And I think we, it comes down to we label things we want to talk about. Those are the, the, the morphemes and words. Those are the things we want to talk about in the world, and we invent those things, and then, uh, we put them together in ways that are, um, easy for us to convey, to process. But that, that, that's, like, a naive view, and I don't ... I mean, I th- I think it's probably right. (laughs) Right? It's naive and probably right, but-

   25. LFLex Fridman23:43

       Well, that's the ... No, I don't know if it's naive. I think it's simple.

   26. EGEdward (Ted) Gibson23:46

       Simple, yeah. Okay, okay.

   27. LFLex Fridman23:47

       I think naive is ... Na- naive is an indication that it's in- incorrect.

   28. EGEdward (Ted) Gibson23:51

       Yeah, yeah.

   29. LFLex Fridman23:51

       Somehow it's a trivial, uh-

   30. EGEdward (Ted) Gibson23:52

       Yeah.
6. 29:40 – 33:00

   Evolution of languages

   1. LFLex Fridman29:40

      it's so, so I think what you're pointing to is that we don't have data on the evolution of language, because many languages have formed a long time ago, so you don't get the chatter.

   2. EGEdward (Ted) Gibson29:50

      We have a little bit of, like old English to modern English, because there was a writing system.

   3. LFLex Fridman29:55

      Mm-hmm.

   4. EGEdward (Ted) Gibson29:56

      And we can see how, how old English looked. So the word order changed, for instance, in old English to middle English to modern English. And so it, you know, we could see things like that. But most languages don't even have a writing system. So of the 7,000, only, you know, a small subset of those have a writing system. And even if they have a writing system, they, it's not a very modern writing system. (laughs) And so they don't have it. So we just basically have for Mandarin, for Chinese, we have a lot of, a lot of evidence from, from, for a long time and for English and not for much else. Not for ma- German, a little bit, but not for a whole lot of like long-term, um, language evolution, we don't have a lot.

   5. LFLex Fridman30:32

      Well, you get-

   6. EGEdward (Ted) Gibson30:32

      We just have snapshots is what we've got of current languages.

   7. LFLex Fridman30:34

      Yeah, uh, you get an inkling of that-... from the rapid communication on certain platforms, like on Reddit.

   8. EGEdward (Ted) Gibson30:40

      Oh, yeah.

   9. LFLex Fridman30:40

      There's different communities, and they'll come up with different slang. Usually, from my perspective, driven by a little bit of humor, um, or maybe mockery or whatever it is. You know, just talking shit in different kinds of ways. And, uh, you could see the evolution of language there, because, um, I think a lot of things on the internet, you don't want to be the boring mainstream. So you, like, want to deviate from the proper way of talking.

   10. EGEdward (Ted) Gibson31:10

       Mm-hmm.

   11. LFLex Fridman31:11

       And so you get a lot of deviation, like rapid deviation. Then when communities collide, you get, like, uh, just like you said, humans adapt to it, and you can see it through the lens of humor. I mean, it's very difficult to study, but you can imagine, like, 100 years from now. Well, if there's a new language born, for example, we'll get really high resolution data.

   12. EGEdward (Ted) Gibson31:30

       I mean, English is changing. English changes all the time. All languages change all the time. So, you know, there's a famous, um, result (laughs) about the Queen's English. So the Queen, if you look at the Queen's vowels, the Queen's English is supposed to be, you know, originally, the proper way for the talk was sort of defined by however the Queen talked, or the King, whoever was in charge. And, uh, and so if you look at how her vowels changed, uh, from when she first became queen in 1952 or '53, when she was coronated the first, I mean, that's Queen Elizabeth, who's go- who died recently, of course. Uh, until, you know, 50 years later, her vowels changed. Her vowels shifted a lot.

   13. LFLex Fridman32:07

       Mm.

   14. EGEdward (Ted) Gibson32:08

       And so that, you know, even in the sounds of British English, in her, the way she was talking, was changing. The vowels were changing slightly. So that's just, in the sounds there's change. I don't know what's dr- you know, we're, I'm interested, we're all interested in what's driving any of these changes. The word order of English changed a lot over 1,000 years, right? So it used to look like German. You know, it looks, it used to be a verb-final language with case marking, and it shifted to a verb medial language. A lot of contact. So, a lot of contact with French, and it became a verb medial language with no case marking. And so it became this, you know, verb initially thing, so. And so that's-

   15. LFLex Fridman32:47

       It's evolving.

   16. EGEdward (Ted) Gibson32:47

       It totally evolved. And so it may very well. I mean, you know, it doesn't evolve maybe very much in 20 years is maybe what you're talking about. But over 50 and 100 years, things change a lot, I think.

   17. LFLex Fridman32:57

       We'll now have good data on it.

   18. EGEdward (Ted) Gibson32:59

       Yeah (laughs) .

   19. LFLex Fridman32:59

       Which is great.

7. 33:00 – 1:17:06

   Noam Chomsky

   1. LFLex Fridman33:00

   2. EGEdward (Ted) Gibson33:00

      That's for sure. Yeah.

   3. LFLex Fridman33:00

      Um, can you talk to what is syntax and what is grammar? So you wrote a book on syntax.

   4. EGEdward (Ted) Gibson33:06

      I did. You were asking me before about what, you know, how do I figure out what a dependency structure is?

   5. LFLex Fridman33:10

      Yeah.

   6. EGEdward (Ted) Gibson33:10

      I'd say that dependency structures aren't that hard, generally. I think there's a lot of agreement of what they, of what they are for almost any sentence in most languages. I think people will agree on a lot of that. There are other parameters in the mix such that some people think there's a more complicated grammar than just a dependency structure. And so, you know, like Noam Chomsky, he's the most famous linguist ever, uh, and he, he is famous for proposing a slightly more complicated syntax. And so he, he invented phrase structure grammar. So he's, um, well-known for many, many things. But in the '50s and early '60s, like, but late '50s, he was basically figuring out what's called formal language theory.

   7. LFLex Fridman33:53

      Mm-hmm.

   8. EGEdward (Ted) Gibson33:53

      So, and he figured out sort of a framework for figuring out how complicated language, you know, a certain type of language might be, so-called phrase structure grammars of language might be. And so he, his, his idea was that maybe we can, we can think about the complexity of a language by how complicated the rules are, okay? And the rules will look like this. They will have a left-hand side and they'll have a right-hand side.

   9. LFLex Fridman34:22

      Mm-hmm.

   10. EGEdward (Ted) Gibson34:22

       Something on the left-hand side will expand to the thing on the right-hand side. So say we'll start with an S, which is like the root, which is an, a sentence, okay? And then we're going to expand to things, uh, like a noun phrase and a verb phrase, is what he would say, for instance, okay? An S goes to an NP and a VP is a kind of a phrase structure rule. And then, and we figure out what an NP is. An NP is a, a determiner and a noun, for instance, and a verb, verb phrase is something else, is a verb and another noun phrase, another NP, for instance. Those are the rules of a very simple phrase structure.

   11. LFLex Fridman34:54

       Mm-hmm.

   12. EGEdward (Ted) Gibson34:54

       Okay? And so he, he proposed phrase structure grammar as a way to sort of cover human languages. And then he actually figured out that, well, depending on the formalization of those grammars, you might get more complicated or less complicated languages. And so he could, he said, "Well, these are things called, you know, um, context-free languages that rule." That he thought, you know, human languages tend to be what he calls context-free languages. Um, and, but there are simpler languages which are so-called regular languages, and they have a more, a more constrained form to the rules of the, of the phrase structure of these particular rules. So he, he basically discovered and kind of invented ways to describe the language, and those are phrase, those are phrase structure, a human language. And he was mostly interested in English initially in his work in the '50s.

   13. LFLex Fridman35:43

       So quick questions around all this. So formal language theory is the big field of just studying language formally.

   14. EGEdward (Ted) Gibson35:49

       Yes, and it doesn't have to be human language there. We could have a computer languages, any kind of system which is generating a, uh, some set of, um, expressions in a language. And those could be like the, you know, the statements in a computer language, for example. So formal- It could be that or it could be human language.

   15. LFLex Fridman36:10

       So technically you can study programming languages.

   16. EGEdward (Ted) Gibson36:12

       Yes, ab- and have been. I mean, heavily studied using this formalism. There's a big field of programming languages within the formal language.

   17. LFLex Fridman36:20

       Okay. And then phrase structure grammar is this idea that you can break down language into this SNPVP, like-

   18. EGEdward (Ted) Gibson36:28

       Yeah.

   19. LFLex Fridman36:28

       ... type of thing.

   20. EGEdward (Ted) Gibson36:28

       It's a particular formalism for describing language, okay? So, and Chomsky was the first one. He's the one who figured that stuff out back in the '50s. And, and, but he- And that's...... equivalent, actually, the, the c- the context-free grammar is actually de- is kind of equivalent in the sense that it generates the same sentences as a dependency grammar would. You know, as, uh, the, the dependency grammar is a little simpler in some way. You just have a root and it goes, like, we don't have any of these... The, the rules are implicit, I guess, in the, in it. We just have connections between words.

   21. LFLex Fridman36:59

       Mm-hmm.

   22. EGEdward (Ted) Gibson36:59

       The phrase structure grammar is a kind of a different way to think about the de- the dependency grammar.

   23. LFLex Fridman37:04

       So-

   24. EGEdward (Ted) Gibson37:04

       It's slightly more complicated, but it's kind of the same in some ways.

   25. LFLex Fridman37:07

       So to clarify, dependency grammar is the framework under which you see language and you make-

   26. EGEdward (Ted) Gibson37:14

       Yes.

   27. LFLex Fridman37:14

       ... a case that this is a good way to describe-

   28. EGEdward (Ted) Gibson37:16

       I think it's the-

   29. LFLex Fridman37:16

       ... language.

   30. EGEdward (Ted) Gibson37:17

       That's correct. That's correct.

Episode duration: 2:50:44