Luís and João Batalha: Fermat's Library and the Art of Studying Papers | Lex Fridman Podcast #209

Lex Fridman 0:00 The following is a conversation with Luis and Joao Batella, brothers and co-founders of Fermat's Library, which is an incredible platform for annotating papers. As they write on the Fermat's Library website, "Just as Pierre de Fermat scribbled his famous Last Theorem in the margins, professional scientists, academics, and citizen scientists can annotate equations, figures, ideas, and write in the margins." Fermat's Library is also a really good Twitter account to follow. I highly recommend it. They post little visual factoids and explorations that reveal the beauty of mathematics. I love it. Quick mention of our sponsors: Skiff, SimpliSafe, Indeed, NetSuite, and Four Sigmatic. Check them out in the description to support this podcast. As a side note, let me say a few words about the dissemination of scientific ideas. I believe that all scientific articles should be freely accessible to the public. They currently are not. In one analysis I saw, more than 70% of published research articles are behind a paywall. In case you don't know, the funders of the research, whether that's government or industry, aren't the ones putting up the paywall. The journals are the ones putting up the paywall while using unpaid labor from researchers for the peer review process. Where is all that money from the paywall going? In this digital age, the cost here should be minimal. This cost can easily be covered through donation, advertisement, or public funding of science. The benefit versus the cost of all papers being free to read is obvious, and the fact that they're not free goes against everything science should stand for, which is the free dissemination of ideas that educate and inspire. Science cannot be a gated institution. The more people can freely learn and collaborate on ideas, the more problems we can solve in the world together, and the faster we can drive old ideas out and bring new, better ideas in. Science is beautiful and powerful, and its dissemination in this digital age should be free. This is the Lex Fridman Podcast, and here's my conversation with Luis and Joao Batella.

Lex Fridman 2:22 Luis, you suggested an interesting idea. Imagine if most papers had a backstory section, the same way that they have an abstract. So, knowing more about how the authors ended up working on a paper can be extremely insightful, and then you went on to give a backstory for the Feynman QED paper.

Luís Batalha 2:42 Mm-hmm.

Lex Fridman 2:43 This was all in a tweet, by the way. We're doing tweet analysis today.

Luís Batalha 2:45 (laughs)

Lex Fridman 2:46 How much of the human backstory do you think is important in understanding the idea itself that's presented in the paper or in general?

Luís Batalha 2:54 I think this gives way more context to the work of, of scientists. I think people, a lot of people have this almost kind of romantic misconception that, uh, the way a lot of scientists work is almost as this sum of eureka moments where all of the sudden they sit down and start writing two papers in a row, and the papers are usually isolated, and when you actually look at it, it's the papers are, you know, chapters of a way more complex, uh, story. And, uh, the, the Feynman QED paper is a good example. So, Feynman was actually going through a pretty dark phase before writing that paper. It was he lost enthusiasm with physics and doing physics problems, and there was one time when he was in the cafeteria of Cornell, and he saw a guy that was throwing plates in the air, and he noticed that there was, when the plate was in the air, there were two movements there. The, the plate was wob- wobbling, but he also noticed that the, the Cornell symbol was rotating, and he was able to figure out the equations of motions, uh, the equations of motions of th- those, uh, plates, and that, uh, led him to kind of think a little bit about, uh, electron orbits in relativity, which led to the paper of, um, about quantum electrodynamics. So, that kind of reignited, uh, his interest in physics, and, uh, and ended up publishing the paper that led to the, his Nobel Prize basically. And I think it's, it's, there are a lot of really interesting backstories about papers that readers never get to know. For instance, we did, a couple of months ago, um, an AMA around, uh, a paper, a pretty famous paper, the GANs paper with Ian Goodfellow, and so we did an AMA where everyone was could ask questions about the paper and Ian was, uh, responding to those questions, and he also, he was also, uh, telling the story of how he got the idea for that paper in a bar.

Lex Fridman 4:48 Mm-hmm.

Luís Batalha 4:48 So, there was also an interesting and, uh, a backstory. Uh, I also read a, a book, uh, by, um, Cédric Villani. Uh, This, uh, Cédric Villani is this mathematician, a Fields Medalist, and in his book, he tries to explain how he got from like, um, a PhD student to the Fields Medal, and he tries to be as descriptive as possible about every single step how he got to the Fields Medal. And it's interesting also to see just the amount of random interactions and discussions with other researchers, sometimes over coffee, and how it led to like fundamental breakthroughs and some of his most important papers. So, I think it's super interesting to have that context of, of the backstory.

Lex Fridman 5:28 Well, the Ian Goodfellow story is kind of interesting and perhaps that's true for Feynman as well, I don't know if it's romanticizing the thing, but it seems like just a few little insights and a little bit of work does most of the leap required. Do you have a sense that for a lot of the stuff you've looked at, just looking back through history, uh, it, it, it wasn't necessarily the grind of like Andrew Wiles with Fermat's Last Theorem, for example?

Luís Batalha 5:54 Mm-hmm.

Lex Fridman 5:55 It was more like a, a brilliant moment of insight. In fact, Ian Goodfellow has a kind of sadness to him almost-

Luís Batalha 6:02 Mm.

Lex Fridman 6:02 ... in that at that time in machine learning-... like, at that time especially, in, uh, for, for GANs, you could code something up really quickly-

João Batalha 6:12 Mm-hmm.

Lex Fridman 6:12 ... on a single machine-

João Batalha 6:13 Mm-hmm.

Lex Fridman 6:14 ... and almost do the invention f- go from idea to, uh, experimental validation in like a single night, a single person could do it. And now, there's kind of a sadness that a lot of the breakthroughs you might have in machine learning kind of require large-scale experiments. So, it was almost like the early days, uh, uh, so, th- I wonder how many low-hanging fruit there are in science and mathematics and even engineering where it's like you could do that little experiment quickly, like you have an insight in a bar. Why is it always a bar?

João Batalha 6:47 (laughs)

Lex Fridman 6:47 But you have an insight at a bar, and then just implement, and the world changes.

João Batalha 6:53 It- it's- it's a good point. I think it also depends a lot on the maturity of the field. When you look at a f- a field like mathematics, like it's a pretty mature field. Uh, a field like machine learning, um, it's- it's growing pretty fast and, um, it's actually pretty- p- p- pretty interesting. I- I- I looked up like the number of new papers on archive with a keyword machine learning and like 50% of those papers have been published on- in the last 12 months. So, you can see just the sense-

Lex Fridman 7:23 Five-zero? Or-

João Batalha 7:24 Five-zero. 50%. So, you can see the- the- the- the magnitude of growth in that field.

Lex Fridman 7:30 (laughs)

João Batalha 7:30 And so, I think like, uh, as fields mature, like those types of- of moments, I think naturally, uh, are less frequent. Um, it's just a consequence of- of that. The other point that is interesting about the backstory is that- is that it can really make it more memorable in a way. And- and by making it more memorable, it's- it kind of sediments the knowledge more in your mind. I- I remember also reading the- sort of the backstory to- to Dijkstra's Shortest Path Algorithm-

Lex Fridman 7:58 Mm.

João Batalha 7:58 ... right? Where- where he- he came up with it, uh, essentially while he was sitting down at a- at a- at a coffee shop in Amsterdam, and he- and he came up with that algorithm over 20 minutes. And one interesting aspect is th- he didn't have any pen or paper at the time.

Lex Fridman 8:13 Mm-hmm.

João Batalha 8:13 And so, he had to do it all in his mind. And so, th- there's only so much complexity that he can handle if you're just thinking about it in your mind. And that- like when you think about the simplicity of Dijkstra's Shortest Path Finding Algorithm, it- it's- you know, knowing that backstory helps sediment that algorithm in your mind, so they don't forget about it as easily.

Lex Fridman 8:33 It might be from you that I saw a meme about Dijkstra. (laughs)

João Batalha 8:39 (laughs)

Lex Fridman 8:39 It's like, he's trying to solve it and he comes up with some kind of random path and then, uh, it's like, "My parents aren't home." And then he does, uh, th- he figures out-

João Batalha 8:48 (laughs)

Lex Fridman 8:48 ... the algorithm for the shortest path. (laughs)

João Batalha 8:52 I- I- yeah.

Lex Fridman 8:52 Just try through words to convey memes, but that's- (laughs) that's hilarious. I don't know if it's in post that we construct stories that romanticize it. Apparently with Newton, there was no apple. Especially when you're working on problems that have a physical manifestation-

João Batalha 9:07 Mm-hmm.

Lex Fridman 9:07 ... or a visual manifestation, it feels like the world could be an inspiration to you. So, it doesn't have to be completely in, um, on paper.

João Batalha 9:19 Mm.

Lex Fridman 9:19 Like you could be sitting at a bar and all of a sudden see something and a pattern will s- will spark another pattern and you can visualize it and rethink a problem in a particular way. Uh, of course, you can also load the math that you have on paper and always carry that with you, so when you show up to the bar some little inspiration could be the thing that changes it. Is there any other people almost on the human side, whether it's physics with Feynman, uh, Dirac, Einstein or computer science, Turing, anybody else? Any backstories that you remember that jump out? 'Cause, uh, uh, I'm also referring to not necessarily these stories where something magical happens, but these are personalities. They have big egos. Some of them are super friendly, some of them are like self-obsessed, some of them have anger issues, some of them, how do I describe Feynman? But he appears to, uh, have, uh, appreciation of the beautiful in all its forms. He has a wit and a cleverness and a humor about him. So, like, uh, does that come into play in terms of the construction of the science?

João Batalha 10:25 Well, uh, I think y- you brought up Newton. Newton is a- it's a good example also to think about his backstory because, you know, there's a certain backstory of Newton that people always talk about, but then there's a whole another aspect of him that is also a big part of the person that he was but, uh, you know, he was really into alchemy, right?

Lex Fridman 10:43 Yeah.

João Batalha 10:43 And that he spent a lot of time, uh, thinking about that and writing about it, and he took it very seriously. He was really into Bible s- uh, interpretation, trying to predict things based on the Bible. And so, there- there's also a whole backstory then and you- of course you need to look at it in the context that, uh, and the time that wher- when Newton lived. Um, but it- but it adds to his personality and it's important to also understand those aspects that maybe, you know, uh, I'm not... uh, people are- people are not as proud to teach to little kids, but it's important. It was part of who he was and- and maybe without those he- who knows what- what he would have done otherwise. So...

Lex Fridman 11:22 Well, the- the cool thing about alchemy, I don't- I don't know how it was viewed at the time, but it almost like to me symbolizes dreaming of the impossible. Like most of the breakthrough ideas kind of seem impossible until they're actually done. It's like achieving human flight.

João Batalha 11:39 Mm-hmm.

Lex Fridman 11:39 It's not completely obvious to me that alchemy is impossible or like putting myself in the mindset of- o- of the time. Uh, and perhaps even still (laughs) like ev- everything that, uh ... you know, some of the most incredible breakthroughs are- are- are- would seem impossible. And I wonder the value of believing, almost like focusing and dreaming of the impossible such that it is actually is possible in your mind and that in itself manifests...... whether the accomplishing that goal or making progress in some unexpected direction. So alchemy almost symbolizes that for, for me.

Luís Batalha 12:18 I distinctly remember having the same thought of thinking, you know, when I learned about atoms and, and that they have protons and electrons, I was like, "Okay, to make gold you just take whatever has an a- an atomic weight below it and then shove another proton in there and then you have a bunch of gold."

Lex Fridman 12:32 (laughs)

Luís Batalha 12:32 So, like, why don't people do that? (laughs) . It seemed like conceptually is like, you know, this sounds feasible, huh, you might be able to do it.

Lex Fridman 12:39 And you can actually, it's just very, very expensive.

Luís Batalha 12:41 Yeah, yeah, exactly. (laughs)

Lex Fridman 12:43 (laughs)

Luís Batalha 12:43 Exactly. So in a sense, we do have alchemy and, and it, and it maybe even back then it wasn't as crazy that he was so into it, but, but people just don't like to talk about that as much.

Lex Fridman 12:53 Yeah.

Luís Batalha 12:54 But Newton in general was a very interesting fella.

Lex Fridman 12:56 Anybody else come into mind? In terms of people that inspire you-

Luís Batalha 13:02 Mm-hmm.

Lex Fridman 13:02 ... in terms of people that you just, uh, are happy that they have once or still exist on this earth.

Luís Batalha 13:09 I think, I mean, Freeman Dyson for me. Y- yeah, Freeman Dyson was, was ... I've had a chance to actually exchange a couple of emails with him. He was probably one of the most humble scientists that I've ever met. And that had a, uh, a big impact, uh, on me. We were trying... we were actually trying to convince him to annotate a paper on Fermat's Library, and I sent him an email asking him, um, if he could annotate a paper and his response was something like, "I have very limited knowledge. I just know a couple of things about certain fields. I'm not sure if I'm qualified to do that."

Lex Fridman 13:45 Yeah.

Luís Batalha 13:46 That was his first response. And, uh, and this was someone that should have won a Nobel Prize and worked on a bunch of different fields, um, did some really, really great work. And then just the interactions that I had with him, every time I asked him a couple of, of questions about his papers and, uh, he always responded saying, "I'm not here to answer your, your questions. I, I just want to open-a more questions."

Lex Fridman 14:10 Yeah.

Luís Batalha 14:10 Um, and, uh, so that had a b- big impact on me. It was like just, uh, um, an example of an extremely humble, uh, uh, yet accomplished, uh, scientist. And Feynman was also a big, big inspiration in the sense that he was able to be, um, you know, again extremely talented and, and, uh, scientist, but at the same time socially he was able to, to ... he was also really smart from a social perspective, uh, and he was able to interact with people. He was also a really good, um, teacher, and was also to ... did a v- awesome work in terms of, um, explaining physics to, to the masses and motivating and getting people interested in physics, and that for me was, was also big inspiration.

Lex Fridman 14:56 Yeah, I like the childlike curiosities of some of those folks, like you mentioned Freeman. I've, uh ... Daniel Kahneman I got a chance to meet and interact with. Some, some of these truly special scientists, what makes them special is that even in, uh, older age, there are still like, there's still that fire of childlike curiosity that burns.

Luís Batalha 15:16 Mm-hmm.

Lex Fridman 15:16 And, uh, some of that is like not taking yourself so seriously that you think you've figured it all out, but almost like thinking that you don't n- know much of it. And that's like step one in having a, a great conversation or collaboration or exploring a scientific question. And it's cool how the very thing that probably earned people the Nobel Prize or, or work that's seminal in some way i- is the very thing that still burns even after, uh, they've won the prize. It's cool to see. And they're rare humans, it seems.

Luís Batalha 15:52 And to that point, I remember like the last email that I sent to Freeman Dyson was like in his last birthday. He was really into number theory and primes.

Lex Fridman 16:00 Mm-hmm.

Luís Batalha 16:00 So what I did is I took like a, a photo of him, picture, and then I turned that into like, um, a, a giant prime number.

Lex Fridman 16:09 (laughs) .

Luís Batalha 16:09 So I converted the picture into a, a bunch of one and eights and then I, I moved some numbers around until it was p- a prime, um, and then I sent him that.

Lex Fridman 16:17 Oh, so the, the visual ... like it still looked like the picture-

Luís Batalha 16:20 It's-

Lex Fridman 16:20 ... and it was made up of a prime. That's tricky to do.

Luís Batalha 16:22 It, it's, it's-

Lex Fridman 16:23 That's hard to do.

Luís Batalha 16:24 It looks harder than it actually is. So, the, the way you do it is like you convert the darker regions into eights and the lighter regions in ones.

Lex Fridman 16:33 Mm-hmm.

Luís Batalha 16:33 And then there's-

Lex Fridman 16:34 Then you just keep flipping-

Luís Batalha 16:35 Yeah. But there are some-

Lex Fridman 16:35 ... numbers until (laughs) -

Luís Batalha 16:37 E- exa- but there's like some primality tests that are cheaper from a computational, uh, standpoint.

Lex Fridman 16:41 Yes.

Luís Batalha 16:42 Uh, but it ... what it tells you is, is i- it excludes numbers that are not prime. Then you end up with a set of numbers that you don't know if they are prime or not and then you run-

Lex Fridman 16:51 Right.

Luís Batalha 16:51 ... the full primality test on that. So you just have to keep iterating on that. And it was, it was ... it's, it's funny because when he got the picture he was like, "How did you do that?" He was super curious to ... and then we got into the details and again, this was ... he was already 90, I think 92 or something, and that curiosity was still there.

Lex Fridman 17:09 (laughs) .

Luís Batalha 17:10 Um, so, uh, you could really see that in, in some of these

Luís Batalha 17:13 scientists.

Lex Fridman 17:14 So, could we talk about Fermat's Library?

Luís Batalha 17:17 Yeah, absolutely.

Lex Fridman 17:18 W- what is it? What's the main goal? What's the dream?

Luís Batalha 17:22 It is a platform for annotating papers in its essence, uh, right? And so academic papers can be one of the densest forms of content out there an- and generally pretty hard to understand at times. And, um ... and the idea is that you can make them more accessible and easier to understand by adding these rich annotations to the side, right? And so we can just imagine a PDF view on your browser and then you have annotations on each side and then when you click on them, a sidebar expands and then you have, uh, y- annotations that support LaTeX and Markdown. Uh, and so the idea is that you can say explain a tougher part of a paper where there's a step that is not completely obvious, um, or you can add more context to it, and then over time papers can become easier and, and easier to understand and can evolve in a way. But it really came from, uh-... myself, Luis, and two other friends. We've been, ah, we've had this, this, long-running habit of kind of running a journal club amongst us. We come from different backgrounds, right? I, I studied CS, Luis studied physics. And so we'd read papers and present them to each other and, uh, and then we tried to bring some of that online, and that's, that's, that's when we decided to, to, to build Fermat's Library. Um, then over time, it kinda grew into, into something, uh, with, with a broader goal. Uh, and really what we're trying to do is trying to help, uh, move science in the, in the right direction. Um, that's really the ultimate goal and, and where we wanna take it now.

Lex Fridman 18:55 Well, so there's a lot to be said. So first of all, for people who haven't seen it, it, th- the interface is exceptionally well done. That's, like, execution is really important here.

Luís Batalha 19:05 Absolutely.

Lex Fridman 19:06 The other thing, just to mention, for a large number of people apparently, which is new to me, don't know what LaTeX is. So it's spelled like latex, so be careful googling it if you haven't before.

Luís Batalha 19:20 (laughs)

Lex Fridman 19:21 Uh, it's, uh, uh, sorry, I don't even know the correct terminology.

Luís Batalha 19:24 Typesetting language?

Lex Fridman 19:25 It's a typesetting language, where it's, you're basically program, writing a program that then generates something that looks, from a typography perspective, beautiful.

Luís Batalha 19:37 Absolutely.

Lex Fridman 19:38 And, uh, so a lot of academics use it to write papers. I, I, I think there's, like, a bunch of communities that use it to write papers. I would say it's mathematics-

Luís Batalha 19:48 Mm-hmm.

Lex Fridman 19:48 ... physics, computer science.

Luís Batalha 19:50 Yeah. That's, yeah, that's the-

Lex Fridman 19:52 That's it?

Luís Batalha 19:52 That's the, the main-

Lex Fridman 19:52 Because I'm collaborating currently on a paper with, uh, two neuroscientists from Stanford.

Luís Batalha 19:57 And they don't know what? (laughs)

Lex Fridman 19:58 (laughs) Um, so I'm using, um, Microsoft Word and, uh, Mendeley-

Luís Batalha 20:05 Hmm. Uh-huh.

Lex Fridman 20:06 ... and, like, all of those kinds of things, and it's, and I'm being very zen-like about, about the whole process, but it's fascinating. It's a little heartbreaking, actually, because, um, it actually... it's, it's funny to say, but, uh, and we'll talk about open science, actually, the bigger mission behind Fermat's Library is, like, really opening up the world of science to everybody, is these silly two facts of, like, one community uses LaTeX and another uses Word, is actually a barrier between them. That's like, it's, like, boring and practical in a sense, but it makes it very difficult to collaborate.

Luís Batalha 20:46 Just on that, like, I think the, if there is some people that should have received, like, a Nobel Prize that, but will never get it-

Lex Fridman 20:52 Yeah.

Luís Batalha 20:52 ... and I think one of those is, like, Donald Knuth-

Lex Fridman 20:55 Yeah.

Luís Batalha 20:55 ... because of TeX and LaTeX then-

Lex Fridman 20:57 (laughs)

Luís Batalha 20:57 ... um, because it had a huge impact in terms of, like, just, uh, making it easier for, uh, researchers to put their content out there, like, making it uniform as much as possible.

Lex Fridman 21:07 Oh, you mean like a Nobel Peace Prize for-

Luís Batalha 21:10 Maybe, maybe (laughs) a Nobel Peace Prize. (laughs)

Lex Fridman 21:12 (laughs)

Luís Batalha 21:13 Maybe a Nobel Peace Prize. Yeah, yeah, yeah. The... I, I, I think so.

Lex Fridman 21:17 I mean, he, at a very young age, got the Turing Award for his work in algorithms and so on and so on.

Luís Batalha 21:21 Yeah.

Lex Fridman 21:22 Like, at an incredibly... Like, when I, I think it's in, it might be even the '60s, but I think it's the '70s that, so when he was really young and then he went on to do, like, incredible work with his book and, uh, yeah, with, with TeX, that people don't know.

Luís Batalha 21:36 And going back just when, um, on the reason why we, we ended up, because I think this is, is interesting, the reason why we ended up using the name Fermat's Library, this was because of, uh, Fermat's Last Theorem. And Fermat's Last Theorem is actually a funny story. So Pierre de Fermat, he, he was, like, a lawyer, and he, he, he wrote, like, on a book, um, that he had a solution to Fermat's Last Theorem, which, um, but that didn't fit the margin of that book.

Lex Fridman 22:03 Mm-hmm.

Luís Batalha 22:04 And so Fermat's Last Theorem basically states that there's no solution... if you have, uh, uh, integers A, B, and C, there's no solution to A to the power of n plus B to the power of n equals to C to the power of w- n, if n is bigger than two. So there's, there's, there's no solutions. And he said that, and that problem remained open for almost 300 years, I believe. And a lot of the most famous mathematicians tried to tackle that problem. No one was able to figure that, that out until Andrew Wiles, uh, I think it was in, in the '90s, uh, was able to publish the solution, which was, I, I believe almost 300 pages long. And so it's kind of an aned- anecdote that, you know, there's a lot of, of knowledge and, uh, insights that can be trapped i- in the margins that you... and there's a lot of potential energy that you can release-

Lex Fridman 22:56 (laughs)

Luís Batalha 22:56 ... if you actually, um, spend some time, time trying to digest, uh, that. And that was the, the origin story for, for the name.

Lex Fridman 23:05 Yes, you can share the contents of the margins with the world-

Luís Batalha 23:08 Exactly.

Lex Fridman 23:08 ... that could inspire a, a solution or communication that then leads to a solution.

Luís Batalha 23:13 But, but, and, and, and if you think about papers, like, papers are, as, as Joao was saying, probably one of the densest pieces of, of text that a- any human can read. And you have these researchers, like, some of the brightest minds in, in these fields, working on, like, new discoveries and publishing this work on journals that are imposing them restrictions in terms of the number of pages that they can have-

Lex Fridman 23:35 (laughs)

Luís Batalha 23:35 ... to explain a new scientific breakthrough. And so at the end of the day, papers are not optimized for clarity and for a proper explanation of, of that content because there are so many restrictions. And so there's, uh, as, as I mentioned, there's a lot of potential energy that can be freed if you actually try to digest a lot of the contents of, of papers.

Lex Fridman 23:57 Can you explain some of the other things? So Margins, Librarian, Journal Club?

Luís Batalha 24:03 So Journal Club is what a lot of people know us for, uh, where we, every week, we release an annotated paper, uh, in, in all sorts of different fields with physics, CS, math. Margins is kind of the same software that we use to, to run the Journal Club and to host the annotations, but we've made that available for free to anybody that wants to use it. And so-... folks use it at, at universities and, and, um, for running journal clubs, um, and, and so we've just made that freely available. And then Librarian is a browser extension that we developed that is sort of an o- overlay on top of arXiv. So, it's about bringing some of the same functionality around comments, plus adding some extra, uh, niceties to, to arXiv, like being able to very easily extract the references of a paper that you're looking at, or being able to extract the bibtex in order to cite that paper yourself. Um, so it's an overlay on top of arXiv. Yeah, it is, and you can have that commenting interface without having to leave arXiv.

Lex Fridman 25:02 It's kind of incredible. I didn't know about it, and once I've learned of it, it's like, "Holy shit." Why isn't it more, more popular given how popular arXiv is? Like, everybody should be using it. ArXiv sucks, in terms of its interface.

Luís Batalha 25:17 (laughs)

Lex Fridman 25:18 Or, it's, uh, let me rephrase that. It's limited (laughs) -

Luís Batalha 25:20 Yeah.

Lex Fridman 25:20 ... in terms of its interface.

Luís Batalha 25:21 arXiv is a, is a pretty incredible project, right? And, and it is in, in a way it's, it, you know, it, it, the growth has been completely n- linear over time. If you look at, like, number of papers published on arXiv, like, you know, it's just been, it's pretty much a straight line for the past 20 years.

Lex Fridman 25:37 Mm-hmm.

Luís Batalha 25:38 Especially for, you know, f- f- like, if you're coming from a startup background and then you were trying to do arXiv, uh, you'd probably try, like, all sorts of growth acts and, like, try to, to then maybe, like, have paid features and things like that. And, and that would kind of maybe ruin it. And so there's, th- there's this subtle balance there.

Lex Fridman 25:56 Yeah.

Luís Batalha 25:56 And I don't know what, what, what aspects you can change about it, and... Yeah. For s- some tools in science, it just takes time for them to, to grow. Yeah. ArXiv is, just turned 30, I believe.

Lex Fridman 26:06 Yeah.

Luís Batalha 26:07 And for pe- for people that don't know, arXiv is this kind of online repository where people put, uh, pre-prints, which are versions of the papers before they actually make it to journals.

Lex Fridman 26:17 A-R-X-I-V...

Luís Batalha 26:20 Exactly. Yeah, that one.

Lex Fridman 26:20 ... for people who don't know. And it's actually a really vibrant place to publish your papers in, in the aforementioned, uh, communities in mathematics-

Luís Batalha 26:29 Exactly.

Lex Fridman 26:30 ... physics and computer science.

Luís Batalha 26:31 It started with mathematics and physics, and any, over the, the, the last 30 years, it evolved, and now, actually, computer, computer science now, it's, it's a more popular category than, than physics and math on arXiv.

Lex Fridman 26:42 And there's also, which I don't know very much about, like a b- m- m- biology, medical version of that, is the bioRxiv.

Luís Batalha 26:48 bioRxiv, yeah, bioRxiv. Um, it's- More recent. ... it's, um, it's interesting because if you look at, like, these, um, platforms for pre-prints, they are a- they actually play a super important role because... For inst- if, if you look at a cat- category like math, for some papers in math, it might take close to three years, um, after you click "upload paper" on the journal website and the paper gets published on the website of the journal. So, this is the, literally the, the longest upload period on the internet.

Lex Fridman 27:20 (coughs)

Luís Batalha 27:21 Um, and during those three years, like, it's, it's, you know, it, the, the content is just, y- you know, locked. And so, it's, it, that's why it's so important for people to have websites like arXiv so that you can share that before it goes to the journal with the rest of the world. There was actually an arXiv that, uh, Perelman published the, the three papers that led to the proof of the Poincaré conjec- conjecture. And then you have other fields like machine learning, for instance, where the, the field is evolving at such a high rate that people don't even wait before the papers go to journals before they start working on top of those papers. So, they publish them on arXiv, then other people see them, they start working on that. A- and arXiv did a really good job at, like, building that core platform to host papers. But I, I think there's a really, really big opportunity in building more features on top of that platform apart, apart from just hosting papers. So, collaboration, annotations, and w- uh, like, having other things apart from, from papers, like code, um, and, and other things. Because, uh, for ins- in the field like machine learning, there's a really big, you know, as, as I mentioned, people start working on, on, on top of pre-prints, and they are assuming that that, that pre-print is correct. But you really need a way, for instance, to, maybe n- n- it's not peer review, but distinguish what is good work from bad work on arXiv. How do you do that? So, uh, like, a commenting interface like Librarian, it's useful for that, so that you can distinguish that, um, at, at, at, in a field that is growing so fast as machine learning. And, um, and then you have, uh, platforms that focus, for instance, on, on just biology. bioRxiv is a good example. Um, bioRxiv is also super interesting because there, there's actually an interesting experiment that was run in the '60s. So, in the '60s, the NIH, um, supported this pro- this, um, this experiment called the Information Exchange Group, which at the time was a way for researchers to share biology pre-prints via mail or using libraries. And that project in the 1960s got canceled six years after it started. And it, it was due to intense pressure from the journals to kill that project because they f- they were fearing, uh, com- competition, uh, from, from the pre-prints, uh, for, in, y- for the journal industry. Crick, uh, was also, uh, was one of the famous scientists that opposed to, to the, uh, Information and x- Exchange Group. And it's interesting because right now, if you analyze the number of biology papers that, uh, appear first as pre-prints, it's only 2% of the papers. And it, this was almost fif- almost 50 years after that first experiment. So, you can see, like, that pressure from the journals to cancel that, uh, initial version of a pre-print repo had a tremendous impact on, on, on the number of papers that are showing up in biology as pre-prints. So, it delayed a lot that, uh, that revolution. And, um, but now platforms like bioRxiv are doing that work. But there's still a lot of room for growth there. And I think it's super important because those are the papers that are open that everyone can read.

Lex Fridman 30:32 Okay. So, but if we just look at the entire process of science as a big system, can we just talk about how it can be revolutionized? So, you have an idea. Uh, depending on the field, you wanna make that idea concrete, you wanna run a few experiments in computer science. There might be some code.

Luís Batalha 30:51 Mm-hmm.

Lex Fridman 30:51 There'd be a data set, uh, for, you know, uh, some of the more sort of biology, psychology. You, you might be collecting the data set that's called, you know, a study (laughs) , right? S- uh, so it's par- that's part of that, that's part of the methodology. And so you are putting all of that into paper form, and then you have some results, and then you, you submit that to a place for review-

Luís Batalha 31:20 Mm.

Lex Fridman 31:20 ... through the peer review process.

Luís Batalha 31:21 Mm-hmm.

Lex Fridman 31:22 And there's a process where, how would you summarize the peer review process? But it's, it's really just like a handful of people look over your paper and comment, and based on that, decide whether your paper is good or not. So, there's a whole broken nature to it.

Luís Batalha 31:36 Mm-hmm.

Lex Fridman 31:37 At the same time, I love the peer review process when I buy stuff on Amazon, be- like, uh ... (laughs)

Luís Batalha 31:45 (laughs)

Lex Fridman 31:45 Yeah. For, like, uh, the commenting system, whatever that is. So, okay-

Luís Batalha 31:47 Mm-hmm.

Lex Fridman 31:48 So, there's a bu- a bunch of possibilities for revolutions there, and then there's the other side which is the collaborative aspect of the science, which is people annotating, people commenting, sort of the low effort collaboration which is a comment. Sometimes, as you've talked about, a comment can change everything. But, you know, or a higher effort collaboration, like, more like maybe annotations or even, like, contributing to the paper. You can think of, like, a collaborative updating of the paper over time.

Luís Batalha 32:20 Mm-hmm.

Lex Fridman 32:20 So, there's all these possibilities for doing things, uh, better than they've been done. Can we talk about some ideas in this space? Some ideas that you're working on-

Luís Batalha 32:31 Mm-hmm.

Lex Fridman 32:31 ... some ideas that, uh, you're not yet working on but should be revolutionized 'cause it does seem that archive and, like, uh, open review, for example, are like the Craigslist of science.

Luís Batalha 32:44 (laughs)

Lex Fridman 32:44 Like, like, yeah. Okay. It's, I'm very grateful that we have it, but it just feels like it's like 10 to 20 years. Like, it doesn't feel like that's a feature.

Luís Batalha 32:55 Mm-hmm.

Lex Fridman 32:55 The simplicity of it is a feature.

Luís Batalha 32:57 Mm-hmm.

Lex Fridman 32:57 It feels like it's a, it's a bug.

Luís Batalha 32:59 Mm-hmm. Yeah.

Lex Fridman 33:01 Uh, but then again, the, the pushback there is, uh, Wikipedia has a same kind of simplicity to it, and it seems to work exceptionally well-

Luís Batalha 33:11 Yeah.

Lex Fridman 33:12 ... uh, in, in, in the crowdsourcing aspect of it. So, sorry, this-

Luís Batalha 33:14 Yeah.

Lex Fridman 33:14 There's a bunch of stuffs going on, on the table. Let's just pick-

Luís Batalha 33:17 (laughs)

Lex Fridman 33:18 ... uh, random things that we can talk about.

Luís Batalha 33:20 Wikipedia, you know, for me, it's the cosmological constant of the internet. It's-

Lex Fridman 33:24 Mm-hmm.

Luís Batalha 33:24 I think we are lucky to live in the parallel universe where Wikipedia exists.

Lex Fridman 33:28 Yes.

Luís Batalha 33:29 'Cause if, if someone had pitched me Wikipedia like a publicly edited, uh, encyc- uh, encyclopedia like a couple of years ago, like, it would be... I don't know how many people would have said that that would have survived. Y- I mean, it makes almost no sense. It's like having a Google Doc that everybody on the internet can edit-

Lex Fridman 33:46 Yeah.

Luís Batalha 33:46 ... and, like, that will be, like, the most reliable source for, for knowledge.

Lex Fridman 33:50 On, I don't know how many, but hundreds of thousands of topics.

Luís Batalha 33:54 Yeah.

Lex Fridman 33:55 It's j- (laughs)

Luís Batalha 33:56 It's insane. It's insa- and like you have... And then you have users, like there's one, a single user that edited one-third of the articles on Wikipedia. So, we have these really, really big power users there are-

Lex Fridman 34:08 Yeah.

Luís Batalha 34:08 ... a, a substantial part of, like, what makes Wikipedia s- uh, um, successful. And so, like, n- no one would have ever imagined that, uh, that could happen. Um, and so that, that's, that's one thing I, I completely agree with what you just said. I also-

Lex Fridman 34:24 So, sorry to interrupt briefly. Maybe let's inject that into the discussion of everything else. I also believe, I've seen that with Stack Overflow, that one individual or a small collection of individuals contribute or revolutionize most of the community. Like, if you create a really powerful system for archive-

Luís Batalha 34:44 Mm-hmm.

Lex Fridman 34:44 ... for, or like open review and made it really easy and, uh, compelling-

Luís Batalha 34:49 Mm-hmm.

Lex Fridman 34:50 ... and exciting for one person who's an, like, a 10X contributor to do their thing, that's going to change everything. It seems like that was the mechanism that changed everything for Wikipedia, and that's the mechanism that changed everything for Stack Overflow-

Luís Batalha 35:04 Yeah.

Lex Fridman 35:04 ... is gamifying or making it exciting or just making it fun or-

Luís Batalha 35:08 Mm-hmm.

Lex Fridman 35:08 ... pleasant or fulfilling in some way for those people (laughs) who are insane enough to like answer thousands of questions or, uh, write thousands of factoids and like research them and check them, all those kinds of things, or read thousands of papers.

Luís Batalha 35:25 Yeah. No, Stack Overflow is another great example of that, and, and it's just... And, and those are both two incredibly productive communities that generate a ton of value and, and, and capture almost none of it, right? And it's... And it- they- in a, in a way, it's almost like counter, um... It, it's very counterintuitive that, that, that people, that these communities would exist and, and thrive. Um, and, and it's really hard to... You, you... There aren't that many communities like that.

Lex Fridman 35:58 So, how do we do that for science?

Luís Batalha 36:00 For science.

Lex Fridman 36:00 Do you have ideas there? Like, what, what are the biggest problems that you see? You're working on some of them.

Luís Batalha 36:05 Look, just on that, there are a couple of really interesting experiments that people are running and an example would be like the Polymath projects. So, this is a so- kind of a social experiment that was, uh, created by Tim Gowers, his f- field m- Fields medalist, and his idea was to try to prove that is it possible to do mathematics in a massively collaborate, collaborative way.... on the internet. So, he, he decided to pick a couple of problems and, uh, test that. And they found out that it ac- it actually... It is possible for specific types of problems, uh, namely problems that you're able to break down in- in little pieces and go step by step. You might need... As- as with open source, you might need people that are just kind of reorganizing the- the house every once in a while, and then, you know, people throw a bunch of ideas and then, you know, you make some progress, then you reorganize, you reframe the problem and you go step by step. But they were- were actually able to prove that it is possible to- to, uh, collaborate online, um, and do, and do progress in terms of mathematics. Um... And so I'm- I'm- I'm confident that there are other avenues that could be explored here.

Lex Fridman 37:14 Can we talk about peer review, for example?

Luís Batalha 37:16 Absolutely. I- I think, like, in- in terms of the peer review, I think we... It's- it's important to look at the bigger picture here of, like... of what does scientific... the scientific publishing ecosystem looks like. Because for me, there- there are a lot of things that are wrong about that entire process. So if you look at, for ins- at the... what, um, publishing means in like a- a traditional journal, you have, uh, journals that pay, um, authors for their articles, and then they might pay, like, reviewers to, um, review those articles, and finally, they pay people toum... or distributors to distribute the content. In the... In the scientific publishing world, you have scientists that are usually backed by government grants that are giving away their work for free in the form of papers, and then you have other scientists that are reviewing their work. Uh, this process is known as the peer review process, again, for free. And then finally, we have, um, government-backed universities and libraries that are buying back all those... all that work so that other scientists can we- can read. So this is... For me, it's bizarre. You have the government that is funding the research, it's paying the salaries of the scientists, it's paying the salaries of the reviewers, and it's buying back all that, uh, product of their work again. Um, and I think the problem with this system, and it's wh- it's why it's so difficult to- to break this suboptimal equilibrium, is because of- of the way academia works right now and the way you can progress in- in your academic life.

Lex Fridman 38:57 Mm-hmm.

Luís Batalha 38:58 Um, and so in a lot of fields, the- the competition in academia is- is really insane. So you have hundreds of PhD students, they are, um, trying to get to a- a professor position, and- and it's hyper competitive, and the only way for you to get there is if you publish papers, ideally in journals with a high impact factor.

Lex Fridman 39:23 In computer science, it's al- it's often conferences are also very prestigious or actually more prestigious than journals now.

Luís Batalha 39:30 Okay, interesting.

Lex Fridman 39:31 So- so that's the one discipline where... I mean, that has to do with the thing we've discussed, uh, in terms of the... how quickly the field turns around. But like NeurIPS, CVPR, those conferences are more prestigious, or at the very least, as prestigious as the journals.

Luís Batalha 39:47 The- the journals.

Lex Fridman 39:48 But doesn't matter, the process is what it is.

Luís Batalha 39:50 And- and- and so with the- the... So for people that don't know how... the impact factor of a journal is basically the average number of citations that a paper would get if it gets published on that journal. But so, um, you can really think that, um... The problem with the- the impact factor is that it's a way to turn papers into accounting units. And- and- and let me unpack this because it's... the impact factor is almost like a nobility title. So pa- because papers are born with impact even before anyone reads them. So the researchers, they don't have the incentive to care about if this paper is gonna have a- a long term impact on- on- on the world. What they care... their goal... their end goal is the paper to get published-

Lex Fridman 40:35 Yes.

Luís Batalha 40:36 ... so that they get that value upfront. So for me, that- that is one of the problems of- of that, and that really creates a tyranny of- of metrics. Because at the end of the day, if you are a dean, what you want to hire is like people, researchers that publish papers on journals with high impact factors, because that will increase the ranking of your university and will allow you to charge more for tuition, so on and so forth. And, um... And- and that- that... and especially when you are in super competitive areas, you know, that people will try to gamify that system and- and misconduct starts showing up. Um, there's a, um, a really interesting book on this topic called Gaming the Metrics. It's a book by a researcher called, uh, Mario Biagioli. It goes a lot into like how these, um... the impact factor and metrics affect science negatively, and it's interesting to think, especially in terms of citations, if you look at the early work of like looking at citations, there was a- a lot of work that was done by a guy called Eugene Garfield, and this guy, um... The early work in terms of citation, they wanted to use... they wanted to use citations as from a descriptive point of view.

Lex Fridman 41:47 Mm-hmm.

Luís Batalha 41:47 So what they wanted to y- to create was a map, and- and that map would create a- a visual representation of in- of influence. So citations would be links between papers, and ideally what they would show, they would represent is that you read someone else's paper and it had an impact on your research. They weren't supposed to be counted.

Lex Fridman 42:08 I think th- this inspired like Larry and Sergey's work-

Luís Batalha 42:11 Exactly.

Lex Fridman 42:11 ... right? For Google.

Luís Batalha 42:12 Exactly. I- I think they even mentioned that. But what happens is like as you start counting citations, you create a market and- and the same way like... And this was... The- the work of E- Eugene Garfield was a big inspiration for Larry and Sergey for the PageRank algorithm that, um, you know, led to the creation of- of Google.

Lex Fridman 42:31 Mm-hmm.

Luís Batalha 42:31 And they even recognize that. And- and if you think about it, it's like the same way there's a- a gigantic market for search engine optimization, uh, SEO, where people try to optimize, you know, the- the PageRank and how high the, uh, uh, web page will rank on Google.... the same will happen for papers. People will try to optimize, like their ci- their, uh, th- the impact factors and the citations that they get. And that, um, creates a really big problem. And if, it's super interesting to actually analyze them, if you look at the distribution of, of the high impa- the impact factors of journals, you have like Nature with, uh, Nature I believe is like in the low 40s. And then you have, I believe Science is high 30s, and then you have a really good, uh, um, um, a, a good set of good journals that f- ... will f- fall between 10 and 30, and then you have a gigantic tail of, of journals that have, uh, impact factor below two.

Lex Fridman 43:25 Mm-hmm.

Luís Batalha 43:26 And you can really see two economies here. You see the, the, um, you know, the universities that are maybe less prestigious, less known that where the faculty are pressured to just publish papers regardless of the journal, "What I want to do is increase the ranking of my university." And so they end up publishing as many papers as, as they, they can in like journals with low impact factor. And unfortunately, this is, uh, represents a lot of, of, of the global south.

Lex Fridman 43:54 Mm-hmm.

Luís Batalha 43:55 And then you have the luxury good economy. So for instance, for ... And there are also problems here in the luxury good economy. So if you look at the journal like Nature, so with impact factor of like in the low 40s, there's no way that you're gonna be able to sustain that level of impact factor by just grabbing the attention of scientists. So what, what I mean by that is like for, for the journals, the articles that get published in Nature, they need to be New York Times great. So they need to make it to the, you know, to the, to the big media. They need to be captured by the meed media-

Lex Fridman 44:33 Yes.

Luís Batalha 44:33 ... big media. And it's because that's the only way for you to capture enough attention to sustain that level of citations.

Lex Fridman 44:39 Yes.

Luís Batalha 44:40 And that, of course, creates problems because people then will try to, again, gamify the system and have like titles or abstracts-

Lex Fridman 44:48 Right.

Luís Batalha 44:48 ... or that are bigger, claim- make claims that are bigger than what is actually can be, um, you know, sustained by, by the data or the, the content of the paper. And you'll have clickbait titles or clickbait, uh, abstracts. And again, this is all a consequence of metrics and, uh, science of metrics and, and this is a very dangerous cycle that I think it's very hard to break, but it's happening in academia in a lot of fields right now.

Lex Fridman 45:17 Is it fundamentally the existence of metrics or the metrics just need to be significantly improved? Because, uh, like I said, the metrics used for Amazon for purchasing, I don't know, computer parts is pretty damn good in terms of selecting which are the good ones, which are not.

Luís Batalha 45:35 Mm-hmm.

Lex Fridman 45:35 In that same way, if it, if it, if we had A- Amazon type of review system in the space of ideas, in the space of science, it feels like that, those metrics would be a little bit better.

Luís Batalha 45:47 Mm.

Lex Fridman 45:47 Sort of when it's, um, when it's significantly more open to the crowd-sourced nature of the internet, of the, of the scientific internet. Meaning as opposed to ... Like my biggest problem with peer review has always been that it's like five, six, seven people, usually even less.

Luís Batalha 46:09 Mm-hmm.

Lex Fridman 46:09 And it's often nobody's incentivized to do (laughs) a good job in the whole process. Meaning, it's anonymous-

Luís Batalha 46:17 Mm-hmm.

Lex Fridman 46:17 ... in a way that doesn't incentivize, like doesn't gamify or incentivize great work. And also, it doesn't necessarily have to be anonymous. Like the- there has to be, um ... The entire system is, um ... doesn't encourage actual sort of rigorous review. For example, like open review-

Luís Batalha 46:42 Mm-hmm.

Lex Fridman 46:43 ... does kind of incentivize that kind of process of collaborative review, but it's also imperfect. But it just feels like the thing that Amazon has, which is like thousands of people contributing their reviews to a product, it feels like that could be applied to science, where, uh, the same kind of thing you're doing with Fermat's Library, but doing at a scale that's much larger. It feels like that should be possible given the number of grad students-

Luís Batalha 47:14 Mm-hmm.

Lex Fridman 47:14 ... given the number of, um, general public that's ge- ... Like for example, I personally, as a person (laughs) who got an education in, in mathematics and computer science, like, uh, I can, I can be a, quote-unquote, like "reviewer" on a lot bigger set of things than, than is my exact, uh, expertise.

Luís Batalha 47:37 Mm-hmm.

Lex Fridman 47:37 If I'm one of thousands of reviewers. If I'm the only reviewer, one of five, then I better be like an expert in the thing. But if, if I, uh ... And I've learned this with COVID, which is like you can just use your basic skills as a data analyst, as a-

Luís Batalha 47:54 Mm-hmm.

Lex Fridman 47:54 ... and- to contribute to the review process in a particular little aspect of a paper, and be able to comment, be able to sort of, uh, draw in some references that challenge the ideas presented or to enrich the ideas that are presented. Or, you know, and, and it just feels like crowdsourcing-

Luís Batalha 48:12 Mm-hmm.

Lex Fridman 48:12 ... the review process would be able to allow you to have metrics-

Luís Batalha 48:18 Mm-hmm.

Lex Fridman 48:19 ... in terms of how good a paper is, that are much better representative of its actual impact in the world, of its actual value to the world, as opposed to some kind of arbitrary, gamified, um, version of its impact.

Luís Batalha 48:35 I, I agree with that. I, I, I think we ... There's definitely the possibility at least for more resilient, a more resilient system than what we, we have today and that's, I think that's kind of what you're describing, Alex. And, and I feel, I mean to an extent we, we kind of have like a, a little bit of a-

João Batalha 48:49 ... uh, Heisenberg uncertainty principle, when you pick a metric, as soon as you do it then maybe it works as a good heuristic for, for a short amount of time, but soon enough people would start gamifying and, and, uh-

Lex Fridman 48:59 Yeah.

João Batalha 48:59 ... but, uh, but then you can definitely have metrics that are more resilient to, to gamification and they'll work as a better heuristic to, to try to push you in the, in the, the best direction. Um...

Lex Fridman 49:10 But I guess the underlying problem you're saying is, uh, is there's shortage of positions in academia.

João Batalha 49:16 That's a big problem, right.

Lex Fridman 49:17 Yeah. And, and that... and so, they're going to be constantly gamifying the metrics.

João Batalha 49:22 Yeah. It's a bit of a zero sum game.

Luís Batalha 49:23 It's very compet- it's what? It's a very competitive field, and, and that's what usually happens in very competitive, uh, fields.

Lex Fridman 49:30 Yeah. Yeah.

Luís Batalha 49:32 Uh, but I think some of, like, the peer review problems, like, scale helps, I think, and, and it's interesting to look at, like, what you're mentioning, breaking it down maybe in, in, like, smaller parts and having more people jumping in. Um, but th- th- this is definitely a, a problem. And, and the peer review problem, as I mentioned is, is correlated with the problem of, like, academic career progression and it's all intertwined and it's wh- that's why I think it's so hard to, to break it. Um, there are, like, a couple of really interesting things that are being done right now. There are a couple of, for instance, journals that are overlay journals on top of, um, platforms like arXiv and, uh, bioRxiv that want to remove, like, the more traditional journals from the equation. So, essentially, a journal is just a collection of links to papers. And, and, um, and what they are trying to do is, like, removing that middle man and trying to, to make the review process a little bit more transparent, um, and, and, and not charging universities. Like, uh, there's, there's a couple of... there are a couple of more famous, um, ones. There's one, Discrete Analysis in mathematics. There's one, uh, called The Quantum Journal, which we are actually working with them. We have a partnership with them. For the papers that get published in Quantum Journal, they also get the annotations on Formas, um, and they're doing pretty well. They've been able to grow substantially. The problem there is getting to critical mass, so, it's again, convincing the researchers and especially the young researchers that need, need that impact factor, need th- those publications to have citations to not publish on the traditional journal and go on a, on an open journal and, and publish their work there. There... I think there are a couple of really high profile scientists, so people like Tim Gowers, they are trying to li- sin- incentivize, like, famous scientists that already have tenure and that don't need that to publish that to increase the reputation of those journals so that other maybe younger scientists can start publishing on, on those as well, and so they can try to break that vicious cycle of, of, um, the more traditional journals.

Lex Fridman 51:35 I mean, another possible way to break this cycle is to, like, raise public awareness and just by force, like, ban paid journals.

João Batalha 51:45 Mm-hmm.

Lex Fridman 51:45 Like, what exactly are they contributing to the world? (laughs) Like, basically making it illegal to, uh... Forget the fact that it's mostly federally funded, so that's-

João Batalha 51:57 Mm-hmm.

Lex Fridman 51:57 ... that's a super ugly picture too, but, like, why should knowledge be so expensive? (laughs)

João Batalha 52:04 Hmm.

Lex Fridman 52:05 Like, where everyone is working for the public good and then there's these gatekeepers that, you know, most people can't read most papers-

João Batalha 52:13 Mm-hmm.

Lex Fridman 52:14 ... without having to pay money, and that's... that doesn't make any sense. Es- like, that, that should be illegal.

João Batalha 52:22 I mean, that's... What you're saying is exactly right. I mean, for ins- uh, right? I, I went to school here in the US and Luis studied in Europe, and, uh, you would sit... like, you would ask me all the time to download papers and send it to him because he just couldn't get it and, like, papers that he needed for his research. And so, this-

Lex Fridman 52:37 But he's a student, like, he's-

João Batalha 52:39 Yeah.

Lex Fridman 52:39 Like, uh, I mean-

João Batalha 52:39 He's a grad student.

Lex Fridman 52:40 He was a grad student, but that, you know... I'm even referring to just regular people.

João Batalha 52:44 Oh, yeah. Okay, that too.

Lex Fridman 52:45 (laughs) Like-

Luís Batalha 52:46 Yeah.

Lex Fridman 52:46 And, uh, I think, uh, during 2020, because of COVID, a lot of journalists put down the-

João Batalha 52:53 Mm-hmm.

Lex Fridman 52:53 ... walls for certain kind of coronavirus-related-

João Batalha 52:55 Uh-huh.

Lex Fridman 52:55 ... papers. But, like, that just gave me an indication that, like, this should be done for everything, th- it's, it's, it's absurd. Like, people should be outraged that there is these gates, because... So the moment you dissolve the journals, then there will be an opportunity for startups to, uh, build stuff on top of arXiv. It'll be an opportunity for, like, Formas Library to step up, to scale up-

João Batalha 53:23 Mm-hmm. Mm-hmm.

Lex Fridman 53:23 ... to something much even larger.

João Batalha 53:25 Mm-hmm.

Lex Fridman 53:25 I mean, that was the original dream of, um, Google, which I've always admired, which is make the world's information accessible.

João Batalha 53:32 Mm-hmm.

Lex Fridman 53:33 Actually, it's interesting that Google hasn't... uh, maybe you guys can correct me, but they, uh, have put together Google Scholar, which is incredible, but they... and they've... the, the scanning of books, but they've haven't really tried to make science accessible-

João Batalha 53:48 Mm-hmm.

Lex Fridman 53:48 ... in the, in the f- in the following way. Like, besides doing Google Scholar, they haven't, like, delved into the papers, right?

João Batalha 53:57 Mm-hmm.

Lex Fridman 53:58 Like, they-

João Batalha 53:58 Which is especially curious given what Luis was saying, right? That it's kind of in their genesis, there's- Yes. ... this, like, you know, research shows very connected with all papers reference each other and, like, building a n- a network out of that.

Luís Batalha 54:09 (laughs) And, uh, interesting enough, like, Google... uh, I think there was a, uh, there was not intend... Google+ was like the Google social network-

Lex Fridman 54:16 Yes.

Luís Batalha 54:16 ... that got canceled. Was used by a lot of researchers.

Lex Fridman 54:18 Yes, it was.

Luís Batalha 54:19 Uh, where there I think was, uh, just a, you know, side, kind of a side effect, but then a lot of people ended up migrating to Twitter, but it was not on purpose. But yeah, I agree with you. Like, they haven't, um, gone past, uh, Google Scholar and-

Lex Fridman 54:31 Well-

Luís Batalha 54:31 ... I don't know why.

Lex Fridman 54:32 ... th- that said, Google Scholar is incredible.

Luís Batalha 54:33 Yeah.

Lex Fridman 54:33 For people who are not familiar, it's one of the best, um, aggregation of all the scientific work that's out there, and especially the network that connects to all of them, what cites what, and also trying to aggregate all of the versions of the papers that are available there and trying to merge them in a way that, uh, one particular work, even though it's available in a bunch of places, comes as-... you know, like a central hub of what that work is across the multiple versions. But that almost seems like a s- fun-

Luís Batalha 55:03 Mm-hmm.

Lex Fridman 55:03 ... pet project of a couple of engineers within, (laughs) within Google-

Luís Batalha 55:06 Sure.

Lex Fridman 55:06 ... as opposed to a serious effort to make the world's science accessible.

Luís Batalha 55:11 But, but going back to just the, the, the journals, when you were talking about that, Lex, I th- I believe that in that front, I think we might be past the, the event horizon. So, I think the, the, the model, the business model for the journals, you know, doesn't make sense. They are a middle layer that is not adding a lot of value. And you see a lot of motions, whereas, like, i- in Europe, a lot of the, the papers that are f- get f- are funded by, uh, the European Union, they will have to be, um, uh, uh, open to the public. And I think there's a lot of, uh-

Lex Fridman 55:44 Bill Gates too. Like, the, the-

Luís Batalha 55:45 Yeah.

Lex Fridman 55:45 ... what the Gates Foundation funds, like they, they demand that it, then it w- that it's, uh, accessible to, to everybody. Oh, interesting.

Luís Batalha 55:52 So, I think it's, it's a question of time before that, that wall kind of falls. And, and that is gonna open a lot of possibilities, um, b- because, you know, imagine if, if you had like the layer of, like, that gigantic layer of papers all available online, um, you know, that unlocks a lot of potential as a platform for people to build things on top of that.

Lex Fridman 56:14 But I think it, it, to what you're saying, it is weird. Like, you can literally go and listen to any song that was ever made, uh, on your phone, right? You open Spotify, and you might not even pay for it. You might be on the free version, and you can listen to any song that was ever made, pretty much. But there's, like you, you don't have access to a, a, you know, huge percentage of academic papers, which is just like this fundamental knowledge that we're all funding, but you as an individual don't have access to it. And, and somehow, you know, like, the problem for music got solved, uh-

Luís Batalha 56:48 Yeah.

Lex Fridman 56:48 ... but for papers, it's still like ... It's just not yet. It could be ad supported, all those kinds of things.

Luís Batalha 56:53 Mm-hmm.

Lex Fridman 56:53 And that hopefully, that would change the way we do science. That's the, the most exciting thing for me is, uh, especially once I started like making videos and this silly podcast thing, I started to realize like that if y- you want to do science, one of the most effective ways is to do a, like couple the paper with a set of YouTube videos, like explaining it. Like ...

Luís Batalha 57:18 Yeah. That also seems like there's a lot of room for disruption there. What is the paper 2.0 gonna look like?

Lex Fridman 57:24 Right.

Luís Batalha 57:24 I think like LaTeX and the PDF seems like if you (laughs) ... it's interesting. If you look at the first paper that got published in Nature, and if you look at the paper that got published in Nature today, if you look at the two side by side, they are fundamentally the same. And, uh, even though like the paper that gets published today, you know, you get ... a- a- even, even code, like right now, people put like code, like on, on a PDF, like ... and y- there are so many things that are, uh, related to papers today, you know? You use ... you have data, you have code. Um, you might need videos to, to better explain the concepts. So, it, it's, it ... I th- I think for me, it's natural that there's gonna be also an evolution there, that papers are not gonna be just the static PDFs or LaTeX. There's gonna be a, a next, uh, a next interface.

Lex Fridman 58:14 So, in academia, a lot of things that are judged, you're judged by is often quantity, not quality. I, I wonder if there's a opportunity to have like ... I tend to judge people by the best work they've ever done, as opposed to the ... (laughs)

Luís Batalha 58:29 (laughs)

Lex Fridman 58:29 I wonder if there's a possibility for that, to encourage sort of, um, focusing on the quality, and not necessarily in paper form, but maybe a subset of a paper, a subset of idea, almost even a blog post or an experiment.

Luís Batalha 58:41 Mm-hmm.

Lex Fridman 58:41 Like, why does it have to be published in a journal-

Luís Batalha 58:44 Yeah.

Lex Fridman 58:45 ... to be l- legitimate?

Luís Batalha 58:47 A- and, and it's interes- interesting that you mention that. I also think like, yeah, it's why a- why, why is that the only format? Why can't a blog post or, uh ... we were even experimenting, experimenting with this a, a few months ago where c- can you actually like publish something, uh, um, or, um, like a new scientific breakthrough or, um, or something that you've discovered in the form of like a set of tweets?

Lex Fridman 59:14 Yeah.

Luís Batalha 59:14 A Twitter thread.

Lex Fridman 59:15 Yeah.

Luís Batalha 59:15 Why can't that be possible? And, um, we were expending- experimenting with that idea. Uh, we even, um ... yeah. We r- ran a couple of, of ... like some people submitted a couple of those-

Lex Fridman 59:27 (laughs)

Luís Batalha 59:27 ... like I think the limit was three or four tweets.

Lex Fridman 59:30 Yeah.

Luís Batalha 59:30 Uh, maybe it's a new way to look at a, a, you know, a proof or something. But, uh, I think it, it just serves to show that there should be other ways to publish-

Lex Fridman 59:38 Yes.

Luís Batalha 59:38 ... like scientific discoveries that don't fit the paper format.

Lex Fridman 59:41 Well, y- but, so even with the Twitter thread, it would be, it would be nice to have some mechanism of formalizing it and making it sta- making it into an N- NFT (laughs)

Luís Batalha 59:53 Maybe (laughs) .

Lex Fridman 59:54 Like, a, a concrete thing that you can reference, with a link, that's-

Luís Batalha 59:58 Yeah.

Lex Fridman 59:58 ... unique.

Luís Batalha 59:59 Oh, yeah.

Lex Fridman 59:59 B- because, um, I mean, everything we've been saying, all of that, uh, whi- while being true, it's also true that the, um, constraints and the formalism of a paper works well. It like forces you, constraints, forces you to narrow down your-

Luís Batalha 1:00:19 Mm-hmm.

Lex Fridman 1:00:20 ... thing and, and, uh, literally put it on paper. But, you know ...

Luís Batalha 1:00:25 I agree.

Lex Fridman 1:00:25 Uh, make concrete. And that's why ... I mean, i- it's not broken. It just could be better.

Luís Batalha 1:00:33 I agree.

Lex Fridman 1:00:33 And th- that's the main idea. I think there's something about writing, whether it's a blog post or Twitter thread or a paper, that's really nice to, to concretize a particular little idea, that that can then be referenced by other ideas, then it can be built on top of with other ideas.

Luís Batalha 1:00:53 Mm-hmm.

Lex Fridman 1:00:55 So, uh, let me ask, you've read quite a few papers.You've, uh, annotated quite a few papers. Let's talk about the process itself. How do you advise people read papers? Or y- or maybe you wanna broaden it beyond just papers, but just read concrete pieces of information to understand the insights that lay within.

Luís Batalha 1:01:18 Uh, I would say for papers specifically, I would, I would bring back kinda what Luis was talking about, is, is that it's important to keep in mind that papers are not optimized for ease of understanding. And so, (laughs) right? Th- there's all sorts of restrictions and size, uh, and format and, and, and language that they can use, and so it's important to keep that in mind. And so that if you're struggling to read a paper, doesn't m- it, it might not mean that the underlying material is actually that hard. And so, so that's definitely something that, that especially for us that we, we, we read papers and most of the times it'll be papers that are completely outside of our, of our comfort zone, I guess, and, and, and so it'll be completely new areas to us. Um, so I always try to, to keep that in mind.

Lex Fridman 1:02:05 So, there's usually a certain kinda structure, like abstract introductions-

Luís Batalha 1:02:09 Mm-hmm.

Lex Fridman 1:02:09 ... methodology, uh, depending on the community and so on. Is there something about the process of, like, how to read it, whether you wanna skim it to try to find the parts that are easy to understand or not, uh, reading it multiple times?

Luís Batalha 1:02:25 Mm-hmm.

Lex Fridman 1:02:26 Is, is there any kinda hacks that you can comment on?

Luís Batalha 1:02:29 I remember, like, Feynman had this, this kind of hack when he was reading papers, where he would basically, um... he would dr- I think, I believe he would read the conclusion of the paper, and he would try to just, um, see if he would be able to figure out how to get to the conclusion in, like, a couple of minutes by himself, and, um, and he would read pa- a lot of papers that way. And I think F- Fermi also did that almost. A- and Fermi was known for doing a lot of back-of-the-envelope calculations, so he was a master at, uh, doing that. Um, and, and in terms of, like, uh, especially when, uh, when reading a paper, I think a lot of times people might f- feel discouraged about the first time you read it, you, you know, it's very hard to grasp, or you, you don't understand a lo- a huge fraction of the paper, and I think it's having read a lot of papers in my life, I think I've in peace with, like, the fact that you might spend hours where you're just reading a paper and jumping from paper to paper, reading citations, and, um, like, your level of understanding of, uh, sometimes of the paper is very close to 0%, and all, all of a sudden, you know, everything kind of makes sense then in, in, in your mind, and then, you know, you have this quantum jump where all of a sudden you, you, you understand then the big picture of the paper. And, uh, I, I... and th- and this is an exercise that I have to when reading papers and especially, like, more complex papers, like, okay, you don't understand because you're just going through the process, and just keep going. And, like, and it, it's f- might feel super chaotic, especially if you are jumping from reference to reference. You know, you might end up with, like, 20 tabs open and you're reading a ton of other papers, but it's just trusting that process that at the end, like, you'll find light. Um, and I think for me that's a, a good framework when reading a paper. It's hard, because, you know, you might end up spending a lot of time, and you, it looks like you're lost, but, uh, but th- that's the process to actually, um, you know, understand what they are talking about in a paper.

Lex Fridman 1:04:36 Yeah, I think that process, uh, I enjoy... I've found a lot of value in the process, especially for things outside my field, uh, reading a lot of related work sections and kinda go- going down that path of getting a big context of the field, because what's... especially when they are well written, there's opinions injected into the rel-

Luís Batalha 1:04:57 (laughs)

Lex Fridman 1:04:57 ... the related work, like what work is important, what is not. And if you read multiple related work sections that cite or don't cite each other, like the papers, uh, you, you get a sense of where the field, where the tensions of the field are-

Luís Batalha 1:05:10 Understand.

Lex Fridman 1:05:11 ... where the s- where the field is striving, uh, and that helps you put into context, like, whether the work is radical, whether it's overselling itself, whether it's underselling itself, all those things, uh, and on t- added on top of that, I find that often the related work section is the most kind of accessible and readable part of a paper, because it's kind of, uh, it's brief, to the point, it's trying to... like summarizing, it's almost like a Wikipedia-style article. The introduction is supposed to be a compelling story or, or whatever, but it's often like overselling. There, there's like an agenda-

Luís Batalha 1:05:49 Mm-hmm.

Lex Fridman 1:05:49 ... in the introduction. (laughs) The related work usually has the least amount of agenda except for the few, like, elements where you're trying to, uh, talk shit about previous work or you're trying to sell that you're doing much better. But other, other than that, when you're just painting where the, where the field, uh, came from or where the field stands, it's, uh, really valuable. And also, again, just w- to agree with Feynman, the conclusion. It's like I get a lot of value from the breadth-first search kinda-

Luís Batalha 1:06:17 Mm-hmm.

Lex Fridman 1:06:17 ... read the conclusion, then read the related work, and then, uh, go through the references and the related work-

Luís Batalha 1:06:24 Mm-hmm.

Lex Fridman 1:06:24 ... read the conclusion, read the related work, and just go down the tree un- until you, like, hit dead ends or run out of coffee.

Luís Batalha 1:06:32 (laughs)

Lex Fridman 1:06:32 And then through that process, you go back up the tree, and now you can see the results in their proper, uh, in their proper context, unless of course the paper is truly revolutionary, which even that process will help you understand that is, uh, in fact, uh, truly, uh, revolutionary.

Lex Fridman 1:06:48 You've also, um... uh, you, you talked about just following your Twitter thread in a (laughs) in a depth-first search. You talked about that you read, um, the book on, uh, uh, Grisha Perelman, Grigori Perelman.

Luís Batalha 1:07:01 Mm-hmm. Mm-hmm.

Lex Fridman 1:07:02 And then you would... you, you had a really nice Twitter thread on it-

Luís Batalha 1:07:05 Mm-hmm.

Lex Fridman 1:07:06 ... and you were taking notes throughout, so...At a high level, is there suggestions you can give on how to take good notes? Whether it's, we're talking about annotations or just for yourself to try to, uh, put on paper ideas as you progress through the work in order to then like understand the work better.

Luís Batalha 1:07:24 For me, I always try not to (laughs) underestimate how much you can forget, uh, within six months-

Lex Fridman 1:07:30 Right.

Luís Batalha 1:07:31 ... after you've read something. (laughs)

Lex Fridman 1:07:32 Oh, I thought you were gonna say five minutes, but yeah, six months-

Luís Batalha 1:07:34 (laughs)

Lex Fridman 1:07:34 ... is good.

Luís Batalha 1:07:35 Yeah, or, or, e- or even shorter. And so, that's something that I always try to keep in mind. And, uh, and it's, and it's often, I mean, w- every once in a while, I'll, I'll read back a paper of, that I annotated on Fermat and it's, and, uh, and I'll read through my own annotations and it's, uh, and I've completely forgotten what I had written. And, uh, but it also, it, it also, it's interesting because in a way after you just understood something, you're kinda the best possible teacher that can teach your future self. Uh-

Lex Fridman 1:08:08 Yeah.

Luís Batalha 1:08:08 ... you know, after you've forgotten it, uh, y- you can, y- you're kind of your own best possible teacher at that moment. And so, it's, it can be great to, to try to capture that, uh-

Lex Fridman 1:08:19 It's, it's, it's brilliant. You just made me kinda realize it's really nice to f- to put yourself in the position of teaching an older version of yourself-

Luís Batalha 1:08:30 Exactly.

Lex Fridman 1:08:30 ... that returns to this paper, almost like thinking it literally.

Luís Batalha 1:08:33 That's under-explored, but it's, it's super powerful, because you are the person that you c- like, if you, if you look at the scale from like one, not knowing anything about the topic, and 10, like you are the one that progressed from one to 10 and you know which steps you struggled with. So you are the, really the best person to help yourself make that transition from one to 10. And, um, and a lot of the times like, eh, and, and we don't ... I, I really believe that the framework there we have to, like esp- expose our- ourselves to like be talking to, like us when we were an expert, when we were taking that class and we knew everything about quantum mechanics, and then six months later you don't remember half of the things. Like how could we make it easier for, like do we have those conversations between you and your past sel- past expert self? Um, I, I, I think there, there might be, you know, it's an under- underexplored idea. I think notes on paper are probably not the best way. I'm not sure if it's a combination of like video, audio, where it's like you have a guided framework that you follow th- to extract information from yourself, so that you can later kind of revisit to make it easier to, to remember. But that's, I think it's an interesting idea worth, worth exploring that not ... I've n- I haven't seen a lot of people kind of trying to, uh, distill that problem.

Lex Fridman 1:09:54 Yeah, only cr- creating the kind of tools. I find if I record, it soun- so- sounds weird, but I'll take notes, but if I record audio, like, um, like little clips of-

Luís Batalha 1:10:04 Uh-huh.

Lex Fridman 1:10:04 ... thoughts, or like rants, that's really effective at capturing something that notes can't.

Luís Batalha 1:10:12 Mm-hmm.

Lex Fridman 1:10:12 Because when I replay them, for some reason it loads my brain back into where I was when I was reading that, in a way that notes don't.

Luís Batalha 1:10:21 Mm-hmm.

Lex Fridman 1:10:21 Like, when I read notes, I'll often be like, "Wha- what?"

Luís Batalha 1:10:25 (laughs)

Lex Fridman 1:10:26 "What was I tr- what was I thinking there?" But when I listen to the audio-

Luís Batalha 1:10:30 Yeah.

Lex Fridman 1:10:30 ... it brings you right back to that place. So there might ... And maybe with video, with visual-

Luís Batalha 1:10:35 Mm-hmm.

Lex Fridman 1:10:35 ... that might be even more powerful.

Luís Batalha 1:10:37 I think so.

João Batalha 1:10:37 Yeah. And I, and I think just the process of, you know, verbalizing it s-

Lex Fridman 1:10:42 Yes.

João Batalha 1:10:42 ... that, that alone kind of makes you, you have to structure your thought and, and, and put it in a way that somebody else could come and, and understand it, and, and just the process of that, i- is useful to, to organize your thoughts and, and, and, um, yeah, just, just that alone.

Lex Fridman 1:10:58 Does the Fermat's Library Journal Club have a, like a video component or no?

João Batalha 1:11:03 We ... N- not natively. We sometimes will include, uh, videos, but it's always embedded.

Lex Fridman 1:11:08 Do people like build videos on top of it to explain the paper? 'Cause you're doing all the hard work of understanding deeply the paper.

Luís Batalha 1:11:16 Not ... We haven't seen that happening too much.

Lex Fridman 1:11:19 Oh.

Luís Batalha 1:11:19 But, uh, we were, we were actually playing around with the idea of creating some sort of podcast version where we try to distill the paper, uh, on an audio format that not ... Maybe you could have access to-

Lex Fridman 1:11:30 That might be tricky.

Luís Batalha 1:11:31 Might be trickier, but you c- there are definitely people that could be interested in the paper and that topic, but are not willing to read it, but they might listen to a 30-minute episode on that paper.

Lex Fridman 1:11:41 Yes.

Luís Batalha 1:11:41 You could reach more people and, and you might even bring the authors to the conversation. But it's tricky, and especially for like more technical papers. We've, we've thought about that, doing that, but we haven't, uh, like converged. Not sure if you have any (laughs) uh, tips for that.

Lex Fridman 1:11:55 Well, I'm, I'm gonna take that as a, a, a small project to take one of your, one of the Fermat's, almost like half advertisement and half as a challenge for myself-

Luís Batalha 1:12:03 (laughs)

Lex Fridman 1:12:03 ... to take, take one of the annotated papers and like use it as a basis for creating a, a quick video. Uh, I, I've seen like, um ... Hopefully I'm saying the name correctly, but Machine Learning Street Talk?

Luís Batalha 1:12:17 Mm.

Lex Fridman 1:12:18 I think that's the name of the show, the, that I recommend highly (laughs) , if that's the right name.

Luís Batalha 1:12:23 (laughs)

Lex Fridman 1:12:23 But, uh, they, they do exactly that, which is multiple our breakdown of a paper-

Luís Batalha 1:12:27 Mm-hmm.

Lex Fridman 1:12:27 ... with video component, sometimes with authors. Um, people love it. It's very effective.

Luís Batalha 1:12:33 There's, there's also, I've seen ... I haven't seen the entire ... in its entirety, but I've seen like the, the founder of Comma.ai-

Lex Fridman 1:12:41 Mm-hmm.

Luís Batalha 1:12:41 ... George?

Lex Fridman 1:12:41 Yeah, George.

Luís Batalha 1:12:42 I've seen him like just-

Lex Fridman 1:12:43 Yeah.

Luís Batalha 1:12:43 ... taking a paper and then, uh, uh, you know, distilling the paper, uh, and, and coding it, coding it sometimes during 10 hours.

Lex Fridman 1:12:51 Yeah.

Luís Batalha 1:12:51 And, um, he was able to, you know, get a lot of people interested in, in that and viewing him, like-

Lex Fridman 1:12:57 So I'm a huge fan of that. (laughs) Like, uh, George is a personality. I think a lot of people, like, listen to this podcast for the same reason. It's not necessarily the contents. It's they- they- they- they like to listen to, like, a- a- a silly ru- l- Russian who's, has a childlike brain and s- mumbles and all those, like, struggle with ideas, right? And George is a madman who people just enjoy, like-

João Batalha 1:13:21 (laughs)

Lex Fridman 1:13:21 ... "How is he going to struggle in implementing this particular paper? How is he going to struggle with this idea?" It's fun to watch and that actually pulls you in. The personality's important there.

João Batalha 1:13:29 True, but there's al- there's, you know, I agree with you, but there also, it's visible, like it's th- there's an extraordinary ability that is there, like he's talented and h- he need to have-

Lex Fridman 1:13:41 Yes.

João Batalha 1:13:41 ... if there's a craft and- and this guy definitely has talent and he's doing something that is not easy-

Lex Fridman 1:13:46 Yeah.

João Batalha 1:13:46 ... and I think that also draws the attention of people.

Lex Fridman 1:13:48 Oh, yeah. Yeah.

João Batalha 1:13:49 And- and- and, like, the other day, we were actually, r- we ran into this YouTube channel of this guy that was restoring art, right?

Lex Fridman 1:13:57 Oh. Yeah.

João Batalha 1:13:57 Um, and- and, um, it was basically just a- a video of him, like his v- the production is really, uh, like really well done and it's just him taking really old, um, pieces of- of art, like, and then paintings and then restoring them. But he's really good at that and he describes that proc- and it, that draws attention, uh, draws the attention of people-

Lex Fridman 1:14:18 Yeah.

João Batalha 1:14:18 ... regardless of your craft, be it, like, annotating a paper or, like, restoring a-

Lex Fridman 1:14:23 Craftsmanship.

João Batalha 1:14:23 Yeah.

Lex Fridman 1:14:23 Excellence. Yeah, like George is incredibly good at programming, uh, like quick, like you know those, uh, competitive programmers, like-

João Batalha 1:14:32 Mm-hmm. Uh-huh.

Lex Fridman 1:14:32 ... Topcoder and all those kinds of stuff? He has the same kinda element where the brain just jumps around really quickly and that's, uh, yeah, j- j- just like with art -

João Batalha 1:14:40 And it's also motivating.

Lex Fridman 1:14:41 ... restoration. It's mo- yeah, it's motivating, but- but i- i- and you're right. In- in watching people who are good at what they do, it's motivating even if the thing you're trying to do is not what they're doing. It's just, like, contagious when they're-

João Batalha 1:14:53 Exactly.

Lex Fridman 1:14:53 ... really good at it. And the same kind of analysis with the paper. I think the- not just, like, the final result, but the process-

João Batalha 1:15:01 Mm-hmm. Yeah.

Lex Fridman 1:15:01 ... of struggling with it. That's really interesting.

João Batalha 1:15:03 Yeah. I think, I mean, I think Twitch proved that, like-

Lex Fridman 1:15:06 Yeah.

João Batalha 1:15:06 ... you know, that there's (laughs) really a market for- for that, for watching people do things that they're really good at and, uh, and you will just watch that, you will enjoy that, that- that might even, uh, spike your interest in that specific topic. And, yeah, and people, people will enjoy, uh, watching sometimes hours on ends of- of, yeah, great craftsmen.

Lex Fridman 1:15:27 Do you mind if we talk about some of the papers? Do any papers come to mind that, uh, have been annotated on Fermat's Library?

João Batalha 1:15:34 The- the- the papers that we annotated, uh, can be about completely random topics, but that's part of the, what we enjoy as well. It forces you to explore these topics that otherwise maybe you'd never run into. Uh, and so, s- and so the ones that come to mind, uh, to me are- are fairly random, but one that I- I really enjoyed learning more about is, um, a paper, uh, written by a mathematician actually, Tom Apostol, and, uh, about a, uh, a tunnel in a Greek island off the coast of Turkey. (laughs)

Lex Fridman 1:16:09 (laughs) I like this already.

João Batalha 1:16:10 So it's very random. Uh-

Lex Fridman 1:16:11 Yeah.

João Batalha 1:16:11 ... so this, uh, okay, so what's interesting about this tunnel? So this tunnel, um, was built in the sixth century BC and, um, and it was built in this, uh, uh, in the island of Samos, uh, which is, uh, as- as I said, off the coast of Turkey, and, um, y- right, they had w- the city on one side and they had a mountain and then they had, um, a bunch of springs on the other side and then they wanted to bring water into the city. Um, did, building an aqueduct would be pretty hard because of the- the way the mountain was shaped and it would also, you know, if they, if they were under a siege, like, it, they- they could just, um, easily destroy that aqueduct and then the water w- wouldn't have any water supply. The- the city wouldn't have any water supply. And so they decided to build a tunnel, and they decided to try to do it quickly. Um, and so the, they started digging, uh, from both ends at the same time through the mountain, right? And so, like, when you start thinking about this, it's- it's a fairly difficult problem and this is like sixth century BC, so you had v- very limited access to- to, you know, the- the mathematical tools that you had at the time were very limited. And so what this paper is about is about the story of how they built it and about the fact that for about 2,000 years, kinda the accepted, the accepted explanation of how they built it was actually wrong. And so th- this tunnel has been famous for a while. There are a number of historians that talked about it since ancient Egypt, and, um, and the method that they described, uh, for- for building it, um, is- is, um, was just wrong. And- and so these- these researchers went there and- and were able to figure, figure that out. Um, and so basically kind of the way that they thought they had built it was basically if you can imagine looking at the mountain from the top and you have the mountain and then you have both entrances, um, and so what they, what they thought and what, this is what the ancient historians described is that they, uh, effectively tried to draw a- a right angle, a right angle triangle, um, with the two entrances at each end of the hypotenuse.

Lex Fridman 1:18:25 Mm-hmm.

João Batalha 1:18:26 And the way they did it is like they would w- go around the mountain and kinda walking in a grid fashion and then you can, you can figure out, uh, the two sides of the triangle. And then after you have that triangle, you can effectively draw two smaller triangles at each entrance that are, uh, proportional to that big triangle, uh, and then you kinda have arrows pointing in each way.

Lex Fridman 1:18:49 Got it.

João Batalha 1:18:50 And then you can, you- you know at least that these, that you have a line going through the- the mountain that connects both entrances.

Lex Fridman 1:18:57 Mm-hmm.

João Batalha 1:18:58 The issue with that is, like, o- once you, once you go to this mountain and you start thinking of doing this, you realize that especially given that the tools that they had at the time, that your error margin would be too small.

Lex Fridman 1:19:10 Hmm.

João Batalha 1:19:11 ... you wouldn't be able to do it. Uh, it, you, you, you, the, just the fact of, of trying to, to build this triangle in that fashion, the error would accumulate and you would end up missing. You'd start building these tunnels and they would miss each other.

Lex Fridman 1:19:24 So, the task ultimately is t- to figure out like really perfectly as c- as close as possible the direction you should be digging.

João Batalha 1:19:30 Mm-hmm.

Lex Fridman 1:19:31 First of all, that it's possible to have a straight line through-

João Batalha 1:19:34 Exactly.

Lex Fridman 1:19:34 ... and then what that, the direction would be-

João Batalha 1:19:37 Yeah.

Lex Fridman 1:19:37 ... and then you're trying to infer that by constructing a, a right triangle by doing ... I, I'm not exactly sure about how to do that rigorously, like, by tracing the mountain? By walking along-

João Batalha 1:19:50 Uh-huh.

Lex Fridman 1:19:50 ... the mountain. How to ... you said grids?

João Batalha 1:19:52 Yeah. You, you kinda walk as if you were in a, in a grid and so you, you just walk in right angles.

Lex Fridman 1:19:57 I see.

João Batalha 1:19:58 And so, right, so-

Lex Fridman 1:19:58 But then you have to walk really precisely, then.

João Batalha 1:20:00 Exactly.

Lex Fridman 1:20:00 You, because you have to use tools-

João Batalha 1:20:02 To measure this.

Lex Fridman 1:20:03 ... and then the terrain is probably-

João Batalha 1:20:05 Yeah, very

Lex Fridman 1:20:05 Under this. ... a mess. So, this makes more sense in 2D. In 3D, it gets even weirder.

João Batalha 1:20:09 Yeah.

Lex Fridman 1:20:10 So, okay. Gotcha.

João Batalha 1:20:12 But so this method was described by, like, an ancient Egyptian, uh, uh, historian. I think Hero of Alexandria. And, um, and then for about, like, yeah, for about 2,000 years that's, that's how, like, that's how we thought that they had built this, this tunnel. Um, and then, uh, and yeah, and then these researchers went there and, and, and found out that actually they, they must have had to, to use other methods and, and then in this paper, they describe these, these other methods and th-