Skip to content
The Twenty Minute VCThe Twenty Minute VC

Alex Lebrun: Why the EU's AI Regulation is a Disaster; How Zuck Prepares for Meetings | E1027

Alex Lebrun is the Co-Founder and CEO of Nabla, an AI assistant for doctors. Prior to Nabla, he led engineering at Facebook AI Research. Alex founded Wit.ai, an AI platform that makes it easy to build apps that understand natural human language. Wit.ai was acquired by Facebook in 2015. Prior to Wit, Alex was the Founder and CEO of VirtuOz, the world pioneer in customer service chatbots, acquired by Nuance Communications in 2013. ------------------------------------------- Timestamps: 0:00 Intro 0:20 Who is Alex Lebrun? 3:27 How Mark Zuckerberg Prepares for Meetings 6:56 Does founding a startup get easier over time? 10:05 The AI Hype Cycle 15:48 Evaluating Emad Mostaque’s Predictions 17:14 AI Startups vs Incumbents 18:17 AI Models: Pre-Trained vs Fine-Tuned 27:03 Open-Source vs Closed-Source: Which will dominate AI? 28:27 National Data Sets 36:14 Will AI replace doctors? 59:43 Do French startups sell too soon? 1:05:25 AI in China ------------------------------------------- In Today’s Episode with Alex Lebrun We Discuss: 1. Third Time Lucky and Lessons from Zuckerberg: How did Alex make his way into the world of startups with the founding of his first company? What worked with Alex’s prior companies that he has taken with him to Nabla? What did not work that he has left behind? What were the single biggest takeaways for Alex from working with Mark Zuckerberg? How does Mark prepare for meetings? How does Mark negotiate so well? 2. Open vs Closed: Why does Alex believe the winning AI models will always be open? Why are open models not as transparent as people think they are? What are the biggest downsides to both open and closed models? Does Alex agree with Emad @ Stability that we will have “national data sets”? 3. Incumbent vs Startup: Who wins in the AI race; startups or incumbents? How important is access to proprietary data in winning in AI today? How does Alex respond to many VCs who suggest so many AI startups are merely “a thin layer on top of a foundational model”? Is that a fair critique? Which startups are best placed to challenge incumbents? Which incumbents have been most impressive in adopting AI into existing product suites? 4. Models 101: Size, Quality, Switching Costs: Why will the best companies switch the models that they use often? Will any models in action today be used in a year? How important is the size of the model? How will this change with time? In what way is new EU regulation around models going to harm European AI companies? 5. Location Matters: Who Wins: When looking at China, US and Europe, who is best placed to win the AI war? What are the biggest challenges Europe and China face? Why is the US best placed to win the AI race? What does it have to overcome first? If Alex were a politician, what would he do to ensure his country were best positioned? -------------------------------------------- #AlexLebrun #Nabla #HarryStebbings #20vc #venturecapital #artificialintelligence

Harry StebbingshostAlex Lebrunguest
Jun 19, 20231h 16mWatch on YouTube ↗

EVERY SPOKEN WORD

  1. 0:00 – 0:20

    Intro

    1. HS

      Will AI replace doctors?

    2. AL

      So AI will not replace doctors, but doctors who use AI will replace doctors who don't. (instrumental music)

    3. HS

      Alex, I am so excited for this. I've been looking forward to this one for a long time. I've been thinking with a couple of the AI shows, I can't wait to do this in person with Alex. So thank you so much for joining me today.

    4. AL

      Thanks, Harry.

  2. 0:20 – 3:27

    Who is Alex Lebrun?

    1. AL

    2. HS

      Now, I would love to start with a little bit of context because we're three startups in at this point. So take me back, how did you first make your way into the world of startups?

    3. AL

      So, um, 22 years ago, I, I fell in love with a chatbot. Her name was Sibyl.

    4. HS

      (laughs)

    5. AL

      And really, really, I was, you know, summer night in Paris in my basement alone with my computer. And I, I, I, I found-

    6. HS

      This could go in many directions. (laughs)

    7. AL

      (laughs) Right. Up to you. Um, and, uh, I was mesmerized by this chatbot, you know, the fact that the machine is trying to understand you, language, and can generate some words. And I, I was really, really struck by this thing and I decided I would, okay, I would spend my life building, uh, chatbots. And so I founded, um, a company doing customer service bots 22 years ago, very early, um, called Virtuozz. And this is how I started, um, you know, this series of companies in the, in the domain.

    8. HS

      Can I ask you, given it was 22 years ago, and I know this is off schedule straight away, but just how do you think about market timing today?

    9. AL

      Well, for, for 20 years, you know, when, when we started, uh, I thought chatbots would become very, very intelligent after three, four years and get to AGI and replace humans in call centers. And the more I worked on the problem, the more I realized it was really, really difficult to do that. Actually, the first time we released a bot in Europe was in '24, uh, 2004, to the French railway company. And we were very, very happy to, to have this customer, and the deputy CEO tried it. And she, she asked, "My name is wrong on the, on the reservation." And the, our bot answered, "Hello. Wrong on the reservation." And then we realized, okay, there is still a lot of work. (laughs) It's not smart at all. And so the more I worked on these things, the more I realized it's not ready yet. And so suddenly for the last year, two years, uh, we reach a point where market timing might, might be finally, um, finally good, you know, where the product is ready and people have evolved too, maybe 10 years after Siri, people have changed-

    10. HS

      Mm-hmm.

    11. AL

      ... and they, they are ready to meet this kind of technology. So maybe the market timing is now.

    12. HS

      So if we project forward in thinking of market timing, how did Nabla come to be, and what was that founding moment?

    13. AL

      So the founding moment of Nabla, I was sitting at, at terrace at Facebook, um, headquarters in, in Menlo Park. You know, we joined Facebook through the acquisition of my second startup, always in the, with the AI in this field. Um, and sitting at this terrace with like cocktails, looking at the sunset, suddenly it reminded me of a scene in Silicon Valley. You know the show?

    14. HS

      Yeah, I love it.

    15. AL

      Where they go on the roof, uh, at Google, I mean Hooli, and there, there are these people doing barbecue and say, "Yeah, we, we are vest and rest." (laughs) And, and I saw myself in this vest and rest situation, and after four years at Facebook, uh, I suddenly realized, okay, it's time to go out and back to the arena. And we've learned so many things at Facebook AI Research, it's time to try to push these things to the real world.

    16. HS

      Can I ask, what

  3. 3:27 – 6:56

    How Mark Zuckerberg Prepares for Meetings

    1. HS

      would you say are the biggest takeaways from your time at Facebook and at the AI Research kind of lab?

    2. AL

      So first thing that amazed me when I arrived at Facebook is I, I thought all big companies were slow. Um, and actually when we arrived at Facebook in 20, early 2015, there was maybe 6,000 employees. And it was going so fast, you know, it was a machine, a very well-oiled machine. Um, engineering was so efficient and I, I, I, I list-, I learned it's possible to be that big and very, very efficient. And when you are in this position, you can, there is nothing you cannot do. Um, just after I joined, I started to work on a project and I wanted to hire 200 people to, uh, human concierge, to train my bots (laughs) in real time. It was, it was not in the budget, of course. And, but a week after I had a meeting with Mark and, and in, in about 10 minutes, Zuck, you know, challenged a little bit my idea and he said, "Okay, let's do that." And I could hire 200 people in Menlo Park. And when you have this, um, this speed and agility and, and the like, unlimited resources of a company like Facebook at the time, uh, so I was really shocked by this, uh, at Facebook.

    3. HS

      What did you learn from working with Mark? You mentioned that meeting there and the speed of decision-making from him. Is there anything that you took away from working with him?

    4. AL

      Many things. Um, one thing I like, he spent not a lot of time outside the company, you know, he had like 30 minutes meetings back-to-back from morning to evening. Um, and he prepared really well every meeting. So you have to send in advance like a short note about why, what decision you are expecting from him. If you fail to send this document 24 hours, you know, by the minute before, your meeting get canceled. And so in many cases, you arrive in front of him, he's read your document, he's, he's gathered all the information, the data he needs. And so the, the meeting is fast, very fast, you know, he will challenge, um, your, what you ask for maybe, and, and make a decision. So this way it work, uh, I'm, I'm trying, you know, I think there is much to learn, (laughs) but it, it influenced me.

    5. HS

      I, I totally get you. I love that in terms of structuring his thought process before the meeting. (laughs)

    6. AL

      I'll give you an, uh, an example of that. Um, when I wanted to hire my 200 human concierge, um, my, my reasoning was we will train this AI and it will, uh, work really well and we will steal business from Google in terms of search result, like exactly what ChatGPT may do in the, in the, in the, uh, in the time to come. Um, and I prepared like answers to every possible objection.... he could, he could object of, about everything I was ready to, to say why my idea would work. And I arrive at the meeting, and first minute he tells me, "Alex, I agree with you, it will work. So instead of hiring 200 concierge, let's hire 100,000 and we will kill Google tomorrow." And then I, I completely reverse my course and say, "No, no, no, Marc, w- I'm, actually, I'm not sure if it will work. It would be crazy to hire 100,000 people." (laughs) And I, I, I gave all my reasoning on, on the opposite sense like, "We should be careful. This thing might not work," and so on. So very smart of him. I don't know if it was on purpose, but I had to e- e- learn more about th- the true, the truth of my projects, attacking it from this direction, this angle. And, um, it was, it was very, very interesting.

    7. HS

      That's such a good way to reverse your thinking.

  4. 6:56 – 10:05

    Does founding a startup get easier over time?

    1. HS

    2. AL

      (laughs)

    3. HS

      Can I as- and we mentioned kind of Nabla being your third company. What worked with the prior ones that you've taken with you? What didn't work that you've left behind? And does it get easier?

    4. AL

      It's not getting easier. Um, you just, y- I mean, you learn some lessons. You don't do the same mistakes again, hopefully. But you, you get new, new dangers, you know, new, new issues. Um, the one I suffered from when I started Nabla is what I call the Kim Jong-un, uh, you know, entourage-

    5. HS

      (laughs)

    6. AL

      ... trap. (laughs) And the fa- you know, you, you see this, this video of Kim Jong-un, and he's visiting something, and all the generals are around him with a notebook, and everything he say will be written down. If you fail to do that, I, I guess you disappear the next day. And, um, nobody of course, will ever, ever say, "I, I disagree with you," or challenge any of his decisions. And, uh, starting my third company after two exits, I felt like my investors were always agreed, my team always agreed, um, people around us. And so I wasn't challenged enough, and we made some early mistakes, uh, that probably, uh, um, were not challenged enough.

    7. HS

      What were some early mistakes?

    8. AL

      Uh, so when we decided to, um, to start in healthcare, uh, we were a v- en- engineering team, you know, very strong in AI and engineering, but not so much in healthcare. And we made actually a good decision to start as a B2C healthcare company to learn. So we, we started a clinic. So I think this one was a good decision, but like in, it's an expensive decision. B- um, and we started this B2C virtual primary care clinic, and we learned everything like that. But then we lost our, o- o- our way a little bit, and, uh, in terms of product, of direction. We went too far before realizing that the, m- it was not the right path, and that we should go back to a B2B business model. We probably took too much time to, to make these decisions. Just an example.

    9. HS

      Can I ask you, what do you do then to stop the Kim Jong-un paradox or problem? (laughs) Like, how do you surround yourself then with people that do challenge you? What did you change?

    10. AL

      So, so first with external... The, the solution first is, I think is with, uh, external people. You know, even my investors who are too close to me, but if you go to people further away, they have l- less, and, and who have, who are very successful, and they have no incentive or, you know, to please you, and you get more, uh, information, more data from them. And internally, it's a matter of, uh, just telling your team, "Okay, um, you, you are, you should challenge, uh, our decisions." And we, we did that a lot at Nabla, and sometimes too much. You know, they challenged everything after that. Uh, also, after you make your, one or two mistakes, the team will more naturally come forward. Uh, and this is, uh, before you reach, uh, the right level of, uh, challenger.

    11. HS

      It's funny, one of my friends is Gustav Soderstrom, who's the CPO at Spotify, and he says, "Talk is cheap, and so we should do more of it." (laughs) Which I like as a saying. Are you... We mentioned market timing. I wanna start on kind of where we are in terms of, you know, uh, the landscape

  5. 10:05 – 15:48

    The AI Hype Cycle

    1. HS

      itself, and we look at the developments today. And I think ChatGBT has brought around kind of consumer excitement to a level that we haven't seen obviously in AI for years and years. Have there been fundamental developments in the last 18 months technologically that have led to where we are today with our excitement level? Or is it the continuation of years of behind-the-scenes development from amazing people like you?

    2. AL

      So e- fro- from the, from the, the outside, from, for the general public, it looks like a very big step function with huge advancements every 10 years. Um, I think from the inside, it's much more continuous. Um, so for instance, ChatGPT, you know, is based on GPT-4, which is based on GPT-3, was released three years ago. Uh, GPT-3 is a large language model. It was invented m- before that. Transformers, where the paper was released in 2016. Um, and so the progress is continuous, I think, more or less. Um, but the public perception of it is, is very, very discontinuous.

    3. HS

      Can I ask you, what do you make of the VC hype cycle? Like now, everyone, like, you know, I did an investment the other day, and it's in an enterprise SaaS company. Nothing to do with AI, and I genuinely felt like a true contrarian adventurer (laughs) by not doing an AI deal. What do you and what do the teams around you and the kind of true AI OGs think when they say, "W- see all the VCs just chasing everything AI?"

    4. AL

      Honestly, for, f- from our standpoint, it's, it's really ridiculous. (laughs) And I think, I've been, you know, I've been building AI companies for, uh, 22 years, so I- I've seen this cycle several times. So after a while, you're not surprised anymore. Uh, when I first raised money from my first startup in 2002 or '03, it was already, there, there were two cycles before, and it was in the AI winter, uh, timing, where in 2002, if you have AI in your deck, you, you will never raise anything-

    5. HS

      (laughs)

    6. AL

      ... because they lost money at the end of the '90s. And then, um, 2005, '06, you had to put back AI in, in your deck, and, and so on and so on. Uh, there was a huge, um-... bubble in tw- 2012, when deep learning, uh, came out as better than, uh, other techniques, and then in 2016 with the chatbots, when we joined Facebook.

    7. HS

      I remember the chatbot bubble in 2016.

    8. AL

      So it's-

    9. HS

      That was big for three months. (laughs)

    10. AL

      As an entrepreneur, the only thing you need to know is when to add or remove AI from your deck, and this will change about every three or four years.

    11. HS

      (laughs) I love that. Um, can I ask you, when I used to be into ambassadors, and, you know, obviously as an ambassador today, I, I'd probably say this, but, you know, the common criticism, especially of generative AI, is, you know, bluntly, it's a thin, relatively valueless layer on top of kind of foundational models that are open to everyone. Is that fair or not when we look at generative AI applications today?

    12. AL

      I, I don't think it's fair. Um, you know, when, uh, the C language came out around 1972, uh, some people said, "Hey, now it's so easy to build software, you know, in C compared to Assembly. Every software is just a thin layer on top of C." And of course it's not true, and we know it. (laughs) It's the same when databases came, uh, you know, in the early '80s. Uh, so I think it's, it's really, uh, not fair to think that any AI application on top of LLM is just a thin layer. Um, it's, LLM is a new kind of resource. It's like a new kind of infrastructure that everybody can access, sure. But then it's completely novel. Um, it's very hard to control. You have hallucinations. It, it's non-deterministic. Um, it's very hard to, to configure. You have several knobs you can change, you know, between, uh, uh, fine-tuning or, uh, how you prompt it, and, and, and many other ... You can do reinforcement learning and so on. So, um, it's very hard to control, and also it, it changes every week. You have new LLMs coming out that claims to be better than the one from the week before. How do you decide when to change your LLM? How do you do it without breaking your existing users and product? You know, so there are rules. There, there is a new game. There is a new infrastructure, there is a new game, new rules, and the companies who are the faster to understand these rules and, and, and play a- along these rules will win, and, and build the best products.

    13. HS

      So if we have two ... Sorry, I'm just trying to understand. So if me and you start a new startup today in the travel and expense management category, and we're using fatne- uh, whatever model we choose to use, do, do you have an advantage over me if we're both using the same model?

    14. AL

      I think I have, because I know the, I know how it works internally, so I know the limitations. When something is wrong, I know where I should poke, you know, to find a solution.

    15. HS

      Mm-hmm.

    16. AL

      I also know when there are many, many, and more and more LLMs available, which one will probably be the better suited to my p- problem. Um, and I know what kind of machine learning I have to build around my LLM to, to, um, balance for the, the, the, the weakness of it.

    17. HS

      Sorry, just so I understand. So will we see companies switch between LLMs very frequently? Is that ... Like, what, is that how it works? Sorry, I'm just trying to understand.

    18. AL

      Y- it's how it will work, of course.

    19. HS

      Yeah.

    20. AL

      Um, yeah, yeah, I mean, every two weeks, incredible things coming out of research. And if you just assume it's aesthetic, in, in a year from now, your product will be very dumb compared to your competitor's products.

    21. HS

      Is it-

    22. AL

      Or, or m- more expensive to run or very slow.

  6. 15:48 – 17:14

    Evaluating Emad Mostaque’s Predictions

    1. AL

    2. HS

      Uh, we had, um, Emad on the show from Stability, and he said h- You mentioned hallucinations. He said hallucinations are a feature, not a bug. I didn't really get this. (laughs)

    3. AL

      I, I don't know what you mean. (laughs)

    4. HS

      D- do you understand that?

    5. AL

      I mean, by construction, the LLM has to output something, and so if, if there is nothing to say, uh, it will build something. It will make something up that looks natural, so this is probably what Emad meant. By, by design he, he cannot not output something, and so this would be a hallucination.

    6. HS

      And you also mentioned kind of the changing nature of LLMs and models. Um, he said that no models today would be used in a year. Do you think that's right?

    7. AL

      Absolutely agree with that. You know, will you drive the car you drive today in 10 years? I don't think so. And 10 years in car industry is like one week in, uh, machine learning gravity zone. (laughs)

    8. HS

      (laughs)

    9. AL

      And so there is, of course there is so much progress so fast that I don't see why we would use the same in one year.

    10. HS

      Got you. We, we mentioned your proprietary knowledge, if me and you were to start a startup in the same space at the same time. I clearly obviously agree with you in terms of winning and having an advantage over me. But so many investors are like, "Ah, but it's all about the proprietary data that a company has that they can leverage on top of an existing model."

  7. 17:14 – 18:17

    AI Startups vs Incumbents

    1. HS

      How much of an advantage is it to have existing proprietary data, say being a four to five year old company with four to five years of customer data, versus being a new startup with no existing proprietary data?

    2. AL

      I think VCs are always one train late. (laughs)

    3. HS

      (laughs)

    4. AL

      But s- so proprietary ... Having a lot of pr- proprietary data was very, very important for the l- last, uh, cycle five years ago. Maybe it's, it's less and less true. Um, you need some to bootstrap your models. So for instance, at Nabla, we acquire, you know, we, we build a dataset of, uh, 30,000 consultat- medical consultations, uh, with patient consent. We have to pay doctors. It's very hard to build this, because we need this to bootstrap our product. But then, uh, w- with the new, you know, pre-trained models, uh, fine-tuning is very efficient, uh, with a little bit of data. There is a ... I don't know if you've heard about, uh, LIMA pro- model that came out three weeks ago. Um-

    5. HS

      Can I, can I actually just interrupt you and just ask, for those that don't know,

  8. 18:17 – 27:03

    AI Models: Pre-Trained vs Fine-Tuned

    1. HS

      um, forgive me for the base questions.

    2. AL

      (laughs)

    3. HS

      Um, what's pre-trained and what's fine-tuning? Just so people understand nomenclature.

    4. AL

      Uh, so large language model is trained, uh, in two phases.... first, first phase is unsupervised, uh, pre-training. So it's just fed with huge amount of text and it's trying to, to predict the next word, basically, the next token's the next word. And so this is what we call unsupervised, uh, pre-training. Unsupervised because you don't need any human intervention. The, the text by itself is enough because it will try to predict the next word and on and on and on. Um, and then the second phase is, uh, fine-tuning is, uh, when you use different machine learning techniques, uh, so for instance, reinforcement learning for, uh, ChatGPT, where, um, you train your model to, um, to follow instructions. Like from ChatGPT is when I s- ask a question, you should tell me something that looks like a right answer or s- looks like the, a reasonable answer to what looks like this question. Uh, and so the fine-tuning is the second phase where you give more precise instruction on how it should behave.

    5. HS

      Mm-hmm.

    6. AL

      And, uh, to do that, yeah, there are several techniques. Um, and what, uh, recent papers... So everybody's still learning, you know. It's like huge ba- black boxes, these things, and everybody's still discovering new things. But, uh, three weeks ago, there was a, a paper, uh, about, uh, LIMA. So, so this, uh, this LIMA paper shows that, uh, with, uh, only 1,000, uh, question and answer examples, so very, very small data sets, um, they get something for fi- use for fine-tuning. So the second stage. They get something that performs better than GPT-3 and almost at the level of GPT-4, uh, with only 1,000, uh, QA, you know, for fine-tuning. So the, the thing with LLM is it looks like a, a huge pre-training with text. It, it, it's learning mostly through pre-training with huge amount of text and then with a small amount of data, with high quality data, you can get a lot, you know, you get... you can get what you need, uh, and train your model to do what you expect with very small data sets.

    7. HS

      Can I ask, how low does that go? You mentioned a thousand Q&A there. Small. Like, does that get to a hundred or does that reach some asymptotic point where it's like, "No, a thousand is as small as it will go"? I'm just intrigued.

    8. AL

      It f- it's very hard to, to answer. Um, probably depends on the domain you are, you are teaching your model. Probably no, you know, won't go less than a hundred or something like that because you still need to give in, you know, information... You cannot compress information, uh, infinitely, and you still need to tell your model what you want and so you... There is a limit somewhere. I don't know where. (laughs)

    9. HS

      Mm-hmm. Can I as... Uh, and we mentioned models there and we mentioned kind of size of models there and actually, uh, how you only need actually very small amounts today. Will there be new foundational model companies created do you think, or do you think we have the existing incumbents already?

    10. AL

      So, we probably have most of the existing companies. Um, maybe a few will, will be getting started. Uh, there is a lot of hype and it's public, so I can spill the, the bean about Mistral, a new one created by, um, uh, three engineers, two of them from, from Facebook. And I know them and, you know, you can, you can still build this foundational model, but you need-

    11. HS

      How can you... Sorry, I... I know them too and they're fantastic. But how can you do it without being so far behind?

    12. AL

      So to, to make, uh, a, a successful foundational model company you need incredible scientists and a huge amount of money.

    13. HS

      (laughs)

    14. AL

      And if you check these two boxes, you, I think you can still do it. Um, and you need focus. Uh, you know, the, the... I k- I know Yann discussed that and I totally agree what he said here, that Google and Meta could have done what OpenAI did, you know, easily and they didn't because it was not a priority and there is all the legal, uh, people around and... But, uh, I think a new startup with enough talent and enough money, and when I say enough money I'm talking about billions, just to be clear, then I'm sure you can do a, as m- at least a- as, as well as OpenAI.

    15. HS

      Can I as... We, we mentioned incumbents, we mentioned startups, they're kind of encroaching on incumbent space. I think the big thing that I think about is like, Bluntney, who wins in this next wave? Is it startups, uh, creating amazing new products leveraging foundational models? Or is it actually Adobe? Is it actually Apple? Is it Google with Bard? How do we think about where value accrues to startup or incumbent? With that wonderful movement of hands.

    16. AL

      (laughs)

    17. HS

      (laughs)

    18. AL

      So incumbents have a huge advantage through distribution. That's uh, w- one thing that is hard for us, uh, as startups. Um, but, uh, incumbents have many, uh, disadvantages. I think first, they are very slow. And you, you mentioned Adobe, I was sure you would because this is the one example that everybody has, but then who else is, is, w- was as fast (laughs) as Adobe? So I think most of the incumbents, um, won't move, uh, fast. And i- in many cases-

    19. HS

      Do you not think they're moving faster? I mean, I- I don't know if you call Notion an incumbent, but like Notion have moved very fast, Adobe moved very fast, Navan, the travel expense management company now is like solely on OpenAI. I think actually they have moved very fast. And do a- argue back with me, I know nothing.

    20. AL

      Hap- No, no, that's, that's fair. You know, some of them can move fast and it's, it's not fair to say they are slow and that's it. I think that incumbents suffer from problems because in many cases, th- they will do, you know, AI enhanced features.

    21. HS

      Mm-hmm.

    22. AL

      And this is what all the one you mentioned-

    23. HS

      Yeah.

    24. AL

      ... did. You know, the, the Notion, you still have a document, you have edit, you have a cursor. You write, "Hello." By the way, you can call ChatGPT and to, to, to summarize a paragraph is what I call spreading a li- a little bit of AI dust on the, magic dust on your existing product.... who knows how differently you can think about, um, building knowledge for your company. It- it- it probably something will come and destroy Notion and Google Docs and all, all of them with a totally new paradigm that is made possible by AI, and certainly these incumbents won't, won't do it. So I think the, the best incumbents can benefit from AI to, uh, to be competitive, you know, in their existing markets, but I don't think they will invent totally disruptive things that will kill them. You know, Kodak, Kodak invented the digital camera, and of course they never released it because it would kill their main revenue, which was, uh, the, the films. And-

    25. HS

      Do you think that's why Google didn't innovate in the way that they could have done, because actually the cost per query, doing it, you know, bluntly like ChatGPT does is so much more expensive than the way that they do it today? And actually it would have cannibalized their whole business-

    26. AL

      (laughs)

    27. HS

      ... if they were like, "Hey, let's embrace this new cost structure, which is so much more expensive."

    28. AL

      I think the reason is, is, is simpler than that. Uh, e- the reason is nobody could predict that LLMs would be so useful and, and powerful before you train one at, at this scale, and who in the Google org chart had the incentive to invest 500 million dollars and just to see this without any business benefit for the company? If you add to that the, the legal department who is not that happy that you are, like, releasing random chat bots who can say anything-

    29. HS

      (laughs)

    30. AL

      ... um, then nothing happens. And this is what happened at Google, and probably also at Meta where nobody has a, the incentive to, to do that. And so OpenAI, a- as, as a private company, they need to, you know, to show something, to build products, and they have this, this curiosity and the financial power, and they did it.

  9. 27:03 – 28:27

    Open-Source vs Closed-Source: Which will dominate AI?

    1. HS

      versus closed?

    2. AL

      Open, obviously. (laughs) Uh, no, LLM are, AI in general is an infrastructure in the future. And, and like every infrastructure, I'm just paraphrasing Yann, but is, is open, uh, open wins always with infrastructure. Uh, and, um, so obviously I think the financial mo- the foundational model, uh, that will win will be open. But that being said, you know, we should be careful because even an, an open model trained with open data to me is not that open, because when you have like 300 billion parameters and it's a huge black box, you don't understand why the output is what it is, is it really open? And so I'm just putting a little bit of salt on, on what I hear on the like, open models are you understand everything, it's explainable, it's predictable. It's not true. You know, open, an open model can be as hard to predict, uh, tha- that and that a closed model.

    3. HS

      Is there anything that can be done to change that then? And is that a problem?

    4. AL

      Sorry?

    5. HS

      Is that a problem that it's as hard to predict as a closed model? Is it... yeah.

    6. AL

      It's, it's a, it's a, yeah, it's a very good point, Raymond. And when I mentioned the, the limitations of LLMs and why when you do a real world application, and you know, it still works in healthcare, you need to work a lot to, uh, circumvent these

  10. 28:27 – 36:14

    National Data Sets

    1. AL

      limitations.

    2. HS

      Can I ask you, in terms of ... we mentioned kind of data sets before, and I, I did just want to touch on it 'cause I think it's important. You know, and I am actually going back to Iman's episode, but he said about the importance of national data sets. And actually for certain things, I think it really is actually very clear that that might be the way, whether it's, um, neobanks, you know, Western economies are very different to, you know, Eastern economies. Um, healthcare, consumer healthcare, you know, if you have sunlight exposure, it's very different to if you have no sunlight exposure, or diets or whatever. Do you think we'll have national data sets moving forward given how different nations have such different data?

    3. AL

      I'm not sure about national data sets. I mean, if you want to sell something to a nation, it's good to say that. But, uh-

    4. HS

      (laughs)

    5. AL

      ... I'm more... I, I think like industry, you know, per industry, per vertical, as I said, of course will improve the quality of the model. So if you, if you train an LLM only on medical data, like we did with Pan, you get a better... with MedPan, you get a better LLM for medical applications. But even you have to understand, I think not, uh, many people make the mistake, even if you feed an LLM with curated data, you don't, it does, it's not guaranteed that the output will be, uh, will be good, will be perfect, that you can trust the output. So feeding an LLM with trus- trusted data do- doesn't make the output trustable because of how LLM works, you know? Uh, uh, so there is... there, there are lots of misconception about this.

    6. HS

      Help me understand that then, 'cause I think a lot of people uns- will misunderstand that. So you have great high quality, trusted data. What leads to then high quality output and good decision and outcome?

    7. AL

      So an, an LLM is a huge probabilistic machine. It's like an autocomplete, I think is-

    8. HS

      Mm-hmm.

    9. AL

      ... the, the best image. Um, and it will complete the beginning of your sentence at any cost. It will always complete it. And you may have like 10 facts, 10 sentences, let's say, in the input data set that are true, but maybe the LLM will start with the beginning of the first sentence and switch to the end of the se- another sentence just because it looks good. And, but logically, the, the, factually this output will be totally wrong. And I, I simplified a lot, but this is exactly how it works. Uh, so-... you may have, and the form is perfect. You, you, it always, always looks like, sounds like a very good answer, but the, the, the reasoning, the, the facts may be totally off, invented, or wrong. And even if the input that I said is, is wrong. So when people say, "ChatGPT gave me a wrong output because it was trained on Reddit, and Reddit is, is, uh, a lot of noise." Yes, but only partially. Even if you remove Reddit from the, uh, training data set, it doesn't mean you won't have, like, random answers.

    10. HS

      Yeah, no, I totally get you. Can I ask, uh, we're g- we're kind of jumping around, um, from Emad to Yann, and picking different points, but Yann said in particular that it's a big jump to assume there's a correlation between intelligence and the desire to dominate. When we think about the most common question being the fear of AI really overcoming human power and dominating us, how do you think about Yann's statement about that correlation? Do you agree and, or disagree? And how do you think about that?

    11. AL

      Uh, I fully agree with Yann, and not just because he was my, my boss, my boss at Meta, but... (laughs)

    12. HS

      (laughs)

    13. AL

      Yeah, I think if you really understand how machine learning works, and you're not looking for free publicity, I don't, I don't see why you would say something like that, that, that getting more intelligent will make them want to kill humanity. I, I don't even understand the paths leading to this thinking.

    14. HS

      Can I be blunt? Why does someone like Geoff Hinton then feel that?

    15. AL

      So I, I, I'm not in the head of, uh, Geoff. I, I don't know.

    16. HS

      (laughs)

    17. AL

      Um, I, I don't know. And I, I know Yann is also surprised and, um...

    18. HS

      Yeah.

    19. AL

      Yeah.

    20. HS

      Do you think Elon Musk's decision to be very proactive in terms of the petition to pause AI development, how do you think about that?

    21. AL

      I think it's very hard and, and dangerous to try to comment and understand what Elon Musk is saying. (laughs)

    22. HS

      (laughs)

    23. AL

      In general. Uh, but, um... And it's weird because he, at the same time he said that, I know he was, he's trying to build a team to compete with OpenAI. He hired somebody from DeepMind, like, the same day where he announced we should pause for six months. And it looks to me that the people who are proponent of this pause, or to more regulation, are the one who feel they are in advance. And so it's like a way to say, "Guys, l- let us, we are in front." (laughs) "I don't want more people to start the race." Maybe I'm, uh... (laughs)

    24. HS

      I love that. Can I ask you, you know, you've been really in this space for decades, if you don't mind me, uh, slightly aging you. And there's a lot of people entering who are also very vocal. (laughs) We were talking about it before. How do you feel about that? And I guess, how does it make you and the community around you feel who've been actually working and doing the hard yards for decades?

    25. AL

      So on the one hand, uh, we are proud because we feel, okay, w- I was right before everybody else. You know, I was talking about chatbots to my parents 23 years ago. (laughs) And, uh-

    26. HS

      And they were going, "What the fuck?"

    27. AL

      And, and 20, 22 years ago, they, they discover ChatGPT and say, "Okay, we finally understood what you're, what you are trying to do, Alex." Um, so on one hand, uh, it feels good. On the other hand, it's sometimes frustrating because, you know, like, everybody's jumping in, and some people are really good at spinning narratives, telling stories, you know, better than we are because we are engineers and it's not our culture. And, and, like, things about you, you mentioned where the risk for humanity, things like that. We, we feel it's, it takes a lot of time to debunk. Um, and we feel that most people, not all, you know, some of them are very credible, of course, but, uh, mostly, uh, the people who defend these theses are not, uh...

    28. HS

      You mentioned creating chatbots 22 years ago, and kind of the, the length to where we are today. I always think it takes longer to adopt than we think. Bluntly, when we look at where we are today, are we really at the precipice now? Or actually, is it another 10 years before we see that adoption cycle? I'm just intrigued.

    29. AL

      I think f- finally it's here. Uh, when I see people using ChatGPT and learning to do their job differently with the help of ChatGPT, um, I really feel we are on the verge of finally having a, a huge impact with, with chatbots. Um, for the first 10 years, I, I tried to convince people that chatbots were ready and to, to buy my chatbots. And they say, "No, it's not." And, and then for the 10 following years, people told me, "Oh, chatbots are incredible. We can replace human with chatbot." And I said, "No, no, no, no, (laughs) don't do that. It's not ready." And finally, now, I feel we, we are at a point where, um, many things will, will be impacted by chatbot, which, which doesn't mean, you know, replacing people, um, with them. You have to be... I think it's a great tool, but it doesn't mean it should, because of the chat form, it's tempting to think, "Okay, let's just replace this guy, this worker with a bot." In many cases, I don't think it's ready for that, but it's, but it's, it will still have a huge impact of how work is done.

    30. HS

      You said it's not ready to necessarily replace. When we think about NABR and healthcare, um... (laughs) and I seem to be quoting him a lot, but Emad: "With AI, you can change the nature of a doctor." Fantastic. Thank

  11. 36:14 – 59:43

    Will AI replace doctors?

    1. HS

      you, Emad. Will AI replace doctors?

    2. AL

      So, AI will not replace doctors, but doctors who use AI will replace doctors who don't.

    3. HS

      Mm-hmm.

    4. AL

      Definitely. Uh, AI is making a huge... It can have a huge impact on doctors, the way they work. And those who will embrace this change will, will, you know, will thrive.

    5. HS

      Can I ask you, in the next one to two years, how will those doctors who embrace it, how will they use it?

    6. AL

      So, the, the, the, the first thing I learned when I started to operate a healthcare company and, and a healthcare clinic, because we were a clinic initially, um, is that for the first 50 years of computerization...Um, computers have been a bad news for doctors. Before computers, they see patients, they, they, they care, they ha- there is a little bit of admin work, but it's done by somebody else, so you know, secret- medical secretary, the assistant, and not so much, uh, paperwork. And, and they're very, very, very happy. And suddenly, people like us come to them and say, "Okay, now there is a electronic patient record you need to document. Uh, you need to fill this form. You need to, to do this and that. And we, we have trouble to get the money from the insurance company, so we need to document more to, to make sure the claims go through," and so on, so on, so on. So, you know, the other day I visited a, a small clinic, uh, in, in, in Paris, and all the doctors looked very happy. And I said, "What happened?" You know, it's not normal. (laughs)

    7. HS

      (laughs)

    8. AL

      They say, "Oh, the computer system is down today, so we, we are just using, uh, papers and pen." (laughs)

    9. HS

      (laughs)

    10. AL

      And they were so happy, all of them. Um, and so you have to realize that the, the state today is a very bad state where computers were bad, bad news for them. And because they are drowning, you know, under this administrative work, documentation, um, for many reasons, um, they spend... Today, doctors spend on average, uh, 49% of their time doing this kind of adminis- yeah, administrative task, as opposed to caring for patients.

    11. HS

      Mm-hmm.

    12. AL

      It's not... It's bad for everyone. It's bad for them. Uh, three out of four doctors suffer from symptoms of, of burnout. It's bad for patients. And-

    13. HS

      Thre- three out of four suffer from burnout.

    14. AL

      I- I- yeah, three out of four doctors have suffered from burnout symptoms for the... in the last year. And, and the, the, the main reason of this is they all, you know, mention the, the pressure of all this admin work, basically.

    15. HS

      What is that admin work? I'm... I don't live this life.

    16. AL

      (laughs) So, so the, the biggest, uh, admin work they have to do is clinical documentation. So they, they have to document everything, they hear, the- their decisions. And in, in big systems that we call EHR, you know, electronic health records, which are like dinosaur, uh, systems. Um, and the reason they need to do this documentation more and more is... first it's a financial reason. To get reimbursed by the insurance companies, especially in the US, you need to have like a s- very solid file with all that. Otherwise, the insurance will take the first opportunity, the first pretext not to pay your claim.

    17. HS

      Mm-hmm.

    18. AL

      The second reason is for legal protection, uh, because medical, uh, malpractice trials, you know, it's so expensive. You... Your best protection is to document this in the right way. And so this is why p- mainly, uh, admin work started to, to grow bigger and bigger. And the old... you know, these EHRs are very, very old and clunky systems. Um, I checked the other day, there was one specific clinical action that took 227 clicks, mouse clicks, to be performed in the system.

    19. HS

      (sighs)

    20. AL

      And I posted this on LinkedIn, and many doctors r- uh, answered and said, "No, actually it's more than 300 clicks." (laughs)

    21. HS

      (laughs)

    22. AL

      Um, and so this is the state today.

    23. HS

      Uh, so when we look at that though, the thing that strikes me there is like bluntly, okay, we could actually have, um, recordings of meetings, transcript, you know, speech to text, and then auto suggestions, meeting summary notes, whatever we wanna do. The trouble is, it's healthcare data. It's not me and you chatting about shopping, where we could have preferences. No one really gives a shit actually so much about, you know, what you wanna buy from a supermarket. Um, with healthcare data, does regulation and consumer protections get in the way of efficiency?

    24. AL

      Yeah, so, so first, uh, the solution, I think how AI will impact doctors, it, it will, it will bring an AI assistant to every doctor, to every clinician. Um, this AI assistant will be aware of what's, what's happening, what the patient is saying, what, what data we know about this patient, what documentation, what work needs to be done. How do we-

    25. HS

      How... H- sorry, just to break it down, how does it know what the patient is saying? We record meetings?

    26. AL

      We don't record the meeting, but we capture the audio. We don't store it, we just capture it and then drop it.

    27. HS

      Uh-huh.

    28. AL

      Um, and so yeah, it gets a lot of context from the audio environment, what we call ambient, uh, system.

    29. HS

      Uh-huh. And, uh, the data itself from the consumer, how do we do that? From wearables? From... How do we ingest the data?

    30. AL

      So we... This AI c- assistant is connected to the EHR, so it will get the data from the existing source of truth, which is, uh, this patient's records.

  12. 59:43 – 1:05:25

    Do French startups sell too soon?

    1. AL

      France, we are really good in, um ... We have very good AI engineers, machine learning engineers, because the education system is free, it's very focused on mathematics. Um, so we produce lots of good engineers, but we suck at growing companies, really. We are n- bad at that. I don't know why. Maybe we lack discipline. Maybe we-

    2. HS

      Is it you suck at growing companies or you just sell too soon? Like something also comes in, and actually 50 million, it's a lot of money. You've only raised one round. You'll take home 10, 15 each as co-founders. C'est bon?

    3. AL

      Exactly, c'est bon. It's enough. (laughs)

    4. HS

      (laughs)

    5. AL

      And so I think it's hard to ... If you are normal, it's hard to refuse a $1 billion offer if you haven't done, you know, 10 million before.

    6. HS

      Yeah.

    7. AL

      Uh, but this is what Zuck did, what Google founders, you know, Sergey and, and Larry did. So maybe we're not f- focused enough or disciplined enough to ... And we don't have examples around us, uh, to grow this company. Uh, hopefully it's changing. We have very, very good, uh, scale-ups in France now. But it, it takes some times to ...

    8. HS

      What do you think Europe needs to do to keep pace with the US in terms of AI and being an ecosystem that attracts the best AI talent?

    9. AL

      So Europe is probably, uh, 10 years late compared to US and China, and with the new regulation, we are probably going to take 50 years, uh, more. (laughs)

    10. HS

      Talk to me about that. Why, why do you think that is? Because the regulation is so prohibitive? W- unpack that for me.

    11. AL

      So the new regulation is, is, is a disaster. Uh, if you look at it, there is no way ... People who wrote this, I don't, don't understand really the, the ... They went too fast, you know. Good regulation is about good timing and involving the right people who are actually doing stuff in this domain. Um, I think Europe being late felt like, okay, let's be at least first in regulation, which-

    12. HS

      What was it about the regulation which made this so bad?

    13. AL

      So the regulation ...Some of the aspects say, for instance, that you should kind of be accountable. I'm simplifying, but you should be accountable of the... all the data you use to train your model, for instance. You should make sure that you have a proper license, uh, like, like, explicit consent from everywhere it comes from and which in pra- I see why it's a beautiful idea. Uh, who would disagree with that? But in practice, it means that 100% of the LMs that were trained these last three years would be illegal in Europe. And so, I think it, it's out of, out o- it's, it's, it's not connected to the reality. The, the intent is good, but the, the, um, the limited puts on how to train a model and how to operate a model makes it... in practice what, you know, compared to what we do today, makes everything illegal.

    14. HS

      Does it mean that... Uh, so it makes everything illegal, as you said, what do European startups do then?

    15. AL

      We move to the UK. Uh, maybe for one reason, Brexit-

    16. HS

      Move to the UK?

    17. AL

      Yeah, finally, Brexit is, uh, maybe was good. (laughs) Uh, uh, you know, I'm half joking. If the regulation, uh... it's not, it's not final yet, but if it's really like this and nobody can challenge it, uh, startups like us may have to move. Or, or, uh, maybe we can keep the team in France, but we have to physically train the models elsewhere. I don't, I don't know, but it's, it's totally, uh, it's a big, huge danger.

    18. HS

      What would you do if you were in charge? If I put you in The Hague or in the EU and you were given the regulatory, uh, powers over the next five years of AI in Europe, what would you do?

    19. AL

      Uh, first, I, I, I'd, I'd wait a l- a bit because it's too early. Um, we are still in an- an- a demo phase of AI. You know, LLMs, it's not really used a lot, you kn- uh, just beginning to be used. And as I said, you know, regulation timing is key. You should- you shouldn't be too late, but if you're too early and nobody knows w- what are the risks that are actually... um, uh, what, what actually are the risks? No, not the risk you think because you read some science fiction books, but the... is a real risk. Nobody knows yet. Um, and so I think first I would wait, uh, maybe a year or two to learn from the field what should be regulated and why, and then I would involve people who build these models, of course, but also people who are using it, you know, the users, the, the general public, which was not done so far. So, they have, like, experts who are, who are smart people, but there is no reality of LLMs yet. So, if you regulate too early, uh, it cannot be good.

    20. HS

      Can I ask, do you feel a bit sorry for them? Because, like, there's never been such a large chasm between government's knowledge and then the private sector's knowledge. Are government adequately placed to instill regulation on AI?

    21. AL

      Whi- which government, uh-

    22. HS

      Well, I mean, any, because bluntly scientists, researchers are not in government, and so you have a load of bureaucrats and politicians saying, "Oh, we need you to, uh, be accountable for all of your models." Do they know what a model is?

    23. AL

      Uh, I think some people know what a model is, you know, uh, they, they, they talk to, to some smart people, but, um-

    24. HS

      Have you spoken to many politicians? (laughs)

    25. AL

      Not enough, probably. (laughs)

    26. HS

      (laughs) But do... say, uh, uh, you kind of have to say, "AI is this. That's where we're at."

    27. AL

      Yeah. I, I, I'd love to, um, uh, spend more time, you know, to take them to the field and show them how it works and, uh, and the benefits and the... and the, uh, real risks because I'm not saying there is no need for regulation or no risk, but yes, there is a huge disconnection.

  13. 1:05:25 – 1:16:43

    AI in China

    1. AL

      Um-

    2. HS

      What do you think about China? We talked about Europe there. China, how do you think they've embraced AI and the next wave of AI?

    3. AL

      So, so China, they have, like, a, a key advantage that... like, there is no, uh, GDPR or very few regulation internally.

    4. HS

      Mm-hmm.

    5. AL

      Um, and so they... the, the data quality they have and the amounts, both the quality and, and quantity of data is, is incredible in China. Um, and so for instance for healthcare data, they probably have everything about every individual in China, and if you have the right... if you're supported by your government, I'm sure you have access for, for research, uh, to all this. So, they have this... they are lucky, uh, for that.

    6. HS

      Yeah. No. I, I, I totally agree. Okay, if there's anywhere that, like, bluntly, if you think about geographies, who's the winner and who's the loser in the next 10 years China, US, Europe?

    7. AL

      Well, they all have issues, their issues. So, we talked about the, the advantages of China, but th- they are very close, they are getting more and more close. They have maybe the wrong incentives at the research level. Uh, so it's hard to predict what will come out of, of, uh, China. We talked about Europe where regulation is probably the, the biggest problem that will, uh, keep Europe behind, and we are already behind. That's, that's my concern. The US, you know, is, is, is, uh, like, always in a very good position. Immigration may be lows, maybe something that will, uh, be a problem eventually in the US because talent is so distributed. Uh, with the new... you know, you can learn deep learning very easily from everywhere in the world on, on a tablet today, and it'll, it'll be, it will be more and more, uh, like this. And, and so if it's so hard to go to the US to work for a US company, and eventually it will have an impact, I think.

    8. HS

      Yeah. And then the US, what's the biggest downside of the US?

    9. AL

      So, immigration laws-

    10. HS

      Yeah.

    11. AL

      ... I think are the, are, are the biggest, uh, downside of the US.

    12. HS

      I wanna move to a quick fire round. I've peppered you already. I say a short statement, you give me your immediate thoughts. Does that sound okay?

    13. AL

      Okay. (laughs)

    14. HS

      Okay. So let's start with what do others not know that you know to be true?

    15. AL

      That we live in a simulation.

    16. HS

      Unpack that for me.

    17. AL

      (laughs)

    18. HS

      You-

    19. AL

      I can- I cannot compromise my sources. (laughs)

    20. HS

      (laughs) Do you agree that some of the biggest businesses to be built in AI will be built in services businesses helping enterprises implement AI?

    21. AL

      I, I disagree.

    22. HS

      Huh.

    23. AL

      Uh, I, I think AI will enable a new generation of, of players in every industry that will kill the incumbents eventually.

    24. HS

      What do you think is that time scale?

    25. AL

      Five years.

    26. HS

      Five years?

    27. AL

      Mm-hmm. And, and I mean, existing services companies, like consulting companies, we know will, are embracing AI. They will eventually, they, they will remove the big data from their website and put AI, and then they will put LLM to replace the words. And they, they will take this business, I think. Um, there was a company called Element AI few years ago in Canada, very ambitious, uh, company who tried to do services, uh, like that. And, uh, they eventually failed. They were acquired by ServiceNow for, um, the amounts they raised.

    28. HS

      What is the biggest element you'd like to change about the AI community?

    29. AL

      Make it more diverse. I'm, I'm really tired of talking to people like, just like me. (laughs)

    30. HS

      (laughs)

Episode duration: 1:16:43

Install uListen for AI-powered chat & search across the full episode — Get Full Transcript

Transcript of episode Po6sNSXAhNo

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

Add to Chrome