a16zMarc Andreessen & Amjad Masad on “Good Enough” AI, AGI, and the End of Coding
EVERY SPOKEN WORD
85 min read · 16,970 words- 0:00 – 0:37
Intro
- MAMarc Andreessen
We're dealing with magic here that we, I think probably all would have thought was impossible five years ago-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... or certainly ten years ago. This is the most amazing technology ever, and it's moving really fast, and yet we're still like really disappointed. Like it's not moving fast enough-
- AMAmjad Masad
Mm-hmm
- MAMarc Andreessen
... and like, it's like maybe right on the verge of stalling out. We should both be like hyper excited, but also on the verge of like slitting our wrists-
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
... because like, you know, it's-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... the gravy train is coming to an end.
- AMAmjad Masad
Right. It is faster, but it's not at computer speed, right?
- MAMarc Andreessen
Right.
- AMAmjad Masad
You know, what, what we expect computer speed to be. It's sort of like watching a person work.
- MAMarc Andreessen
It's like watching John Carmack on cocaine. [laughs]
- AMAmjad Masad
The world-- Okay. The world's, the, the world's best programmer on a stimulant.
- MAMarc Andreessen
Yeah, on a stimulant. Yeah, that's right. [upbeat music]
- 0:37 – 3:00
Programming in Plain English
- MAMarc Andreessen
So let's start with, um, let's assume that I'm a sort of a novice programmer, so maybe I'm a student, um, uh, or maybe I'm just somebody, you know, I took a few coding classes and I've hacked around a little bit or like, I don't know, I do Excel macros- [laughs]
- AMAmjad Masad
Mm-hmm
- MAMarc Andreessen
... or something like that. But I'm like, not as, as well as I'm not like a master craftsman of coding. Um, and you know, people, somebody tells me about Replit and, and specifically AI, um, uh, AI and Replit. Like, what, what's my, what's my experience, uh, when, when I launch in with, with what Replit is today with AI?
- AMAmjad Masad
Yeah, I, I would, um, I, I think the experience of someone with no coding experience or some coding experience is largely the same when you go into Replit.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
The first thing we try to do is get all the nonsense away from like setting up development environment and all of that stuff, and just have you focus on your idea. So what do you wanna build? Do you wanna build a product? Do you wanna solve a problem? Do you wanna do a data vis-visualization? So the prompt box is really open for you. You can put in anything there. So let's say you wanna, you know, build a startup. Let-- You have an idea for a startup. I would s- I would start with like a paragraph-long kind of description of what I wanna build. Uh, the agent will read that. It will punch it out.
- MAMarc Andreessen
You just, you t- you just type the, the-
- AMAmjad Masad
Just type it
- MAMarc Andreessen
... standard English.
- AMAmjad Masad
Standard English.
- MAMarc Andreessen
In standard English, you just type it in.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
"I wanna build a... I wanna sell, I wanna sell crepes." Uh, [laughs] "I wanna sell crepes online."
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
So you just like-
- AMAmjad Masad
You can do that
- MAMarc Andreessen
... type in, "I wanna sell crepes online."
- AMAmjad Masad
You can, you can... It literally could be that-
- MAMarc Andreessen
Okay
- AMAmjad Masad
... four words or five words.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
Or it could be if you're, if you have a programming language you prefer or a stack you prefer, you could do that. But, uh, we actually prefer not, for you not to do that, because we're gonna pick the best thing for... We're gonna classify the best stack for that request.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So it's a... If it's a data app, we'll pick Python or Streamlit, whatever. If it's like a web app, we'll pick JavaScript and Postgres and things like that.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So you just type that.
- MAMarc Andreessen
Or you can decide, you can decide, you can say, "And I wanna do it in-"
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
"I know Python," or, "I'm learning Python at school, and I wanna do it in Python."
- AMAmjad Masad
That's right. The, the-
- 3:00 – 5:15
The Vision Behind Replit
- AMAmjad Masad
uh, everyone would want to build software.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And the thing that's kind of getting in, in people's way is this, all the, uh, what Fred Brooks called the accidental complexity of programming, right?
- MAMarc Andreessen
Right.
- AMAmjad Masad
They're like essential complexity, which is like, how do I bring my startup to market, and how do I build a business, and all of that. Accidental complexity is what package manager do I use, and all of that stuff. We've been abstracting away that for so many years, so you can just, um... A- and the last thing we had to abstract away is code.
- MAMarc Andreessen
Right.
- AMAmjad Masad
I had this realization last year, which is, I think we, you know, built an amazing platform, but the business is not performing, and the reason the business is not performing is that code is the bottleneck.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Like, yes, all the other stuff is important to solve, but syntax is still an issue. Like-
- MAMarc Andreessen
Right
- AMAmjad Masad
... you know, syntax is just an unnatural thing for people. So ultimately, English is the programming language.
- MAMarc Andreessen
Right.
- AMAmjad Masad
I, I-
- MAMarc Andreessen
But so just to... Does it work with other, other, uh, world languages other than English at this point?
- AMAmjad Masad
Yes.
- MAMarc Andreessen
Does it-
- AMAmjad Masad
You can, you can write in Japanese, and we have a lot of users, especially Japanese users.
- MAMarc Andreessen
Amazing. Okay. [laughs] That tends to be very popular. So does it support these days, like for the, does, does A- does AI support every language, or is it still, do you still have to do like custom work to craft a new, new language?
- AMAmjad Masad
No, most, most, uh, you know, uh, mainstream languages that has like 100 million plus people, well, speak it, AI is pretty good at it.
- MAMarc Andreessen
Okay. Yeah.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Wow.
- AMAmjad Masad
So, uh, I, I, I did a bit of, a bit of historical research recently for so- for some reason. I just wanna just understand the moment we're in, and because it's such a special moment, it's, as I think it's important to contextualize it. And I, I, I read this quote from Grace Hopper. So Grace Hopper invented the compiler, as you know. Uh, at the time, people were, uh, you know, programming in machine code, and that's what programmers do.
- MAMarc Andreessen
Yeah, of course.
- AMAmjad Masad
That's what the specialists do.
- MAMarc Andreessen
Yes.
- AMAmjad Masad
And she said, "You know, specialists will always be the specialists. They have to learn the underlying machinery of computers. But I want to get to a world where people are programming English."
- MAMarc Andreessen
Right.
- AMAmjad Masad
That's what she said. That's before Karpathy, right? That's like, you know, 75 years ago. Uh, and, and, and that's why I invented the compiler. And in her mind, like C programming is English.
- MAMarc Andreessen
Right.
- 5:15 – 7:00
From Machine Code to English Code
- AMAmjad Masad
step.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Instead of typing syntax, you're actually typing thoughts.
- MAMarc Andreessen
Right.
- AMAmjad Masad
You know, which is what we ultimately want.
- MAMarc Andreessen
And the machine writes the code.
- AMAmjad Masad
And the machine writes the code.
- MAMarc Andreessen
Right. Right.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Um, yeah, I remember it, too. You're, you're probably not old enough, uh, to remember-
- AMAmjad Masad
Mm-hmm
- MAMarc Andreessen
... but I re- I remember when, uh, when I was a kid, it was, um, you know, there, there were, were higher level languages, you know, uh, by the '70s, like, like Basic and so forth, and Fortran, and C and C.
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
But, um, uh, there was still... You know, you still would run into people who were doing assembly programming-
- AMAmjad Masad
Right
- MAMarc Andreessen
... or assembly language. Which by the way, you still do, you know, like game companies-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... or whatever, still do assembly to, to, to get-
- AMAmjad Masad
And they were hating on the kids that were doing Basic. [laughs]
- MAMarc Andreessen
Oh, well, so the, so the assembly people were hating on kids doing Basic, but there were also older coders who hated on the assembly programmers for doing assembly and not, and not, and not-
- AMAmjad Masad
Not punch cards? [laughs]
- MAMarc Andreessen
... and not do-- Oh, no, no, no, not doing direct machine code.
- AMAmjad Masad
Right. Right.
- MAMarc Andreessen
Not doing direct zero and one machine code.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
'Cause, 'cause asse- 'cause assembly la- assem-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... so if people don't know, assembly language is sort of this very low level programming language that sort of compiles to actual, actual machine code.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
And if you, and if it's, it's, it's incomprehensible gibberish to most programmers-
- 7:00 – 9:30
Building Apps with AI Agents
- MAMarc Andreessen
It's, like, just, uh, you know, people never change.
- AMAmjad Masad
Okay. Got it.
- MAMarc Andreessen
Okay, so you s- you're, you're ty- you're typing English, "I wanna sell crepes online, I wanna do this, I wanna have a t-shirt," whatever the, the business is.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Okay, what, what happens then?
- AMAmjad Masad
Yeah, and then, uh, uh, Replit agent will show you what it understood.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
So it's trying to build, um, a common understanding be-between you and it, and I think there's a lot of things we can do better there in terms of UI, but for now it'll show you a list of tasks.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
It'll tell you, "I'm gonna go set up a database," because you need to store your data somewhere. Uh, we need to set up Shopify or Stripe because we need to accept payments. Uh, and then it shows you this list, and gives you two options initially. Do you wanna start with a design so that we can iterate back and forth to get locked d- design down, or do you wanna build a full thing?
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
Hey, if you wanna build a full thing, we'll go for 20, 30, 40 minutes-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... uh, and a- and i- it be like, the agent will tell you, "Go-- Here, install the app."
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
Uh, "I'm gonna go set up the database, do the migrations, write the SQL, you know, build the site. I'm gonna also test it." So this is a recent innovation we did with, um, Agent 3, is that after it writes the software, it spins up a browser and goes around and tests in the browser, and then any issue, it kind of iterates, kind of goes and fix the code. So I'll spend 20, 30 minutes building that. I'll send you a notification. It'll tell you the app is ready.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
So you can test it on your phone. You can ba- go back to your computer. You'll see, maybe you'll find an, a, a bug or an issue. You'll describe it to the agent. It'll say, "Hey, it's not exactly doing what I expected." Uh, or if it's perfect and, and you're ready to go, and that's it, you know, 20 minutes la-- By the way, there's a lot of examples where people just get their idea in 20, 30 minutes, which is amazing. Um, you just hit Publish.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
You hit, uh, you hit Publish. Um, it, uh, couple clicks, you'll be up in the cloud. We'll set up a, a virtual machine in the cloud. The database is deployed. Everything's done, and now you have a production database.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
So think about the steps needed just two or three years ago in order to get to that step. You have to set up your local development environment. You have to sign up for an AWS account. You have to provision the databases, the virtual machines. You have to create the entire pipe, deployment pipeline. All of that is done for you, and it just, you know, a kid can do it, a layperson can do it. If you're a programmer and, uh, you're curious about what the agent did, the cool thing about Replit, because we have this history of being an IDE, you can peel the layers.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
You can open the file tree and you c- look at the files. You can open Git. You can push to GitHub. You can connect it to your editor if you want. You can open it in Emacs. So the cool thing about Replit,
- 9:30 – 11:00
When the Agent Becomes the Programmer
- AMAmjad Masad
yes, it is a vibe coding platform that abstracts away all the complexities, but all the layers are there for you to look at.
- MAMarc Andreessen
Right. So l- let's go back, let's go back to, um, th- that was great. L- but l- let's go back to you said it, it, it gives you, the, the a- the agent gives you, you s- you say, "I've got my idea."
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
You plug it in, and it says, it gives you this list of things.
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
And then you s- and then when you describe it, you said, "I'm gonna do this, I'm gonna do that."
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
The, the I there in that case was the, the agent-
- AMAmjad Masad
Agent. That's right
- MAMarc Andreessen
... as opposed to the user.
- AMAmjad Masad
Yes.
- MAMarc Andreessen
And so the, the agent lists the set of things that it's going to do, and then the agent actually does those things.
- AMAmjad Masad
Agent does those things.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
So yeah, that, that, that's a-
- MAMarc Andreessen
Yeah
- AMAmjad Masad
... that's a, that's a very important point.
- MAMarc Andreessen
Right.
- AMAmjad Masad
When we did this shift, we hadn't realized internally at Replit how much the actual user stopped being the human user, and it's actually the agent programmer.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So one, one really, uh, funny thing happened is we had servers in Asia, uh, and we-- The reason we had servers in Asia, because we wanted our Indian or, you know, Japanese users to be, to have a, you know, a shorter, uh, time to the servers. Uh, when we launched Agent, their experience got significantly worse, and we're like, "What happened? Like, it's supposed to be faster." Well, it turns out it's worse. It's because the AIs are sitting in, uh, in United States, and so the, the programmer's actually in the [chuckles] United States. It's, you're sending the request to the programmer, and the programmer is interfacing with a machine across the world. And so yes, suddenly the agent is the programmer.
- MAMarc Andreessen
Okay. So like the, the term- term, you know, n- n- new terminology, agent is a software program that is basically using the rest of the system-
- AMAmjad Masad
That's right
- MAMarc Andreessen
... a- as if it were a, as if it were a human user, but it's not. It's a, it's a bot.
- AMAmjad Masad
That's right.
- MAMarc Andreessen
Right.
- AMAmjad Masad
It has
- 11:00 – 13:45
Long-Horizon Reasoning and Coherence
- AMAmjad Masad
access to tools such as write a file, edit a file, delete a file, uh, uh, search the package index, install a package-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... uh, provision a database, provision object sto- object storage. It is a programmer that has the tools and interface. It has its sort of an interface-
- MAMarc Andreessen
Right
- AMAmjad Masad
... that, that is very similar to a human programmer.
- MAMarc Andreessen
And then, um, you know, the, uh, uh, we'll talk more about how this all works, but a, a debate inside the AI industry, um, is with the, these, was kind of this, the, you know, this idea now of having agents that do things on your behalf and then go out, you know, go, go out and kind of accomplish missions. Um, th- there's this, you know, kind of debate, which is, okay, how, like obviously, you know, it's a big deal even to have an AI agent that can do relatively simple things. To do complex things, of course, is, you know, one of the great technical challenges of the last 80 years, you know, to, to, to do that. And then there's this sort of this question of like, can the agent go out and run and operate on its own for five minutes, you know, for, for 15 minutes, for an hour, for eight hours? And, and w- meaning like how, sort of like how long does it maintain coherence? Like how long does it actually like stay in full control of its, [chuckles] of its faculties and not kind of spin out? Uh, 'cause at least the earl- early agents or the, the early AIs, if, if you set them off to do this, they might be able to run for two or three minutes, and then they would, they would start to get confused and go down rabbit holes and-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... you know, kind of s- kind of spin out. Um, more, more recently, uh, more recently, um, uh, you know, we've seen that, that, that, that agents can run a lot longer and, and, and do more complex tasks. Like where are we on the curve of agents being able to run for how long and for what complexity of tasks before-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... before they break?
- AMAmjad Masad
That's, uh, that's absolutely the, the, I think the main metric we're looking at, even back in 2023, you know, I've had the idea for software agents, you know, four or five years ago now. The problem every time we attempt them, the, the problem of coherence.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
You know, they'll, they'll, they'll go on for a minute or two, and then they'll just, you know, they just compound in errors in a way that they just can't recover. Um-
- MAMarc Andreessen
And you can actually see it, right? 'Cause they actually, they actually, if you wa- watch them operate, th- they get increasingly confused and then-
- AMAmjad Masad
Yes
- MAMarc Andreessen
... you know, [chuckles] maybe even deranged.
- AMAmjad Masad
Yeah, they very derange.
- MAMarc Andreessen
Yeah.
- AMAmjad Masad
And they g- they go into very weird areas, and sometimes they start speaking Chinese and do-
- MAMarc Andreessen
Right, right. [chuckles]
- AMAmjad Masad
... doing really weird things.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And um, but, uh, I would say sometime around last year, weMaybe cross a three, four, five-minute mark.
- MAMarc Andreessen
[chuckles]
- AMAmjad Masad
And it felt to us that, okay, we're on a path where long re- you know, long-horizon reasoning is getting solved.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
Uh, and so we made, we made a bet, and I te- I tell my team-
- MAMarc Andreessen
So, so sorry, long-horizon reasoning meaning, reasoning meaning like dealing in like facts and logic-
- AMAmjad Masad
Mm-hmm
- MAMarc Andreessen
... um, i-in a, in a sort of complex way, and then long horizon being over a long period of time-
- 13:45 – 17:30
Reinforcement Learning and Problem Solving
- AMAmjad Masad
talking to itself. It's like, "Oh, now I need to go set up a database. Well, what, what kind of tool do I have? Oh, there's a tool here that says Postgres. Okay, let me try using that. Okay, I used that, I got feedback. Let me look at the feedback and read it," and it'll read the feedback. And so the, the, that prompt box or context is where both the user input, the environment input, and the internal thoughts of the machine are all within. It's sort of like a program memory in, in memory space. And so reasoning over that was the challenge for a long time. That's when AIs just like went off track, and now they're able to kind of think through this entire thing and s- and maintain coherence. And there's, uh, there's now techniques around, uh, compression of context.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
So there's still, uh, the context length is still a problem, right? So I would say LLMs today, you know, they're marketed as a million, uh, token, uh, length, which is like a million words almost. Uh, in reality it's about two hundred thousand, and then they start to struggle. So we do a lot of, uh, you know, we stop, we compress the memory. So if a memory-- if, if a portion of the memory is saying that I'm getting all the logs from the database, you can summarize, you know, paragraphs of logs with one statement or the database set up.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
That's it, right?
- MAMarc Andreessen
Right.
- AMAmjad Masad
And so every once in a while we'll compress the context so that we make sure we maintain coherence. So that there's a lot of innovation happened outside of the foundation models as well in order to, to enable that long context coherence.
- MAMarc Andreessen
And what was the, what was the key technical breakthrough at the, in the foundation models that made this possible, do you think?
- AMAmjad Masad
I think it's RL.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
I think it's, uh, reinforcement learning. So the way pre-training works is, you know, uh, the, uh, pre-training is a, uh, the first step of training a large language model. It reads a piece of text, it covers the last words and tries to guess it. That's how it's trained. That doesn't really imply long context reasoning. It, it, you know, it, it, it, it turns out to be very, very effective. It can learn language that way. But the reason we weren't able to move past that limitation is that that modality of training just wasn't good enough. And what you want is you want a type of problem-solving over a, uh, over long context. So what reinforcement learning, uh, uh, especially from code execution gave us is the ability to, for the machine to-- f-for the LLM to roll out what we call trajectories in AI. So a trajectory is a, uh, step-by-step reasoning chain in order to reach a solution. So, uh, the way, uh, as I understand reinforcement learning works is they put the LLM in a programming environment like Replit and say, "Hey, here's a pr- here's a, a code base, here's a bug in the code base, and we want you to solve it." Um, now the human trainer already knows what the solution would look like. So we have a pull request that we have on GitHub, so we know exactly, or we have a unit test that we can run and verify the solution.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So what it does is it rolls out a lot of different trajectories, those-- they sample the model, and maybe one of those trajectories will reach, and a lot of them will just go, go off, off track, but one of them will reach the solution-
- MAMarc Andreessen
Right
- AMAmjad Masad
... by solving the bug.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And it reinforces on that. So that, that gets a reward, and the model gets trained that, okay, you know, this is how you solve these type of problems.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So that's how we were able to extend these reasoning chains.
- MAMarc Andreessen
Got it. And, and how, uh, so two-part question is how, how, how good, how good are the models now at long, long, long reasoning? Uh, and h- and I would say, and how do we know? Like how, how is that established?
- AMAmjad Masad
Um, there is a nonprofit called Meter,
- 17:30 – 21:15
The Verification Loop and Multi-Agent Systems
- AMAmjad Masad
um, that is, um, measuring, uh, u-useful t- It has a benchmark to measure, uh, how long a model runs w-while maintaining coherence and doing useful, useful things, whether-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... it's programming or other ta- benchmark tasks that they've done. Uh, and they put up a paper, I think, uh, late last year that said every seven months-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... uh, the n- the minutes that a model can run is doubling.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
So you go from two minutes to, you know, four minutes in seven months. I think they vastly underestimated that.
- MAMarc Andreessen
Is that right?
- AMAmjad Masad
Vastly.
- MAMarc Andreessen
So it's doubling, it's doubling more often than seven months.
- AMAmjad Masad
We, so Agent 3, we measure that, uh, you know, very closely. Uh, and we measure that in real tasks from real users, so we're not doing benchmarking. We're actually doing A/B tests, and we're looking at the data that how users are successful or not.
- MAMarc Andreessen
Right.
- AMAmjad Masad
For us the, the absolute sign of success is you made an app and you published it, because when you publish it, you're paying extra money. You're saying, "This app is economically useful, I'm gonna publish it."
- MAMarc Andreessen
Right.
- AMAmjad Masad
So that's as clear cut as possible.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And so what we're seeing is in Agent 1, the agent can run for two minutes-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... and then, and then perhaps struggle. Agent 2 came out in February. It ran for twenty minutes.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
Agent 3, two hundred minutes.
- MAMarc Andreessen
Okay. Right.
- AMAmjad Masad
Two hundred min-- And so some users are pushing it to like twelve hours and things like that. I'm less confident that it is as good in, when it goes to these stratospheres, but at like two, three hours timeline, it is really, it's, it's, it's, it's insanely good. And, and the main innovation outside of the models is a verification loop.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
Uh, I remember reading, um, a research paper from NVIDIA. So what NVIDIA did is they're trying to, uh, write, um, GPO kernels, uh, using DeepSeq, and that was, like, perhaps seven months ago-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... when DeepSeq came out. And what they found is that if we add a verifier in the loop, if we can run the kernel and verify it's working, we're able to run DeepSeq for, like, twenty minutes.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
And it, it was generating actually optimized kernels.
- MAMarc Andreessen
Mm-hmm. Right.
- 21:15 – 23:45
Watching AI Work Like a Human Programmer
- AMAmjad Masad
It's not at computer speed, right?
- MAMarc Andreessen
Right.
- AMAmjad Masad
At w- what we expect computer speed to be.
- MAMarc Andreessen
It's like watching a per-- like if you watch the, if you, if it's describing what it's doing, it's sort of like watching a person work.
- AMAmjad Masad
It's like watching John Carmack on cocaine work. [laughing]
- MAMarc Andreessen
The world-- Okay. The world's, so, so say that the, the world's best programmer.
- AMAmjad Masad
[laughs] Yeah.
- MAMarc Andreessen
The world's best programmer on a stim-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... on a stimulant.
- AMAmjad Masad
On a stimulant, yeah, that's right.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
Uh, and so-
- MAMarc Andreessen
Working, working for you.
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
Working for you.
- AMAmjad Masad
Yeah. So the-
- MAMarc Andreessen
Yeah
- AMAmjad Masad
... it's very fast.
- MAMarc Andreessen
Yeah.
- AMAmjad Masad
And you can see the, uh, file diffs running through, but every once in a while it'll stop, and it'll start thinking. It'll show you the reasoning.
- MAMarc Andreessen
Yeah.
- AMAmjad Masad
It's like, "Oh, I did this, and I did this. Am I on the right track?" It kind of really tries to reflect.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Uh, and then it might review its work and decide the next step, or it might kick into the testing agent or, you know. So, so you're seeing it do all of that, and every once in a while it calls a tool. For example, it stops and says, "Well, we ran into an issue. You know, Postgres, um, fifteen is not, um, uh, compatible with this, you know, database ORM package that I, that I have."
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
"Um, okay, this is a problem I haven't seen before. I'm gonna go search the web."
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
So it has a web search tool.
- MAMarc Andreessen
Right.
- 23:45 – 26:00
From Stochastic Parrots to Real Reasoning
- MAMarc Andreessen
there are three. Um, so, um, so it, it, it was this thing, and so people were-- and there was even this term that was being used, kind of the, the, the slur that was being used at the time was stochastic parrot.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Right.
- AMAmjad Masad
I was thinking clanker. [laughing]
- MAMarc Andreessen
Well, well, clanker is the, is the new slur. Clank-clanker, clanker is just the full-on racial slur aga-
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
... against AI as a species. Um, but the, the technical critique was so-called stochastic parrot. stochastic means random.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Uh, so sort of random parrot, mean-meaning basically that this thing was sort of a, the large language models were like a mira-- they were like a mirage-
- AMAmjad Masad
That's right
- MAMarc Andreessen
... where they were, like, repeating back to you things that they thought that you wanted to hear, but they didn't-
- AMAmjad Masad
And in a way, it's true in, uh, in the pure pre-training LLM world.
- MAMarc Andreessen
Right, for the v- for the very basic layer. And, but then what happened is, as, as you said, over the last year or something, there, there was this layering in of, of r- of reinforcement learning, and then, and, but the key to-
- AMAmjad Masad
It's not new, crucially. It's like-
- MAMarc Andreessen
Okay, go ahead
- AMAmjad Masad
... it's AlphaGo, right?
- MAMarc Andreessen
Right.
- AMAmjad Masad
So, uh-
- MAMarc Andreessen
Describe, so describe that for a second.
- AMAmjad Masad
Yeah. So we, we had this breakthrough before in, uh, twenty fifteen was the AlphaGo breakthrough, I think, twenty fifteen, twenty sixteen, where it is a merging of sort of, uh, you know, the, the, the, you would know a lot better than me, the old AI debate between the connectionists, uh, the, the, the people who, who thinks neural networks are the true sort of way of doing AI, and the s-symbolic systems, I think-
- MAMarc Andreessen
Right
- AMAmjad Masad
... or, like, the people that think that, you know, discrete reasonings or if statements and knowledge bases, whatever-
- MAMarc Andreessen
Right
- AMAmjad Masad
... this is the way to go. And so there was, there was a merging of these two worlds, where the way AlphaGo worked is it had a neural network, but it had a Monte Carlo tree search algorithm on top of that. So the neural network would generate, uh, would, would, like, uh, generate a list of potential moves. Uh, and then you had a more discrete algorithm sort those moves and find the best based on just, uh, tree search, based on just, uh, trying to verify. Again, this is sort of a verifier in the loop, trying to verify which move might yield the best based on more classical way of doing algorithms. Um, and so that, that's a resurgence of, of that movement, where we have this amazing generative, uh, neural network, that is the, uh, the LLM, and now let's layer on more discrete ways of trying to verify whether it's doing the right thing or not, and let's put that in a training loop. And once you do that, the LLM will start gaining new capabilities,
- 26:00 – 30:15
Why Coding Is Advancing Faster Than Other Fields
- AMAmjad Masad
such as, uh, reasoning o-over math and code and things like that.
- MAMarc Andreessen
Exactly, right. Okay, and then that's great. And then, and then the, the key thing there, though, for, for RL to work, for LLMs to reason, the, the key is that it be a, a problem statement that there is a def-defined and verifiable answer.
- AMAmjad Masad
That's right.
- MAMarc Andreessen
Is that right? And so, and, and, and you might think about this as, like, uh, let's give a bunch of examples. Like in medicine, this might be, like, um, you know, a diagnosis that, like, a panel of human doctors agrees with.
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
Um, or, or, or by the way, or a diagnosis that actually, you know, solves the condition. Um, in law, this would be a, um, you know, cor-- a, a, a argument that in front of a jury actually results in an acquittal.
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
Uh, or, or something like that. Um, in, um, math, it's an equation that actually solves properly. Uh, in physics, it's a result that actually works in the real world.
- AMAmjad Masad
Mm-hmm.
- MAMarc Andreessen
I don't know, in civil engineering, it's a bridge that doesn't collapse, right? So, so, so there, there, there's always some, some test of correctness-
- AMAmjad Masad
Caveat is that-
- MAMarc Andreessen
Okay, go ahead
- AMAmjad Masad
... the first two do not work very well-
- MAMarc Andreessen
Okay
- AMAmjad Masad
... just yet.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
Like the, the, like the, the, I would say, uh, law and healthcare, they're still a little too squishy, a little too soft.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
It's unlike math-
- MAMarc Andreessen
Okay
- AMAmjad Masad
... or code. Like the way that they're training on math, they're using this, uh, sort of like a program language, uh, provable language called Lean for proofs, right? So you can run a Lean statement. You can run a computer code. Uh, perhaps you can run a physics simulation or civil engineering, uh, sort of physics simulation, but, uh, you can't run a diagnosis.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
So, uh, I would say that-
- MAMarc Andreessen
But you could verify it with human answers or n- or not.
- AMAmjad Masad
Yeah. So that, that's a more-
- MAMarc Andreessen
Or-
- AMAmjad Masad
... RL HF in a way.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
So it is not the like sort of autonomous RL train-
- MAMarc Andreessen
Okay
- 30:15 – 33:45
Verifiable Domains: Math, Code, and Physics
- MAMarc Andreessen
So squ-squ-- So software domains meaning like domains in which it's har- it's harder, harder or even impossible to actually verify correctness of, of result-
- AMAmjad Masad
Yeah, like-
- MAMarc Andreessen
... in a sort of a deterministic, factual, grounded-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... non-controversial way.
- AMAmjad Masad
Like if you have a, a chronic disease, you could, you could have, you know, you have, uh, POTS or, uh, you know, whatever, EDS syndrome or, y- And, and they're all, they're all clusters, and it's-
- MAMarc Andreessen
Right
- AMAmjad Masad
... because it, it is the domain of abstraction. It is not as concrete as code and math and things like that.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So I think there's still a long ways to go there.
- MAMarc Andreessen
Right. So sort of the more concrete the problem, like it's the concreteness of the problem that is the key variable, not the difficulty of the problem. Would that be a way to think about it?
- AMAmjad Masad
Ye-yeah, I think the, the, uh, concreteness i-in a sense of can you get a true or false ver-verifiable output.
- MAMarc Andreessen
Right. But like in any domain, in any domain of, of human effort in which there's a verifiable answer, we should expect extremely rapid progress.
- AMAmjad Masad
Yes.
- MAMarc Andreessen
Right. Okay.
- AMAmjad Masad
Yes, absolutely, and I, I think that's what we're seeing.
- MAMarc Andreessen
Right. And that, and that for sure includes math. That for sure includes physics, for sure includes chemistry, for sure includes large areas of code.
- AMAmjad Masad
That's right.
- MAMarc Andreessen
Right. What, what else does that include, do you think?
- AMAmjad Masad
Bio, like-
- MAMarc Andreessen
Yeah. Yeah
- AMAmjad Masad
... we're seeing with a protein folding.
- MAMarc Andreessen
Like genomic, genomic, yeah, yeah. Okay.
- AMAmjad Masad
Genomic.
- MAMarc Andreessen
Protein folding, right.
- AMAmjad Masad
Yeah, yeah.
- MAMarc Andreessen
Okay.
- AMAmjad Masad
Things like that. I think some, some, uh, uh, areas of robotics.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Um, uh, there's a clear outcome.
- 33:45 – 37:45
The AGI Debate: Are We on Track?
- MAMarc Andreessen
And like, you know, we should both be like hyper excited, but also on the verge of like slitting our wrists-
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
... 'cause like, you know-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... the gravy train is coming to an end.
- AMAmjad Masad
Right.
- MAMarc Andreessen
And, and I always wonder, it's like, you know, on the one hand, it's like, okay, like, you know, not all, I don't know, ladders go to the moon. Like, just 'cause something, you know, looks like it works or, you know, doesn't mean it's gonna, you know, be able to sc- you're gonna be able to scale it up and have it work, you know, to the fullest extent. Um, uh, you know, so like, it, it's important to like recognize practical limits and to not just extrapolate everything to infinity. Um, on the other hand, like, you know, we're dealing with magic here that we, I think probably all would've thought was impossible five years ago-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... or certainly ten years ago. Like I, I didn't, you know, look, I, I, you know, I got my CS degree in the late '80s, early '90s. I, I never th-- I didn't think I would live to see any of this.
- AMAmjad Masad
Yes.
- MAMarc Andreessen
Right? Like, this is just amazing that this is actually happening in, in, in my lifetime. Um-
- AMAmjad Masad
But, but, but there's a huge bet on AGI, right? Like, whether it's the foundation models, uh, I think, you know, now the entire US, uh, economy is sort of a [laughs] a bet on AGI. And, and there are crucial questions to ask whether are we on track to AGI or not.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Because there are some ways that I can tell you it doesn't seem like we're on track to AGI because we, uh, because there doesn't seem to be transfer learning across these domains that are, that are, you know, significance, right?
- MAMarc Andreessen
Right.
- AMAmjad Masad
So if we get a lot better at code, uh, we're not immediately getting better at like generalized reasoning. We need to go also fi-- you know, get training data and create RL environment for bio or chemistry or physics or math-
- MAMarc Andreessen
Right
- AMAmjad Masad
... or law or... So, so, and, and this, this has been the sort of point of discussion now in the AI community after the, uh, Dwarkesh and Richard Sutton, uh, interview where, uh, you know, Richard Sutton kind of poured this cold water on the, um, uh, on the bitter lesson. So everyone was using this, uh, essay that he wrote called "The Bitter Lesson." The idea is that there are, um, infinitely scalable ways of, uh, doing, uh, uh, AI research. And, a-and, and, and, and any time you can pour more compute and more data and get more performance out, you're just, y-you know, that's the ultimate way of getting to AGI. And some people inter-- you know, interpreted that interview that perhaps he's doubtful that even, we're even on a, on a bitter, uh, lesson path here.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And perhaps the current training regime is actually very much the opposite in which we, we are so dependent on human data and human annotation and, and all of that stuff.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So I think the, the... I, I agree with you. I mean, as a company, we're, we're [laughs] excited about where things are headed, but, but there's, there's a question of like, are we on track to AGI or not? And-
- MAMarc Andreessen
Right. Right
- AMAmjad Masad
... be curious what you think.
- MAMarc Andreessen
So, uh, so a-and, you know, Ilya, I think, you know, Ilya Sutskever makes a, makes a specific form of this argument, which is basically like we're, we're just literally running out of training data.
- AMAmjad Masad
It's the fossil fuel argument.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Like we've, we've, we've slurped all the training da- We-- Fundamentally, we've slurped all the data off the internet. That is where almost all the data is at this point. There's a little bit more data that's in like, you know, private and dark pools somewhere that we're gonna go get, but like-
- AMAmjad Masad
Right
- 37:45 – 41:15
Transfer Learning and the Limits of Human Intelligence
- MAMarc Andreessen
It, well-
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
At one, at one point. At one point. At one point. It, it-- Let's get, yeah, let's say e-even if, even if he's a brilliant-- Well, this is, so this is the thing, like what does that mean? Like, uh, should a brilliant economist be able to extrapolate, you know-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... the, the, the internet is a, is a good question. But, um, th- but the point being like even if he is a br- you know, or take anyb- take, take anybody... Oh, by the way, or like Einst- like Ein- like Einstein's like actually my favorite example. Like Br- I think you'd agree Einstein was a brilliant physicist.
- AMAmjad Masad
Definitely, yeah.
- MAMarc Andreessen
He was like a, he was a, he was a Stalinist. Like he was this, he was a-
- AMAmjad Masad
Was he?
- MAMarc Andreessen
Yeah, he was a socialist and he was a Stalinist.
- AMAmjad Masad
Oh.
- MAMarc Andreessen
And he was like, well, he thought like Stalin was fantastic.
- AMAmjad Masad
Well, he did order his out, so [laughs]
- MAMarc Andreessen
Yeah. Yeah, okay. All right.
- AMAmjad Masad
True socialism, I mean.
- MAMarc Andreessen
All right, Ei- all right, Einstein. You know, I'll, I'll, uh, I'll, I'll, I'll, I'll, I'll, I'll, I'll, I'll take your word for it.
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
But like once he got into politics, he was just like totally loopy. Or, or, you know, or even right or wrong, it's just, you know, he just sounded like all of a sudden like an undergraduate lunatic, like s- somebody in a dorm room. Like he-- There, there was no transfer learning from physics into politics. Uh, like he, he, uh, was it right or wrong, he didn't-- There was no g-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... there was clearly, there was nothing new in his political analysis.
- AMAmjad Masad
Yeah. Yeah.
- MAMarc Andreessen
It was the same rote, routine bullshit you get out of, you know-
- AMAmjad Masad
Yeah. So, so in a way, your, the argument you're making is like w-we may, may be already at human-level AI. I mean, perhaps the definition of AGI is, is, is something totally different. It's like above human level that something that truly generalizes across domains. It's, it's not something that we've seen.
- MAMarc Andreessen
Yeah, like we've ideal-- Yeah, as, as I said, we, we've, we've... And, and, you know, look, we should, we should shoot big, but we've, we've idealized a, a, we've idealized a goal, um, that may be idealized in a way that like it, it-- Number one, it's just, it, it's, it's like so far beyond what people can do-
- AMAmjad Masad
Right
- MAMarc Andreessen
... that it's, it's no, you're no longer, it's no longer a relevant comparison to people.
- AMAmjad Masad
Right.
- MAMarc Andreessen
And, and usually AGI is defined as, you know, able to do everything better than a person can.
- AMAmjad Masad
Right.
- MAMarc Andreessen
And it's like, well, okay, so if doing everything better than a person can, it's like if a person can't do any transfer learning at all-
- AMAmjad Masad
Right
- 41:15 – 45:20
Functional AGI and Automating Labor
- AMAmjad Masad
on top of that or train the same foundation model on top of that, and, and we'll, we'll go, we'll target every sector of economy, and, and you can automate a big part of labor that way.
- MAMarc Andreessen
Right.
- AMAmjad Masad
So I think, I think, yeah, w- I think we're on that track-
- MAMarc Andreessen
Right
- AMAmjad Masad
... for sure.
- MAMarc Andreessen
Right. Um, you tweeted after GPT-5 came out that you were feeling the diminishing returns.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
What, what were you expecting and, but, and, and what needs to be done? Do we need another breakthrough to get back to the pace of growth? Or what, what are your thoughts there?
- AMAmjad Masad
I mean, this, this whole discussion is, is sort of about that, and, and my feeling is that, uh, you know, GPT-5, uh, got good at verifiable domains. It didn't feel that much better at anything else. The more human angle of it, it felt like it regressed, and like you had this, uh, sort of, uh, Reddit pitchfork, uh, sort of, uh, movement against, against Sam and OpenAI because they felt like they lost a friend. GPT-4.0 felt a lot more human and closer, uh, whereas GPT-5 felt a lot more robotic, you know, very in its head, kind of trying to think through, through everything. And, um, and so I, I, I would've just expected like when we went from GPT 2 to 3, it was clear it was getting a lot more human. It was, uh, a lot closer to our experience. It can, you can feel like it's actually, oh, it gets me. Like, it, there's something about it that understands the world better. Similarly, 3 to 4. 4 to 5 didn't feel like it was a better overall being, as it were.
- MAMarc Andreessen
But, but is that, is that, is, is that, is that a, the, is the question there, like, is it emotionality? I-is it-
- AMAmjad Masad
Par-partly emotionality, but, but again, partly, like, I like to ask models, like, very controversial-
- MAMarc Andreessen
Mm-hmm
- AMAmjad Masad
... uh, things. Um, can it reason through, uh, I don't know how deep you wanna go here, but like, um-What happened with World Trade Seven?
- MAMarc Andreessen
Right, right. [laughs] Sure.
- AMAmjad Masad
It's, it's an interesting question, right? Like, I'm not, I'm not putting out a theory.
- MAMarc Andreessen
Right.
- AMAmjad Masad
But like, it's interesting, like how did it-
- MAMarc Andreessen
Right
- AMAmjad Masad
... you know, and, and can it, can it-
- MAMarc Andreessen
Right
- AMAmjad Masad
... think through controversial questions-
- MAMarc Andreessen
Right
- AMAmjad Masad
... in the same way that it can go think through a coding problem?
- MAMarc Andreessen
Right.
- AMAmjad Masad
And, uh, there, there hasn't been any movement there, like the, all the reasoning and all of that stuff. Heaven sa-- And not just that, you know, that's a cute example, but like, um, COVID, right?
- MAMarc Andreessen
Sure.
- AMAmjad Masad
Like, you know, w- the origins of COVID.
- MAMarc Andreessen
Right.
- AMAmjad Masad
You know, go, uh, you know, dig up GPT-4 or other models and go to GPT-5. You're not gonna find that much difference of, okay, let's reason together, let's try to figure out what was the origins of COVID, because it's still an un-
- MAMarc Andreessen
Right
- 45:20 – 53:10
GPT-5, Diminishing Returns, and Lost “Humanity”
- MAMarc Andreessen
like, went out and did that work, like it would maybe be that good.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
Um, but then, but then of course the significance is it's like, it's like, you know, at least for, it's, it's, this is true for many domains, you know-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... kind of PhD and everything. And so-
- AMAmjad Masad
But, but this is synthesizing knowledge, not trying to create new knowledge.
- MAMarc Andreessen
Well, but this, this, this gets to the, the sort of, you know, of course, you, you get into the angels dancing on the head of a pin thing-
- AMAmjad Masad
Right
- MAMarc Andreessen
... which is like, what, what, you know, what's the difference? How many, how much new knowledge e-ever actually is there anyway?
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
What do you actually expect from people when you ask them questions? Um, and so what, what I'm looking for is like, yes, explain this to me in like the, the clearest, most sophisticated, most complex-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... most like complete way that is possible for somebody to ex-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... you know, for a real expert to be able to, to, to explain things to me. Um, and that's what I use it for. And at le- and again, as far as I can tell from the cross-checking, like I'm getting, you know, like almost, like basically a hundred out of a hundred. Like I don't even think I've had an issue in months-
- AMAmjad Masad
Yeah, yeah
- MAMarc Andreessen
... um, where it's like-
- AMAmjad Masad
For sure
- MAMarc Andreessen
... had, had a, had a, had a problem in it.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
And it's like, yeah, you can say, yeah, synthesizing is supposed to create new information, but like it's, it's generating a forty page-- It's basically generating a forty page book-
- AMAmjad Masad
That's amazing. Yeah
- MAMarc Andreessen
... that's like incredibly like fluid. It's, you know, it's-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... it's, it's, it's, it's, you know, the, the lo- the logical coherence of the entire-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... like it's a, it's a great write. Like if, if you, if you evaluated an, an, a, a human author on it, you would say-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... "Wow, that's a great author."
- AMAmjad Masad
Yeah.
- 53:10 – 57:30
Creativity, Reasoning, and Finding Truth in AI
- AMAmjad Masad
lot of progress or outcome there. But I, I watch it kind of from far.
- MAMarc Andreessen
Although, you know, for all we know, it's already, there's already a bot on X somewhere.
- AMAmjad Masad
What's that? Maybe.
- MAMarc Andreessen
You know? You know?
- AMAmjad Masad
Perhaps.
- MAMarc Andreessen
You never know. It might not be a big announcement. It might just be a, you know, one day there's just, like, a bot on X that starts winning all the arguments.
- AMAmjad Masad
Yeah. It could be. It could be.
- MAMarc Andreessen
Or, or a co- as I was saying, or a co- a coder, uh, a user at Reddit all of a sudden that is-
- AMAmjad Masad
Yeah
- MAMarc Andreessen
... generating incredible software. Um, okay, let's, uh, let's spend our remaining minutes. Let's, let's, let's talk about you. So, uh, so, uh, so how-- So yeah, take us, start from the beginning-
- AMAmjad Masad
Mm-hmm
- MAMarc Andreessen
... with your, uh, with your life, and how, how did you get, how did you get from being born and being in Silicon Valley?
- AMAmjad Masad
Okay. [laughs] Um-
- MAMarc Andreessen
In two minutes.
- AMAmjad Masad
Yeah.
- MAMarc Andreessen
I'm just, I'm joking, but...
- AMAmjad Masad
Yeah. I, I got introduced to computers, uh, very, very early on. And so for whatever reason, so I was born in Amman, Jordan.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
And for whatever reason, my, my dad, who was just a government engineer at the time, uh, decided that computers were important. And he didn't have a lot of money, took out a debt, bought a computer. It was the first computer in our, in our neighborhood, first computer o-of anyone I know. And I just-- one of my earliest memories, I was six years old, just watching my dad unpack this machine and sort of open up this huge manual and kind of finger type CD, LS, MKDIR. And like, I would, you know, be behind his shoulder and just, like, watching him, you know, type these commands and seeing the sort of machine kind of respond and do exactly what he's asked it to do. Um-
- MAMarc Andreessen
Popping, popping Tylenol as you-
- AMAmjad Masad
[laughs] Popping Tylenol. Exactly. [laughs]
- MAMarc Andreessen
Autism activated.
- AMAmjad Masad
Of course. [laughs] You have to. [laughs] You have to. Exactly.
- MAMarc Andreessen
What kind of, um, what kind of computer was it?
- AMAmjad Masad
Uh, it was, uh, an IBM, as far as I remember.
- MAMarc Andreessen
IBM PC.
- AMAmjad Masad
It was IBM PC-
- MAMarc Andreessen
So what year was this about?
- AMAmjad Masad
... MS-DOS, uh, 1993.
- MAMarc Andreessen
1993?
- 57:30 – 1:03:00
The Origins of Replit and Early Coding Days
- AMAmjad Masad
computer engineering and-
- MAMarc Andreessen
Oh, okay
- AMAmjad Masad
... and, and did that for a while, uh, but then rediscovered my love for, for programming, uh, reading Paul Graham essays on LISP and things like that and, uh, started messing around with Scheme and programming languages like that. Um, but then I found it incredibly difficult to just, like, learn different programming languages. I didn't have a laptop at the time, and so every time I'd go to, like, wanting to learn Python or Java, I would go to the computer lab, download gigabytes of software, try to set it up, type a little bit of code, try to run it, you know, run into missing DLL issue or... And I was like, "Man, this is so primitive." Like, at the time, it was 2008, something like that, you know, we had, uh, Google Docs. We had Gmail. You could, like, open the browser, uh, and probably thanks to you, and be able to kind of, uh, use software on the internet, and I thought the web is the ultimate software platform.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Like, everything should go on the web. Okay, who's building an online development environment?
- MAMarc Andreessen
Right.
- AMAmjad Masad
And, and no one.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And it felt like I w- I found, like, a hundred dollar bill on the, you know-
- MAMarc Andreessen
Right
- AMAmjad Masad
... on the floor of Grand Central Station.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Like, surely someone should be building this.
- MAMarc Andreessen
Right.
- AMAmjad Masad
But no, no one was building this. And so it's like, okay, I'll, I'll try to build it. And I got something done in, like, a couple hours, uh, which was a text box. You type in some JavaScript. We-- And there's a, there's a button that says Eval. You click Eval, and it evaluates the Java-- It shows you in a, in an alert box [laughs] .
- MAMarc Andreessen
Right. Right.
- AMAmjad Masad
So one plus one, two.
- MAMarc Andreessen
Right.
- AMAmjad Masad
I was like, "Oh, I have a programming environment."
- MAMarc Andreessen
Yeah.
- AMAmjad Masad
I showed it to my friends. People started using it. I added a few additional things like saving the program. I was like, "Okay. All right. This is-- There's, there's a real idea here. People love it." And then again, it took me two, two to three years to actually be able to build anything because, you know, the browser can only run JavaScript, and it took a breakthrough at the time. Um, Mozilla had a research project called M Scripton that allowed you to, uh, compile different, uh, programming languages like C, C++ into JavaScript.
- MAMarc Andreessen
Mm.
- AMAmjad Masad
And for the browser to be able to run something like Python, I needed to compile C Python to JavaScript, so I was the first to do it in the world.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
Uh, so built, uh, contributed to that project and built a lot of the scaffolding around it, and we c- uh, my friends and I compiled Python into JavaScript, and it was like, okay, we did it for Python. Let's do it for Ruby. Let's do it for Lua. Let's do it... And that's how the emergence of the idea for Replit came, is that when you need a REPL, you should get it. You should REPL it.
- MAMarc Andreessen
Mm-hmm.
- AMAmjad Masad
And so a REPL is, is the most primitive programming environment possible. So I added all these programming languages, and again, all this time, my friends were using it and excited about it. And I was on GitHub at the time, and just my standard thing is, like, when I make a piece of software, is to open source it. And so I was open sourcing all the things. I was, you know, years building just, like, this underlying infrastructure to be able to just run code in the browser, and then it went viral.
- MAMarc Andreessen
Right.
- AMAmjad Masad
Uh, it went viral on Hacker News, and it coincided with the MOOC era.
- MAMarc Andreessen
Right.
- 1:03:00 – 1:08:00
Hacking His University and Getting Caught
- AMAmjad Masad
I went into my parents', uh, uh, basement, uh, and, uh, implemented, uh, the polyphasic sleep. Are you familiar with that?
- ETErik Torenberg
I, I, I, I am.
- AMAmjad Masad
Uh, Leonardo da Vinci's, uh, polyphasic sleep. I didn't h-hear it from Leonardo da Vinci. I heard it from Seinfeld, 'cause, uh, there's an episode where John Kammerer goes on, [laughs] on polyphasic sleep.
- ETErik Torenberg
So you sleep, what, 20 minutes every four hours?
- AMAmjad Masad
Yes, 20 minutes every four hours.
- ETErik Torenberg
Every four hours.
- AMAmjad Masad
Yeah.
- ETErik Torenberg
And, and yes, and this, this somehow is gonna work well, and it, it-
- AMAmjad Masad
Yeah. A- and, and hacking-
- ETErik Torenberg
It's, it's definitely-
- AMAmjad Masad
... if you've ever done any-
- ETErik Torenberg
As, as the meme goes, this, this has never worked for anybody else, but it might work, but, but-
- AMAmjad Masad
But it might work for me? Yes. [laughs] And a lot of what hacking is, is that you're, you're coming up with ideas for, like, finding certain security holes and, like, writing a script and then running that script, and that script will take you, like, uh, 20, 30 minutes to run. And so you'll take that, you know, 20, 30 minutes to sleep and go on. And so I spent two weeks just going mad, like, trying to hack into the university database. And, uh, finally, I found, um, a way, I found a SQL injection somewhere on the site, uh, and I found a way to, like, be able to edit the, the records. But I didn't wanna risk it, so I went to my neighbor who was going to the same school. Uh, I think till this day no one caught him. But I went to him and I said, um: "Hey, uh, I have this way to change grades. Like, would you want to be my guinea pig?" And I was honest about it. I was like, "I'm not gonna do it. [laughs] Are you open to do it?" [laughs] He's like, "Yeah, yeah, yeah."
- ETErik Torenberg
They call this human trials.
- AMAmjad Masad
[laughs]
- ETErik Torenberg
This is how medicine works.
- AMAmjad Masad
[laughs] So, so we, we went and, and, and, uh, we went in and changed his grades, and he, he went and pulled his transcript, and the, you know, the update wasn't, wasn't there, and went back to the basement. Well, it turned out that I had access to the, uh, slave database.
- ETErik Torenberg
Mm-hmm.
- AMAmjad Masad
I didn't have access to the master database. So found a way through the network, privilege escalation. It was an Oracle database that had a vulnerability, and then found the real database, and then I just, you know, did it for myself, uh, changed the grades, and went and pulled my transcripts. And sure enough, it actually changed. Went and p-bought the, the, the, the gown, went to all the graduation parties.
- ETErik Torenberg
[laughs]
- AMAmjad Masad
Uh, did all that, and we're graduating. Um, and then one day I'm at home. It's, like, maybe 6:00 or 7:00 PM. I get a, you know, the, the telephone at home rings. You know the-
- ETErik Torenberg
[laughs] Ominous, ominous, ominous ring sound.
- AMAmjad Masad
Yes. [laughs] Uh, well, um, hello, and he's like: "Hey, this is the university registration system." And I knew the guy that run it. Uh, he's like: "Look, you know, we, we, we're having this problem. The system's been down all day, and it, it keeps coming back to your record. There's an anomaly in your record where you're both pass-- you have a passing grade, but you're also banned from that, uh, final exam or subject." I was like, "Oh, shit." Well, it turns out the database is not normalized, so typically they, when they ban you from an exam, the, the grades reset to 35 out of 100. But apparently there's a Boolean flag, and by the way, all the column names in the database are single, are single letters. So [laughs] that was the hardest thing is security by obscurity.
- ETErik Torenberg
Right.
- AMAmjad Masad
And it turns out there's a flag that I didn't check. So when, when, when you go over attendance, um, uh, wh-when you don't attend and they, they, they wanna fail you, they, they ban you from the final exam. So I changed the grades, and that, that, that created, uh, a-an issue and brought down the system. So they were calling me, and I thought at the time, I was like, you know, I could, I could potentially lie and it'll, it'll be a huge issue, or I just like, I'll just, I'll just fess up. I'll just fess up. Yeah.
- ETErik Torenberg
[laughs]
- AMAmjad Masad
So I said, "Hey, listen, look, um, yeah, I might know something about it. Hey, let me, let me come, uh, tomorrow and kind of talk to you about what happened." So I go in, and I open the door, and it's the deans of all the un- all the schools. It's like computer science, computer... They, they were all working on it for, like, days because it's like, it's like, it's a very computer heavy-
- ETErik Torenberg
Mm
- AMAmjad Masad
... you know, university, and it, it was, like, a problem. And they're all kind of really intrigued about what happened. And so I pull up a whiteboard and started explaining what I did.
- ETErik Torenberg
[laughs]
- 1:08:00 – 1:11:55
The Redemption and Lessons Learned for the AI Age
- AMAmjad Masad
gonna let you go, but you're gonna have to help the system administrators secure the system-
- MAMarc Andreessen
There we go
- AMAmjad Masad
... uh, for the summer." I was like, "Yeah, happy to do it." And I show up, and all the programmers there hate me.
- MAMarc Andreessen
Yeah, I'll bet.
- AMAmjad Masad
Hate my guts.
- MAMarc Andreessen
Yes, yes, 100%.
- AMAmjad Masad
And, uh, they, they would lock me out. Like, I would see them. They would be outside. I would knock on the door, and no one would listen. [chuckles] It's like they don't wanna let me in. I tried to help them a little bit. They, they weren't collaborative. And so I was like, "All right, whatever." Uh, and so, uh, it, it came time for me to actually graduate. It was the final project, and one of the computer science dean ca-came to me and he said, "Look, I, I need to call a favor. I was a big part of the reason we kind of let you go, and we didn't kind of prosecute you. Uh, so I want you to work with me on the, um, on the final project.
- MAMarc Andreessen
[chuckles]
- AMAmjad Masad
And it's gonna be around security and hacking." I was like, "No, I'm, I'm done with that shit."
- MAMarc Andreessen
[laughs]
- AMAmjad Masad
Like, I just wanna, I just wanna build programming environments and things like that. Uh, and he's like, "No, you have to do it." I was like, "Okay." So I, I thought I'd do something more productive, so I wrote a security scanner, uh, that I was very proud of that, that kind of crawls the different site, that tries to do SQL injection and all sorts of things. Um, and actually, my security scanner found another vulnerability in the system.
- MAMarc Andreessen
Amazing. Amazing.
- AMAmjad Masad
And so I went to the defense, and he's like, "You need to sh-run this security scan live and show that there's a vulnerability." And I didn't understand what was going on at the time, but I just, "Okay." So I give the presentation about how the system works, and I was like, "Oh, let's run it," and it showed that there's a security vulnerability. "Okay, let's get, let's try to get a shell." So the system automatically runs all the security stuff, and it gets you, gets you a shell. And then, w-well, the other dean that turned out, he was giving the mandate to secure the system, and now I started to realize I'm a pawn in some kind of rivalry here. And, and his, his face turned red, and he's like, "No, it's impossible. You know, we secured the system. You're lying." I was like, "You're, you know, you're accusing me of lying? All right, what should we know? Should we know your, um, uh, your salary or your password? What do you want me to look up?" And I was like, "Yeah, look up my password." So I, I look up his, his password, uh, and it was like gibberish. It was encrypted. And I was like, "Oh, that's not my password. See? You're lying." I was like, "Well, there's a decrypt function [chuckles] that the programmers put in there." So I w-- I do decrypt, and it shows his password, and it was something em- uh, embarrassing. I forgot, [laughs] I forgot what it was.
- MAMarc Andreessen
Yeah.
- AMAmjad Masad
And so he gets up, really angry, shakes my hand, and leaves to change his password. Uh, and so that, that I, I was able to hack into the university another time. Luckily, I, I was able to graduate, give them the software. They secured the system. But, um, but yeah, later on I would realize that, yeah, he wanted to embarrass the other guy [laughs] which was why I was in the middle.
- MAMarc Andreessen
Economic politics. I, well, I think the m- the moral, the moral of the story is if, if you can successfully hack into your school system and change your grade, you deserve the grade and you deserve to graduate.
- AMAmjad Masad
I, I think so.
- MAMarc Andreessen
And, and, and, and just for any pa- for any parents out there, just, yeah, or children out there-
- AMAmjad Masad
[laughs]
- MAMarc Andreessen
... you can just, you can, you can cite, you can cite, cite me as the-
- AMAmjad Masad
Marc Andreessen
- MAMarc Andreessen
... as the moral, you can cite, you can cite Amjad and me as the mor- moral authority, moral authority on this. Yeah.
- AMAmjad Masad
One maybe lesson I think that is very relevant for the AI age, uh, I think that the traditional sort of more conformist path is paying less and less dividends.
- MAMarc Andreessen
Right.
- AMAmjad Masad
And I think, uh, you know, kids coming up today should use all the tools available to be able to discover and chart their own paths. 'Cause I feel like just, you know, listening to the traditional advice and doing the same things that-
- MAMarc Andreessen
Yeah
- AMAmjad Masad
... people have always done is just not as, uh, it's not working out as much as, as we'd like.
- MAMarc Andreessen
Yeah. That's right.
- AMAmjad Masad
Amjad.
- MAMarc Andreessen
Yeah.
Episode duration: 1:11:56
Install uListen for AI-powered chat & search across the full episode — Get Full Transcript
Transcript of episode g-WeCOUYBrk
Get more out of YouTube videos.
High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.
Add to Chrome