No Priors Ep. 33 | With Replit's CEO & Co-Founder Amjad Masad

This week on No Priors, we explore Replit and its new AI tool, Ghostwriter. Replit now has more than 22 million users and recently started a partnership with Google. With Replit, anyone from beginners to experts can quickly build and release fully functional live apps in seconds, now with the help of AI. Amjad Masad, CEO, co-founder, and head engineer at Replit joins Elad and me this week on No Priors. Welcome.

Thank you.

Amjad, just to start, you were working as an engineer at Facebook. What made you decide to start Replit?

My journey with Replit actually starts way back in, in college. I was really interested in program languages, and I found it really difficult to just, like, keep setting up the development environment. I didn't have a laptop at the time, and so I was like, you know, "Why can't I open a browser window and start coding?" Turns out nobody has done that, and I was, I was kind of naive and, and thought I, you know, could probably try to do it, and got a prototype up pretty quickly. But my friends loved it in college. Everyone started using it. It was just, like, a simple webpage with a text box and, and a button that says run, which is still kind of in a lot of ways what Replit is. But then that started me on a, like, multi-year journey to actually build the first kind of in-browser sandbox, and at the time, we had a bit of a breakthrough. We were the first to compile Python, Ruby, and other languages to, to JavaScript. I open-sourced that project. Uh, that got me a job at Codecademy. I was a founding engineer there. And after that, I went to Facebook, and I was one of the founding engineers in React Native. We, like, made React and, uh, Jest, uh, Babel, and, uh, a bunch of tools that, uh, JavaScript developers use today. Our goal was to make mobile development as fun and easy and fast as web development, which I think we succeeded at. You know, towards the end of my time at Facebook, I just wanted to see what to do next and kind of looked around and, you know, whatever happened to this online coding space. And there was a bunch of online editors, but none of them really, uh, felt like web native, like you can just share a URL and someone can hop in and code with you, kind of like how Figma or Google Docs work, and, uh, we decided to start in 2016.

You have made a huge bet on AI as, uh, a company. Can you just talk about Ghostwriter and how it came about and the investment in this area?

Yeah, throughout my career, working on code that handles code, right? Whether it's at Codecademy for teaching programming, whether it's my own project, whether it's React and building the runtime around React Native, I always felt like our tools that were handling code, whether it's, like, compiling it, parsing it, minifying it, all that stuff, were very kind of rigid and very laborious. Um, in a lot of ways you're building sort of a classical intelligence system, very algorithmic, um, and a lot of heuristics and, and all of that. And I always thought that you can probably apply machine learning to it, and I started reading around whether anyone had done it. There was this seminal paper in 2012 called On the Naturalness of Software, and basically a bunch of researchers try to apply NLP to code. And what they found is that actually code can be modeled like any language. That's why they call it naturalnesses, because, hey, code is kind of repetitive like language you can do, things like n-gram, basic n-gram model can actually s- start to generate code that, that is compilable. That had a huge impact on me. And every year or two while starting Replit, I would go look at the state of the art on ML on code, and nothing ever really worked o- all that well up until GPT-2. And you can, like, take GPT-2 and fine-tune it and, like, try to make it write code and was, like, kind of okay, but obviously GPT-3 was like, "Okay, it's here." And that's when we started building what became Ghostwriter. Initially it was a bunch of tools kind of sprinkled all over the Replit IDE, explain code, you know, generate code here. And then we wrote our kind of autocomplete product, which is kind of like Copilot. As you're coding, it kind of gives you suggestions. And then we added the chat product, and that all became sort of Ghostwriter, our, our AI suite.