Rajat Monga: TensorFlow | Lex Fridman Podcast #22

The following is a conversation with Rajat Monga. He's an engineering director at Google, leading the TensorFlow team. TensorFlow is an open source library at the center of much of the work going on in the world in deep learning, both the cutting edge research and the large-scale application of learning-based approaches. But it's quickly becoming much more than a software library. It's now an ecosystem of tools for the deployment of machine learning in the cloud, on the phone, in the browser, on both generic and specialized hardware, TPU, GPU, and so on. Plus, there's a big emphasis on growing a passionate community of developers. Rajat, Jeff Dean, and a large team of engineers at Google Brain are working to define the future of machine learning with TensorFlow 2.0, which is now in alpha. I think the decision to open source TensorFlow was a definitive moment in the tech industry. It showed that open innovation could be successful, and inspired many companies to open source their code, to publish, and in general, engage in the open exchange of ideas. This conversation is part of the Artificial Intelligence podcast. If you enjoy it, subscribe on YouTube, iTunes, or simply connect with me on Twitter, @LexFridman, spelled F-R-I-D. And now, here's my conversation with Rajat Monga. You were involved with Google Brain since its start in 2011 with, uh, Jeff Dean. It started with this belief the proprietary machine learning library and turned into TensorFlow in 2014, the open source library. So, what were the early days of Google Brain like? What were the goals, the missions? How do you even proceed forward once there's so much possibilities before you?

It was interesting back then, you know, when I started out, when you, you were even just talking about it. The idea of deep learning was interesting and intriguing in some ways. It hadn't yet taken off, but it held some promise and it showed some very promising and early results. I think the, the idea where Andrew and Jeff had started was, what if we can take this, what people are doing in research, and scale it to what Google has in terms of the compute power? And, uh, also put that kind of data together, what does it mean? And so far, the results have been if you scale the compute, scale the data, it does better, and would that work. And so that, that was the first year or two, "Can we prove that out right?" And with this belief, when we started the first year, we got some early wins, which, which is always great.

What were the wins like? What was the wins where you were, "There's some promise to this, this is gonna be good"?

I think the two early wins were, one was speech that we collaborated very closely with the speech research team who was also getting interested in this, and the other one was on images where we, you know, the cat paper as we call it-