No Priors Ep. 32 | With NEAR’s Illia Polosukhin

(music plays) A blockchain operating system might just be the key to a democratized Web3. In fact, more than 25 million users are already getting a taste of this, thanks to NEAR. This week, Elad and I are joined by Ilya Polosukhin, the co-founder of NEAR and co-author of the landmark Transformers paper, to discuss the interaction of blockchain and AI technologies, what we should expect from AI agents, how to handle the content authenticity problem, and why the alignment problem in AI is really a human problem. Ilya, welcome to No Priors. Thanks for doing this.

Thanks for inviting me.

You are one of the authors of the original Transformers paper.

Mm-hmm.

We've also had Noam and Jacob on. How did you get involved with that seminal work in AI?

I worked on a team wi- on natural language understanding that focused on question answering, and the state of the art at this time was LSTMs, recurrent neural networks, which you could not launch in production at all, because they're too slow and take fair bit of time to process as documents scale. So Jacob at the time was using attention for query similarity, and he had this idea, like, using attention for encoder-decoder type. Um, I kind of jumped into it and, uh, with Ashish were playing around with, can we actually get it to train and understand the order of words and do translation just based on, you know, attention. So yeah, it was pretty cool to explore that, and obviously grew into something very interesting and awesome.

You originally co-founded NEAR in, I think, 2018, meaning for it to be an AI-focused company. What was that initial mission, and how did it become a blockchain company?

Yeah, so we started with the idea that we wanted to teach machines to code. You know, we had Transformers coming out. There was a lot of kind of really interesting push in '17, '16, '17, around AI, and so our expectation was we kind of would ride the exponential growth of AI, which has happened in (laughs) this year. We thought it will happen in '17, '18, and so with that, we got a really interesting dataset around language to code. But more interestingly, we had a whole community of developers, mostly students, who were doing crowdsourcing for us. So we would give them code. They would write descriptions. We would give them descriptions. They would write code for them, write tests, like all kinds of tasks. And we actually faced a challenge of paying them, because a lot of them were in China, in Eastern Europe, and kind of other countries where there's monetary control problems. People don't have bank accounts, and so we started looking into blockchain just, like, to solve our own problem. The AI kind of, uh, exponential explosion didn't happen at the time, and so we saw an opportunity of, we can actually build a blockchain that we would use to solve this first and focus on that, uh, while kind of waiting out the AI thing to, uh, really happen. And as you go into the blockchain rabbit hole, you realize there's a lot more that meets the eye.