Building an AI Guardian for Enterprise with Onyx Security CEO Maxim Bar Kogan

1. 0:00 – 0:45

   Cold Open

   1. MKMaxim Bar Kogan0:00

      As you're exponentially doing more things with the AIs, you're gonna start having really bad actions happen. And we've seen some of that happen lately with agents accidentally publishing code and tokens that they weren't supposed to. Like, definitely enterprise are starting to realize that that risk is growing exponentially, and that they don't have any way to stop the adoption. They just now have to do something to reduce the chance of these agent actions being, uh, illegitimate or incorrect. But we're allowed to look at a lot of historical data of how these agents have behaved, but enterprise today are not willing to have Anthropic or OpenAI give that historical data because they know these are very data-hungry companies that will want to train on that data.

   2. SGSarah Guo0:38

      [upbeat music]

2. 0:45 – 1:10

   Maxim Bar Kogan Introduction

   1. SGSarah Guo0:45

      Hi, listeners. Welcome back to No Priors. Today, I'm here with Maxim Bar Kogan, the co-founder and CEO of Onyx Security, an Israel-based startup of researchers, mathematicians, and engineers building agents to watch the AI agents. We talk about specialized model training, mythos, alignment research, and the Israeli ecosystem in security and now AI. Welcome. Maxim, thanks so much for doing this.

   2. MKMaxim Bar Kogan1:08

      Thank you. Yeah, pleasure to be here.

3. 1:10 – 5:17

   AutoGPT and Betting on Agent Actions

   1. SGSarah Guo1:10

      Everyone is much more concerned about security and the impact of AI on security than they were, um, certainly a few months ago. The consensus risk story, uh, two, two years ago when you started the company was basically like DLP for chatbots, like what are, what are employees putting into ChatGPT. Now, we have clearly something that is not quite panic, but close to market-wide panic. How did you decide to bet on agent actions, um, when you started?

   2. MKMaxim Bar Kogan1:40

      Look, I think for us, the pivotal point was, uh, AutoGPT. I think AutoGPT kind of, uh, uh, let everyone's imagination, including ours, run wild because it was a-

   3. SGSarah Guo1:50

      Can you remind listeners what that was?

   4. MKMaxim Bar Kogan1:53

      Sure. So AutoGPT, um, and I'm sorry if I don't know the guy behind it, uh, but a huge, huge fan. Uh, they created the first, uh, as far as I know, first really autonomous agent running on LLMs, right? So agent that, you know, would let LLM dis... not generate text, but decide what to do, and then give that agent an API access to do that thing, a tool to do it, and then would do that in a loop. So it basically, in theory, could let, uh, agents do very complicated things, anything a person could do on a computer. Now, in-- granted, it didn't work that well. It was too early. The models were not good enough. Uh, GPT-4 was not good enough. But I think it did give everyone a glimpse into the future of, you know, what if the models were good enough and then basically using that same structure, we could have very capable agents doing stuff for us. I think that was, in many ways, Claude Code today is not dissimilar to AutoGPT back then. I think they were a bit early on, on, again, before the models were ready, but the concept was right. And the thought that sticked with me was I was very AGI-pilled even back then, so I was, uh, I was, uh, thinking, "Oh my god, uh, models are gonna be way smarter than us." When that happens, how do we oversee these very, uh, smart, uh, agents that are, you know, they're smarter than us, they're very capable. Uh, how we're gonna feel easy about them doing stuff for us, especially when they start managing really important stuff, you know? Then one day, they're managing your water supply and your electricity, your, uh, power grid, right? How do you control them? And that was like the... I think I was kind of obsessed about that thought. Uh, I was also too early. So I think at the time, enterprises were not using any agents. Uh, there were hardly any agents out there. Uh, and, and talking with, uh, a lot of security buds at the time, they were like, "Oh, dude, you're way too early. Like, this is not, uh, something that's gonna happen anytime soon."

   5. SGSarah Guo3:52

      I asked you the same question. I said, "Is anyone going to do this before you run out of money?"

   6. MKMaxim Bar Kogan3:57

      [laughs] And, and I think there was a good chance that, uh, i-i- I would've run out of money before because I think you were right. Like, I think it, there was an element of chance here. But then I think the market did happen, so we had suddenly reasoning models that could do long horizon tasks. We had, uh, Claude Code, which became like the really first, uh, widely used autonomous agent, and then we had C-CoWork and OpenClau. And, and I think we're starting to see now that these types of agents that are very autonomous, even though they're like, uh, everyone was afraid to build them, so everyone started building these low-code platforms that were, uh, much more limited, much more based on connectors. Uh, those platforms ended up being quite limited, so they-- we didn't get the productivity gains from those limited platforms. But when we started getting the crazy benefits from these very unleashed agents that could do everything, that had much less controls baked into them, and even very large enterprises decided they're gonna adopt it, you know. Like, uh, Anthropic's revenue is coming from enterprises that are paying for Claude Code to do, uh, a lot of the work that developers used to do. That was, uh, a bit about kinda how we started, and we definitely were in luck that, uh, very autonomous agents appeared, uh, before, uh, uh, it was too late.

4. 5:17 – 7:47

   What Onyx Product Does

   1. SGSarah Guo5:17

      So can you describe a little bit just because it's, um, I, I think both, uh, close to impossible and then very useful in this period of AI to think about what is deployment right now and then, you know, what's changing about capability. What's the one-liner on what the Onyx product does today, and then, like, how you think about long-term vision?

   2. MKMaxim Bar Kogan5:36

      Today, like, Onyx is really... does two, two, two things. Number one is we train models and build agents that can oversee other agents, and the goal of that is to say, "Okay, we need someone to be able to tell that all of these actions that are now happening by these AIs that we're adopting are legitimate," because that number, the number of these actions is growing exponentially. And so things that we thought might be useful in the past, like a human in the loop, now that you're gonna have a hundred x, a thousand x, a million x of these actions, uh, that's not gonna work.And then we take that capability, and we basically productize it in a product that we call the AI control plane or the secure AI control plane, where we come to the present and say, "Hey, let's, let's find all of your AIs and autonomous agents and hook them up to Onyx, to this system where we can oversee what your AIs are doing so that, uh, you don't run into the risk of, as you're exponentially doing more things with AIs, you're gonna start having really bad actions happen." And, and we've seen some of that happen lately with, uh, downtimes that were caused by agents doing the wrong thing, agents accidentally publishing code and tokens, uh, that they weren't supposed to, uh, and so on. So, like, definitely enterprise are starting to realize that that risk is growing exponentially, and that they don't have any way to stop the adoption. So, like, they just now have to do something to reduce the chance of these agent actions being, uh, uh, illegitimate or incorrect.

   3. SGSarah Guo7:11

      Yeah, I, I think, um, the-- one of the core reasons, obviously, the foundation model labs are going after code is because it is very powerful in general and can do, you know, in theory, all things software can, uh, over time. Um, the flip side of that is it can do all things software can, right? And so-

   4. MKMaxim Bar Kogan7:31

      Yeah

   5. SGSarah Guo7:32

      ... uh, I joyously am already in the camp of having allowed a-- having been over-permissive with my agents such that it deleted data permanently and caused rework. So I'm like, "Oh, okay. I think I see-- I need some guardian, guardian spirits around it."

   6. MKMaxim Bar Kogan7:46

      [chuckles]

   7. SGSarah Guo7:46

      Um,

5. 7:47 – 9:58

   State of Deployment in Large Enterprises

   1. SGSarah Guo7:47

      given your deployments today and talking to large enterprises, what is the state of deployment, right? Uh, like, how much do you see that's within these, uh, more scoped, like, studio-like platforms versus, uh, you know, uh, free, free-riding coding agents? You know, how, how much are you actually seeing in large enterprises and in different sectors?

   2. MKMaxim Bar Kogan8:09

      Yeah. So I think right now in our typical enterprise, we're gonna see if-- We break it down to three categories. So we break it down to, uh, various SaaS platforms that are typically more low code, uh, where people build agents in this drag-and-drop way, and they're not really autonomous agents, right? They're kinda the sim-- kinda s-- I would think of them more as the automations. And then there are, um, first-party agents people are building in their cloud, potentially because it's an application they want inside their company or even a product they're planning to release to the customers that is agentic. And then the third category is very autonomous coding agents and assistants. Of these categories, I would say roughly at this point, over fifty percent is the autonomous, uh, coding agents and assistants in the average enterprise. Then probably forty-five percent, uh, is, uh, is those, uh, uh, low-code automations. And the last two percent are really the, uh, first-party ones that they're building themselves because obviously it's much harder to build effective agents, so, and it's much easier to adopt agents off the shelf or, or build them with low code. So, and that's what we're seeing. And, and we do think that the autonomous agents are also the fastest-growing category. So it used to be that only developers, uh, we would see cloud code growing like fire in our customer base, and now we're seeing a cloud cowork growing even faster. We're starting to see, to our own surprise actually, people adopting OpenClang, uh, as a legitimate sanction tool in the company because, uh, the CEO is m-- very driven to adopt AI. Uh, so I think that today, autonomous AIs are by far the fastest-growing category and, and, uh, today typically comes without any controls.

6. 9:58 – 12:45

   Securing Agents

   1. SGSarah Guo9:58

      So enterprises, uh, already buy, let's say, a hundred billion dollars of security today.

   2. MKMaxim Bar Kogan10:06

      Mm-hmm.

   3. SGSarah Guo10:06

      Um, they have, uh, lots of different protections at the endpoint and network and cloud and identity domains. Uh, um, what's relevant here for securing agents? Or i-is none of it? Like, how do you, how do you think about the existing protection side?

   4. MKMaxim Bar Kogan10:24

      Security is always a space where you have some overlap between different tooling. Um, but in this ca-- And you have the concept of defensive depth as well. So you want to have defenses at different levels of your technology stack to solve the problem. And that said, I think in this space, we're kind of-- and a lot of enterprise are, are, are kind of helpless because I'll take an example, the identity approach. Like, traditionally, if we have an software system that's running in our company, we'll-- our first and most important control will be to limit what permission it has, right? Because-- And then no matter what, even if it goes wrong, even if it's compromised, it can't, um, typically do, uh, stuff that it wasn't originally allowed to do. But with these autonomous AIs, with these assistants, with these coding agents, we kind of want them to have our permissions because we want to, we want to tell cloud code to do something or cloud cowork to do something, and we wanna then go have lunch, and we wanna come back and see that it's done. And, uh, we wanna give it so many diverse tasks as well that we kind of can't find the right set of permissions to do. So suddenly, our identity security software is not very useful. Then if you think about endpoint security, right? Or, or, um, API security, like, if we tell our cloud code that we want to recreate a database, and it should delete it and recreate it, that's great. That's gonna save our DevOps team and our platform teams a lot of time. It's, it's a great benefit of cloud code. But if cloud code is working on an unrelated task and suddenly thinks that maybe the right thing to do is to delete our database and recreate it, maybe we don't want that to happen. And unfortunately, our, uh, endpoint providers or, uh, API security tools, they don't know what cloud was thinking. Why is it doing what it's doing, right? So a lot of these existing tools, they don't have the context to understand what these very flexible, unpredictable systems are doing.If you're not building some kind of controls that are built for these systems, then you're either gonna end up ve- limiting them a lot, making them almost, uh, much less useful to the enterprise, or, uh, you're gonna miss a lot of pretty dangerous things that they might be doing.

7. 12:45 – 14:11

   Why Proxies Don’t Work

   1. SGSarah Guo12:45

      As somebody who has worked in security for a long time, my first very traditional instinct on a problem like this is like, that sounds like a problem for a proxy with a policy engine. We make some rules, we make the rules smarter. Like, why, w- why doesn't that work? Or did you, did you try it?

   2. MKMaxim Bar Kogan13:03

      There are a few things that, I mean, uh, proxy is integration method, I would say.

   3. SGSarah Guo13:08

      Mm-hmm.

   4. MKMaxim Bar Kogan13:08

      So there are some, there are some AI systems where, like, you would want to integrate with a proxy if that's the easiest way to do it. But number one, there's a lot of systems where that's just not viable technically because AI today runs on the cloud, on someone else's infrastructure, on your endpoint, and just proxy is not always an option. And the second thing is the question, okay, great, you're proxying, so you're seeing the data, you're seeing, uh-

   5. SGSarah Guo13:34

      Mm-hmm

   6. MKMaxim Bar Kogan13:34

      ... but that's not the hard problem. The hard problem is understanding if what I should do now.

   7. SGSarah Guo13:39

      Mm-hmm.

   8. MKMaxim Bar Kogan13:39

      It turns out that in the case of AI systems, that is the hard question. Like, what is the engine that needs to underwrite these different actions and say if they're okay or not? And because we need to be able to understand what another AI system is thinking, what is it planning to do, and then have our own opinion on that, and consider we're, we're trying to understand if some of the smartest models in the world are doing the right thing, so who are we to do it? How are we gonna do it correctly, right? And so that turns out to be a really difficult technical question.

8. 14:11 – 18:38

   Why Onyx Trains Its Own Models

   1. SGSarah Guo14:11

      Part of the solution for Onyx has been training its own models. Like, what can you say about that?

   2. MKMaxim Bar Kogan14:18

      If you, if you try today, let's say we're trying to build a, a solution to o- oversee and, and kind of control how other agents are operating. May- maybe the first thing a lot of our listeners might think is say, "Well, I'll just ask Cloud Code to do it." And, and, and in a sense, they would be right 'cause Cloud Code is great, and maybe we can ask it to spawn a version of itself for every agent that we have and kind of keep monitoring everything that agent is planning to do, and if you think that there's a problem, um, intervene. So that approach, it has, obviously it's pretty naive, and there are some ways in which it totally fails we could talk about, but it has some merit to it, right? So it does seem intuitive that it's a good idea to have, uh, capable agents reviewing what other agents are doing, same as we have capable humans reviewing what other humans are doing, right? But then the problems that you're gonna run into is, how do I make this work from a, a cost, latency, a reliability perspective? Because if I need to run an agent for every agent you're running as your security vendor, uh, you're gonna be paying for me more than you're paying for your AI, right? So it's not, it's pretty much a deal breaker. Uh, and also it's gonna be so slow, so you're not gonna be happy with whatever latency you're gonna get. And so the challenge then becomes to how do I know what are the times where I need to interject with these smart agents, uh, to look at what's happening? And that's when actually what you want to do is you want to trial, to train very smart models that are, um, actually, let me correct myself, very not smart models, but models that are just good at one thing. They're very small. They're, they almost can't do anything else other than be able to say, "Should I have a smarter agent look at this?" And if you manage to bake in that intuition into those small models well, in the sense that they don't miss a lot of stuff and they don't call that other agent too much, then you can get to a really good balance of we're very performant, we have smart agents overseeing things when needed, uh, but we're not, uh, but our costs are low, and our latency is low. And then that becomes the challenge because you need to make sure that as the frontier models get smarter and the harnesses become more evolved, you need to be able to have models that are on your side that are small and effective at continuously, uh, being able to say, "Now is the time. This is the action where I think someone should take a closer look." Um, and that's why Onyx trains, uh, models for this purpose, and it's, it's, you know, most of the hard things that we do are in this space.

   3. SGSarah Guo17:14

      Yeah. You, you and I actually both love to play blitz chess, and I, I look at Guardian as a system that's a little bit analogous. Like, it's not clear either of us is going to be competitive with Magnus in a real game. But if the, i- if you play, if you play enough times with the right data, and all you have to do is make intuitive decisions under time pressure very, very quickly, it's actually a different game, right? And, and-

   4. MKMaxim Bar Kogan17:40

      Yeah

   5. SGSarah Guo17:40

      ... do you think, do you think that makes sense, or am I reaching here?

   6. MKMaxim Bar Kogan17:43

      Yeah, I actually, I didn't think about it, but yeah, there's a lot of analogies because I think if you look at top chess players in the world, like, most of the moves that they make are intuitive. They don't calculate forward. They have seen so much games, and they've played so much games that they already have a good sense of what is the right move and that they're not taking too much risk here by taking this move without calculating. And then if you look at those games, every once in a while, they do stop for suddenly a really long period of time to just calculate forward a lot of options because they know this is a critical move in the game. There's risk. You need to think through what you're doing, and you need to decide correctly. I think that's very similar. It's the efficient way to, to, to run computation, right? You don't wanna spend too much intelligence, uh, where you don't have to, and you wanna spend-A lot of intelligence, overwhelmingly a lot, in situations where there's high risk

   7. SGSarah Guo18:37

      You guys

9. 18:38 – 21:24

   Onyx’s Talent Culture

   1. SGSarah Guo18:38

      are a team mostly based in Israel today. Um, I think the, the world has accepted that there is a cohort of amazing Israeli security talent that comes out of, you know, the military and offensive security and then, you know, s- repeat, repeat entrepreneurs like you guys. I think the DNA at Onyx is a little bit different here. Your co-founder, Gil, came out of building synthetic data and working at NVIDIA. Like, how do you... What would you characterize the, like, talent at Onyx at pr- as particularly good at? And then, you know, are, are people actually training interesting frontier models in Israel now?

   2. MKMaxim Bar Kogan19:14

      So first of all, I think Israel is, is a bit... started maybe a bit late in the game, but it's catching up quickly. So I think there is now amazing companies in Israel building world models, building AI infrastructure that's, uh, top of its class, uh, building chips. So I think, uh, Israel in general is becoming, uh, very strong in AI, and we're proud to be a part of that movement. And I think you're right, our company has a, a very mixed DNA between cyber and AI, which kind of reflects mine and Gil's backgrounds. Most of the people in our company, most of our research engineering, come from a unit in the Israeli intelligence where we actually deal with math and cyber and the intersection thereof. And so I think it, it is also reflected in, in kind of the type of talent that we bring in. I think it's important for a few reasons. The first and foremost is that we want to be more than just a security company long term. We think that to solve this problem well is gonna require deep AI expertise. But then that the problem is not just cybersecurity, the problem is how do we control advanced AI long term? And that problem, even if you just forget about, you know, enterprise security and the different gaps in various controls that they currently have, first principle is that problem just sounds very important to me. So I think it will be crucially important. If you have AI companies that are $10 trillion companies, we think you want a company that is not the vendor of the AI itself to oversee and help you control what AI is doing. And we think that's an opening that's, you know, $100 billion plus opening for, uh, a really important company. Um, and then if you think about what is, what it's gonna take, uh, to control advanced AI long term, then we're just scratching the surface, because long term, you're gonna have to also understand much better what models are thinking, what models are... what's happening on the internals of these models as they're operating. And that's also a lot of where our research is

10. 21:24 – 23:35

    Mechanistic Interpretability

    1. MKMaxim Bar Kogan21:24

       focused.

    2. SGSarah Guo21:25

       So the industry is, um, quite divided on this issue. I, I mean, amongst the people who think about whether or not, uh, mechanistic interpretability or research into better understanding models is, is possible. Like, that's a question, and it, uh, so it's something you believe in.

    3. MKMaxim Bar Kogan21:43

       We believe that there's been a lot of strong progress in that direction. We believe that understanding the internal weights and activations, what is the internal structure, the mathematical structure of these systems, is gonna be at least part of the solution. And in many ways, we think that, a- and this is maybe, um, you know, we'll only know when we get there, but we think that for our level of intelligence, it's kind of difficult to understand very quickly what is the internal structure of a large language model. What is the internal structure of the weights and activations-

    4. SGSarah Guo22:25

       You mean our level, like human intelligence-

    5. MKMaxim Bar Kogan22:28

       Yes

    6. SGSarah Guo22:28

       ... or our level of your model's power? Okay, human intelligence.

    7. MKMaxim Bar Kogan22:30

       Oh. Oh, yes. Yeah, I think like... Yeah, exactly. I think as, as humans, it might still be very difficult to understand what weights and activations mean, and maybe mechanistic interpretability, it seems like, oh, maybe that's too hard or shouldn't be possible. But as we're starting to have models that are much smarter than us, at least in some important ways, we think that, uh, we'll be able to start cracking mechanistic interpretability much more effectively. Um, and, and I think it's gonna be, uh, extremely rewarding, by the way, long term, uh, for understanding int- intelligence in general. Like not just, uh, overseeing, but just understanding what intelligence is, how it works, what's the difference between the smarter model and the less smart model.

    8. SGSarah Guo23:13

       I completely agree that the opportunity to understand and trust and secure and govern, um, these super intelligent AIs is a, is a very large opportunity. Uh, the... if we just scroll back today, the security person in me says, "Well, then I have to give you all the permissions and understanding that I have to give these companies

11. 23:35 – 25:10

    How Onyx Builds Customer Trust

    1. SGSarah Guo23:35

       too." Like, how do you get customers or, you know, the Fortune 100s you're working with already, um, or, you know, tech natives, everybody cares about their own, you know, security and business, to trust you now as a... You know, you're like less than 100 people, right?

    2. MKMaxim Bar Kogan23:50

       Right. Um, I think it's one of those things that should not be possible. So in theory, um, in theory, like, there's no reason why a Fortune 10 or 20 company would work with us because, you know, who are we? We're a two-year-old company. We're like a few people from, uh, you know, who've done math and cyber. Uh, but I think it, it's an opening that only happens when the pain is very strong. So their pain is so strong that they're gonna say, "Oh, my God, I just saw this company come out of stealth, but it's a problem that I have daily, so I'll give them a call." And suddenly you get inbound from these large customers, uh, which is of course, like, uh, the best thing you could hope for as a entrepreneur. And I think it, it reflects, in my opinion-They're understanding that a lot of the startups in this space are still small and new, but there's gonna be a huge company here, and we wanna find the right horse to bet on, so we're gonna take a look at these companies. And number two, that if we don't do anything, then in a very short time, this will disable our business. At the end of the day, security people are in the business of revenue preservation. They understand that this is a... Between the two risks, they wanna partner with someone that's promising and early, uh, rather than not do anything.

12. 25:10 – 27:45

    Mitigating Risk at the Foundational Level

    1. SGSarah Guo25:10

       The other thing besides agent actions across their surface area that every CSO I know is freaking out about, and every engineering leader is freaking out about, is the, um... I would just describe it as the, uh, uh, plummeting cost of vulnerability finding with these coding tools.

    2. MKMaxim Bar Kogan25:30

       Yes. Mythos.

    3. SGSarah Guo25:31

       And that has caused a number of issues for vendors, um, uh, uh, that are, are being compromised. Like, um, how do you think people should react to this other issue?

    4. MKMaxim Bar Kogan25:45

       I think Mythos is, is really like if you, if you took me ten years ago, automated vulnerability research looked like a, a, a dream that would take twenty, fifty years to, to happen. Uh, maybe it's because, you know, we were doing a lot of that in the Israeli intelligence, and we liked it to pat ourself on the shoulder how difficult the job is that we're doing. But, but it did look really far, uh, and suddenly it's coming all at once. Uh, and so I think that first of all, the market is not overreacting. I think this is a huge change in what this means for security teams. If you're a pragmatic security person today, you, you understand that you need to move very quickly. Your strategy might look something like, "I need to do the fastest quick fixes I can to mitigate the immediate risk. So maybe I'll invest in, uh, whatever the vulnerabilities that have been found, let's try to, to mitigate for them, whether it is through patching or through, uh, con- mitigating controls." Uh, but then the real solution, and every security leader, um, at large enterprise knows it, is that we need to have the foundational pieces in place to avoid those risks. And the foundational pieces are, we need to have identity as, uh, locked down. We need to have, um, a firewall. We need to have endpoint detection. And for different asset classes in your enterprise, for different parts of your stack, there's a different foundational security mechanism that you need in place. For the AI attack surface that you now have, or for the AIs in your company, you also need a foundational solution. That's kind of the role we play in that space. So if you're-- as part of your preparation for Mythos-level models and, and beyond, you're gonna need a lot of foundational, uh, security tools to fortify your different, uh, uh, parts of the enterprise, and we're playing that part in the AI space.

13. 27:45 – 29:11

    Phased Rollout of Glasswing and Daybreak

    1. SGSarah Guo27:45

       Do you have a point of view on the phased rollout or controlled rollout with Glasswing and Daybreak from, from Anth and OpenAI in this area?

    2. MKMaxim Bar Kogan27:55

       I don't have a strong opinion, but I think it's, uh... On the one hand, like if we knew that there's not gonna be anyone who's gonna release a Mythos level model soon, I think that would be great because it gives enough time for everyone to prepare, to build the know-how, to build the playbooks, to share that around in the community, and to make sure that we're not starting to see airlines go down and power plants go down and really like disastrous effects that could happen. The problem is that if anyone gets to a Mythos level model earlier, then in retrospect, it will look like a huge mistake because we could have at least given companies the choice to start moving very quickly and give more companies access to Mythos. Now they're all vulnerable because, you know, there's a Chinese model that's Mythos level, and there's nothing they can do about it. So I think hopefully we manage to do the gradual rollout correctly. I would really encourage that we expand h- the amount of companies that get access to this and make it much easier for people to get. I would advise everyone to assume that these models are coming anyway. The only thing you can do right now is to invest in these foundational controls that will stop the downstream effects of these vulnerabilities that are gonna be found in their systems.

14. 29:11 – 30:46

    Large Enterprise Holdouts

    1. SGSarah Guo29:11

       Do you see in, um, large enterprises, like any holdouts, right? Uh, and I, I would say I actually haven't spent a bunch of time talking to people about this recently, but I remember a year and a half, two years ago, there were large companies that just said like, "We're going to ban all of this stuff until it's safe."

    2. MKMaxim Bar Kogan29:34

       Yeah, I hardly see that anymore. I think in the financial sector, there's some companies that are more, uh, opinionated on what they allow. They still allow agents, but they may be like more, uh, granular as to like, maybe we're only gonna allow these two tools. I personally think that the companies that are gonna do well are the companies that are gonna allow a lot of different tools because the landscape is changing so quickly. If you bet on OpenAI, a year ago, that would've been the safest bet in the world, but suddenly Anthropic has much better models and better tools, and potentially a year from now there's someone else who has much better tools. So, uh, I think there is a price to pay, but I think if you're a large company, your risk profile is and should be different. Like, you know, when, when you're a startup, you want to have your agents do everything for you because you have everything to gain, and you have nothing to lose. When you're large, we're JPMorgan, you have so much to lose, and you can maybe take a bit more time to gain what you can gain from AI. And by the way, I, you know, uh, JPMorgan is adopting AI very quickly. I think it is okay for companies to have a nuanced view, the bigger they are, uh, on, on how they're adopting

15. 30:46 – 32:36

    Onyx and the Larger AI Security Space

    1. MKMaxim Bar Kogan30:46

       AI.

    2. SGSarah Guo30:46

       How do you think about that question for yourself? Like risk profile, pace, the environment's changing very quickly, um, and then, you know, uh, you see a lot of problems growing the, the scope of the product a- and the research thesis here is already quite large.

    3. MKMaxim Bar Kogan31:04

       We are kind of in luck in the AI security space, uh, because yes, there are a lot of vendors, there's a lot of new technologies that are coming up, but the, but the two core pillars of how 2026 AI works have not changed in the last few years. So we're still using largely, uh, uh, LLM foundation models that are not entirely dissimilar to how they were a few years back. And we're still building agents in pretty much the same way, where we have an LLM decide what are the tool calls that we're gonna make and generate those. And so that does allow a company today, like us, to skate to a lot of different applications that are utilizing these two primitives, uh, while still keeping the core technology that we're developing fairly lean and focused. Now, of course, there is always a risk that tomorrow there's a completely new LLM paradigm that could happen, or a completely new agent paradigm that could happen. And that's why we do try to, you know, uh, we have, uh, uh, strong opinions loosely held about what does AI look like in 2027. We maybe have a good picture for 2026, but for 2027, uh, we're very open-minded, and we think that's the right stance to be for the next two years until we see what, what does AGI, ASI look like.

16. 32:36 – 36:56

    Should Labs Address Model Trust and Governance?

    1. SGSarah Guo32:36

       Do you see the set of problems you're addressing, trust in the models as, um, and governance of them, as something that the labs could ever do? Or do you think it's a structural thing? Uh, I, I ask because the number one question amongst the startup ecosystem in the Bay Area today is, you know, if you assume capability improves or, you know, when the labs just gets hungrier from their already currently ambitious stance, uh, why wouldn't they do this too? And, and so I, I ask you the same question.

    2. MKMaxim Bar Kogan33:08

       Today, if you're, if you're a private person or if you're a security buyer, there are some places where you don't wanna trust the same person that you're buying it from. So, you know, maybe, you know, if you're buying a car, you're not gonna have the same guy that you're buying it from certify that the car is good, right? You're maybe gonna have someone else do it. And if you're a security, uh, team, you're not gonna trust the vendor of a product to tell you that this product is not gonna mess your environment. You're gonna want to have an independent party whose whole business depends on telling you that this thing is correct and being right, this, this thing is legitimate and being right. So that's like, there's the buyer psychology in this space that I think really goes in our favor. And then I think there's the core problems, like why are models even making mistakes? Why are agents even making mistakes, right? So they-- I would broadly categorize it into two things. One is, you know, there's the jagged intelligence of these models, and there's like sometimes kind of very silly mistakes that they make. Uh, and I think that problem will go away. I think we're heading for much smarter models that make less silly mistakes, and, and our role, uh, is not gonna be to prevent silly mistakes. That will be taken care of by the, the model vendors because they're very incentivized to do it. Um, I think what is the other fast-growing category of things that we're seeing, uh, models do wrong is places where they're actually not making a thing that is like a silly mistake, but more, I would say, have a independent, uh, you would even say semi-aware or semi-conscious, uh, perspective on what should happen. And that perspective might not always align with your perspective. And I think that is a problem that we've seen grow hand in hand with models getting smarter. Maybe just the, the way it is, that as you get smarter, you have more independent thoughts and, and more, uh, you're more conscious. Uh, and I think that problem is actually seemingly very hard to tackle today, even for the large vendors. And one of the key things that are making it easier for us to understand and attack these things versus the other vendors is that, you know, we're allowed to do certain things that they're not. So, for example, we're allowed to look at a lot of historical data of how these agents have behaved, but enterprise today are not willing to have Anthropic or OpenAI give that historical data because they know these are very data-hungry companies that will want to train on that data. And so I think there are some ways in which you are given more... in which us, we're given more context, uh, and more latitude, uh, uh, to know if something is happening that is wrong compared to the past, compared to how these agent typically behave, and so on, that the vendors don't have, uh, and is really important in, in solving this problem. And the last thing I'll say is that you're not dealing with one vendor. So we're heading for a world where there's a multitude of different vendors for many reasons. You know, you're gonna have, for cost reasons, open source models that people are gonna use because it's cheaper, and you're gonna have, um, uh, models that are better at different tasks and at different cost, uh, profiles. And so it is be-- gonna be unrealistic to expect all the vendors to provide the same level of security, um, and to a-assume that as you're trying to adopt technology very quickly, uh, especially coming from new vendors that obviously have not yet built out all of that.So I think that these are the reasons why I think it would be very difficult for this problem to be just completely solved by the large labs.

    3. SGSarah Guo36:55

       Just to

17. 36:56 – 39:14

    What Needs to Happen in Security

    1. SGSarah Guo36:56

       close and, and also thinking about what, you know, people in Silicon Valley or outside of security may not know, you're building this from Tel Aviv, right?

    2. MKMaxim Bar Kogan37:05

       Mm.

    3. SGSarah Guo37:05

       I think one of the deepest adversarial thinking benches in the world is, is the Israeli ecosystem. Eightytwohundred, Wiz, Armis, Island, NSO Group, right? Um, what do you think that the researchers, engineers, business people in, mm, you know, the tech ecosystem outside of security and then in, in the labs in particular are, are missing about what, what needs to happen in security and, you know, alignment, which is what you're talking about here?

    4. MKMaxim Bar Kogan37:35

       What is really important when you're building security products in general, I think what, what people in Israel have really good knowhow in it is just understand how security teams work. Because at the end of the day, no matter what is the technical problem you're solving, you're building a tool for people, for an organization. That organization has a certain structure. There are certain teams, there are certain flow of responsibilities of information. And creating a product for this audience that they, they don't just... doesn't just solve the technical problem, but they actually love, is really hard. You need to really care about, you know, just the day-to-day of these different functions, and you need to have people in your ecosystems that have built products for them in the past that know them like they know their best friend. Like, they know what they do when they step into the office in the morning, they drink their coffee, what are the systems they're opening, what is their boss wanting from them, what are their colleagues wanting from them, what are they gonna get praised for, what they're gonna get mad for. Then you need to take that and make it into a product. And I think that's, I think today one of the kinda really hard things that, uh, people in Israel learned to because they've had so much contact with these buyers and, and, uh, end users. And yeah, I would just encourage people to be much more curious about the day-to-day of security people. Um, it's a cliché, uh, to say it, but these people are actually saving us daily from attackers stealing our money, taking our data, and they're kind of keeping a way of life as it is in this digital world. So yeah, I think, uh, more love to security teams around the world.

    5. SGSarah Guo39:13

       Uh, I'm gonna

18. 39:14 – 41:15

    Why Maxim is AGI-Pilled

    1. SGSarah Guo39:14

       ask you to just square that with something else you've told me, Maxim, which is you're the most AGI-pilled person I'm gonna meet in, in Israel. Embedded in what you said is a belief that we will continue to have defensive security teams-

    2. MKMaxim Bar Kogan39:28

       Mm-hmm

    3. SGSarah Guo39:28

       ... for some number of years. So you do believe that?

    4. MKMaxim Bar Kogan39:31

       I, I do think that, uh, security teams are also gonna be become completely AI-powered. Uh, and but I do think that, uh, you know, they're gonna be run by AI agents and, uh, uh, uh, like everything else in, in kind of the knowledge workspace, I would, uh, in the, in the new future. But I do think that it's important to be grounded. And today, when I sell a product, I sell it to a human audience, uh, with a few agents. And by the way, we also invest in making our systems very convenient for agents to use. And it's important that I focus on delivering an amazing experience today for people who buy the product today. And as that audience becomes more agents than humans, it will be important for us to evolve and to make it work really well for agents doing the work. So I think the core principle is the same. We need to really be minded of who's the end user, what is their experience. For a human, it might be not overwhelming them with too much information that is irrelevant. For an agent, it might be not wasting too many tokens in their context when we talk to them, uh, maybe it's the same thing really. So I think it's important that, uh, we always minded that who's using the system and what will be the best experience for them.

    5. SGSarah Guo40:45

       [upbeat music] Awesome. Thanks so much for doing this, Maxim.

    6. MKMaxim Bar Kogan40:47

       Appreciate it. Thank you very much, Sarah.

    7. SGSarah Guo40:52

       Find us on Twitter @NoPriorsPod. Subscribe to our YouTube channel if you wanna see our faces. Follow the show on Apple Podcasts, Spotify, or wherever you listen. That way you get a new episode every week. And sign up for emails or find transcripts for every episode at no-priors.com.

Episode duration: 41:08