Stanford CS153 Frontier Systems | The Road Ahead: Resilience Required

1. JSJoe Sullivan0:10

   I, I have two kind of themes that I wanna touch on that I hope at the end of this, uh, session, um, kinda get a little bit in your brain. Uh, I have been working in technology since the 1990s, uh, when I got out of school. I, um, moved, uh, t- here to Northern California in 1995, and when I got to San Francisco in 1995, I was working for the US Department of Justice. And it's funny, Mike said-- had asked me, "How did, how did you get, like, into doing technology, uh, for the, for the government?" And it was because I asked the Department of Justice if they would give me a direct internet connection to my desk in 1995, and they said, "A-absolutely not. We can't let our network touch the internet." So I just kept asking, and eventually they let me have a separate computer to use on the internet, and then I was the only person in the office who had a computer that was connected to the internet, and I became the gatekeeper to everything. Um, but let me tell you a little bit, uh, about... Let's see, how do we... Here's my background. So I spent my first eight years with the Department of Justice, and then I, uh, in 2002, I went to eBay. Back then, eBay was kinda like the hottest company in Silicon Valley, and it was a really fun place to work for a few years. Right after I got there, we acquired, uh, PayPal, and so I spent a bunch of time, uh, for eBay and PayPal, um, building out kind of both the legal side and the, um, safety and security side of those companies. And then in 2008, I went to Facebook, uh, when it was smaller than MySpace. Uh, it was here in downtown Palo Alto. We were scattered in a bunch of little, uh, uh... I don't know. I was, like, in an old law firm office where I was working with a, with a group of other people. And, uh, we-- It took us years to get to having a campus. Um, and so I was at Facebook until we became basically the company you know now after we'd integrated Instagram, WhatsApp, Oculus, and all that. And then I went to Uber and became their first head of security. So at Facebook, I g- I inherited three engineers and built it up to a large group. Then I went to Uber, inherited three engineers, and built it up to hundreds. And then in 2018, I went to Cloudflare, inherited three engineers, and built it up again. So today, that's a lot of what I do. I work with startups that need to scale security and technology really fast. So I have my own company, and, uh, we work with three or four c- startups at a time, helping them scale. I also advise, uh, s- cybersecurity companies, startups, and, uh, some non-security companies on, on security best practices. Um, I, I'm a venture partner at Costanoa Ventures, and I am the CEO of a nonprofit helping kids in Ukraine. So that's kinda my background in a nutshell. But I'm gonna take you through, uh, in particular, uh, s- something, uh, that I had to go through when I was at Uber. If you look at my roles and my career, there's one theme, which is, uh, I've been at the intersection of where government and technology tech companies meet. And, uh, I've spent a lot of time... Like, when I, when I was that federal prosecutor here in Northern California, I would go around to all the tech companies and I would say, "Tell me about your cybercrime. I wanna prosecute it." And they would all say, "We don't have any." There was no incentive. If you're a company and you're having bad things happen to you, why would you tell anybody about it? Is it good for your brand? Is it good for your business? Not at all. So the companies would always say to me, "Oh yeah, we have the..." And so they would tell me about all these other issues they had. I ended up, like, prosecuting the... It was actually a, a guy from Stan- who'd gone to Stanford. He was a joint, uh, he had a law degree and MBA joint degree from Stanford, and he ran all of business development for Cisco. And he felt that the CEO of Cisco didn't appreciate him enough, apparently, so he stole 40 mil- As they acquired companies, he created his own, uh, subsidiary called Cisco Systems Inc.'s Bahamas, and he, um, uh, when, when they would divide up the stock portfolio, he would put about half of it in actual Cisco and half for himself. And then eventually we figured it out, and, and so I prosecuted him. And I was like, "That's not exactly cybercrime," uh, but you know, it was interesting. Uh, and then I had to build trust with the companies, and then they would actually start telling us about the real issues when they understood they could trust us to actually just go prosecute and not do big negative PR against the companies. Then I switched over and was on the company side, and at eBay, our number one problem was trust. Like, if you remember, m-maybe you don't remember before PayPal, but the business model of eBay when I joined was identify an item, win the auction, put money in an envelope, mail it to the seller, and hope they send you the goods. That was literally the eBay business model when I joined the company. A small percentage of transactions were going through this little startup called PayPal, and we had our own competitor to PayPal. And then eventually, you know, digital payments caught up, and now, um, we are able to use credit cards and things like that and have assurances on our transactions. But I went to 46 of the 50 states for eBay to talk to regulators and trying to get them to work with us and, uh, to enable this platform. I trained law enforcement in, like, a dozen different countries on how they could, uh, prosecute somebody for doing bad things on eBay. So we were, like, trying to pull law enforcement and government to pay attention to what happened on the internet in the early days.By the time I got to Facebook, it was, it was still the same thing, uh, but there was a little bit more tension. There was this whole, uh, situation with that guy, Edward, uh, Snowden, and, you know, he left the NSA, and he revealed all these documents that made it look like, uh, Silicon Valley was sharing everyone's data behind the scenes with the NSA. Um, that wasn't the actual full story. I ended up in the middle of all that basically as the face of, of Facebook interacting with the NSA because I had managed our relationship with them all along. And so that was the backdrop when I got to Uber in, uh, 2015. And Uber was kind of like, um, the beginning of that mobile explosion. If we think about like the transition and how technology has become such a bigger part of our lives in the last 20 years, it was really like phase one was like regular internet, uh, "Oh, we can do e-commerce. Wow, this is amazing." And then part two was that mobile explosion. Uber couldn't exist until there was an iPhone. And, uh, it's like led to this next generation of explosion of technology companies really taking over the world. And when this happens, when technology becomes the most important thing, all of a sudden the government folks really start to care about technology, and that's what's happened in the last decade. We've seen a lot more initiative going back to around the time of the, uh, first Obama administration, 2008 to 2012. They really started trying to figure out, "How do we get closer to Silicon Valley?" Um, President Obama came and visited us at, at Facebook. Uh, so did George W. Bush and Al Gore. Uh, and so like you started seeing, uh, a lot more of that interaction. So I was at Uber. Everything seemed to be going okay, and then one day I got this text or email. It was from Eric Newcomer, who's a reporter at Bloomberg, and he w- messaged me 'cause he wanted to know about me getting fired from Uber. I had no idea what he was talking about. I was on vacation with my family up by Lake Tahoe. It was Thanksgiving week. I'd taken the week off. Um, after, uh, getting that, this was the headline I saw. He wrote, published an hour later, "I paid hackers to delete stolen data on 57 million people," according to the news, and it just blew up across the planet. My phone started going crazy with people texting me, trying to call me, and right in the middle of that, my phone stopped working, 'cause my phone had been issued by Uber, and my team had put software on that, and then my team used that software to brick my phone and my computer, because the company had decided to fire me. So I was all of a sudden like the most famous person in cybersecurity for the wrong reason, uh, about a decade ago. And, um, that hurt a lot. Uh, I w- I, um, I'm still involved in litigation related to that. But I, um, I went into hibernation for about two months, grew a beard, didn't wanna show my face, and then in early 2018, I decided I gotta get off my butt and get back going in life. And so I went out and tried to apply for some jobs, and that was when I got, um, hired. Well, the funny thing was after, after going through this, the first three companies to contact me about working for them and running security, Huawei, WeWork, and ByteDance. I'm dead serious. They would love to have me despite all this. Uh, instead, I chose to go, uh, work at a small startup called Cloudflare. Uh, and Matthew Prince, uh, I think maybe speaks to this class. Um, Matthew did his due diligence. He talked to Travis, who had been my CEO at, and manager at Uber, and a lot of other people, and decided he would take a chance on me. So I went to-- So I went and worked at, um, Cloudflare starting in, uh, spring of 2018. And then 2018 was the midterm elections, and t- 2016 was when President Trump was elected for the first time, and then there was the midterm elections. Cloudflare got so much negative heat because I got doxxed, and this, uh, organization, uh, these group of organizations I'd never heard of went after me. Pl- Don't go to that URL 'cause you'll see the entire doxxing of me, because Google refused to take it down, uh, even though I submitted a takedown request. But, uh, I guess if you go there, you'll see, uh, I have six brothers and sisters. You can see all of their addresses. You can see my family's information. Uh, there's a whole timeline. There's all kinds of information about my mom, who worked for the CIA, like the, uh, and, and lots of other stuff that I didn't even know about myself. Um, and so like just me, because of what I'd gone through before, I ended up inflicting this on Cloudflare. And, um, the thing I'll say about Cloudflare is that is a company that really cares about transparency. Um, when I joined the company, uh, I had my first security incident, and, um, I had been through a lot of other security incidents where we don't get to control the communication about the security incident on the security team. It's a cross-functional thing. You're supposed to work with, you know, the communications team and the legal team. Legal says what can go out. Communication team polishes it up. The CEO has to sign off. That's the way communications work in companies, right? So at Cloudflare, I had my first security incidentI call Matthew, our CEO. It's a Friday night, 'cause security incidents only happen on Fridays, um, so that your team has to work all weekend. Um, it's, it's, it's a science. It's been proven. Uh, and so on that Friday night, I call Matthew and I say, "We have a security incident." And he said, "Who's writing the blog post?" And I always remember that. I'm like, "What do you mean who's writing the blog post? We- we're bleeding here. I need to make sure we stop bleeding and make sure that w- our customers are safe." And he's like, "Who's writing the blog post?" I was like, "I, I'll figure that out later." And so I hang up. Five minutes later, who pops onto the Zoom but our CTO. I'm like, "John, why are you on this?" He's like, "I'm writing the blog post." Like, our CEO had made our CTO just join my incident response kind of working room just to write down and document everything so we could be transparent. A year later, we had our first big real outage as a company. I was over in London, and it, it was our local team in London, uh, uh, pushed a, uh, a rule to our WAF that basically took down half the internet. And fortunately, most of the United States was asleep because of, of the timing of it. But John and I had to-- We called every large customer that we had. We put out a detailed, uh, blog report. We had literally disrupted the entire internet, and a day later, if you went online and you looked at how c-Cloudflare was being discussed, they were praising us for transparency. Instead of break- getting, like, slammed for breaking the internet, we were getting praised for being transparent. And I think there's this constant tension between transparency and not around technology, what-- the good and bad of it, and I think we need to bias more and more the way Cloudflare has towards this transparency. So after that, what ha- uh, it's twen- now 2020, and, uh, the FBI issues this press statement saying that they have arrested me. My oldest daughter, uh, was moving into her dorm at UT Austin at the time, and she calls me because a friend of hers had heard on NPR that I'd been arrested. And so she is freaking out, and she calls me. I'm sitting at my desk here in Palo Alto. I live in, by Midtown in Palo Alto. I was sitting at my desk on a Zoom for Cloudflare. I hadn't been arrested. Um, so we have to add one thing to this. Um, so, uh, for-- I hadn't been arrested, but what I had been, uh, was charged with a crime. So I've never been arrested, but I did get charged. I got charged with obstruction of justice and misprision of a felony. Without going into all the details, what it basically means was I was being personally held responsible for the company's failure to be transparent with the government in 2017 or '16 when that security incident happened. Um, so I wanna take you through the security incident a little bit. I'm gonna skip the legal stuff. I went to trial against the government in September of 2022. One of my daughters drew this picture because you're not allowed to take cameras in federal courts. Uh, so this was during the trial. This was, uh, uh, the person on the stand there was a lawyer from Uber, coincidentally, when my daughter drew this picture. This is the chief privacy, the head of privacy and regulatory for legal. And she testified, "It's my team's job to tell the government about security incidents, and, and my team owns responsibility, and my team was the one that, uh... And I personally knew about that security incident. And yes, we did not tell, uh, the government agency that was investigating us about the security incident." So she said all that, but I was the one who was the defendant sitting in the courtroom wearing a mask because it was COVID times. The jury never actually saw my face through the whole trial. They only saw a guy in a suit with a mask on. Um, so what, what was the case actually about? Uh, I really believe in this concept of responsible disclosure and trying to get the hacker community to work well with corporations. So when I-- In 2007, when I was at PayPal, we published a responsible disclosure policy. It was the first time a company published one. If you do security research, you know what this, these policies are. If you don't, you've probably never heard of them. But what we d- what we said in 2007 at PayPal was, "If you find a vulnerability, please tell us about it. We promise we won't sue you. We promise we won't tell law enforcement about you. We wanna have an open dialogue." So we did that in 2007 at PayPal, and other companies started to follow suit. I, uh, went to Facebook in 2009- 8, and we published a responsible disclosure policy there right after I got there, 'cause it was something that I cared about. Then a couple of years later, there was this movement in the hacker community that was like, "Wait, that was nice that you said you won't prosecute us, but why don't you actually pay us money? 'Cause you're fi- you're finding vulnerabili- we're finding vulnerabilities. We're making you safer." And I remember the first time I got that email from, uh, a hacker, and it said, "Pay us money, and we'll tell you about the vulnerability in your systems." And if you own the system, and you own security for that system, and you get that message, you get kinda mad. And I used to be a prosecutor, so I was like, "I'm g- this..." I get double mad. Uh, and I start thinking, "How can I use the law against you?" Right? It's like... And then, and then my team's like, "Joe, shut up." Like, we should be paying these people. And I came around to that. And so I think it was 2011, t- 10 or 2011, at Facebook, we launched the third-ever bug bounty program. Like, bug bounty programs are a, a thing everywhere now. Google last yearPaid out I don't know how many millions of dollars in bug bounties, and they just announced a new, uh, program where you can get $250,000 for a single vulnerability. And so, like the world has been evolving to this place where we recognize that our goal should be the best possible security, uh, and that we should cultivate these relationships. So when I got to Uber in 2015, we published a responsible disclosure policy. Uh, and I should add that when I went from Facebook to Uber, about 40 of my team came with me, and so we brought not just me-- I, I went not by myself, but over the course of a few months, a lot of my team, so much so that, uh, the general counsel from, uh, Meta sent me that warning letter that you sometimes get. Um, and then, uh, we published a bug bounty program. Uh, and, uh, we had it running in private for like a year before we launched it publicly in the spring of 2020-- 2016. And in the fall of 2016, this is the email I got. "I found a major vulnerability. I was able to dump database and other things." And I did what I always do when I get this email, 'cause I've gotten a lot of this email over the years. I forwarded it to the product security team that manages the bug bounty. Member of our security team emailed and said, "Hey, we've s- we use HackerOne for our bug bounty program, uh, but we're also happy to work with you even if you do it otherwise." This is the email from Rob Fletcher, who's now a startup founder somewhere. Uh, but he, uh, he, he led the interaction with this person who wanted to be anonymous, and they showed us that they had actually found a vulnerability in the way our AWS was configured related to some old databases that we d- like my team didn't even know existed because they had been deprecated before we got there. Um, we treated it like a security incident. We documented everything. We had a centralized tracker, uh, and all my team's notes are still there from it, uh, because I was going to trial over this. These are all slides from the trial actually, from like my lawyer's closing argument. It was showing like, "Here are all the people in the company who knew." I went to the CEO. He signed off on us paying the bug bounty, 'cause we paid $100,000 to these researchers. It was all approved. Legal was-- Three lawyers were in the loop. Communicate-- Uh, two lawyers, the communications team, all in the loop. Um, and we actually had written formal policies and documentation, and it said, "Legal's responsible doing the investigation, reporting it, et cetera." And we ran the whole thing by legal, and they said, uh, "We don't think we have to disclose it." The communications team had already prepared documents for if they were gonna disclose it. They put those aside. Uh, I said to my team, "The, these people are still anonymous. Can we find out who they are and actually go interview them and make sure that they have deleted the data?" So my team did an investigation. I'm not gonna go through all the details here, but long story short, we were able to figure out who they were and where they were. Um, it turns out at the exact same time that we were doing this investigation, the FBI was also doing the same investigation because, uh, these two guys, uh, 19 and 20 year old, 19-year-old down in Florida and 20-year-old up by Toronto, who had met in gaming community, where they, they had found vulnerabilities in a few companies of the same type. And so they reached out to a few companies. I think they reached out to five companies and said, "We found vulnerabilities." We worked with them, paid them, fixed the vulnerabilities. Another one of the companies, which was, uh, LinkedIn, decided to contact the FBI. The FBI then tried to find them. We didn't know any of this was going on at the time. The FBI couldn't find them. My team was able to. Um, and, uh, my, my team and I still get involved in working with the government on situations like that, um, because we're really good at that stuff. And so, uh, we were able to find these guys, and, um, I had a retired CIA, uh, intelligence officer who's specially trained in interrogation, a top trainer from-- He trains other people from the CIA on how to do interrogation. So I sent him down to interview, uh, Brandon. Well, actually, Matt from my team sent this email. We basically figured out who Brandon was, where he was, work-- uh, living, uh, down in, um, Florida, and we sent him an email and said, "You gotta be really careful in these situations. You'll be viewed as an extortionist. We don't think you're an extortionist. We think that you should be paid." And, uh, by the way, one of my team-- Oh, he didn't know we knew his name was Brandon when we sent him this email. So this was kinda like we send you the email, and we sent it to his real email address instead of his ProtonMail. So you imagine you're Brandon. You wake up that day, and there's an email saying, "Hi, Brandon." [chuckles] "Uh, this is, this is Matt from Uber. Uh, and one of my team members is right around the corner. Can you guys meet today?" Um, that's wha- that happened, and then my team member, the trained CIA interrogator, went in, and he prepared for me a, a, a, I think it was like a six-page psychological profile of the guy and documented and validated, uh, that the data was deleted, that our customers were protected. So this is a situation where at the end of the day, legal had signed off on, uh, the communication side, and my team had done the work where I felt comfortable our customers were protected. And we closed the chapter on the case, um, until 2020 when I got charged with a crime. I didn't know until much later that apparently, you know, people were agitating behind the scenes from Uber and others to, to g- to get the government to go dig into this. So I go to trial. Uh, we come through the trialMy lawyers at the end of the evidence say, at the end of the government's case, they said, "Joe, we don't even need to put on a defense. We totally won." I was like, "Okay, sounds good, but like let's just call a couple witnesses to fill in these little gaps." We did. So we barely put on a defense, and then the jury goes out and they deliberate for a few days, and I'm just like, "Guys, if, if, if it was such a easy slam dunk victory for us, what's going on?" And then this question comes out, uh, "With regard to this, uh, hacking statute, does Uber have the right to extend authorization after the access?" So, uh, under 18 USC 1030, there's-- this is basically the computer hacking statute. It says like, "So if I access your computer without your permission, I violated the law." And then there's various levels of significance beyond that point. And so the legal question was, when Brandon and the other guy accessed Uber's AWS, could we after the fact give them permission, or was it automatically a crime the second that they accessed our computer, and do we have the ability to unwind it? All the advice I'd ever gotten, and we'd discussed this a million times before with lawyers, is it's like the o- they would say, "Oh, it's like the old trespass statutes." You know, if somebody steps into your front yard and you can be like, "Oh, hey, come on in," that kind of effectively by law means it's no longer a trespass. And so that was the advice that like the bug bounty platforms and our lawyers had always told us. But then when the jury asked this question, the judge was not so sure, and the government was arguing at that time, "No, we can't, uh... Uber couldn't give permission." So the jury basically got this instruction, "Uber cannot give, uh, permission." So effectively, it just basically gutted our whole defense. Um, and so I could be held accountable, uh, for a criminal, uh, obstruction supporting the bad guys, even if I had gotten legal approval and didn't think that we did anything wrong. So we lose the trial. Uh, it's now October of 2022. I went through that period in 2018 where I had to climb back on my feet, and in 2022, it was a lot harder because I had just lost a trial. So like I called all these different nonprofit-- 'cause I, I was sitting, sitting around at home moping again, and I called all the different nonprofits who always wanted to work with me, and they were like, "Uh, yeah, we can't be associated with you this time." And so I, um, I, I had been helping Ukraine through my role at Cloudflare, and I realized that the only people who were willing to work with me in the fall of 2022 were the Ukrainians, uh, because they had nothing to lose, and they didn't care about my case. Uh, so I joined, uh, a nonprofit called Ukraine Friends and, uh, became their CEO. Uh, w- I started a program called Digital Wings. I realized that at every tech company, we have these piles of laptop computers that are sitting behind the help desk, 'cause, you know, we hire a bunch of people, half of them don't last two years, but we're not gonna give those computers to the next new employee. So the piles of computers get bigger and bigger. On my first trip to Ukraine, a friend of mine was the CISO of Robinhood at the time. H- he gave me 20 of their, uh, uh, cleaned up, uh, used computers, and so I brought them in my carry-on. You know when you get to the airport and they're like, "Do you have any lithium ion batteries?" I'm like, "Yeah, I got 20." [laughs]

2. SPSpeaker28:06

   [laughs]

3. JSJoe Sullivan28:06

   Uh, they, they didn't know what to do. They just let me on the plane. Uh, and since I'd already been convicted of a crime, uh-

4. SPSpeaker28:15

   [laughs]

5. JSJoe Sullivan28:16

   ... I was like... Anyway, um, I'm just kidding. I actually, I really take seriously the shipping. Uh, uh, I've shipped thousands of computers to, to Ukraine at this point, and I've learned everything about safe shipping of lithium ion batteries and the like. And it's, it's really important you take those things seriously, uh, because there have actually been fires and things on planes. But public service announcement aside, um, I got to Ukraine with a bunch of laptop computers, and I realized, uh, what a need there was. So my nonprofit, we get kids-- uh, we bring computers to kids who've lost a parent in the war. My last trip to Ukraine was two weeks ago. I was there two weeks ago for the week. Uh, uh, TD Bank had donated over 1,000 computers, and so I was there to kind of like oversee the distribution of those. Uh, and, uh, we work directly with military units so that some of the soldiers in the unit can give the laptops to the kids of their fallen brothers. Uh, and, you know, the, the people who survive feel like almost a sense of responsibility for the families of, of, of, of those who didn't survive. And so we like to work with them to help them. Uh, and, uh, you know, what the people in Ukraine have been going through, it's incredible, their resilience. I, I come back inspired every time I go. I've been six times in the last three years, and, uh, I wish I could go more frequently. Um, so I'm doing this work in Ukraine and I'm waiting and my sentencing keeps getting postponed. I had the most amazing thing happen. I'm in like this funk. No one will hire me. I'm volunteering in Ukraine, seeing sad stuff, and I'm waiting for my sentencing hearing. And the government says, "We're gonna argue that you should get three years in pri- in federal prison." Uh, I guess I'd still be in federal prison if, if they had gotten that. Um, there's a process that you go through, though, for, uh, before you get sentenced, uh, and that in the federal system is there's somebody called-- there's a probation office, and they prepare a pre-sentence report where they kind of review your whole life. And so it's like, it's like a 75-page document of everything about me so that the judge can make an informed decision.And by the time the probation office got through documenting, like th- Joe's been a volunteer for the federal government 17 different times since he left the government, doing all these different things, like, um, and involved in these different nonprofits and helping people in Ukraine, et cetera. The probation office came in with a recommendation to the judge, "You should just give Joe probation and let him go live his life." And the prosecutors, when they heard that, they dropped down and instead they argued that I should get 18 months. But so during that process, um, I had the most amazing thing happen, which was I got these emails, and attached to each email would be a letter to the judge. I got over 200 separate letters to the judge sent to me by people who'd worked with re- me through my career, uh, by people who were upset about my case. One letter was signed by 60 people in the cybersecur-security community, another by 50, another by 40. It was like this mass uprising of support because they felt that the case was unfair, or even if they, they didn't know anything about the legal stuff, they s- they, they wanted me to be out and, and doing what I do. And so I had a sentencing hearing on May 4th, 2023, so literally three years ago and a couple weeks, or a week. Um, and the judge said it wasn't a cover-up. That was the best thing I ever could have heard. Uh, the judge then went on to, you know, basically yell at the prosecutor in some sense, saying, "Why, why-- If you're charging a company, why wouldn't you charge the CEO? The CEO was in the loop. Uh, CEO supported all the decisions. If we're gonna hold corporations accountable, let's start at the top." Uh, he, uh, also yelled at the prosecutor, like, "There was no financial incentive for Joe to do this. Why, why do you, why do you think he would do this? Do you think he needed to protect himself for his career?" Stuff like that. He just said, "I've never seen a case like this in my life." And then he sentenced me to three years of probation and a small fine and sent me on my way. So I actually finished my probation, uh, a week ago. I got a letter saying I'm off probation. Um, [audience applauding] thank you. I still get secondary inspection every time I come into the country, but, uh, my daughters really enjoyed it the first time. They were like, "Dad, this is so cool." Uh, but yeah, so I, I landed on my feet. I started my security consulting business. I still do the nonprofit stuff. I've been working with some VCs. Costanoa made me a venture partner. I've been advising a bunch of comp- startups. Uh, this slide's actually outdi- dated because, well, four of, four of these companies have recently gotten acquired, and so I no longer advise them. Um, but I was happy they got acquired. Um, I get to go do keynotes. I, I got, I get paid to speak all over the world. I-- This year, I keynoted a big AI conference in January in, uh, Tokyo. This was a very, a keynote in Australia. Um, and so I get invited and, uh, and paid to go do these things that I love to do and talk about, uh, things like this case. And, and, and I just wanna, like, spend, like, five more minutes on, like, the wor- Cybersecurity and the world of cybersecurity has changed so much since I got involved. When, when that Uber case happened in 2016, it was like the worst-case scenario. Data had left the building. Like, in cybersecurity, that's all we cared about for the longest time. And then something new happened around 2018 and '19, which is ransomware. So now in 2025, 2026, cybersecurity is... Still we care about data leaving the building, but we also have to care about operational resilience. Does anybody know what happened to Jaguar Land Rover last year? They got hit with probably one of the biggest cyberattacks. It was a ransomware attack, and last, I, I think it happened last August. They literally had to shut down all of production for all of Jaguar Land Rover for three months. The UK government had to do a bailout of over a billion dollars. A bunch of their supply chain companies, so, you know, a Jaguar is not just all the parts made by Jaguar. They're made by hundreds of little companies. When Jaguar couldn't pay, pay them for three months, a lot of those companies went out of business. So, like, the impact of a cyberattack cost the UK economy literally billions of dollars, billions of pounds, and anybody who owned a Jaguar Land Rover during those months couldn't even take their car into a mechanic shop. So that happened. Uh, cybersecurity became about operational resilience. And then also, you know, what's going on with AI. Uh, I, I just got back from spending the last three days in meetings in Washington, DC, uh, 'cause I do some volunteer support, uh, for a couple of government agencies now. And, um, it'd just be like, it's weird. I'm under, I'm, I'm on probation and under investigation by one part of the government, but I usually am helping a different at the same time. I, I have had these conversations where it'd be like in the morning I'm with the FBI talking about something, and in the afternoon I'm with the FBI talking about them putting me in jail. It's been, it's been pretty surreal. Um, but so I was there earlier this week, and the amount of pressure the government is feeling right now about AI, you know... I, I work with some companies that have access to Mythos, the, the cyber model from, uh, the cyber used model from Anthropic that's so powerful, and it, it, it is, it is as powerful as everybody says. Like, we're finding things, uh, that are amazing, uh, and scary. And so the government knows that and really needs cybersecurity to step up in the next six months, because those mo- that type of model that's being held close right now is gonna be publicly available in six months, even if it comes from the open source guys. So that's the future we're facing. All of a sudden, every CEO really cares about cybersecurityI get a call a day, "Joe, this CEO needs a head of security right now. They need somebody who's, like, has the experience you have, where you can... You're comfortable reporting to a CEO, sitting in the exec room, co-running a company." That's the kind of people we need in cybersecurity right now, and I, I don't even have enough people to, to refer. At the same time, governments are tightening up on the regulatory side. A lot of other countries are thinking about doing enforcement actions like the ones against me. So it's this weird situation where a lot of my peers call me. Like, I hear from every CISO in every bad situation, and I also hear from... You know, like, they call me when they're like, "Joe, my... We just had a ransomware, and the CEO's forcing me to sign something to go to all our customers saying that everything's fine, and I know everything's not fine. What do I do?" Like, I get questions like that every week from people in the role. And then the other question they get is like, "I'm being asked to take the top seat. Do I even want it?" Uh, because it's really scary to be a cyber security leader in this environment right now. And the thing I'll say is I've been through a lot, and one of the things I've realized is that you have to have resilience. And I don't care if you're going into cybersecurity or what other jobs y'all decide to go into, you're gonna get punched in the face sometimes. And you gotta think about, "How am I going to handle getting punched in the face?" Like, you... Whe- when a boxer goes into the ring, they know they're gonna get punched in the face, and they think they still have a plan. I think leadership in 2026 and beyond is about that resilience. I, I... These four people... Like, ever since I went through my thing and I've had people say like, "Oh, you're a model of resilience," I started looking. There are a lot of really good models. Like, these four people all got punched in the face when they thought they were at the peak of their career, and they thought they were at an amazing place, and then they end up going 10 times higher in their career. And you can find so many people like that. And so the thing I would talk... You know, I do a lot of work with organizational leaders, and I... You know, I g- I did, like, a four-hour how do I prepare for... I have, like, a literally a four-hour program on how do you as an executive prepare yourself, your team, and your company to deal with crisis before it happens. I'm not gonna go into all that stuff with you, but I want you to think about and remember that we don't write into the job description resilience and crisis management. But if you're working in technology in 2026, we're so highly visible, there's so much pressure on us, we have to be ready to get punched in the face. And that means thinking about what are the key elements for success in a crisis. I think the number one element for success in a crisis is actually how well you communicate. I, I, I brought up how Cloudflare has handled crisis over the years. They always err on the side of transparency, and it always builds trust. Uh, companies that choose, like, say, Uber in 2016, not to be transparent, it leads to this boiling negativity over time. So my last thought for you is this. Run towards those opportunities, run towards those stressful situations, 'cause the more you go through them, the better you'll handle them. I get invited to work at companies, the coolest companies on the planet, because they have confidence that I have wisdom from g- having gone through the bad things. If you try and steer your career to never go through bad things, you'll never get the wisdom and experience you need to really succeed.

6. SPSpeaker40:23

   So the question is, how do you rebuild your, your reputation, which is clearly, you know, world, world-known?

7. JSJoe Sullivan40:28

   Yeah. I, um... It was interesting. So I, I, I consider... Like, I consider I lost the trial in the fall of 2022, but I won the sentencing in the spring of 2023. And it was really, um, my wife, who's here in the front row, who's a Stanford grad, she came along today. Um, she was there with me through it all. And having, having, having strong support at home, number one, um, was really important. But then I had a lot of support from the community. Like I, I mentioned those letters. Uh, I joke that it was like a, a... I got to sit through my own Irish wake, you know. Uh, the idea that, like, um, I got to hear all these people say good things about me while I was still alive. And I, I bring it up a lot with leaders because what you don't realize when you're a leader is how much the little things you do or don't do, um, your team picks up on. Like, w- I had people write in these letters to the judge talking about things that I, I didn't remember at all. It's like, I didn't remember I had lunch with that guy and my team's kid who was thinking about cybersecurity, but apparently I did. You know, I didn't remem- Like, there were just lots of examples like that. And so, um, so after I won the trial, I reached out to a couple of people. I decided I should... I couldn't talk for seven years. My lawyers wouldn't let me talk, so I was just all negative for seven years. And so a- after it was over, I reached out and, um, I reached out to the guy who runs the DEF CON conference, uh, who started it in Vegas back, whatever, 30 years ago, and he'd started Black Hat as well. So they're two of the most well-known cybersecurity conferences. And I said, "I'd love to get a chance to tell my side of the story." And I, um... And he, he contacted me back a week later and he said, "At Black Hat, we have a CISO summit, so, like, all, all the security leaders from the biggest companies will be there. Uh, you can do an off-the-record talk there if you'll do an on-the-record talk at DEF CON." And so those were the first two times I was talking about my case. It's funny, my dad emailed me the other day 'cause he found the DEF CON talk and watched itUh, three years later, and, um, and he emailed me about it. And it ma-- And I was just reflecting on, I was so nervous. I was so nervous because a friend of mine, I went... who lives here in Palo Alto, he'd been on the early Facebook team with me. I went walking with him, and he'd said, "What are you gonna do if you get booed?" And so I mentally going into the speaking was worried that I was gonna get booed. Uh, but I just did it. I got up and went and did it. It was the same thing as, like, in January of 2018, the first time I went to a security conference after getting fired on global news. I felt very sheepish and awkward and uncomfortable, but I got through it. When I spoke at Black Hat at that CISO summit, I got-- ended up getting a standing ovation from, like, my peers, the best security leaders in the world. And so that just gave me the confidence and courage to, to go forward. Uh, I started my own consulting business, and then I had success doing it. It was mostly what I've learned is that I was mostly able-- Like, large companies can't be associated with a felon, um, although I do work with some large companies, but they prefer that we keep it under NDA. Um, and so I started embracing working with startups even more 'cause startups don't care. They just wanna have the best security-

8. SPSpeaker43:59

   Mm-hmm

9. JSJoe Sullivan43:59

   ... they can get from somebody who understands them. So I just, I just have been building it ever since.

10. SPSpeaker44:04

    Good. Next question.

11. JSJoe Sullivan44:05

    Next.

12. SPSpeaker44:06

    So the question is, what are the security issues around vibe-coding, and what should we be thinking about?

13. JSJoe Sullivan44:11

    Yeah. I actually joined the board of an OpSec company, uh, last fall, and, and with at-- over at the VC, we've been looking a lot at how application security is evolving, and I've been thinking about... And the, the, uh, the companies that I advise and work with are obviously in different stages of, of, um, embracing it. Financial services is really slow on embracing it, but some of the other companies I work with are really deep in, uh... Like, a large percentage of their code is being generated through, um, these tools. Uh, the, the first challenge is just the sheer volume of code being generated, uh, has gone through the roof. Uh, like one small southeast bank that, uh, we work with, uh, they went from, like, 250,000 lines of code a month to, like, 1.25 million lines of code a month in, in, like, a two-month period after G- uh, so challenge number one is the sheer velocity of, uh, um, of code. Challenge number two is that, uh, one of the other companies I work with here in the Bay Area, their, um, their CISO called me, and he was like, "We just had our, the first marketing person merge into production, and there was a vulnerability, and we tried to kick it back to marketing, and they don't know how to fix the vulnerability." So in like-

14. SPSpeaker45:27

    [chuckles]

15. JSJoe Sullivan45:27

    You know, whereas a, a, a software engineer would actually like, "Okay, here..." You know, security could send, could send them a proposed fix, and then they would... Typ- the typical OpSec model is the security sends a proposed fix, and then the t- the engineer actually looks at it and thinks about the bigger context. But it's somebody from marketing, you can't really do that. Um, so that's the second challenge. Um, the third challenge is, um, it's not just a vibe-coding, but like Cloud Cowork, for example, is... I mean, which is really Cloud Code with a wrapper. Um, Cowork, uh, it's getting non-technical employees to be even more ambitious with connecting externally, and the way that they'll solve problems is if they don't have the API key, they'll go out and try and, you know, literally they'll go try and set up their own remote external server so that-- and create their own API key, and you're like, "There's no way an engineer would do this." So we're seeing all kinds of crazy things.

16. SPSpeaker46:20

    [chuckles]

17. JSJoe Sullivan46:21

    There is no one silver bullet solution. I'd say companies are walk- are coming at it from two different directions. Some companies are doing YOLO and then trying to clean up. But a lot of companies, and smart companies in particular, are, are starting out with pilots and constraining to just software engineers who, who know better and then are slowly adding, um, different groups. I really believe that we can't solve, um, a-- We can't solve the headaches, uh, the security headaches of agents inside our environment just by putting guardrails on them 'cause it's not... You can't say, "Okay, y-you can have access, you can have write access to my email for purpose A but not purpose B." It's like we just can't do that. And so we have to have, um, kind of like anomaly detection around. I think of it like-

18. SPSpeaker47:09

    [chuckles]

19. JSJoe Sullivan47:09

    ... um, agents inside companies are like toddlers in-inside a house. They're running around. They can run, but every so often they're gonna, you know, if you ever seen a parent of todd-toddlers, they're kind of running next to them. It's like real time, runtime. Um, and that's what I think we're gonna have to get to in, in, um, agentic solutions. It's like we'll put some guardrails, but it's not that they have access, it's what they do with the access that we have to pay attention to.

20. SPSpeaker47:33

    Hmm. Interesting. So, so the question is, what would you have done differently if you were back leading security at Uber?

21. JSJoe Sullivan47:40

    Yeah. So from a technical operational side, my team, my team-- I was so happy that we actually got to get to the trial so that the world could see what my team did technically. Um, I think everything we did, I would do the same. I wish we had more documentation. I'm actually an advisor to a company now called BreachRx, which, uh, creates a platform that forces legal and, um, communications to work more directly with security. And I started working with them, uh, before they g- even got their seed investment because I really believe that it is about how you get the different teams inside the company to work together on transparency. Um, like in the middle of a security incident, the security leader doesn't have the credibility around communication or legal issues to say, "We should be public about this." You have to work through that stuff ahead of time. So operationally, I wouldn't change anything. We should be paying those researchers. Um, we should be, uh, fixing things. We should be working with legal. Um-I think I spend much more time now educating the other executives at the companies I work with, n-not just the security team. Like there's-- When you become a leader of a company, you don't actually work on your team anymore. You work on the leadership team of the company. When I mentor a security executive, uh, I always start out with a, a question that's a-actually a trick question. The first time I'm meeting with someone new, I say, "Tell me about your team." And they immediately start talking about, "I got this team that does detection. I have this team that does application security. I have this..."

22. SPSpeaker49:11

    Mm.

23. JSJoe Sullivan49:11

    I'm like, "No, no, I mean your team." They're like, "What do you mean?" I'm like, "The other executives at the company."

24. SPSpeaker49:17

    Mm.

25. JSJoe Sullivan49:18

    When I was at Facebook, I had an exec coach, and she told me that I should be spending 50% of my team with the other executives instead of with the security team. And I actually think for a security leader, it needs to be even more, 'cause our world is dark and scary and confusing. It's not very measurable by metrics, and you only hear the bad stories. And so it's our job as security leaders to get out and really build trust with the other executives at the company so that in the crisis moment, they'll trust us more. Yeah.

26. SPSpeaker49:47

    Questions around quantum cryptography.

27. JSJoe Sullivan49:50

    I'll tell you that, uh, this comes up all the time. Like, I was in Florida last week for a closed-door group of, like, 20 security executives, including from a bunch of the large, um, uh, ga- uh, gas and energy world, oil, gas, energy world. And we had a whole session talking about, like, what are we doing about the quantum risk and opportunity? For the most part, companies are not doing a lot right now. I think the reality is that we could, you know, if we look at how the pace of AI, uh, has, like, sped up from predictions, uh, quan-quantum seems like it could be here by 2030. And so arguably, we should be doing stuff. But for the most part, when you think about, uh, where cryptography exists in our environments, I think that, um, most of the work that needs to be done needs to be done at the Googles, the AWS. Um, like the biggest risk probably to most of us right now is that, um, agencies of governments have vacuumed up a lot of historical communication data that has been encrypted by non-quantum resistant, uh, encryption. And so if you were part of a terrorist group five years ago, you might have some trouble in five years. Uh, that kind of stuff. Um, like the quant- uh, most of our environments that are the main infrastructure companies supporting them are gonna be quantum resistant. Um, and also, if you flip it around, it's a little bit like the Mythos situation. Once we get quantum, it's not gonna g-gonna be like all of a sudden every data center is a quantum data center. Quantum machines require extreme cold and all this other stuff. So it's gonna be like a few people have quantum before everybody has quantum.

28. SPSpeaker51:44

    Mm.

29. JSJoe Sullivan51:44

    And then there's going to be a period of time, uh, and ho- so hopefully it'll be the good guys get quantum before the bad guys, and then they can do kind of what Anthropic and OpenAI have been doing with their new cyber models.

30. SPSpeaker51:55

    Actually, just I have a question. On those models, like, uh, with the Mythos, like what is your opinion on how those tools should be released early and, and kind of like what, what's the right kind of process there, do you think?

Episode duration: 1:05:18