Episode 379 Deep Dive: Sean Duca | Confidence Is Not Permission - Rethinking Authority in the Autonomous SOC

Sean Duca [00:00:00]: Being able to isolate a server and being allowed to isolate it are 2 different questions, and only one of them is being asked. KB [00:00:07]: From KBI Media, I'm Karissa Breen, and this is KBKast. My guest today is Sean Duca, co-founder and CEO of getsoteria.ai, with 25 years in cybersecurity and former senior security roles across Cisco, Palo Alto Networks, and Intel Security. We talk about why confidence isn't the same as permission, what governed autonomy really looks like once the agents are live. And the question no vendor seems to be really answering is who is actually allowed to act. If today's episode lands for you, do 2 things: hit follow and send it to one person in your network who needs to hear it. That's how shows like this grow. Alrighty, let's get into it. So Sean, welcome back to the show. KB [00:00:58]: I want to start perhaps with, are security vendors confusing AI capability with AI authority? And are customers actually about to pay the price for that mistake? There's a lot of conversations going on in the market, but what's your view? Sean Duca [00:01:13]: Yeah, good question. I think it's a couple of parts. So yes, it's probably the first part. And I think the confusion is probably a structural one rather than probably something that's careless. Vendors typically ship a model. They can isolate a host and then treat the ability as permission. Being able to isolate a server and being allowed to isolate it are 2 different questions, and only one of them is being asked. I think for years, when I think about security technology, having been in security now for 25+ years, everyone's always worked on this notion of purely based on confidence. Sean Duca [00:01:45]: On whether customers actually pay, they really are. So if you think about IBM's data breach report, like the most recent breach report they sort of came out with, around 97% of organizations that have suffered an AI-related security incident lacked a proper AI access control. And 63% of them had no AI governance policy whatsoever. So I think that the reality actually, we're seeing more and more challenges popping up where AI is actually being used. And I think this is where we're seeing challenges actually sort of popping up. People are actually being impacted and there is no real sort of formal governance in place. And governance is not necessarily a little sort of set and forget document or something like that. It's actually something that needs to be inherently part of the capability that people are actually using. KB [00:02:32]: That's interesting. I literally interviewed someone today about governance and policies, but would you say that even if someone writes this nice policy around AI, et cetera, like how are people going to start enforcing it? Because people, then we have shadow AI and all sorts of things, but What does that then look like? Because people have been trying to go around policies for years and now it's like, wow, we're dealing with this AI, this live wire situation. Like there's obviously a lot more risk attached to it. So what would be your view to making sure, like, I don't wanna say it, but like adhering to the policy there, I said it, but what, what can people actually start to do practically to make sure that this doesn't become a massive risk? Sean Duca [00:03:12]: So look, that's where I kind of think you need to look at it from a capability standpoint, which is probably gonna be part of a, part of a moral part of a solution. But then the other aspect is authority is a decision that's done by the business. So the industry keeps on letting the first one be answered for the second, if that makes sense. So what I actually mean by that is there is confidence, there is permission or the authority to act. And I think some people conflate the two together to say like, it's just one of the same. And you go, no, no, it's actually two different things because you can be 99% sure about something. It doesn't necessarily mean that you've got the right to act and the ability to act on a particular system or a post or to do a particular type of action as well. I think that's the fundamental difference. Sean Duca [00:03:55]: And that's the thing more and more when I kept on thinking about what autonomy actually looks like, you know, what does governed autonomy actually look like? And I think that's where I started to land on this whole notion that you need to have both confidence and permission. KB [00:04:07]: Okay. So talk to me a little bit about that. What does governed autonomy mean in your eyes? Because people, well, if you look at, for example, I'll give context. So if you look at agentic AI now, everyone's like, okay, well, this is the next thing that's coming down the pike, but then people are still freaking out saying, well, If I can't, if I don't actually know, I can't intervene at any point and kill it, that's a massive problem because things could start making its own decisions and we know how that goes. So what's your definition then of governed autonomy? Sean Duca [00:04:37]: Yeah, sure. So even if you break it down, like the 2 words that most of the industry treat as opposites, you know, it deliberately fused together at the best of times. So autonomy on its own means a system that acts without a human in the loop. You know, decides and executes on its own. Governance effectively means every one of those actions is constrained by rules and the system cannot escape. So when you put the two together, governed autonomy means the machine acts by itself, but only inside a boundary it did not set and cannot cross itself. The distinction that it makes is more than a slogan. It's more one of those, the whole thesis we need to sort of focus on, which is autonomy is what most vendors are shipping. Sean Duca [00:05:16]: And I'll be brutally honest about that one. An AI capable enough to take a containment action with its own confidence score deciding when to pull the trigger, that's effectively saying the one that makes the decision is the one that's taking the action as well. And you go, hang on, that's not right. There should always be some sort of, you know, secondary approach to say like, okay, is this truly right? And the person that decides, you know, shouldn't necessarily be the one that takes that action. You know, think about in a bank, there's separation of duties. You could go into a bank teller and say, hey, I want to take out $1,000. They'll give you $1,000. If you want to take it to $20,000, it's going to be a second person that's going to come in to kind of authorize that transaction. Sean Duca [00:05:54]: So again, think about that as like a governed transaction that's actually taking place. The machines that could be capable of— a machine could be highly capable and highly confident and still be permitted to act because the permission is a separate decision made by a policy, not by the model itself. So with that notion with the bank, you'll see that those, there's boundaries and there's decisions around who can do what, who's allowed to do what as well. KB [00:06:18]: Okay. There's a couple of things in there. Number one, going back to human in the loop. Now, you know, everyone's talking about human on the loop, in the loop. Now people seem to be, they're off the loop thing. That's what I'm hearing recently of people saying, well, people are lazy. They're just going to say, yes, it's accepted, et cetera. But then also someone said to me, yeah, but KB. KB [00:06:37]: Does that human have the expertise? So going back to your bank example, is the person that above the first bank teller actually have the level of authority as well to be able to provide that person with a $20K? Those questions don't seem to be answered yet because it's like, well, we've just gotten rid of half of our workforce. So we're going to have to get, you know, young Joe in and he probably doesn't have the level of expertise and authority, but that's going to be the person in the loop that's going to be responsible for making this decision. Sean Duca [00:07:02]: So think about this way. So human in the loop is a required step inside that particular sort of cycle. So the machine cannot complete the action without a human acting first. That's the best definition of like human in the loop. It will propose something, a person approves it, then someone executes it, and then it executes. You know, removing the human and nothing happens. If you think about like humans on the trigger in that sort of that war sense, there's always going to be someone there that actually pulls the actual trigger. Human on the loop is a little bit different. Sean Duca [00:07:32]: This is where the human sits above the actual process. Yeah, the machine decides and acts on its own, and the human monitors and retains power, and it has the ability to intervene, pause, and override anything that actually takes place. Yeah, the actual system will complete by default. The human's role is effectively to catch what goes wrong, remove the human, and the machine will keep on running. But the thing that's actually different here, this is where I always tend to sort of Give people examples of this. Human in the loop is where pretty much the whole world is right now. Everyone's going to go off and start to roll out agents en masse. And when you do that, great, you're going to have thousands, potentially hundreds of thousands of agents that are coming back saying, hey, Carissa, you need to do this. Sean Duca [00:08:13]: Hey, Chris, you need to actually do this. Hey, Chris, you need to do this. And it's going to keep on going to a point where our cognitive load will never actually be able to catch up. And you sit there and go like, how do we actually start to scale and do something different? And I think about it in the world of security operations. You know, we know that the threat is going to keep on growing. We also know that a threat actor can also turn around and overwhelm us with different types of events and attacks and incidents that have to be processed. So that whole human in the loop, all you're really going to be doing is exacerbating a problem because you've got more agents that are trying to deal with these challenges. Human on the loop, you're actually deciding and you're putting rules around what should actually happen. Sean Duca [00:08:52]: You know, how you could do this. So with that human-in-the-loop process, you know, if every action needs an approval, right, those 3,000 alerts a day that you're actually dealing with, we're automatically going to start rubber stamping. Yeah, let's just let all that through. Let's just bulk approve all of those pieces. But in a world where you've got a human on the loop, the human has got time to actually see what are the actions, be able to intervene before any harm lands. That's 2 o'clock in the morning, if there's a particular threat that actually comes out, you know, we're going to be able to see what's actually happening in real time based on certain types of criteria. So when is an agent allowed to act? And that could be on a, hey, we've seen 20 events pop up on a workstation. Let's just simply allow autonomy to take place. Sean Duca [00:09:36]: But if something was to happen and there was an event that took place on one of my domain controllers, that's when I want a human to be involved. That's when I'm going to escalate it straight away and say, Human, you actually need to do something at this point right here. KB [00:09:48]: This is where it gets interesting because people are saying to me, KB, let's focus on human in the loop. People's getting lazier. Yep. Except because like you said, it's creating more cognitive load because there's so many things that a human needs to make the final decision. Getting back to the human on the loop aspect side of things, what I'm hearing, and maybe you can, you'll probably know more about this than me, is apparently you can actually timebox these agents that every quarter, they will, they'll check back and say, hey, Sean, these are my credentials. This is this, this is my privileged account management. That way they're not going rogue, doing whatever they want, making decisions that don't make sense. Those sort of things are a way in which, yes, of course, aligned to the policies, for example, but there's that constant check to make sure, oh, our agent mini Sean is actually doing things and these crazy credentials that we never authorized in the first place. KB [00:10:38]: with the potential to do something else. Would you then agree that's what people are trying to do now to make sure they're governing these agents appropriately? Sean Duca [00:10:46]: Yeah, absolutely. And I think the governance piece is not to basically let an agent sort of run wild and sort of go off and do stuff. You're setting clear boundaries and policies around what they're allowed to do. You know, if you think about the world of what an agent actually is great at, and let's just say an agent or AI in particular, an AI effectively works on a Notion of being probabilistic. Probabilistic is a good example of that. I could ask ChatGPT a question and I could ask it 5 times and I will get 5 different answers. That's probabilistic. In a world where you've got security operations, you want to be deterministic. Sean Duca [00:11:19]: You want to be able to have the same outcome every single time. If it looks and smells like this, this is what you should actually be doing. So it's a way of trying to wrap the probabilistic nature of AI and the beautiful part about AI and its reasoning capability. In a very deterministic policy around what it can and can't actually do. That human judgment when it comes into like, you know, do people actually have the expertise here? That's the point. So we want to make sure that an agent is able to actually go off and do a lot of the work. And then when something actually comes up based on the criticality of an asset, based on the type of threat, based on the way that, you know, we are very confident about something, but we may not have permission to act. Right. Sean Duca [00:11:58]: That should actually go straight to a human. And that's what we actually want to make sure that a human is always intervening at that key point. KB [00:12:04]: Okay. So I'm then curious to know if a model, for example, is 95% confident, but the wrong 5%, for example, could take down a domain controller that you mentioned before at 3 AM, should confidence ever be enough to then trigger the action? Sean Duca [00:12:21]: No, they should always be separate. So you should always have 2 independent gates to effectively go through. So we could be extremely confident. It doesn't necessarily mean it gives us the permission to act on that particular host. You should never actually have that ability. And that's the key part. So that was the thesis of kind of where I sort of landed when, you know, when I started to think about how do you build an autonomous SOC? I literally started to visualize what would this actually look like? And, you know, walking into a room, I don't have a room that has all these screens. I don't have agents that I can actually look over their shoulders and see like what they're actually doing. Sean Duca [00:12:55]: So you start to think about how does this world act? How would you allow an agent to run around and deal with all these challenges? And the most obvious thing was sort of those dual independent gates, confidence and permission. And confidence is a world that we've typically always focused on, which is always, can I see what the actual threat is? Yes, it looks like this. This is what it actually does, but it doesn't actually mean that it has the ability to actually go through and approve that. And I think that's where it started to really, you know, shine a light, I guess, on what the actual problem was that you needed to have these 2 independent gates because it's not necessarily, it was always missing. It was fundamentally needed in a world of AI and autonomy. KB [00:13:34]: So, Sean, do you think people get the whole 2 independent gates? Do you think people understand that fundamentally? Sean Duca [00:13:40]: No. So, so the more I kept on looking at how do we actually do this, I didn't see anything that was actually on the market at the time to try and solve this. And that's where we really started to focus on this is what we had to do. Everyone is really focused on trying to speed up the SOC. You know, everyone's trying to work out how do we actually get faster AI to effectively help do the triage, try and work out what are the threats and challenges that are actually out there and speed up the human. And I thought, let's actually flip that around. You know, if you truly want to create an autonomous SOC, how would you do that? How would you build it from the ground up? And it's kind of akin to, if you're going to build a self-driving car, do you really need to have the steering wheel and the seat actually there? Sure. Maybe for an aesthetic thing, maybe for the whole key piece of that, you still want the human on the loop. Sean Duca [00:14:23]: You know, and a human on the loop on a full self-driving car is where I could sit down in the driver's seat and not actually hold onto the steering wheel, but as and when needed and instructed by the system, it's going to say, hey, you need to grab hold of the steering wheel right now. And that's actually what happens in a full self-driving car. And that's kind of the exact same notion of what we've built with the, our own sort of autonomous SOC. Where we're looking at 2 gates of confidence and permission. Because you could always be 99% sure. A model could even tell you that it's extremely confident. You know, and models can be very confident in the way that they actually can respond. They could still be lying to you as well. Sean Duca [00:14:59]: But again, do they have the permission to act? And when you think about the type of events that are coming through, I don't have business context of your ERP server. I don't have business context on your domain controller, your payment gateway, your root CA, whatever it may be. But that's the point that a human should be involved at that point because an agent sure as hell will not have the context around, is it, you know, a root CA or is it just a standard workstation there? So you need to have those 2 gates. KB [00:15:27]: So looking at the permission side of the gate, as we know with humans, when there's an incident, there's always somebody goes, why did you do that, Sean? When you're doing like a postmortem about an incident and a breach and you're going through that, why did that team do that? So obviously there's still going to be The machine acted like this, we allowed it, we thought it was best case. So I don't think that problem goes away because we've been dealing with this problem for years on end around, well, why did that team make that decision? We didn't agree with it, but they've done it now. So do you think that the same problem is kind of there, but just with the machine? Sean Duca [00:16:00]: Look, the same problem could always be there, but again, it comes down to it. Like, am I allowed to act in that particular instance? You know, the system can treat, is confident enough the actual thing. And I just sit there and go, no, because you can't actually be, you know, very confident, but still be not allowed to act. I could be very sure that something's going to happen. It doesn't necessarily mean it gives me the right to act on something. And that's where I think we have to be looking at a number of different factors. Think about the whole asset class that's actually there. Do you really want to allow an agent to act on your most critical and sensitive crown jewels to an organization? Probably not. Sean Duca [00:16:35]: Yeah, that's where you still want a human to be involved at that point. You know, if something was to happen on a domain controller, it could cause a business outage. And that's where kind of one of the areas that I started to think about from the beginning was there was this notion that I came up with called the autonomy control paradox. You know, there's the cost of hesitation, react too slowly and it potentially causes a breach. Then the other side of it is, you know, the cost of an unchecked action, you know, let AI act unchecked. And it potentially causes a business outage. And that's where it's that whole notion of, I could be confident, but if I was to be allowed to act on a domain controller, I could actually bring down the server. So confidence is not permission, you know, and being sure isn't enough. Sean Duca [00:17:14]: Before the platform can actually act, those 2 separate checks both have to say yes. Is it confident and is it allowed to act? One green light is not a go. And that's the fundamental key piece around autonomy-control paradox. Because we're constantly going to keep on seeing more and more events coming through. We have to be fundamentally sure what is it. And if we're sure about something, great. Are we allowed to act? And that's the key piece. Most security technology, and I'll say most because the plethora of technology that's actually out there always fail in the way of they'll fail open, meaning that if the system actually doesn't know how to deal with something or the system sort of fails or whatever it is, It's just simply going to fail and it's going to allow things to pass right through. Sean Duca [00:17:56]: Think about your typical network security devices that are actually out there. Yeah, they typically will fail open, let the traffic through uninspected. We've taken the approach of saying, why don't we fail closed? It will always degrade to a human. So every single time that something actually pops up and we don't actually know, or something actually, we hit a brick wall of some sort, it's always going to go to a human. It doesn't mean the system stops. It just means that there's always going to be human oversight to every single thing that we do. And I believe that's the thing that's actually been lacking. You know, we've always tried to say like, look at the river of different threats and challenges that are out there. Sean Duca [00:18:28]: And we try and work at it, keep up with the deluge of threats and challenges, but we need to be able to say, okay, the things that can actually be done with autonomy, let it happen. That could be your average workstation. It could be the laptop that probably you and I are both using right now. But the things that are most critical to a business, they're the ones that we want to make sure there is always human oversight and humans actually do some of the work as well. KB [00:18:49]: We'll come back to that after a quick word from our sponsor. Engineering teams are increasingly being pulled into the compliance conversation, and it's not where you want your sprint hours going. Vanta automates the painful bits of ISO 27001, SOC 2, and GDPR, so your team can spend less time on audits and more time building things that matter. Visit vanta.com/kbts. That's vanta.com/KBKast to learn more. So can I ask more of a rudimentary question? If that's the case, then do people come back to you and say, Sean, I'm tired. I'm backed up because all of the system needs my human brainpower to make a decision. And there's 3,000 of these things that I need to deal with and I can't deal with it because I'm going on holidays next week. KB [00:19:39]: So what do we do? We have to divert to the other guy who's junior and doesn't get it. Sean Duca [00:19:42]: Yeah. KB [00:19:43]: And then I'm fatigued now, Sean, because I'm getting so many of these alerts and these things that I have to approve or not approve. Are we back in that whole alert fatigue sort of days again with trying to use technology to reduce the noise and the friction? But then what I'm hearing from people are saying, there's all these other things that people have to intervene on anyway. So it's causing more brainpower for people to actually start to make those decisions because they've got to think through it. They can't just willy-nilly make it like, yes, I'm going to approve that sort of thing. Yeah. Sean Duca [00:20:12]: Yeah, good question. And think about it this way. So a human authority, the reality is that they're still human. So at the end, you know, your analyst is going to be fatigued at 2 o'clock in the morning trying to approve, should we be containing this particular issue or this particular action over here? Why is the policy— we are cognizant about the fact that a policy could be set and that's going to force someone to be sort of fatigued and effectively maybe even rubber stamp some of the challenges that are there. Humans do provide that real oversight. And the key piece there is it's more around making sure that your policy is correct. It's like, how do we use AI to help us and not necessarily just speed up the work for us to have a greater cognitive load? It's more around how do we get them to actually take some of the work off our plate? If you think about what's the largest amount of probably systems in most organizations, you're probably going to have, sure, the world has changed and every organization is going to be different. But you think about it, there's more workstations than there probably are servers. Sean Duca [00:21:09]: You know, workstations in my mind is where autonomy should actually run rife. Like we should allow agents in a very deterministic way to actually go off and execute tasks on those workstations. But for things that are very critical to a business that could cause an outage, think about it. My laptop could actually die. It doesn't necessarily mean the whole business dies, but my domain controller falls over, that's actually going to cause an issue where no one can really do their work. You know, if it's the payment gateway, if it's our root CA or something like that, that's going to cause an issue. So again, it's making sure that you've got policies like defined, and it's only the things that should actually be escalated to a human that should actually go there. Right now, all humans are dealing with all issues that are actually popping up in a SOC. Sean Duca [00:21:53]: Even with this whole world of the autonomous SOC capability that's out in the market, all we're really doing is speeding up the amount of threats and challenges that people have to deal with. And there has to be a point in time when you go, let's take some of that noise away and allow autonomy to actually happen. And I'm not trying to say just blind oversight to, I'm not going to look at that. It's just going to happen on its own. You've got full control of it. And I question the people right now to say, do you even have control or have you even truly turned on autonomy in your autonomous SOC solution? That's the biggest thing. Because in speaking to a lot of people, they are fatigued. They were overwhelmed with the number of different threats and challenges that are actually out there. Sean Duca [00:22:32]: But the key part is we're trying to make sure that we're moving the human up the layer. And it doesn't mean that we're sort of moving them away from the problem or relocating them into a place that looks like it's theater, so to speak. It's more around making sure that they're actually focusing their energy and efforts on the places that actually matter the most. KB [00:22:49]: Quick question, going back to the 3 AM example, just say you're the person that's going to deal with the issue. Fine. We can't get ahold of Sean. You're in Timbuktu on a holiday. There's no phone reception. Then is it part of the policy that it escalates to someone else? But then also you're losing time then, because time, as we know, is critical to something going down or staying up, for example. So what then happens in that scenario? If you are the main person that needs to be informed, you're away, something happens, then it's gotta divert to someone else. What does that then look like though? Sean Duca [00:23:24]: So in a world where you've got human in the loop, this is where every alert is gonna be a fresh demand for judgment on, you know, a human. This is where someone's always gonna be, that whole compound of the volume and the threats and challenges, especially at 3 o'clock in the morning, is gonna be probably the hardest. Yeah, this is exactly where the analyst is potentially gonna be probably at its weakest as well. So there's no way to make this human reliable because the failure mode is gonna be the volume. Yeah, this is where the human in the loop, in my mind, looks like theater. When you apply policy at that point, this is where it's going to start to help the human at 3 o'clock in the morning. Because again, you're trying to balance that whole, the cost of hesitation, which is reacting too slowly to the breach, and the cost of unchecking the action, which is effectively, you know, if I let AI just do whatever it needs to do, it's going to cause that business outage. So the key part is actually making sure that there is no cost of hesitation. Sean Duca [00:24:15]: You will actually want to be able to make an informed decision at all times. And you should be, you know, it shouldn't necessarily be a case of, well, there's a lot of threats that are coming in at 3 o'clock in the morning. What do I do now? I'm paralyzed. I'm sort of the turtle on its back, so to speak. You know, you want to be able to have the ability where you say these threats and challenges should be dealt with according to confidence and permission. These other ones, maybe there is no permission, and that's where it goes to that human in the loop. So we're not trying to say that we're taking the human away from it. But we're trying to say, let's actually move the human up the stack, so to speak. Sean Duca [00:24:48]: You know, get them in a place where they're not making every single decision. They're making the few decisions, but they're making the right decisions for a business. KB [00:24:54]: What happens if the human was supposed to intervene, but it is 3 AM in the morning, can't get ahold of the person, they're asleep, they're on holiday, whatever's happened. Does the machine then go, okay, I haven't heard from Sean in X amount of time. I need to do something else. If then what? Or does it start to go back to the confidence side of things to say, I'm 95% confident, no one's responded or intervened, therefore it's okay to do the next action? Sean Duca [00:25:21]: It won't act. So period. So if you think about it in that notion, which is no different to the way that we would actually be working today. So if someone basically received a request and say, hey, you need to intervene in, let's just call it the current sort of world that we live in, nothing's going to happen. In a world where you've got confidence and permission. Doesn't make a difference if it's 3 o'clock in the morning. Doesn't make a difference if the person is not available. It still will not act. Sean Duca [00:25:43]: Like, that's the reality of it. And that's the key point there. It's more around places that it's allowed to act, it will act. Places that it's not, it won't act ever. And it's not even a case of, oh, but I've waited 25 minutes. It's simply just going to be routing that particular issue to a human and it sits in the human's queue. But the key part around that one is it's not going to be overwhelming the humans. With this abundance of different threats and challenges that are actually popping up. Sean Duca [00:26:09]: Because if the system is acting correctly based on policy, only the things that really matter are going to be moving up to the human. So we're trying to take away that whole deluge of threats that are actually landing on the plate of a human analyst, more so than what we're probably seeing right now with a lot of different solutions that are speeding up a SOC and speeding up the triage and giving them more things to think about. We're trying to take that problem away. KB [00:26:33]: And I guess from what you're saying, from what I'm hearing, it's a moot point because even if humans couldn't intervene at that time in the morning, but then even if going from birding cobbers back in the day, people probably wouldn't be intervening that quick anyway. They've got to get the guy up. They've got to think about what we're doing anyway. So then my next question goes, the whole cost of hesitation then. So do you think we're going to see a world in the next 12 months, Shaun, saying, well, Shaun didn't intervene with the thing because he was asleep. which is fair enough, but now we've just lost a couple of hours and now we've got a massive problem on our hands. So do you think that's gonna be the next issue that opens up? There's gonna be more pressure down from executives and boards to say, we should have intervened, the machine should have done something, 'cause now we are dealing with a massive outage and we've got media people all over us. Sean Duca [00:27:17]: Look, I think there's gonna be a couple of different things. And look, without sounding like the alarmist, I think we're gonna see more threats and challenges pop up than we've ever seen. And the reason why I say that is just look at the last 25 years of threats and challenges, and you know that there's an exponential growth that's there. So let's just put that out there, is that's just normal. When we've already started to see AI-driven attacks that are effectively working autonomously as well, and you go, that will end up being a constant. Like we will end up seeing more and more of this popping up. And then if you apply all of that to the existing way, this is the way that we've always done security, and we're always relying on the human to kind of go, Good, bad, indifferent, good, bad, indifferent. Let's actually sort of move on from there. Sean Duca [00:27:58]: That is only gonna overwhelm people. Where I think we need to get to is an understanding that the threat is gonna be a constant, that we need to be using AI because if it's a machine speed-based attack, we need to be using machines to actually help us deal with that. That does not mean that the human disappears. It just means that the human is elevated into a position that it should actually act on the things that actually matter and machines act on the places that it's allowed to act. And that's the fundamental distinction. You know, making sure that the machines are working in the right place and the humans are dealing with the right stuff. Because if you think about it today in a SOC, we're dealing with 100% of the problems. You know, why don't we get into a world where maybe 80% of all the challenges are dealt with by a machine and the 20%, the things that matter the most are the ones that a human actually deals with. Sean Duca [00:28:47]: That will actually in turn reduce probably the amount of threats and challenges that they have to deal with on a day-to-day basis. And the machines are allowed to act in the right place. And that gives you the scale to exponentially grow as well. KB [00:28:59]: Okay. I want to talk to you now about autonomous SOC. We've sort of touched on it, but as you know, a lot of companies pushing that this is their capability. This is what they do. What is your gripe perhaps with autonomous SOC, would you say? Sean Duca [00:29:13]: The biggest challenge that I probably have seen is having looked and worked in this space for a number of years, people are buying technology and I don't think there's actually trust in the system. And it's not necessarily the trust in the particular vendor or, or something like that. Trust of what actually happens. No one wants to wear it if something goes wrong. And that's why some people have that hesitation of actually fully turning on this capability. If you think about it right now, there's probably about 60 different vendors out on the market that are sort of Touting their wares as an autonomous SOC, you know, and is there trust? And how do you actually get trust in a system? You know, typically with an AI model, like the models are great and the models are getting even better as well. And you go like, what's actually the problem there? And you think about it, AI is actually that black box. It's opaque. Sean Duca [00:29:55]: You can't see inside it. You don't understand what's actually going on there. My challenge is more around the fact that people are buying technology and not really extracting the full value that's actually there. And the reason for that is because of the fact that there's no trust in the system. As far as I'm concerned, trust is built, but it's a way of how do you actually prove that to someone? How do you actually show what's going on? So the approach of what I always thought about was how do you actually earn trust? And that's where I thought, well, it takes time. So what does time look like? So the way that we have solved the problem and the way that we're sort of solving the problem is we work in a notion of what we call shadow mode. So for the first 90 days, we're actually looking and seeing the different types of threats and challenges that are popping up. And we're just learning and understanding what's actually going on. Sean Duca [00:30:38]: A human is actually covering everything at that point. But for the first 45 days we go through, so the first 45 days of the 90 shadow mode days is where we're actually looking and seeing. The second part of the 45 days is where we're actually giving recommendations. Again, human is still there, but we're giving recommendations around, this is actually what we would have done in an autonomous world. And again, Where we would act is where we're allowed to act. So we're going to give recommendations at that point. So we're going through and earning our trust and showing the customer at the end of the day, this is actually what we've seen. This is how we would do this. Sean Duca [00:31:14]: We're focusing on a world where we will always be at a certain level of confidence that we can actually start to even show the customer and say, based on all of the events that we saw, this is where autonomy would've kicked in. And this is a bar we set for ourselves. And we have to keep clearing. Every determination our agent makes, our own threat hunters grade. Did we get this right? Would we have made the same call? And we don't hit that number once and we walk away. We have to hold better than 85% agreement continuously. Otherwise, we'll always degrade to a human. The day we fall below that, the system will always revert straight back to 100% humans always looking over every single event that we keep on seeing. Sean Duca [00:31:52]: It keeps on getting looked at. time and time again. That's the deal we've made from day one. That's the whole point of ShadowMOBE. Autonomy isn't a switch you flip. It's earned and it stays earned as long as you keep on clearing that, that thing on file. The day we don't, the humans get everything back. And a lot of organizations we've built for don't even have a threat hunter of their own. Sean Duca [00:32:11]: So the burden's on us to prove it, not on them. KB [00:32:13]: So going back to the autonomous SOC, if people are buying this capability, what you're saying is Companies need to use it fully. So don't sort of put your foot in both camps, or we're too scared to use it fully, or we're not using it, it's a waste of money. You might as well do it fully, do it properly, or don't do it at all. Or don't take the leap just yet. Do the reconnaissance in the space, ask the vendors the questions or whatever people need to do to make sure that they are confident. Or else, would you say, as a result of trying to not fully invest in something, you're muddying the waters then a lot? Sean Duca [00:32:44]: Yeah. KB [00:32:44]: With your team, with the technology, with the cost it takes to invest in this sort of stuff? Sean Duca [00:32:50]: Look, absolutely. And you think about it, I think the running stat is any type of enterprise technology that's actually ever been released from, you know, from the beginning of time, people extract about 15% value from it. Challenge yourself. Anyone who's actually using a capability right now, how much value do you actually extract from it? And if you think about what probably the autonomous SOC space, what people are actually doing, Sure, it has the ability to do a lot of different things, but what people are actually using the autonomous SOC capability for is doing things like summarization, adding the context for a human to actually start to understand what the threat looks like a little bit more, maybe doing some of the triage. That's probably about it. But there could be a raft of different other capabilities that are built into it, but people don't have the confidence for it to go off and act. They're still solely relying on the human that's there. Secondly, the solutions that are actually built are all about augmenting and complementing the work that a human actually does. Sean Duca [00:33:45]: And that's fine, but that's also akin to sprinkling AI on top of humans and expecting an amazing outcome. As we've seen in any type of AI capability that's actually deployed out there, even put security aside for a second. If you use that same notion of sprinkling AI on a broken workflow or process, You will get marginal gains at best. You know, and we've already seen that. This is where people are seeing AI productivity gains are not really sort of keeping up with what everyone's sort of saying that what AI is meant to be delivering. I actually said, there we go, flip around on its head. Think about a world of like, how do we truly build an AI-native SOC? What would that look like? Humans will always be there, but let's actually take them out of the trenches and actually get them to focus on the things that matter truly in a SOC. But allow AI to intervene and act where it is allowed to act. Sean Duca [00:34:33]: And I think that's the very big fundamental piece. It's the permission piece that's always been lacking. You know, how do we give permission, the right type of permission on the right assets for it to actually act and not just simply give it carte blanche access of like, do whatever you want to do. Because that's where it's focused on, where people have conflated confidence and permission as one thing. KB [00:34:52]: Do you think customers' expectations are exorbitant? And what I mean by that is apparently ISACA has repulsed poll report that 22% met their expectations, which is quite low considering how much adoption that there is and everyone's talking about it. So do you think that to your point, people just think, I'm just going to flick on the switch, it's all there and I don't need to worry about it anymore. And then they get the bill and, you know, you've heard about the whole token problem, tokenization and all that sort of stuff. So do you think that perhaps people are just maybe a little bit immature in this space in their thinking? It's not their fault, but they're not reaping the benefits. They're not fully immersing themselves. They don't know, perhaps they don't know what to expect. And then they're not, your point, marginal gains. They're thinking, hey, we're going to get 70% uptick and we're only getting 15%. KB [00:35:41]: So how is this going to sort of play out now moving forward, given your 25 years in the space, your pedigree? What do you think's happening now? Sean Duca [00:35:50]: Look, I think that autonomy is a statement. You climb to based on evidence. It's not a toggle switch in any solution that's actually out there. Autonomy is the system is allowed to execute an action. You know, it's allowed to sort of go off and initiate something inside a permission boundary of particular policy that's been defined. When you apply that definition to most of where the market is, it will fail. You know, a human-approved action is always going to be augmented, not autonomous. That shadow mode recommendation that I talked about is a recommendation. Sean Duca [00:36:22]: It's not autonomy. And as an action taken at that sort of governance layer is when something is, when the system is actually available, it's going to actually sort of go off and act. When it's not available, it's going to go to a human. The key thing there is speed is a symptom of maturity, never the definition of it. So when you think about the point that you said, that those 20 sort of percent things like that, we're seeing this across the board with anything AI, not even just in the security realm. People just thinking it's the auto magic, turn it on and it's going to do stuff and you go, great. But are you going to allow it to actually run and act on its own in the middle of the night on anything inside your own environment? Probably not. And the reason for that is people don't want to wear it if something actually happens that it does. Sean Duca [00:37:02]: So if it does something wrong in the middle of the night, people are not going to want to wear that. So that's the key piece. And I think we're seeing this time and time again. I think about some of the type of work over the years that I've done around AI as well. People just simply say, oh, it's the auto magic switch. You turn it on and it's going to do stuff. Great. But what is it actually going to do for you? I think that's the big part. Sean Duca [00:37:22]: You know, you look at the whole concept of the forward deployed engineer. Why do we need forward deployed engineers all of a sudden for this AI world? And it's because of the fact that we're trying to make sure that we're redefining the way that AI could be used inside an organization. We're thinking about a way of how do you actually get capability into people's hands so they can actually maximize the investment they've made into it. That's why the whole concept of the forward deployed engineer is so successful. Palantir were the ones that really sort of pioneered this over 20 years ago. And now all of a sudden, every single vendor is running around trying to have forward deployed engineers. It's because they're trying to get the technology to work properly and not just simply turn on the auto magic switch and hope for the best. KB [00:38:03]: Okay. So here's a question for you. Do you think Whether it's human in the loop, on the loop, whatever, someone intervening. Do you think that by default where we are in this space, we're going to start to manufacture people that are just going to say, yeah, it's okay. And then something happens because people don't even read things anymore. They're always like antsy about, give me the summarized version. People can't even watch TikTok videos without someone saying, I want the summarized version in the comments. Like people are naturally getting lazier. KB [00:38:33]: So what really are the chances then of something that does need to be looked at in great fidelity to make an informed decision of someone saying, couldn't be bothered, I want to go have, you know, beers with my buddies, looks all right, confidence is 95. What about that from the human side of it then? Sean Duca [00:38:52]: So I think right now the cognitive load of everyone is probably, you know, at its peak. It's probably sort of burning people out as well. Yeah. Think about it in the days of old, like I'll say days of old for me of being sort of in an office every single day between meetings, you would actually stop and talk to people. You would always probably have those moments that you would sit there and think like, oh, okay. Like, you know, you'd start to strategize. You'd think you'd probably start to collaborate with your colleagues and start to kick around ideas. And you had that time to sort of like, let's call it decompress. Sean Duca [00:39:23]: And you started to think. Right now, every single time between meetings, people are always sort of jumping online. They're using AI to help them with nearly every single task. And look, that's fine. But you now think about it, every little waking second that you have, every gap in the day that you have, you're actually jumping onto AI to try and find out an answer. The more we keep on doing that, the more we're overwhelming ourselves with information. That's where we have to get to a place that we're saying, okay, how do we hand off the right type of work? To AI, to autonomy, to actually be allowed to do something in the right places. So that way people's cognitive load can actually reduce and focus on the things that matter the most. Sean Duca [00:40:02]: That's the whole 80/20 piece that I talked about earlier. You know, let the agents go off and do 80% of the work. So we focus on the 20%, the things that actually matter, the things that do require you to read a little bit more. You know, people turn around and say like, you know, is AI making us dumb? There's been many articles that are coming out around that one. I think sometimes people are skim reading a lot more because they're just overwhelmed with the amount of information that they're just constantly looking at right now. And not necessarily saying that they're dumb. I don't believe that at all. But I just think that we're slowing our brain down with the amount of information that's actually being presented to us all the time. Sean Duca [00:40:35]: And we have to fix that. Yeah, that's where we have to do that right handoff onto where is autonomy allowed to act and operate? And then what are the things that the humans should be focusing on and actually start to relevel what that looks like as well? Because right now all we're doing is we're playing catch and we're playing catch for every single ball that's being thrown at us. Yeah, it's time that we actually change that. KB [00:40:56]: Okay, Sean, closing comments, final thoughts, hit me. Sean Duca [00:40:59]: The key missing piece has always been that authority model. And this is the part I want to leave people with. This is not about building better AI. And I say that deliberately because I don't think better AI fixes this. Here's the reason why. If the authority model lives inside the reasoning system, then every time you make the reasoning better, you've just made it better at reasoning its way around the constraint. So the authority has to sit outside the thing doing the reasoning. It has to be the layer the reasoning can't talk its way past. Sean Duca [00:41:28]: That's the piece this industry has been missing. Not a smarter model, a separate authority. And that's what we've been building. KB [00:41:37]: That was Sean Duca, everybody. What's staying with me is super simple. Confidence and permission are 2 different gates, and most of the market is only building one. So if you're a CISO or you're sitting on a board, here's the question to take away before you switch on anything autonomous. Ask the vendor who actually holds the authority to act and make sure that authority sits somewhere the AI I can't reason its way past. VO: I read every reply. If you've got some thoughts on this one, send me a message on LinkedIn. KBKast, Cyber For The C-Suite.