Episode Transcript
Andre Durand [00:00:00]:
if permission escalation were sent through email and approval was just clicking on a link in email, but, you know, for example, a cowork agent has access to email and can just go to the email and click it, it can approve its own permission. It can initiate the permission and approve the permission. And that's a problem.
VO:
From KBI.Media, I'm Karissa Breen, and this is KBKast.
KB [00:00:30]:
Andre Durand, founder and CEO of Ping Identity is back on the show. He spent decades working out who gets let in and what they're allowed to do. We talk about who carries the liability when an agent gets it wrong, why logging in stops being enough once agents act at machine speed, and who guards the guardrail when an agent can approve its own permissions.
VO:
If you find these conversations useful, hit follow. It's the single best way to make sure the next one lands right into your feed, and it helps other execs find the show.
KB [00:01:02]:
Alrighty, let's get into it. Andre, welcome back. Companies are obsessing over what AI agents can do. So what I'm curious to understand is, are they avoiding the main question around who gave the agent authority and how far can that authority travel? So perhaps to extend on that, if agents can create subagents and delegate work, for example, should everyone have a named human sponsor who remains personally accountable for the entire chain? Or walk me through this. I know that's a very long-winded question.
Andre Durand [00:01:44]:
Well, they always say follow the money. In this particular case, maybe follow the liability. At some point, we can't be in a situation where an agent does something and nobody's responsible. Company's not responsible for it. If the company's responsible for it, then who at the company's responsible for it? At some point, holding agents accountable doesn't necessarily mean anything. There's no, there's no liability in that construct. They would simply possibly apologize. I see that all the time.
Andre Durand [00:02:15]:
Sorry, I lied. Sorry, I was wrong. Sorry, I made a mistake. So yeah, so I think in this particular case, it's one thing to have agents advise us and humans decide and humans take action. And another thing to allow an autonomous machine or an autonomous thinking entity to take action and then to mistakenly take action. So ultimately, yeah, we will have to have custodianship and accountability that is line of sight to individuals.
KB [00:02:44]:
And this is interesting because I've been interviewing a fair few CEOs like yourself for the last few weeks, and this whole accountability question seems to be very top of mind. What are your thoughts on, in terms of people owning that accountability? Will people be resistant to own that simply because they might be like, well, I need to understand all the details. What are those agents doing? I need to understand what are the guardrails that are in place, and are we governing them effectively? So, do you think that How does that sort of look in your eyes given what you are doing each day and the people that you are speaking to?
Andre Durand [00:03:17]:
I mean, yes, yes, and yes. The accountability without authority or accountability without control doesn't make any sense. I can't hold you accountable for a set of actions that I can't monitor and/or have some say in what you actually do, which is why the identity controls and the governance model, both from an identity, from an agentic lifecycle point of view, as well as from an agentic runtime point of view, we will have to have answers for that. So I think, how do we know what an agent does at every moment across every surface? I mean, ultimately we're going to have to have answers for that. So from a human trust point of view, we've talked about zero trust as a holy grail for quite some time, better part of a decade. But truth be told, if we're real about what companies have been willing to invest to actually ensure zero trust for humans, I would argue that we've kind of chosen with our wallets, the pearl wasn't necessarily worth the dive, meaning the expense both in tooling and in complexity to get to zero trust with our human actors. We've been moving towards it, but there's a number of things that we have not invested in. Because for the most part, humans have consequence.
Andre Durand [00:04:40]:
They do self-manage their actions. And for the most part, they're trustworthy. And so we've operated within a trust framework that extends just enough trust for the amount of fraud and abuse we see in the system. Not to say we don't see fraud and abuse, we do, but I'd say the level of investment. With agentic, we're going to have to accelerate our trust model to zero trust. And what that ultimately boils down to is not trusting what an agent will do or only trusting what we allow the agent to take as its next action. Like, I trust you as much as your next action. If an agent did something appropriately 5 times in a row, I wouldn't necessarily trust that it would do the exact same thing on the 6th time.
Andre Durand [00:05:29]:
The probabilistic nature of AI, the undeterministic nature of it says we can't be guaranteed what it's going to do on the 6th time. As a result of that, we must have a runtime control and governance model for agentic that is significantly more stringent, as you call them, guardrails. than we've had for humans.
KB [00:05:52]:
Okay, this is interesting. So there's a couple of things in there that you mentioned. I just wanna go back just for a moment. You said there's areas that we just haven't as an industry invested in. What are those areas would you say, Andre?
Andre Durand [00:06:01]:
The big one is authorization. So we've said for some time, identity is the new perimeter, maybe a decade. So it's not new anymore, but we've been using the term for a while. And we invented that term about the time that cloud and mobile hit. And enterprise resources started escaping their network. They started moving beyond the company's firewall into cloud, and the users started accessing corporate information on mobile phones. So cloud and mobile kind of broke the traditional perimeter security model, and who was in and who was out was determined by, did you have an account and did you authenticate? And location disappeared as a factor of gaining access.
Andre Durand [00:06:46]:
So, if you look at that world, identity is the new perimeter. What we really meant was that we had 2 control gates, 2 entry points into identity. One gate was essentially identity management. So, did you have an account? If you didn't have an account, you didn't exist. If you didn't exist, you couldn't interact. So, identity management was really an administration gate. And you were in or out depending on whether or not you had an account. The second gate that we created was a runtime gate, and we call that login or authentication.
Andre Durand [00:07:20]:
And if you successfully logged in and you authenticated, you had a session. If you had a session, you were good to go. So you actually had to have both. You had to have an account, you had to authenticate to the account, and you were in. When we talk about agentic, knowing that an agent exists and authenticating an agent is no longer sufficient to govern and control and guarantee agentic action when it's in our environments. Would we feel comfortable, for example, saying the agent authenticated 2 hours ago? Well, what's it been doing in the last 2 hours? Hold on for a second. Let me find out what it's done in the last 2 hours. If that's the security model that we create for agentic, we're all in trouble.
Andre Durand [00:08:04]:
So the trust model where we gave kind of broad permissions to humans and broad swaths of time for humans to interact in our systems, we need to tighten the guardrails around agentic. So the new gate for agentic is authorization on the runtime. And what I mean by that is it's the decision an agent will say, I want to do X, Y, or Z. We will check our authorization policy. And if it adheres to our policy, we'll allow the agent to take the next action. So the new gate is authorization. It evolves from login and authentication was good enough for humans, no longer good enough for agentic. Authorization will become the new runtime gate.
KB [00:08:48]:
Okay. So, okay, this is interesting. So I want to get into going back to, do we trust what the agent's doing then on the 6th time? Now what was coming to my mind as you were talking there would be people now are trying to create efficiencies and getting the machines to do the work, all the stuff that, you know. So do you think that this creates fatigue for people to be like, well, every time I've got to keep checking, like I thought on the 5th or the 6th time that it's okay because all the other times it's been right. Does that then create, is that creating a rod for our back and sort of counterintuitive to what we're trying to achieve as an industry?
Andre Durand [00:09:24]:
It does, and we'll have to break through that model. So if we want to move at agentic speed, you can't insert humans in all of the approvals for obvious reasons. Now we have what's called human in the loop, which is agents can do a lot of work, automate a whole slew of things, but the final step in the action, the human needs to approve. So for example, I could tell an agent, I would like to move money from account A to account B. And the agent could prepare the summary of that movement, but not actually authorize the movement. The human has to authorize the movement. That would be an action of agent does some of work on behalf of the human, but the human authorizes the risky and final transaction, if you will. And we're going to see a lot of that. Anywhere humans are in the loop, to be clear, we will be slowing down to a certain extent
Andre Durand [00:10:16]:
the speed advantages of agentic, not entirely, but to some extent. And so trusting agentic at some point and moving at agentic speed, which is what we are all in the pursuit of, will require we create some trust models or governance frameworks that don't have human in the loop, where the goal is to get the humans out of the loop, yet still trust agentic. And it'll be a little bit like how auditors trust the numbers, the books. And I was an auditor way back when. We didn't foot every number. We did test sampling. If there were too many transactions to add them all up to make sure they added up to the number, we did a sampling of transactions and we extrapolated the sampling to extrapolate the error rate. And as long as the error rate was immaterial, we said, okay, it's accurate. We probably are going to end up at a similar scenario with agents where we will create frameworks where as long as the framework is running, the guardrails of the framework, if we want, if we want to trust agent output, we will have to create new mechanisms where humans aren't the slowest arbiter of that speed.
Andre Durand [00:11:25]:
And there's a number of things that we are currently doing to do exactly that. So for example, in development, say one of the AI coding tools, we develop, we use different models and different instances of AI to review each other's code. So we didn't code it, an agent coded it, but we use another agent to review the quality of the code. And we might use even a third agent to approve the code or the commit. So this is AI for AI. We want to move at AI speed. The AI is generating the activity or the action, and AI is then judging and/or approving that output.
Andre Durand [00:12:06]:
So we will end up with models like that, and we will have to trust our frameworks, and we will have to ensure that there is not collusion. We do know that AI models collude. We know that one model might not have permission and another does, and the one that doesn't will ask the one that does to go do something for it. And so that would be an example of collusion to get around permissions in the pursuit of an outcome. And so we will have to design systems where segregation of duties and the inability to collude gives us some confidence that what the AI is doing is appropriate.
KB [00:12:43]:
Okay, I want to stay with human in the loop. So recently, and I want to get your thoughts on this, I've been interviewing folks and they're like, I'm over the human in the loop. And I was like, why is that? People say to me, well, couple of things. Number one, like to your point, if it's getting to the end of the line, the human then approves it, what someone said to me was, well, people are starting being conditioned to just hit approve. Yep, it's all good. That's one scenario. The other scenario would be, is, does that person, are they the right person to actually approve that action or that outcome? What if they're, you know, someone that's filling in for someone else or they don't understand in full fidelity what that agent was doing. So therefore it's like, okay, it appears all right.
KB [00:13:24]:
I'm just going to then approve the action or the outcome. What are your thoughts on that? And then someone else said they don't like the term human in the loop. They prefer human oversight. So now it's like everyone's saying human in the loop, and now I've got people saying, no, I don't like that, KB. So I want to get your thoughts here to help me make sense of it.
Andre Durand [00:13:41]:
I think conditioning humans to approve is a real thing, and you end up rubber stamping. And anytime we've seen humans being asked to repetitively do something, you do begin to condition them to just approve. And it's, say, one of the challenges, for example, in my space in multifactor authentication, push notifications can train people just to accept an MFA authentication when in fact it was an adversary that initiated that push. And so, so I do agree with the assessment that, that we need to be cautious about where human-in-the-loop approval occurs and to essentially not condition people to just hit approve. I do think that at some point, until we fundamentally address where does liability ultimately land for agentic action, that there likely is going to be some requirement for human custodians, people responsible to review the policy of what an agent can do. People responsible for the oversight of reviewing agentic actions that the system has flagged as potentially suspicious or in some way not in keeping with the intent of what we had, of what we wanted the agent to do. So I do think a more evolved view, scalable view of this is human oversight, agentic action at the speed of machines with human oversight of the governance model and the framework. I do think that's a more evolved view.
Andre Durand [00:15:14]:
It'll take us a little while to get there. Human in the loop will probably still be appropriate for certain transactions, and human oversight probably is the model that we're moving towards for scale.
KB [00:15:25]:
So do you think as well, just coming back to the liability, do you think if I'm approving something and I know that, hey, for example, my job's on the line if I don't get it right, and I'm just going approve, approve, approve, and I'm hungover and I'm watching, you know, Netflix in the background, I'm not really paying attention. Do you think that that'll change the mindset to be like, actually, I've really gotta look at these things because it could cause me to lose my job. Because at the end of the day, if people don't really have those repercussions, they may not be as invested. I mean, they should be, sure. The reality is people just aren't. So do you think that having that accountability, and I know it's still a gray area around, well, who owns what? Is it the vendor? Is it this? Is it that? Do you think that'll change perhaps?
Andre Durand [00:16:03]:
That's the entire point of accountability. If accountability meant no consequence, it wouldn't mean the same thing. There has to be consequence for action. And the accountability and consequence of humans not doing the right thing is what keeps society in line. And it's why we have regulation and laws that say there's consequence for misbehaving in society. So yes. Now the challenge is there's no consequence for AI misbehaving. There is consequence for humans misbehaving.
Andre Durand [00:16:34]:
So humans do self-govern themselves when they know something is wrong and they know they could be held accountable. And they understand that the accountability comes with consequence. And that's the reason why if you follow the money and follow the liability here, why for some period of time, humans who are accountable, accountable, meaning there is consequence for them not being diligent in the review of agentic action, is probably the closest proxy that we've got to making sure things don't come off the rails.
KB [00:17:06]:
And do you think we're getting there as an industry? You know, all these things are relatively new for people and they're still trying to work it out. And the reason I'm asking this is because let's look at a traditional breach. People never want to admit like, oh, it was us. People try to finger point and blame the other person. It was the vendor's fault. It was this third party that we've never even heard of half the time. So do you think that the pendulum will sort of land in the middle at some point, but it's just going to take a bit of time to your earlier sort of comments before.
Andre Durand [00:17:32]:
They always do. And all the liability shifts always get tested in the courts and different courts take different positions now on what the, where the liability is. Is there liability for an executive acting on behalf and under authority of a company that misbehaves? And what repercussions do society have against that executive? And there are lots of examples where misbehavior and consequence and the law cross and people are made an example of and everyone else takes notice of that. Absolutely. And when they see that happen to peers, they pay a lot more attention to their role in it and their responsibility. So yeah, no, I think all of this stuff, the pendulum finding the balance is an appropriate analogy, I think. And all of this stuff will have to play out. And look, the same thing has played out with autonomous cars, right?
Andre Durand [00:18:28]:
Person gets pulled over in a self-driving car for breaking a law and says, sorry, officer, I wasn't, you know, I didn't have my hands on the wheel. I didn't, I'm not the one that broke the traffic law. And the police officer says, well, you're sitting in the driver's seat of the car. The car is registered to you. And, and if you're in an accident, it goes on your record, not the car's record. So yes, you're responsible whether you had your hand on the wheel or not. Autonomous agents are somewhat analogous to that.
KB [00:19:00]:
Do you think that's a bit of a cop-out though, of people just saying, oh, it was the agent's fault. So technically it wasn't us. Because I know that people have used the Air Canada example a lot. Like, well, technically it was the agent. Yeah, but you're still responsible for it. It was your agent. Now, obviously it did something wrong. You're still responsible or else then that is not, to your point, it's not taking any accountability at all. So do you think that it'll start to change over time?
Andre Durand [00:19:23]:
Society won't function if every one of us can basically abdicate responsibility to agents operating in our control and under our authority, misbehaving and doing harm to others.
KB [00:19:36]:
We'll come back to that in just a moment after a quick word from our sponsor. In fintech, trust is everything and proving it shouldn't slow you down, whether you're dealing with ISO 27001, SOC 2, CPS 234, or GDPR, Vanta helps you demonstrate security and compliance without derailing your roadmap. Used by thousands of fast-moving regulated companies, Vanta automates the hard part so your team can focus on shipping features, not gathering screenshots. Visit vanta.com/kbkast. That's V-A-N-T-A.com/kbkast. Well, I guess time's going to tell and see how it's going to roll out. It's just interesting to observe. Okay, I want to switch gears now for a moment.
KB [00:20:25]:
If one legitimate permission can become an invisible chain of delegated authority, how does an organization know where that authority sort of ends then? So for example, to extend on this, should an agent's permission expire after every task rather than sort of remaining standing? I know we sort of touched on it before, but would love to get into this a little bit more detail.
Andre Durand [00:20:48]:
One of the zero trust principles is no standing privilege, no privilege by default, meaning no entitlements or access by default. Every request comes with a decision. Will I let you do this? Yes or no? So, when I referred to the runtime control plane shifting to authorization or the decision gate, that's exactly what I meant. To trust AI in a long-lived sense. Okay, hold on, AI. Let me give you a bunch of permissions to go do a bunch of things. And then let me just allow you to do as many of those as you want for some long period of time is a recipe for disaster. What's the alternative? The unit of control becomes, oh, you want to do X, Y, or Z? Let me evaluate my policy.
Andre Durand [00:21:36]:
If it adheres to my policy, I'll let you take that next action. And when you come back to do it again, I will reevaluate the policy as well as any other context that is appropriate for the decision. And I'll decide if you can take the next action. So zero trust and runtime control with agentic is really boiling down to a unit of control that we call your next action. You're only as trustworthy as your next action.
KB [00:22:00]:
Do you think people know that, or do you think they're sort of coming to terms with that?
Andre Durand [00:22:05]:
I think once you begin to explain it, they understand it. Now, that's not the way we operate today. We're operating with a lot of trust and the more trust we give agentic, the faster it goes. And for the most part, it's okay. But again, these models are probabilistic, not deterministic, which is what we both love about them and what we can't count on if these agents are taking action on infrastructure. Again, I can ask the same question 5 times and get 5 answers if it's just advising me. But that's very different if I'm asking it to do something like move money. So there we want the answer to be a little bit more deterministic.
KB [00:22:43]:
So then staying with that, most identity systems were designed around predictive human behavior, right? So would you say like nowadays we're retrofitting human-like controls onto the machines? And to your point, these machines that are capable of making, you know, thousands of actions before anyone potentially can intervene. And to your point, we need to be able to count on making the right decision.
Andre Durand [00:23:09]:
We're applying all the human controls plus, again, because agent machine speed is different than human speed. They can do a lot more damage in a lot shorter period of time. You know, many companies have had kind of SLAs in response to security or threats that were perpetrated by humans. And, you know, you might measure it in hours, but the amount of damage that an agent could do if it were given privileges and allowed to operate infrastructure with no unit of control that, as I described, is its next action, it could do a tremendous amount of damage before the human's even alerted, much less diagnose and analyze and hit the kill switch. So all the same, you can't secure what you can't identify. Let's just start there. So you have to know what actors are operating in the environment. We need to register them.
Andre Durand [00:24:03]:
We need to understand who's responsible for them, kind of assign a custodian or an owner, for example. We need to authenticate them when they interact with us. We need to manage the entire lifecycle of their identity if they're coming or going and allowed to operate. We need to govern what permissions, if they have any permissions, or what roles they will take on. And we need to see and be able to recreate what they did in our environment. So we need to be able to audit what they ultimately did. Those are all principles of identity management, and agents are no different. We need to know what they are, where they are, what they're doing, who owns them, what permissions they have, what policies govern their action within our infrastructure.
Andre Durand [00:24:51]:
The only difference is they operate at a completely different speed. They operate at machine speed, not human speed. So the measurement of control needs to be much tighter. And the risk of them doing something, since as they have no consequence, we need to scope their access to a much finer degree. Used to be like, log into the network, and if you're on the network, you can do everything. That concept with agentic is very, very dangerous. So we need to scope their access back to fine-grained authorization, and we need to scope the time in which they can act to their next action.
KB [00:25:26]:
So, Andre, would you say companies are at that stage where they're starting to scope their access to see what could happen, what their permissions look like?
Andre Durand [00:25:34]:
Not yet. There's beginning to be a recognition of how do you apply, what exactly is the method under which we apply zero trust to agentic action? People are now paying attention to the answer to that question. But most are chasing the reality that everyone has been told to go fast with agentic. And in the pursuit of go fast, they're not fully aware of where all the agents are and what all the agents are doing. So we haven't yet built the control plane that will see all of the agents operating in our environment. Would you know if it's a human or agent hitting your APIs today? Do they have the detection signals? If it's a computer-using agent, so the agent has been embedded in your browser and can move your mouse and type your keyboard, and it shows up at your website, would you know that it's an agent clicking the keyboard versus a human? So people are just now at the beginning. They've been told to go fast. As one mentor used to say, be sure your skis don't get too far out in front of you.
Andre Durand [00:26:40]:
They know their skis are out in front. They know that there's some embedded risk, but they also know that they need to go fast for competitive reasons. And now they're coming around saying, okay, how do we discover what agents are operating? How do we discover what they're doing? That's problem number one, visibility, observability. And then step number 2 is, okay, which of these agents present the most risk in what they're doing? And what tools and methods do we have to essentially add the guardrails? I would say the whole industry is in a discovery phase around what I just described.
KB [00:27:16]:
So, your view is that authorization must remain independent of the agent, but if an agent can request broader permissions, for example, influence an approval process, manipulate policy enforcement, et cetera, who guards the guardrail?
Andre Durand [00:27:34]:
Yeah, that's a problem. Like if, for example, if permission escalation were sent through email and approval was just clicking on a link in email. But you know, for example, a cowork agent has access to email and can just go to the email and click it. It can approve its own permission. It can initiate the permission and approve the permission, and that's a problem. So you know, many of these principles, segregation of duty, independent channels, we are going to have to create methods under which we ensure that agents can't essentially escalate their own permissions into a dangerous state. And these are all, right now, that's where a human in the loop is occurring and where, for example, facial biometrics to ensure humanness are being used because we do need to keep the swim lanes separate.
Andre Durand [00:28:25]:
When we allow agents to impersonate a human and we can't distinguish the human from the agent, it's hard to figure out how we're going to keep the agent in its lane and the human in its lane. So we're right on the edge of all of this stuff right now. Doing a facial biometric to prove that you are the one operating the keyboard is a little bit different than cut and paste of an OTP that came in. An agent could technically see the message of an OTP and cut and paste it back into the website. And so does an OTP prove humanness? Not anymore. Not when an agent's running on the endpoint, but yet a facial biometric where we've ensured the camera hasn't been tampered and the biometrics stored in the TPM still is a method of ensuring humanness in the permissioning chain. So anyway, these are all new problems that have emerged literally in the last 6 months.
KB [00:29:19]:
So then my next question for you would be, in your experience, what action should an AI agent never be authorized to take autonomously, sort of regardless of its accuracy, would you say?
Andre Durand [00:29:32]:
Certainly everything related to its own privileges. We touched upon probably the biggest one. We have to make sure agents can't escalate their own privilege. That's how many attacks work, right? They come in with low privilege and they kind of work sideways and they aggregate privilege. Any one privilege on its own might not be risky, but risk accumulates. Privilege accumulation basically accumulates risk until it hits a threshold. So I give you access to one small thing, you can't do a lot of damage. I give you access to 2, maybe it's still moderate damage.
Andre Durand [00:30:07]:
I give you access to 3 or 4 things, you can combine them in ways I couldn't foresee and you can do a lot of damage. I think the privilege part of this equation is a big part of it. I also think that we need to be careful if we're allowing autonomous agents to make decisions. We have to understand that agents have bias. All of these agents have tremendous amount of bias. They are trained by humans after all. And so the fact that they sound human and their entire existence lives in the probability of words, you can see bias play out in these models.
Andre Durand [00:30:45]:
So if we test a model for a bias one direction or another, and it doesn't exhibit it, and the model gets upgraded, can we be sure that a model that was saying one thing says the same thing when the model gets upgraded? Or do we need to test that the new model hasn't all of a sudden taken on a new bias? So there's a lot we need to do in ensuring that our intent is followed through when it goes through the fuzzy logic of a probabilistic engine. If we want to allow agents to think and make decisions, but we want to be responsible for the quality of those decisions and the consistency of those decisions, there are some new things that we're going to have to do again to make sure that the intent and the output match and that they don't drift. Because models drift. The first time you ask, the intent and the output might match. The second time you ask, it might drift a little bit. And on the third time, you might get a completely random answer. So again, we're going to have to match the risk-reward equation here, the governance model that sits on top of where and how we use these decisioning engines. There's a lot of sophistication here we're going to have to build that we haven't had before.
Andre Durand [00:32:03]:
Yeah.
KB [00:32:03]:
Yeah, this is super interesting. So just sticking with the bias side of things, how do companies know if there's any bias even in the first model, even if it gets upgraded? Or do you think there just may be no way of fully knowing other than, like you said, the testing? But if they didn't test it before, they might be going off even the first model and then they're trying to upgrade it to the new one.
Andre Durand [00:32:24]:
Again, you're describing why zero trust probably is the standing principle. Don't trust anything. We've got to gate and guardrail everything. So you don't trust that one model to the next model carries the same bias or that the bias hasn't shifted. When you gate every action of agentic, that is a unit of control that says, I don't trust, I don't trust the model. Let's assume agents get reputations like humans do, because you could see that happening. We do behavioral biometrics on humans, meaning this person always behaves or clicks on a certain thing or behaves a certain way when they log into our system. Let's assume we did the same thing with agents and we build a reputational, a behavioral reputation system for agents.
Andre Durand [00:33:08]:
Would we stand upon that if we saw it? Like, would we trust it? Oh, I see. I've seen this agent before. It's always behaved well in the past. Let it go. Again, when we come back to, they're probabilistic, so you don't know what they're going to do. The models change, so you don't know how they're going to behave through a model change. Here's the other thing. We don't know that they've been compromised.
Andre Durand [00:33:31]:
So now all of a sudden, just knowing that it's the agent, we've seen this agent before, we're sure we've seen this agent before, it's always behaved well, doesn't mean that it hasn't been compromised. Doesn't mean in some way the prompt hasn't been, hasn't changed, the instruction hasn't changed. Again, this all comes back to zero trust principles and why decisioning and authorization becomes the new gate for agentic. So to answer your question, We don't trust that the models haven't changed the behavior. We have to assume they do change the behavior, and we have to have a way to make sure that the output of the agents still are within acceptable limits for what we're asking them to do.
KB [00:34:07]:
Andre, do you think companies are assuming that? And let me caveat that by saying, and we've sort of spoken about this, because everyone's saying, we've got to go faster, we've got to go faster, competitive edge. So maybe there's elements of people not wanting to do that, but they think we've got to cut corners because we need to do other things. So do you think that's sort of in there as well for people?
Andre Durand [00:34:25]:
100%. Yeah, we're all over our skis right now and we are riding upon a lot of trust that will be abused. It just full will be abused. So trust has historically been weaponized and is being actively weaponized now. Anywhere there's trust, those are the first things that get attacked. And so look, it's maybe outta a little bit of curiosity and necessity, right now, but I think we all know that it can't be the standing long-term position. So most new technologies kind of run ahead of their ability to be governed and secured and trustworthy. They're trusted before they're trustworthy from a true guardrails point of view.
Andre Durand [00:35:07]:
And so this is, this is nothing new, but it's just happening at a different speed, at a completely different speed.
KB [00:35:13]:
So what do you think happens now? Look, I know we've discussed a lot of different things here today. What do you think is going to happen? And I know you said some of these things have been happening the last 6 months. What do you think sort of the next 6 months, what can we expect or what do you think is going to be the shift in the market?
Andre Durand [00:35:28]:
Well, companies are going to want to figure out where their central registry for agents are going to be. They're going to want to formalize their discovery of the agents operating in their enterprise or hitting their, their trust boundary, their resources. So there's going to be a lot of, a lot in the visibility and observability space. And then one by one, they will identify which agent use cases pose the most risk, like we do with all of our risk programs, and begin to bring those use cases into compliance, into the governance compliance. So slowly but surely, technology, policies, gateways will be installed to see, authenticate, and authorize agentic action.
KB [00:36:11]:
That was Andre Durand again, everybody. The phrase I keep coming back to is, you're only as trustworthy as your next action. An agent that behaved 5 times in a row tells you nothing about the 6th. So trust has to be checked every single time it acts. Thank you, Andre. If you're a CISO listening to this, find out whether you can even tell an agent from a person hitting your APIs today, because nothing else in this conversation works until you can. I read every reply.
VO [00:36:46]:
If you've got some thoughts on this one, send me a message on LinkedIn.
KBKast - Cyber For The C-suite.
