November 18, 2026

00:36:43

Episode 397 Deep Dive: Ravi Soin | Shadow AI Isn’t a Workforce Problem – Why the Real Gap Is Policy

Episode 397 Deep Dive: Ravi Soin | Shadow AI Isn’t a Workforce Problem – Why the Real Gap Is Policy
KBKAST
Episode 397 Deep Dive: Ravi Soin | Shadow AI Isn’t a Workforce Problem – Why the Real Gap Is Policy

Nov 18 2026 | 00:36:43

/

Show Notes

In this episode, KB asks Ravi Soin what “yes, but safely” actually means. For Ravi, a safe yes needs three things. The request is visible before it happens, it’s scoped to only what it needs, and someone can see what was done with it. Miss one and you’re just hoping.

What they get into:

  • Why so many staff are using AI outside company policy, and why banning tools only pushes it out of sight
  • Why an AI agent switched on by an employee should be treated as its own identity
  • How some companies have blown their annual AI budget in a month
  • Why AI pilots that shine on clean data fall over in production

About Ravi

Ravi Soin, CIO & CISO at Smartsheet, brings more than 20 years of information security and IT leadership at global technology companies, including his role as CIO & CISO at healthcare software leader, Edifecs, and various product roles at Microsoft. His experience successfully building and managing SaaS products for public, private and private equity companies will be invaluable as Smartsheet strengthens its offering. A prominent figure in the Seattle IT and security leadership community, Soin serves on the SeattleCIO advisory board and was recently honored as Seattle CIO of the Year and by CIO100.

Connect with Ravi on LinkedIn. Subscribe to the KBI Dossier. KBKast is presented by Vanta. Learn more at vanta.com/kbkast.

View Full Transcript

Episode Transcript

Ravi Soin [00:00:00]: All of a sudden, he or she has turned on an AI model that you know, has gone rogue without any guardrails. You've got some serious issue where that's much more beyond the employee itself because that agent is now on its own, and in my mind, that's a different identity than the employee. VO: From KBI Media, I'm Karissa Breen, and this is KBKast. KB [00:00:25]: My guest today is Ravi Soin, who is both CIO and CISO at Smartsheet. We talk about what yes, but safely actually takes and why so many people are using AI outside company policy. Then there's the bigger question: Can governance ever keep up with adoption? Ravi has a pretty blunt answer. KB [00:00:55]: All righty, let's get into it. Okay. So Ravi, I wanna start with when CISOs say yes, but safely, what does that actually mean in your eyes? Or what's your definition of that? Ravi Soin [00:01:08]: Yeah. So I think when CISOs are referring to this notion of, yes, but safely, I think the default answer is a yes. But you're saying yes with guardrails, and the guardrails come at the same time, not bolted on afterwards. Ravi Soin [00:01:24]: In my mind, the three things have to be true. Number one, the request that you are making is visible before it happens. Number two, it's scoped to only what it really actually needs, and thirdly, someone can see what you did with it, right? So miss any of these and it's not a yes, definitely not a yes but safely. Ravi Soin [00:01:44]: It's just a yes, and hoping. And the failure I see most is that security shows up in many of these instances when the business has already said a yes, so and by then it's too late, right? And, and yes has already happened in someone else's head, and the guardrails that you put in place are, are really limited to trying to r- really recoup what, what the mistakes have already been made. KB [00:02:07]: And when you say the word hoping, like obviously we know like hope's not a strategy. Would you say people are putting maybe perhaps their eggs too much in that basket of hope? Ravi Soin [00:02:16]: Not because they want to, I think it's not intentional, it just may be how the cards fall at the end of the day. Uh, if I think about the notion of risk broadly, right, and I, I think it really depends on the role that you're in, right? Ravi Soin [00:02:29]: So some, some will say, "Hope that it doesn't happen" as a function of their understanding of what safety looks like, right? So as an example, when you think about legal, legal's version of safe is about liability, right? Compliance version is about a specific regulation. And, and both of those are right in their own ways, but w- when you reconcile what the definition of what safety or safely means, right, a- and not having a common shared definition, that's when you get this notion of you're hoping that things will be fine because at the end of the day, you've not thought through the lens of that shared definition. KB [00:03:08]: And on that note, with the shared definition, what are some of the discrepancies? 'Cause I'm assuming that there's probably discrepancies between like tech, cyber, AI teams versus like a business executive on what they want. So would you say that's where perhaps some of this conundrum really exists? Ravi Soin [00:03:22]: You know, I mentioned the legal example to you, you know, the compliance one, right? Ravi Soin [00:03:26]: I'm, I'm gonna take the engineering example, right? Like their world of, if I think about software engineers and what they're trying to build, their world of what safely looks like is the context of did it break anything, right? Of did I break a build or did I break a customer, a solution that may be in play? Ravi Soin [00:03:43]: And I think the n- the frontline employee that you have is often just sort of, "Will I get into trouble," right? None of them are wrong. But the shared definition is an important one because when you think about that reconciliation in my mind of what that, what that looks like, what is the technical control that you're trying to drive at from a safety definition perspective? Ravi Soin [00:04:04]: And who are the people that are defining what's safe, right? And, and you need to arrive at that one definition, written down, simple enough that s- anyone in the organization can understand it, without calling some legal or security person, right? I think that's when you think about that shared notion, that belief across the organization. KB [00:04:23]: My next question is, are CISOs being asked to say yes to AI before the business has built the governance needed to use it safely? And I ask this because a lot of people I'm speaking to at the moment on the show are saying they are racing to get there because of their competitors. We kinda need to get there, but now we're, we're worried because now we sorta need to pull back because maybe we've over-indexed. KB [00:04:44]: So talk to me a little bit more here, Ravi, about what's going on in your mind. Ravi Soin [00:04:48]: Yeah, no, especially, I know, I think my vantage point of being the CIO in addition to the, the CISO, I, I, I see this a lot with my colleagues in, in the CIO community, right? The business is definitely pushing for a faster adoption around the AI lifecycle. Ravi Soin [00:05:05]: I think today, uh, what you're seeing is many, many organizations are very much in that boat where the business is- isn't necessarily asking for permission around enabling some of these AI functionalities. So someone in, whether it's someone in HR or marketing or sales, has enabled an AI tool to get their job done faster, and governance or no governance, right? Ravi Soin [00:05:27]: And the CISO isn't necessarily approving that, right? And they're just finding out, many of the times, they're finding about what, what has already happened, and, and it's, by that time, it's if they're lucky, first of all, by that time it's too late. So the real question in my mind isn't can governance move faster than adoption. Ravi Soin [00:05:45]: It never will. Adoption by the sheer word, right, in my mind always wins the race. So the question you have is, like, have you built the framework that is clear enough that people are able to make the right judgment, the right calls without asking a CISO? And I think that's, in my mind, the real version of keeping up what's actually possible. Ravi Soin [00:06:07]: So you don't wanna be the hindrance of saying no. You wanna make sure that you have the guardrails and the, and the processes that are necessary for the business to excel as fast as they can. KB [00:06:18]: Okay. So there's a couple things in there. So when you say the, is it clear enough, would you say it is kinda clear enough at the moment? KB [00:06:26]: Or do you think it's still relatively early days and it's gonna take a bit of time? Ravi Soin [00:06:30]: Well, I think it depends on the organization. I think many organizations are just grappling with what AI is and how do they adopt it to their processes and what we can do, right? Within Smartsheet, as an example, we really thought about not only from the product side and what we offer to our customers, right, and the workflows that are enabled and, and ensuring that we have governance from the get-go. Ravi Soin [00:06:52]: Similarly in internal processes wise, right? When, when somebody within the organization, whether it's a marketing or a sales or our people function or anybody in IT are enabling AI tools and processes, the governance framework has it existed from the get-go. It's not being bolted on after the fact. And certainly, you know, we have tools to discover many of these things as well if things go rogue. Ravi Soin [00:07:18]: But at the same time, I think it really depends on the maturity level of the organization and how they think about governance as part of their, their processes. KB [00:07:25]: Then just going back to when people say, "Yes, but safely," what happens if it's like, yes, but safely, but you wanna do 10 of these things, but safely means we're gonna turn off nine of those. KB [00:07:35]: So it's like in theory it sounds like yes, but safely, but it's really under my proviso. Ravi Soin [00:07:40]: Going back to that governance layer, right? If you have enabled the governance layer for organizations to enable on top of it, like as an example, I'll give you the Smartsheet case, right? We've got this notion of citizenship developer. Ravi Soin [00:07:54]: We created a governance structure where anyone in our company, 3,000 employees globally, can build tools and, and AI processes on top of. Whether you are in marketing, whether you're in sales, you wanna go build a next best AI tool, those guardrails have existed for them to enable to do that. And we cannot think of every single use case or scenarios for every single function within the company, but we've enabled a pipeline with where people can build their AI agents on, or workflow automations on, or improving their processes. Ravi Soin [00:08:28]: We've also got sort of this unified notion of integrating everything within Claude and MCP and connecting to those, and so that you have a sheer, you know, a single pane of glass across the data as it moves across the organization. So we, we've enabled those guardrails and the tool sets and enabled our organization to go do that, and I think it really depends on where organizations are in their journey of AI. KB [00:08:50]: And then if you zoom out now for a moment, what would you say is the biggest bottleneck at the moment for companies? Ravi Soin [00:08:57]: The maturity curve, I think is, is really where it, where it comes down to, right? So there are organizations that are very early in the AI life cycle that are piloting all kinds of stuff, right? Ravi Soin [00:09:08]: Whether it's tools that they're enabling AI functionality on, or enabling giving people access to OpenAI or Claude or Copilots or whatever their favorite tool is and letting people discover it. You've got on the other end of the spectrum where for organizations that have actually gone through that pilot phase, seen, you know, their token spend go completely out of whack and really having their finance team go back in and say, "Hey, we're gonna put a stop to all of this AI tooling and functionality because our spend is, is just not something we can sustain the way it is," right? Ravi Soin [00:09:43]: I think the, the, the sweet spot is really organizations that think about the workflows and the ROI of what they're trying to do. So when then I went and then I look about like what is the investment that you're making and what are you trying to do it for? So if I think about a workflow of, let's say I'm kind of... Ravi Soin [00:09:59]: I'm gonna take a, an IT example of onboarding experience, right? Like from the time that the journey of somebody who applies for a job to getting onboarded in the company to getting their payroll set up to their access systems and all that stuff, that now all of a sudden is a very powerful agentic way of creating that automation that you can enable. Ravi Soin [00:10:20]: And so now that's an output that you can say, okay, like I'm gonna cut down that onboarding time from a couple of weeks to a matter of hours, and that's a very phenomenal change that you were able to do as a function of AI. Of course, you know you've gotta have governance, and I keep going back to that from a CISO lens, right? Ravi Soin [00:10:35]: Like you gotta have governance from the get-go to enable that because it, those agents that are acting on your behalf, whether it's working through your HRIS system or your identity system or your access control system, all of those things have to be come in play. So the guardrails have to be set up right up front versus being bolted on. KB [00:10:54]: So since going back to the tokenomics side of things, and I have heard that, and I know that there's companies out there that are like blowing their entire annual budget in like a month, that I'm also hearing, and I'm keen to get your thoughts here, that CFOs being like, "Okay, this is way too expensive. KB [00:11:08]: We've all gone over it. We're just gonna turn it off." But then turning it off also then impinges on people's workload and what they're doing. So do you think that... Are you seeing this happen? And then if so, how are companies sort of- Going back to how they were operating before, 'cause it's not just like, "Oh, we're just gonna turn it off and we're just gonna keep operating the way we were." Ravi Soin [00:11:27]: It's not, it's not like that. When you think about this notion of a business like value prop, right? You gotta, you gotta make sure that I talk about people running pilots and then they do the, the pilots and they say, "Oh, it was, it was a failure or it was a success." What they really, really... What is the end output that you're trying to achieve as it? Ravi Soin [00:11:47]: So it has to achieve a business outcome, right? Within Smartsheet, as an example, anytime we enable an AI workflow or an agent that is doing work on behalf of something, it's really tied to our value creation plan, right? And so the, those VCPs that we have are tied to the business of what, what we're... Ravi Soin [00:12:06]: Whether it's efficiency, whether it's we're trying to drive growth, efficiency in our ability to launch products. Whatever the use cases that you may have, those agents are tied to that. Now all of a sudden, the economics of that equation become much more clear because now you can look at this whole notion of tokenomics is like, well, we tried, experimented, and we blew through that in a month. Ravi Soin [00:12:27]: Well, what were you trying to achieve, right? If you're trying to say, "Okay, I'm gonna do this business outcome in this way with a certain model that's there. This is the data that's gonna... I'm gonna use it for. These are the guardrails that are there to ensure that my agent has the right set of controls around it." Ravi Soin [00:12:45]: That token efficiency and how you look at token maxing and, and all of that stuff, that can all be engineered and architected in a manner that, that is pretty phenomenal. And nowadays it's... You know, I, I go back to the days of cloud operations, right? When you think about people who are moving from data centers to AWS and Azure and moving their data to cloud, nobody had any idea about the cloud spend. Ravi Soin [00:13:07]: It's very much the same world that we're living in today, right? And, and people were able to dial back what is, what does success look like in the context of the business process that you're trying to enable. KB [00:13:17]: And do you think companies know what they're trying to achieve? Now, I s- I know that sounds like a rudimentary question, but I ask that because perhaps what I'm hearing in the market is people saying, "Well, you just gotta jump straight into the AI. KB [00:13:29]: It's nice to hear KB and Ravi talk about it in theory, but you just gotta jump in and you gotta trial it and test it." So do you think that theory has gone, "Okay, we kinda know what our North Star is, but we've just gotta jump in and test it. Oh, but actually now we've blown the budget." So do you think there's a bit of that in terms of mindset in there? Ravi Soin [00:13:46]: I arrived at Smartsheet about a year back as a CIO and CISO in the organization. I think we went through our own journey within Smartsheet, and I think a lot of the organizations are going through that themselves, right? Lots of AI tools, lots of experimentation, people turning on AI functionality within the tool that they're, they're doing with... Ravi Soin [00:14:05]: And what they're realizing is that you can take the AI notion within that tool and certainly turn it on and, and learn from it. But how do you think about the data itself and what does that process look like? And the re- reason I bring that up is like when I think about a workflow like onboarding, what I just talked about, like if I wanna make that much more efficient, or I'm talking about like our sales QBR process and how do you make that much more efficient, right? Ravi Soin [00:14:32]: You're taking a workflow, and that business workflow, you have to look at data from different myriad of different data sets. I think that's where people get caught off guard is that they're turning on AI functionality within their silo and not understanding that the data lives beyond just the world that they're currently in. Ravi Soin [00:14:50]: So as an example of the onboarding, I'm touching four different systems and that business value prop that Smartsheet has created and enabling it across those systems, it's very powerful. So, and I think that's how I would encourage any CIO colleague of mine as they're thinking through this, is like, look at the processes, the business processes that you're trying to enable. Ravi Soin [00:15:11]: Take them one at a time, whether it's efficiency-based or whether it's growth-based, and then look at what that process you want and what is the da- the types of data that you wanna touch. What AI tool are you going to enable to go touch the data across a whole set of different systems to pull it together? KB [00:15:28]: We'll come back to that in just a moment after a quick word from our sponsor. Selling SaaS to enterprise? You'll be asked about your security credentials sooner or later, likely before the contract's even signed. Vanta helps SaaS teams prove their compliance and frameworks like ISO 27001 and SOC 2 without losing months to manual prep. Visit vanta.com/kbkast. That's V-A-N-T-A.com/kbkast to learn more. KB [00:16:00]: So there's some research that I'm aware that Smartsheet has conducted. Now I'm gonna read it out. It says that 70% of operational professionals are using AI tools outside of corporate policy. So does this mean that governance has effectively failed? 'Cause I mean, 70% is quite a lot. Ravi Soin [00:16:19]: Well, I, you know, I push back on the word fail, right? Ravi Soin [00:16:23]: I think, I think that, that number, that- that's a number in my mind is a, is a symptom, not a verdict, right? 94% actually, to be precise, of those same people are already using AI to do their job better. That's not people going gone raw-- rogue in any, any sense of the matter, right? What they're really doing is people are trying to be more productive, right? Ravi Soin [00:16:42]: Like you talked about earlier. So for that 70%, there's no rule yet to ignore, right? Only a quarter of organizations have really fully documented and enforced AI governance policy at all. So I wouldn't call that governance failing, I'd call that governance that never really got built at the speed that adoption happened. Ravi Soin [00:17:02]: So banning tools doesn't close that gap, right? It... frankly, just the opposite. It pushes that same behavior further out of sight. In my mind, the fix isn't cracking down on the 70%, it's really building the other 30% of policy that's much faster, right? Make the approved path the easy path, right? And there's no reason to go around it. Ravi Soin [00:17:23]: I think that's my recommendation to any CISO, right? Like, I mean, that's the key is like, you wanna make your approved path the easier path, 'cause then you're not saying no, you're saying yes, but with guardrails. KB [00:17:34]: Okay. So just a couple things there. So is this sort of the same version, but different of what we've seen with like shadow IT, for example? KB [00:17:42]: Now it's about shadow AI. So is it similar sort of mechanism? Is it a s- similar sort of approach on how to make this not as big of a problem than perhaps it, it is? Ravi Soin [00:17:52]: That's correct. It's very, very, very similar. The shadow AI is very much like shadow IT, and I think you, you gotta make it, like I said, right? Ravi Soin [00:18:00]: The approved path the easy path. You have to enable the path, and the guardrails are already there so that if anyone wants to explore new tools, they're doing it in the context of the guardrails that are already put in place. KB [00:18:12]: So we were talking about policy, and I know that things change each day. New tools pop up- The market shifts, something else happens. KB [00:18:20]: How would you address companies to continuously stay on top of that? And I, and I say this by saying back in the day when things didn't change as much, it was kind of like policy schmolicy. It's we've got this thing, someone's developed it, we kind of adhere to it, sometimes we don't. Someone else in the organization's been there for 40 years, and they want to run this tool, so we just kind of have to allow it despite it going against the policy. KB [00:18:44]: Now things are so much faster. Does that put a lot more pressure then on, like, security teams to, to manage this? Ravi Soin [00:18:51]: I think the organizations that have had a loose practice around governance, right? And I think are, are the ones frankly are going to be even more in trouble i- in the advent of an agentic workforce, right? Ravi Soin [00:19:03]: So when I think about agents that have access to a certain dataset and how that same permission model that exists even whether the employee's there or not, and what else that agent has access to, and correlating data from different myriads of different tools and so forth, that, that scale of what that can do is very, very exciting, but it's also, i- in my mind, a governance nightmare for folks that have, have in the past traditionally not adhered to that. Ravi Soin [00:19:31]: So yeah, so I, I agree with your sentiment, right? I mean, I think it, it's that 40-year employee that, that's been around and allowed to do certain things the way they've had it. All of a sudden, he or she has turned on an AI model that you know, has gone rogue without any guardrails. You've got some serious issue where that's much more beyond the employee itself because that agent is now on its own, and in my mind, that's a different identity than the employee. KB [00:19:57]: What are your thoughts then towards agentic workforce, the agentic workforce? Ravi Soin [00:20:02]: Well, I'm definitely excited. I think the agentic workforce has, I think, allowing us to be, you know, I think that the ability for us to be much more efficient, leverage the, the productivity hardness that we have as a result of it. Ravi Soin [00:20:15]: I have seen so many different scenarios within our, our organizations itself and, and similarly in a similar way that Smartsheet's enabling that for our customers as well. If I think about any of these scenarios that you describe, whether it's, you know, your procurement workforce or your project dashboard, or you're looking at launching the next best product and you have a, a single pane of glass that's looking at data across different things, things that used to take hours and now in, in many cases, weeks and months, have now been able to shr- you know, shrunk down in a matter of minutes and I think that's a pretty tremendous gain in terms of productivity that, that we did not have before. KB [00:20:55]: What are people sort of perhaps cautious about? Because there's people that I'm speaking to that everyone's got different opinions depending on who you speak to. Or why would people be apprehensive about a- the agentic workforce? Ravi Soin [00:21:07]: Well, I think there's a few reasons, right? I think one is, I mean, I'm gonna take the, the security angle of it, right? Ravi Soin [00:21:14]: I think one, you think about from a, from the process of ensuring that you have the right guardrails because people are- Unclear of what an agent can and cannot do. So therefore, it's, it's absolutely paramount that anyone building a solution on any of the agent tech platforms, including Smartsheet for that matter, have the governance layer up to snuff, right? Ravi Soin [00:21:37]: So you know what the agent is doing, you, you know how, what the agent can do, and then once it's done, there's an audit trail that you can go back and really look at, did it do what it's supposed to do, right? So, so all of those three things, like I talked about earlier, are, are in place. I think that's the first thing that people are very cautious about is what does it have access to, and what can this agent do? Ravi Soin [00:21:58]: So that's the first thing. Second thing I think about is sort of this cultural transformation, right? From an, from a workforce perspective about this apprehension where people are certainly wary of like, hey, what is this impact going to do? You have, you have the younger generation coming out that what I would sort of call the AI native generation, which are, which are u- used to using AI front and center for, for so many different tasks across their daily lives. Ravi Soin [00:22:25]: For them, the AI native generation, adopting these tools and technologies is much easier because they're like, "Oh, I can go do this in a much faster way. You're asking me to pull a dashboard together or analyze this data set? Oh, I can, you know, load this data and, and have this out with- within a couple of matter of minutes so I can go do something much more strategic," right? Ravi Soin [00:22:44]: So I think there's a notion of sort of that, that cultural transformation that is, that is happening within, with- within humankind in general, right? And, and the advent of AI that's there. And, you know, on the flip side is, is also true, like you have folks that are very apprehensive because it's-- they're afraid of what it's going to do to the tasks that they currently do. Ravi Soin [00:23:04]: And to them, I would just essentially say I-- in my mind, I think AI, if, if anything, i- is going to enhance the, the level of productivity that we have never seen before, and a pretty exciting time to be living in. KB [00:23:15]: So I want to go back to your comment around did it do what it's supposed to do. Now, if we look at cybersecurity people by trade and history, relinquishing control and losing some of that control could be a thing, especially for people who've been around the, the traps for a while. KB [00:23:31]: So how does that sort of sit with them? Because it's always about we need to know everything that's sort of going on, that way we can control it, 'cause when we can control it, we can prevent things from happening. So is that shift in mindset, is that challenging the way perhaps, you know, security professionals that have been in the space a bit longer, how they're sort of adopting to changing that behavior? Ravi Soin [00:23:54]: Well, I, I, I think security professionals, when you- when they think about... I think they look at the, the controls, the security controls specifically, and, and map the behaviors essentially to those controls, right? I am going to use NIST as a framework. NIST 800-53 is sort of the backbone in my mind of many of the, whether it's your state, federal, or ISO or SOC standards, right? Ravi Soin [00:24:18]: You can do the mapping against those controls pretty easily. That's been around for, for quite some time. I think if anything, we're gonna now see a much more advent of using those controls as a basis of ensuring that we have the right governance in place with all of these tools that are, that people are using. Ravi Soin [00:24:36]: So I don't think it changes the behavior for security professionals. I think if anything, they're gonna lean in, in more on the control framework that they're been using for decades, which is i- in my mind, NIST based, right? So, and leveraging those to ensure that any tools, any change control, any vulnerability management, any procurement processes, whatever the case may be, right? Ravi Soin [00:25:01]: I mean, there's a whole framework across all of those, and they're, they're following that framework down to the T. KB [00:25:06]: And do you think this is gonna become one of those things that just takes time until- People find perhaps that blueprint that works for them and, but being more adaptable in the sense of things do change a lot more frequently than perhaps they did 10, 15... KB [00:25:18]: Well, we know that they are changing more frequently. So do you think that is just always going to be there in the back of their mind? Things aren't gonna be as static. We need to stay on, on top of things in terms of the competitive advantage, et cetera. But if you zoom out, is that more of a different style perhaps than what we've seen traditionally? Ravi Soin [00:25:39]: I would say that the frameworks that, I'm gonna use NIST as an example again, right? I would say that the framework has not changed. I think the liberties that we all took upon time has changed. So you were able to take a chance on a vulnerability being exposed for a longer amount of time, but that time has been hockey stick on the reverse side, right? Ravi Soin [00:26:04]: You've got a, what used to take months or weeks for a bad actor to exploit a vulnerability has gone down from months to weeks to days and now hours, right? And you've heard about Mythos, you've heard about sort of this AI advent of vulnerabilities and so forth. I think if anything, the security professionals are, the fundamentals of what they need to do around vulnerability management or the use of AI and how the governance, that doesn't change. Ravi Soin [00:26:31]: I think what they don't have is, is time on their hand because now the pace of what's coming at them is much, much faster than it used to. I will flip that. The other side, I also think that AI is, is helping security professionals just as much as it, it's exposing the vulnerabilities, right? So I, within, from the n- the aspect of being a CISO, I'm leveraging AI from a SOC perspective to reduce our phishing, you know, false alerts down to a matter of minutes when it used to take us weeks to figure out phishing attempt. Ravi Soin [00:27:03]: Uh, the same is true for our incident response time, right? Our meantime to resolution has come down. Our ability to look at code reviews is much faster as a result of AI and the advent of AI. So I think there's sort of the, the double-edged sword in my mind, but I'm definitely excited about leveraging all the power that, that comes with it. KB [00:27:21]: Just talking on the NIST framework, so people that I know in the industry, they're saying it's okay, like large enterprise got lots of people, but when I'm like one person and I'm now responsible for this and it's a very convoluted, long document... I mean, this was obviously before we had AI tooling and stuff like that. KB [00:27:37]: So do you think that perhaps, yes, there's the framework, but people just don't have the resources perhaps to go and implement some of these things, even if they are using AI tooling and stuff like that? It's just what I've heard. It's a very long document. Some things are applicable, some things are very vague, not relevant to indust- to certain industries, et cetera. KB [00:27:56]: So it means well, but then it's like, well, I'm kind of getting there as a general sort of gist, but perhaps it's not exactly what I need. Ravi Soin [00:28:04]: Yeah, no, I think NIST is a great framework. Of course, there are folks that they'll take a subset of it because not everything is applicable to every industry or for that matter, or I would say even in the scope of what you're trying to do, right? Ravi Soin [00:28:17]: Having said that, I would say that with the, the advent of AI tooling that exists today, uh, that same framework that you apply to, certainly we've been able to do a lot more with a lot less as a result of the AI tooling that exists today, and I'm gonna u- use the example of code reviews, right? What used to take us, uh, weeks or months to do code review for every single line of code that we were checking in, now we're doing this in real time. Anytime a developer within Smartsheet checks in code, it goes through a vulnerability scan. Ravi Soin [00:28:47]: It happens right up front, so I'm not doing this post-release. We do sort of that red teaming exercise that happens within the code base. When a merge request happen, when the code is getting merged into our production and gets deployed, it's getting checked on the other end from an internet facing side of the things. Ravi Soin [00:29:03]: That used to take forever and, and many times weeks and months for us to be able to do this. Now we're able to leverage the power of AI and harness it i- in a matter of minutes, and that's pretty remarkable. And then again, going back to the controls, right? We actually fed our NIST standards and our base control framework that was applicable to Smartsheet, and we're using those same control frameworks within our code base. Ravi Soin [00:29:25]: So I can now look at NIST control and say, okay, like what, what control? What's the evidence around it? Where is the change management around it? Who- which developer checked it, and, and how do I satisfy an external third party audit that's gonna come into play? KB [00:29:38]: So then on that note, the other question that I have, which I've started to ask people, is with certain, like, AI capability, so a human on the loop, in the loop, et cetera, the question that I probably have is if you look at like a traditional developer, they've understood it, they've gone to university, they really get it in terms of the mechanics. KB [00:29:57]: Now you're gonna have another wave of people that are not learning it from the beginning all the way up. So there's gonna still be gaps of knowledge in terms of the mechanics right down the bottom that they may miss. So the question that I have is even if people are sitting there, they're leveraging AI tools and they've gone, "Okay, well, I need to approve something 'cause it said that I'm not sure what to do. KB [00:30:16]: Now I'm gonna engage a human." Do you still think there could be error though, because that potentially that employee doesn't have the pedigree that we would've seen historically? Ravi Soin [00:30:27]: Great point. I, I think this is where I would urge my colleagues to I think part of it is also the, the coaching and the training that needs to happen with the pedigree that you talked about, right? Ravi Soin [00:30:41]: Like, when we hire folks fresh out of college, right, while they are AI native, a lot of it is, like, simple things like we do with code reviews, right? Like, so certainly AI can take a big pass of it, but at the end of the day, having a mentor or, or a lead that's looking at the code that's getting checked in and reviewing it and ensuring whether it's a spot check, whether it's in AI that's doing it and you're l- verifying whatever the percentage of code that you're checking in and verifying that it's doing the right thing. Ravi Soin [00:31:09]: I don't think that replaces human knowledge and the ability for humans to be finally the, the approver or the, of, of what finally gets... Ends up in, in the real world, right? So organizations that are going completely, "Hey, we're not gonna have a human in the loop at all," and I, I don't know if we're ready for that yet. Ravi Soin [00:31:28]: Despite all the, the advancements in AI, I, I still think when you think about that guardrail that I talked about, right, you wanna make sure that you have the right guardrail so you can verify whatever you did can be done by coaches, right? At the end of the day, you're coaching AI native em- employees that are coming from the workforce and, and helping them build that pedigree. KB [00:31:50]: So I think maybe where my mind was going was around, for example, like I work in media, my background's in cyber by trade, and if I was trying to automate something, like I would feel more comfortable if a machine asked me a question that I knew as a human 'cause my area of specialization. But if it's like, hey, there's some regulation accounting thing that you need to respond to, I don't think I'd feel comfortable to request like approval on something like that. KB [00:32:15]: So it's just more that the new wave of people perhaps they're just not... They don't have that fundamental knowledge. It's like a mechanic, like an old school mechanic that understands the vehicle wholeheartedly versus perhaps now. People don't really understand things from the inception of how they were built, so that I still believe there could be that gap in knowledge with companies with new talent coming in. Ravi Soin [00:32:39]: I know. Ravi Soin [00:32:40]: I, I think that's a fair point, and which is why I was saying I, I think it's important for, for the new talent to be coached and guided i- in a way that makes the most sense by a mentor or a leader that's willing to spend the time, right? Fully agree with you. I, I think it's important for folks who've been in the trenches, right? Ravi Soin [00:32:58]: The mechanic model, who knows where the oil leak is happening and, and how to fix that leak is just as important despite it being a fully automated car because you know you've got that engineer mindset of, "Okay, I think this is right, but it's not quite right." Or to your point about like, well, there's an accounting control that I don't quite understand even though AI has given me the answer to it. Ravi Soin [00:33:20]: Let me go reach out to my colleague and get the verification that I need. KB [00:33:24]: So then would you say companies are also calling AI pilots successful because they are operating these sort of pilots in controlled environments versus maybe they are ignoring why they break perhaps in real business operations? Ravi Soin [00:33:40]: I think a lot of time, yes, and, and it's an easy trap, right? I mean, it's not a dishonest one, but a, an easy trap, right? Uh, when I think about a pilot, it's successful because it's small. You know, the data that you have in it is clean. There are a limited number of people that are watching it very closely, right? Ravi Soin [00:33:56]: And none of that is true when it rolls out, right? I mean, the data is m- a lot more messier outside of the pilot case. The edge cases are multiplied significantly, and nobody's really watching it as closely as it was a pilot. And so the tell is really the success metric. So did it work or did it work under real conditions at real scale with governance turned on the way it's actually supposed to run in production? Ravi Soin [00:34:20]: So if a pilot really never really touched messy data, right? Or, or sort of contested access control systems, calling it a success, in my mind, solving the easy version of the problem, right? And it's assuming that the hard version behaves the same way, and it, and it usually doesn't. KB [00:34:37]: Yeah, that's interesting. KB [00:34:39]: So then would you say that people have this preconceived assumption that like, "Hey, it's performing well. Let, let's just go into production now," and then we've got lots of issues, or do you think people are acutely aware of that, but they're mindful? Ravi Soin [00:34:50]: Yeah. No, I think this is exactly right, right? So I think that you need... Ravi Soin [00:34:54]: You start throwing in the whole notion of real conditions, scale with all the governance that's in play, right? All of the sudden you're like, "Hey, why is this not working? It worked great on in a, in a pilot." Well, the pilot was much cleaner data, much smaller. You had people looking at it, and I think it just... Ravi Soin [00:35:10]: So I think it really comes down to what does success metric look like, right? Did you apply the real conditions at it at the scale and the governance that was needed for it to be successful? KB [00:35:19]: So Ravi, final question. What do you sort of think moving forward? Like what comes up in your mind perhaps at the end of the week once you finish your, your work that you think about in terms of the industry? Ravi Soin [00:35:30]: I think we all have to operate in, in my mind as CISOs by saying yes, but with guardrails in place, right? And that needs to be the operating model of how all CISOs across the organization do work. And, and shadow AI isn't a workforce problem, right? It's a policy vacuum. And once you fix that vacuum, you really... Ravi Soin [00:35:49]: That's the, that's what you gotta fix, not the workforce itself. And so I, I'm definitely excited about the possibilities that exist in the future ahead and even in these challenging times. KB [00:36:00]: That was Ravi, everybody. The part I'll be thinking about tonight is how he reframed shadow AI. The staff aren't the problem. The gap is the policy that was never written. If you're a CISO listening to this, stop chasing the people working around your rules and build an approved path that's easier to use than the workaround. VO: I read every reply. If you've got some thoughts on this one, send me a message on LinkedIn. KBKast - Cyber for the C-suite.

Other Episodes