November 13, 2026

01:16:28

From The Field at DASH 2026 | Andrew Krug & Nathan Pitchaikani on Knowing What You're Actually Defending

From The Field at DASH 2026 | Andrew Krug & Nathan Pitchaikani on Knowing What You're Actually Defending
KBKAST
From The Field at DASH 2026 | Andrew Krug & Nathan Pitchaikani on Knowing What You're Actually Defending

Nov 13 2026 | 01:16:28

/

Show Notes

KB recorded this episode of From the Field at DASH by Datadog 2026 at the Javits Center in New York, with two guests who come at security from different angles.

Andrew Krug, Head of Security Advocacy at Datadog, starts with what he calls the most boring truth about breaches: stay in business long enough and you’ll have one. Most companies accept that now, but few have actually prepared. He explains why long-lived credentials and machine identities are still one of the biggest gaps in cloud security, and how AI agents with broad permissions make it harder. He also covers why identity risk rarely makes a CISO’s top two priorities, and how pathfinding.cloud maps the way attackers go from read-only access to full admin.

Nathan Pitchaikani, Senior Security Engineer at Riot Games, explains how his team keeps up with supply chain attacks that now land almost daily. A teammate built an AI tool that scrapes threat feeds, checks Riot’s environment for matching IOCs and flags anything relevant to gaming. Nathan also gets into the downside: vibe-coded tools tripping detections, AI budgets blown in a month, and why peer review still matters. He wraps up on crown jewels, deception technology and running threat intel against SaaS and identity logs.

View Full Transcript

Episode Transcript

KB [00:00:08]: What's up everyone? It's KB, and I'm on the ground at DASH by Datadog at the Javits Center here in New York City. This is the 9th year Datadog has run this conference. This used to be a room that came for dashboards and uptime alerts. 5,000+ people are here now, 60-odd sessions, 20 workshops, and 30+ speakers are gathered here this week. Teams have pushed AI into production, and a lot of them can't see what it's doing once it's running, which means you either have to catch the problem at 3 AM or your customers catch it for you. So across the next few segments, I'm sitting down with the people running and securing these environments to find out what's actually holding up and what's still a slide. This is From the Field in New York City. KB [00:01:12]: Joining me now is Andrew Krug, Head of Security Advocacy at Datadog. Andrew, great to have you here. And today we're talking about identity, access, and what you're calling the borderless cloud reality. So Andrew, thanks for being here. Andrew Krug [00:01:30]: I'm really happy to be here. KB [00:01:33]: Okay, so I'm very curious to know about the boring truth about breaches. Andrew Krug [00:01:39]: I think the most boring truth about breaches is probably the one we haven't really accepted yet, which is that we're all going to have one at some point, provided that we're in business long enough. So as long as I've been working in cybersecurity, we've been discussing this assume breach mindset, but still to this day I interact with folks that I just don't think are considering that in the way that they've built their security strategy. KB [00:02:08]: So would you say that that perception is changing though, given all the headlines and given everyone's talking about it, or do you still think people are of more of an archaic sort of mindset? Andrew Krug [00:02:18]: I think that we're accepting it, but I don't think that necessarily is reflected in preparation. KB [00:02:24]: Tell me more about reflected in preparation. Give me an example of what comes to mind. Like, we can all say get an IRP, et cetera, but what else sort of are you thinking about? Andrew Krug [00:02:34]: So we can accept the fact that there's going to be a breach and we can lean into kind of the, what we've called in the past security nihilism aspect of that, which is it's gonna happen and there's really not that much I can do about it. Versus the other end of the spectrum, which is kind of going through more of a preparedness exercise where you're doing things like incident response tabletop exercises. KB [00:02:34]: Mm-hmm. Andrew Krug [00:02:57]: You are reducing the blast radius of a specific breach as part of an incident. And I think in an era where we're building faster with AI-assisted workflows, we're really shining a spotlight on those things that aren't working well for us. KB [00:03:14]: In a way that we just haven't. And that's an interesting point because now, given where we are, AI accelerating everything and shining the spotlight, do you think that this will be more of the catalyst to customers saying, okay, well now we've gotta do something 'cause things are accelerating so fast? Or do you still think that mindset will still be there? Andrew Krug [00:03:36]: I think I'm gonna go with option 3 here, which is that it's neither of those. And a lot of the change in mindset will be driven through governance and legislation. So if we look at the amazing things that are going on in the APAC market with regard to data privacy and data sovereignty legislation, which has largely been inspired by countries in the EU, where regulation is going to force a lot of folks to kind of catch up as we roll out things like AI agents in, increasingly sensitive spaces. KB [00:04:09]: So there's that whole, that old saying, you attract more with honey than you do with vinegar. Do you think that perhaps we've tried the honey approach, but now the vinegar is the regulation and the compliance side of things? Andrew Krug [00:04:24]: I think that it's very attractive to think about delivering features very quickly because nobody shows up to work desiring to be the root cause of a data breach. They show up to work to deliver features that make their customers happy and ultimately contribute in some way directly to revenue. But with the pile of higher priorities that folks are often working with, we need a legitimate business justification to go through the security hygiene, the process, etc., that really makes a company breach resilient. KB [00:04:56]: So the industry spends a lot of time talking about sophisticated attacks, nation-state actors, and AI-powered threats. But you're sort of saying, Andrew, that many breaches still come down to the stolen credentials. Would you say we, the industry, are overcomplicating then the problem? Andrew Krug [00:05:17]: I would say that stolen credentials, particularly long-lived credentials, are one of the biggest gaps that we're still closing, especially since, you know, cloud adoption became really popular. KB [00:05:17]: Mm-hmm. Andrew Krug [00:05:30]: And if we look at the Datadog State of Cloud Security report year over year, this is actually one of the things that we report on is the trend in how many long-lived static credentials are still present, at least in the environments that we monitor. And while it's trending down, it's still not zero. There are plenty of alternatives for folks to eliminate long-lived static credentials. They just haven't really leaned into those. And the AI era has really brought that brought that back with API, long-lived API keys for things like the large LLM providers for foundational models. And so we're really seeing some of the things that we already learned in cloud security become problematic again. And so I suspect at some point we'll see that come full circle in the AI space as well. KB [00:06:16]: So you said it's trending down and it's still not zero. Do you think it ever will be though, or is that like super optimistic? Andrew Krug [00:06:23]: Absolutely think that it can rapidly approach zero, like almost entirely temporary credentials. KB [00:06:29]: Right, okay. And so when you spoke about long-lived credentials, talk to me more about, are you seeing that a lot in customers or where does that sort of sit with you? Andrew Krug [00:06:40]: So we count the number of IAM users inside of AWS environments or GCP service account tokens, for example. And those are credentials often that are used for machine-to-machine workflows in on-prem to cloud provider environments. So they're often highly privileged credentials that sit on a box that does something really important like infrastructure as code deployment. And those are the types of credentials that we're talking about. We're not talking about the engineer shows up to work and goes to sign in for the day, 'cause those are the ones that we are seeing trend down. Yeah. KB [00:07:17]: Really? So the old school manual. So, okay, so then why do you think that they're trending down then? Andrew Krug [00:07:22]: A lot of companies are moving to single sign-on workflows or they have very mature endpoint management for other parts of the business. And so they're able to rapidly onboard using standards to things like AWS's Identity Center or SAML or OAuth Flow in the case of GCP. So we've got the the developer experience down. KB [00:07:22]: Right. Andrew Krug [00:07:44]: It's really just that machine-to-machine or like the box in the corner that performs this really critical service that's still living in an on-prem data center. KB [00:07:52]: Right. So coming back to the breach, so when people think about a breach, they often sort of imagine some like super highly sophisticated attack chain, for example. So give me the real view though, like how often in your experience does this come back to like basic identity hygiene? Andrew Krug [00:08:11]: I think before we can actually talk about that, we have to talk about what counts as a breach. KB [00:08:11]: Sure. Andrew Krug [00:08:18]: Because different companies will have different internal definitions of what is a breach. So if we were to look at something like healthcare, where the data is very, very sensitive, accessing a single email that contains patient information in an email account could be considered a substantial breach. That's gonna be different for a software company or a company that collects telemetry. So it's a spectrum. And yes, long-lived static credentials are a big part of that. And even with 2-factor auth, sometimes we still see the ability to bypass. Malware has also got increasingly sophisticated. So if we look at things like stealer malware KB [00:08:18]: Mm-hmm. Andrew Krug [00:08:58]: That can pop on a machine, take a session cookie and then leverage that in a way that is a complete bypass for 2FA. We have all kinds of initial access vectors that are really, really relevant in this conversation. KB [00:09:12]: 'Cause one thing I've observed in this space is so many reports still say like malware's still like number one or like quite high on the list. Would you agree with that? Andrew Krug [00:09:24]: Certainly malware that's delivered as part of a phishing campaign or I think this year we've all been surprised in the rise of software supply chain malware. Sure. So I would never have predicted last year that we would already have the number of incidents that we've had in 2026 for software supply chain. KB [00:09:41]: Why do you think that is? And I only ask that because, you know, the supply chain is definitely, the last probably 2 years, been a very hot topic of conversation. But what do you think the change is recently? Andrew Krug [00:09:53]: I mean, certainly we could blame AI-assisted workflows for this. And I don't like to blame AI-assisted workflows because really what they're doing is they are highlighting a problem that already exists. And that is that the upstream repositories themselves don't really have the guardrails or the scalability to vet packages when they're committed to those repositories. So if we just look at how people are consuming packages in all these ecosystems, the guardrails just don't exist. KB [00:09:53]: Mm. Andrew Krug [00:10:24]: And then on the consumer side, they have never really had to be as concerned as they are today with kind of gating those things, using SBOMs, using package repository proxies, creating things like allowlists. So it's a complex ecosystem to begin with, and it's become more complex as malware authors are generating this very bespoke malware with AI and then getting it into the ecosystems. But I don't blame AI for it. I will cite it as a reason for the pace. KB [00:10:24]: Right, of course. Andrew Krug [00:10:58]: But it's not the root cause of the problem. KB [00:11:02]: So what's interesting there is people, companies, vendors, et cetera, are just talking about reducing the complexity, but things are super complex. So what do you think people are generally doing to reduce that? Is there anything that comes to mind? Andrew Krug [00:11:14]: I mean, there is the reduction of complexity, but there's also the very large scale that we're at. KB [00:11:14]: Mm-hmm. Andrew Krug [00:11:21]: So in a small environment, it may have been practical to simply reduce the complexity of that environment. But enterprises, nation states, nonprofits even have increasingly complex infrastructure where we could deliver the best system in the world and it could be not complex, but we could also have a problem prioritizing the order in which we should act in the case of an incident. So when I started my career, the axiom was you can patch everything, like just patch everything on Patch Tuesday or whenever the vendor rolls out patches. We just can't do that anymore. We have to accept that and prioritize. KB [00:11:59]: Okay, so staying with prioritization, I'm hearing that a lot as well. Do you think that people sort of know what that looks like or do you think it's hard because depends on who you ask, like tech risk is prioritizing one thing versus business risk versus the business in general, which is business continuity. How do you all as a company get on the same page? That way you're sort of walking in lockstep? Andrew Krug [00:12:21]: I think business risk has to dictate the technical risk that you pay attention to. And that's like, that's the first part of any equation, custom or otherwise, that you're going to create involving the creation of a risk score. And then we work backwards from that into the technical stack. From there, you know, you have parts of the business that are either legislated or critically important for what is the crown jewels of that business. And then what we do is we look to reduce the blast radius of any single incident by mitigating risk. And that's a lot of words to just say your custom score should be driving what is a priority for you. KB [00:12:59]: Okay, so I want to get into this a bit more because it's super interesting because I've worked on the practitioner side historically. And so we go to these meetings, you know, like tech risk, security risk, business risk, they were all kind of equal footing. But now you're sort of saying that like business risk really has to inform the others of the risk. So do you think that that hierarchy's changing now? Andrew Krug [00:13:24]: I think that security teams are better at understanding that business risk is going to help them achieve their goals faster. KB [00:13:24]: Got it. Andrew Krug [00:13:32]: So this generation of CISOs that we have sitting in leadership roles today is really a business-centric role that's working closely with whomever they report into, whether that be the CTO or in some cases the CFO. CISOs sit in different places. They're really beholden to the business first and then using that to drive their security program. KB [00:13:56]: So I wanna just come back to identity hygiene again. Why do you think businesses continue to sort of neglect identity hygiene? And what's interesting is a lot of my conversations and interviews, people are saying, it's always about the identity, it starts with the identity. But as we know, that sort of remains one of the biggest causes then of breaches. So if it's the biggest cause, why aren't we sort of fixing it? Seems to be overlooked. Andrew Krug [00:14:21]: I think identity in general is a very difficult problem to understand and quantify. KB [00:14:21]: Sure. Andrew Krug [00:14:26]: At least in simple terms that you could go into a boardroom and present. There's also the concept of the identity that you have in daily life and a digital identity. KB [00:14:26]: Sure. Andrew Krug [00:14:40]: And these are really different just in their paradigms. In your daily life, you do not present temporary credentials I would bet very often, to gain access to something or to get some superpower that you get to do for a limited period of time. It does exist. It's not super prominent. So even just as humans, we have a hard time relating to the concept of ephemeral access, digital identity, least privilege access. KB [00:15:07]: Yes. Andrew Krug [00:15:08]: And so just having the conversation, I think, is something that we have to get better at. before we'll see a lot of companies adopt platforms that help them drive down things like identity risk. KB [00:15:20]: And then also to add on those, like machine identities, like there's like thousands of those then. So do you think that companies are trying to find the balance between all of these things and there's so many of them now? It's not a manual process anymore. Andrew Krug [00:15:34]: Yeah, machine identities are incredibly complex in the world that we live in today, especially if a machine does more than one job. And generally the systems that you have that are running your software platform are each going to have access to a variety of sensitive datasets. And when we talk about bringing agents into that, it gets even more complex because agents often operate with the maximum potential permission set that they need for their specific job. KB [00:15:34]: Yes. Andrew Krug [00:16:03]: And then we're reliant on something that's non-deterministic to decide, should I or should I not have access to that dataset? So we're still kind of seeing that space unfold in a way that is going to allow engineers to downscope permissions appropriately. KB [00:16:19]: So when you say that space unfold, and I'm hearing a lot about deterministic and all this sort of stuff, people, it's filtering through now in the interviews. How do you see that space then unfolding as of today and then the rest of the year, for example? Andrew Krug [00:16:37]: So the AI agent space is following some of the patterns that we've been using for machine-to-machine authentication and computing forever. KB [00:16:37]: Mm-hmm. Andrew Krug [00:16:45]: Right? We have a single machine that has a specific job to do, and it only has code that does that job. When we talk about agents, like, agents are going to interpret the request, and then they're going to carry out whatever that model believes is the next best set of actions. So it's, it's gonna go potentially decide that it needs access to a sensitive dataset, for example, and it's going to try to gain access to that dataset using the tools that it has available. So really, like, we're kind of seeing a reality unfold that, that we haven't seen prior in software engineering where you're, you're writing a prompt. KB [00:16:45]: Mm-hmm. Andrew Krug [00:17:24]: Which is gonna go ahead and do something that you can't necessarily predict with something like a unit test. Or an integration test or a pen test. KB [00:17:31]: So I want to move on for a moment. One statistic that jumped out for me from your Datadog research was the number of access keys that are more than a year old. So walk me through that. Why are so many organizations still sitting on like the long-lived credentials? I know we sort of touched on it before, but I want to get into this a bit more just to really drum that point. Andrew Krug [00:17:54]: So there's kind of a couple use cases here for those more than a year-old long-lived access keys. Like one could be that they're just not taking care of it. The other could be that they have introduced a second-factor authentication control. And so rotating that key becomes less important because it is protected with a restriction that they have to apply two-factor access. in our State of Cloud Security report, we don't necessarily differentiate between those 2 use cases. KB [00:17:54]: Mm-hmm. Andrew Krug [00:18:19]: But I think we can assume in many cases that these are actually just long-lived engineer credentials that are still somewhere performing a service, and maybe those teams just don't know what service they're performing, so they're afraid to disable them for whatever reason. 'Cause that's another challenge in identity and access, right, is that if I downscope something or I disable a long-lived access key, the fear is often that I will break the business by trying to do the right thing from a security perspective. KB [00:18:48]: Is that true though? If you just turn something off? Andrew Krug [00:18:49]: It can be. Yeah, it can be true, but there are all kinds of ways today that we can know what that credential is doing. And so observability becomes a very critical part of understanding like, is the key in use? If so, what is it in use for? How do I replace it? And then what is the inventory of places that I need to go in order to get that out of the environment. KB [00:19:14]: So when you mentioned before, perhaps people not taking care of it, if there's that observability function, are people not using that or what would be your sentiments here? Andrew Krug [00:19:25]: I don't think that most teams in even mature companies are focused on their identity risk tools just because there's so many things above the watermark of like, these are my priorities that I need to deal with first. Like either because they're a compliance issue, they're a software security issue that's preventing the rollout of a feature. Basically, they're preventing the business from driving forward, where identity risk is just kind of waiting quietly until it's exploited someday during an incident or a credential leak that becomes an incident. KB [00:20:02]: Okay, so let's shift gears for a second. You've said identity is becoming the new perimeter. What does that actually mean in a world where there isn't really like a traditional like 4 walls perimeter anymore? Andrew Krug [00:20:16]: So we have said this phrase, identity is the new perimeter, which really just means that cloud service providers, the hyperscalers, their APIs are always available. So if you think about the data center days, you know, '90s into the 2000s, what do you do when you want to secure something? You build a fence around it. Cloud is always on, the API endpoints are always available to anyone that has access to that credential. So it's much easier to leverage. And that's what we really mean when we say identity is the new perimeter because the scale and scope of your credential is going to always dictate the potential maximum blast radius of a single credential leak. KB [00:20:54]: Do you think customers or clients Do you think they're aware of that or do you still think that they're thinking a little bit more like in the 4 walls, for example, despite all the conversations that are out there, what you're saying, that sort of common theme in the market? What are your thoughts? Andrew Krug [00:21:12]: Yeah, I think this concept that cloud service providers are always on and always accessible is new to a lot of people and we see it not just on the credential management side, we see it in the way that services are configured. Because services are designed to be public by default. They are not designed in the same way that we thought of them in the data center where if somebody doesn't have physical access to plug into a port in the corporate network, they do have the ability to get to that at a network level. KB [00:21:41]: Do you think as well, just zooming out, out of sight, out of mind, do you think that, I know that sounds silly, but do you think that that sometimes comes into it perhaps? 'Cause like I'm not physically going to a data center. And then if you can't see something, perhaps it gets a little bit relegated. Andrew Krug [00:21:58]: Yeah, I think there is that out of sight, out of mind aspect, which is why I think it's essential that we make that risk visible. We also have to make that risk visible in a way that is understandable. So surfacing identity risk right alongside other findings in terms of like how somebody might exploit an attack path or a complex set of steps through a system is really, really critical to bubbling that up to the top. KB [00:22:27]: So every human identity, every machine identity, and every service account potentially becomes an entry point into the business. So how does that fundamentally change the way security leaders need to think about their architecture now? Andrew Krug [00:22:43]: I think the old quote, attackers think in graphs, is really relevant to that conversation. And so in order to better defend, we need for our cybersecurity leaders, our engineering leaders to think in graphs. And when I say that, think about the ways that something can be abused to get to the maximum potential privilege. We actually just launched a project called pathfinding.cloud on Datadog Security Labs, which is a communication tool for helping people understand these pivot and escalate style identity attacks. So how do I go from entry level, like maybe I could only read something, to full administrator access inside of the account? How many steps does that take me? How hard is it? KB [00:23:24]: Okay, so talk to me a little bit more about that. I know I've been to the keynote, there's a lot of announcements coming out in terms of features, etc., but maybe walk me through a little bit more. What does that sort of mean for people? Andrew Krug [00:23:40]: So it means that they have access for the first time to a database where they can quantify different types of identity risk, either starting with a human identity— KB [00:23:40]: Right. Andrew Krug [00:23:49]: And pivoting to a very privileged service account role perhaps, or starting with a machine identity and gaining access to a sensitive dataset, for example. It's like, what are the potential paths that defenders should be looking for when they think about something like detection engineering? KB [00:24:05]: So then with that, would you say a lot of some of these problems that we've discussed at length today would be heavily reduced? Andrew Krug [00:24:13]: I think it starts with understanding and education, and then we can back ourselves into the strategy of how do we use the existing tools that we have in order to defend. KB [00:24:24]: So with the education piece, what do you think's still missing then? What, what is it that we can discuss here today that perhaps people don't know, or that they can walk away knowing more about it? Is there something that I'm not asking you perhaps? Andrew Krug [00:24:37]: I think it's about just growing that literacy around like, what is identity risk? What are— how does somebody even exploit or leverage one of these things, which is what the project is designed to do. It has a series of lab environments, for example, that teams can spin up so that they can play the role of the attacker and practice one of these complex privilege escalations. But it's also in being able to articulate the risk and the potential detection or defense. Because one of the great things about AI is that we live in a world where we can describe our intent and that intent can be interpreted and actually actioned into security controls. KB [00:24:37]: Right. Andrew Krug [00:25:18]: Without having to onboard another platform or another system, we can really live in an intent-based security kind of reality. And I think that that is the coolest thing about today is just that one statement. KB [00:25:34]: Intent-based security reality. Andrew Krug [00:25:36]: Yeah, where we describe our intent and then that is somehow just like carried out by the complex system that we have. KB [00:25:43]: Wait, when you say somehow carried out, what do you mean? Andrew Krug [00:25:46]: I mean, you have an AI agent or a foundational model that's actually going through and implementing that for you. So if you think about something like detection as code, where detection engineers are writing rules for their SIEM. Previously, you know, they had to have mocked data and they had to understand a complex set of languages. Today you can just hand that sample data to an AI agent, you can have it propose detection rules and very quickly go through a battery of tests at a velocity that you just previously couldn't. So we're able to cover a lot of things much faster by just describing our intent. KB [00:26:19]: So, Andrew, we're seeing more organizations adopt data perimeters, and multi-account cloud environments. Are we finally seeing companies redesign their environments around identity rather than networks nowadays, or? Andrew Krug [00:26:34]: I think when it's greenfield, the answer is yes, especially if you're in a regulated space. If you're coming from an existing space, it's much harder to back yourself into because let's say that you're a company sitting on petabytes of potentially sensitive data. KB [00:26:34]: Mm-hmm. Andrew Krug [00:26:50]: Even the act of classification for that dataset can be incredibly expensive. So we tend to take a bunch of half measures prior to actually going through and doing a very heavy classification effort, like humans manually label data, which we know doesn't work. So we're, we're really coming up with the systems that enable us to build the strong perimeters today. But I think a lot of that is going to be platform moving forward. KB [00:27:17]: So talk to me a little bit more about platformization. I know that we lived in a world where we outsourced everything, then we went to point solution. Now there's tool sprawl, AI sprawl. Now we're coming back to platformization. People want to reduce complexity, etc. How will that continue to shift then moving forward? Andrew Krug [00:27:37]: It really starts with the engineering teams, and I think the best representation of that has been the adoption of developer platforms, so IDPs, where we start with creating the software catalog, we start with the inventory of assets by design, we're observing that, we understand the data classification, and then when that is deployed and running in production, we have the runtime context to know what data is flowing through those systems. Because until we can classify the data, we actually can't create the cloud service provider level data access controls. KB [00:28:14]: So what I've heard as well and observed, people talk a lot about data classification is quite difficult to do. Would you agree with that then, especially if you're looking at like a bank that is like super legacy old data from like 100 years back? How do you sort of think about that then and how companies can start to modernize their data, would you say? Andrew Krug [00:28:35]: I think some data classification is very, very easy because if you look at modern datasets, the bank example is interesting, you know, if we talk about OCR and scanning old records, but in modern datasets, sensitive data tends to follow predictable patterns that we don't need AI to detect. So we have a product, for example, called Sensitive Data Scanner. KB [00:28:35]: Okay. Andrew Krug [00:28:56]: Sensitive Data Scanner finds data that's potentially dangerous in logs, and it's able to sanitize those or alert. That's based on a database that matches patterns in datasets. So some percent is very, very easy. Some data understood in context, much more complex. KB [00:29:20]: So when you say understood in context, tell me, can you elaborate on that a little bit more? Andrew Krug [00:29:20]: So articulating the why that piece of data is sensitive. So if we look at your banking records example, that could be optical character recognition from a photo or a scan. So first we have to be accurate in detecting what the data actually is. And then we have to be accurate in identifying whether that data is— what level of sensitivity do we have to put on that data? And a lot of that is going to be around what type of document is that. So the exercise can be many steps or it can be very simple. KB [00:29:58]: Do you think currently where it is at the moment, that's a little bit of an arduous process? It's not super automated yet? Andrew Krug [00:30:00]: It is an arduous process, but with AI-assisted and machine learning workflows, we've been able to seriously up the game in terms of data classification. KB [00:30:10]: And when you say up the game, just means in terms of process speed, how quickly it gets done? It's, you know, because it's one of those things where it's like people should be doing it, but then it's like, oh, it takes so much time. Where do I start? It's overwhelming. And that's probably when people feel overwhelmed, they just don't do it. Andrew Krug [00:30:25]: Yep. That's the evolving data security posture management space or what a lot of people are calling DSPMs now, and those are going to sit at every layer of the stack that exchanges data. KB [00:30:40]: So zooming out now for a moment, what's the biggest item security leaders still aren't willing to admit about identity risk? Like perhaps when you're out talking to customers or in the field, what sort of comes up for you? Andrew Krug [00:30:47]: I think it's really that, again, that prioritization conversation is like where if I was to ask a senior leader one question, it would be, where is identity risk on your priority list this year? And you can't ask broadly. It has to be this year. Where is it in your list right now? KB [00:31:08]: Where is it on the list? Andrew Krug [00:31:10]: And if it's not number 1 or number 2, we're probably not getting to it in that year. Like most CISOs, for example, have a top priority for that year. Guaranteed, they're going to solve their top priority. And then maybe initiative number 2 or number 3 is going to be a little bit further down the road. KB [00:31:28]: So with the prioritization of identity, would you say generally speaking it is top of the list or not yet? Andrew Krug [00:31:36]: I don't think often it makes the top of the list. KB [00:31:40]: Why do you think that's the case? Andrew Krug [00:31:42]: Either because there's some other burning issue that the business really has to deal with, like maybe it is something that's been surfaced in a pen test or a compliance attestation that they still haven't gotten around to because those have direct impact. For every year that you don't take care of that thing, you're either being penalized or fined in some way that's forcing you to put that at the top of the list. KB [00:32:06]: And that's so interesting because every time I'm interviewing someone like yourself, they're talking to me about a wide range of security problems, issues, items, and I'm like, oh, that should be number one. But then the next person I interview, I'm like, well, that should be number one. How do you think like CISOs or security leaders, how do you think they'll get to the point where they're like, this needs to be top priority? I mean, they could listen to this conversation, but I mean, is it something that they'll get to on their own or is it just that they'll come to that realization with having multiple issues perhaps that perhaps overturn some of the other burning cybersecurity problems? Andrew Krug [00:32:40]: I think sometimes it does in fact take a really impactful breach to bubble that up to the top. If you don't want to have to have a really impactful breach to bubble it up to the top, a tabletop exercise where you go from assuming that one of these things was true and really examining what the impact would be can help with that conversation of why we should bubble it up to the top. Because the CISOs are technical enough to know and understand that it is a risk. KB [00:32:40]: Mm-hmm. Andrew Krug [00:33:09]: But they need consent from their other partners across the entire organization to either staff that, prioritize it if they can't get additional staff, onboard a platform that helps surface and remediate that risk. There are a number of options. It's just about having the conversation to get the buy-in from everyone else. KB [00:33:30]: Okay, so I'm curious, given your advocacy role, how would they get the buy-in? And I know it sounds like a really rudimentary question, and we've been sort of asking the same question here for years, But what do you sort of think, how would they all agree like this makes sense? Andrew Krug [00:33:44]: Yeah, again, I'll just like reference the tabletop exercise is a really great way to ensure that a leadership team is speaking a common language because helping everyone develop the vocabulary is enormously impactful in just having the conversation. The second is really just like literacy in the engineering teams, right? Because engineers generally want to do the right thing. So raising awareness for those things even down the chain so that we can reduce the potential surface area of those identity incidents early on in the software development lifecycle, that's really critical. KB [00:34:20]: Which then enforces up the chain. Andrew Krug [00:34:22]: Yes. As a result. So if the issues aren't making their way into production, we don't have to deal with them later. KB [00:34:22]: Right, right, right. Andrew Krug [00:34:28]: So there's kind of like that 2-prong strategy is we need one to deal with the legacy systemic risk, but at least initially there are guardrails education that we can put down in the software development lifecycle that will at least stop the bleeding. KB [00:34:44]: So perhaps to wrap up, Andrew, a lot of announcements this week and maybe stepping back from some of the individual announcements, what do you think the biggest story— what is the biggest story Datadog is trying to tell this week at Dash, would you say? I know that's a big question and there's probably multiple answers, but what really comes to mind? Andrew Krug [00:35:04]: I think the top message for me is really that we're here to help people with their AI workflows, whether that be using AI inside of the Datadog platform to be better users of Datadog and solve problems faster, or observing their AI workflows. So it's kind of a two-pronged approach. I'm incredibly excited about some of the agentic discovery announcements because I think that is a problem that a lot of people are going to struggle with as they roll out agents. Sort of this idea that they could be anywhere and everywhere inside of your organization. How do you quantify the risk if you can't observe it? KB [00:35:42]: Of course. So talk to me a little bit more about the agentic announcements, perhaps. Andrew Krug [00:35:48]: They kind of fall into those 2 categories. One is the observability of AI and the other is how AI can assist in the product. Those are the 2 broadest categories. On the security side, the AI-assisted workflows are largely helping with triage and consolidation of information. So if you think of all of the toil that security teams go through to correlate single data points during an incident, now with something like Bits AI Security Analyst, we can do that in minutes, which is amazing because you have all that information available in one place. On the agent observability side, I think we're getting ahead of solving real problems in helping folks identify what models do they have, how much do they cost, where are they deployed in the business, what data they have access to. So those are, those are kind of how it breaks down for me and what I'm most excited about. KB [00:36:40]: And then maybe just one other quick question before we depart would be final thoughts, summary, closing comments. What can you share? Andrew Krug [00:36:48]: I think we're all going to be surprised at how rapidly the pace of all this is going to increase. So just predicting things will go faster and faster, and we probably won't see these issues lessen before we see them magnified further. KB [00:37:11]: Andrew, thanks for your time. Andrew Krug [00:37:17]: Thank you. KB [00:37:28]: Joining me now is Nathan Pitchaikani, Senior Security Engineer at Riot Games. Nathan, great to have you here. And today we're talking about threat intelligence, incident response, and what it takes to respond to emerging threats at a high gear in today's world. So Nathan, thanks for joining me and welcome. Nathan Pitchaikani [00:37:46]: Yeah, thanks for having me. Excited to be here. KB [00:37:49]: So I want to start with threat intelligence because it's one of the most talked about topics in cybersecurity. Well, I mean, there's many of them, but one of them— everyone seems to have a threat intelligence platform, a threat intelligence feed, or a threat intelligence strategy. So perhaps let's start right there. What does threat intelligence actually mean in practice to you at Riot Games? Yeah. Nathan Pitchaikani [00:38:10]: I think that's a great question. I think, you know, threat intelligence at Riot Games is really trying to understand, you know, what's affecting us in the landscape, right? Sort of looking at what I like to call like threat data and sort of like, you know, what's important artifacts, like whether it's, you know, IOCs, IP addresses, domains, file hashes, and sort of like getting a lot of that telemetry and then really understanding like actually how is it related to us? KB [00:38:10]: Right. Nathan Pitchaikani [00:38:34]: Like we sort of want to filter that and be like, you know, is it related to the gaming scene, environment, the entertainment industry, and is it useful to us, right? So I think that's like the first thing. We have a bunch of different ways of doing this. We use an open-source tool called MISP, which is sort of like our data warehouse for a lot of our threat intelligence. Other ways of doing threat intelligence, we have started really using AI recently because, to be honest, I think with all the supply chain attacks that's been happening recently, it's sort of impossible these days to sort of manually as a human sort of keep up with everything that's going on. So we started using AI basically to sort of start pulling in, giving it basically like articles, giving it feeds, giving it websites, giving it Twitter accounts, and a bunch of other different things that we can use so the AI can go and scrape and sort of pull the data from there. KB [00:39:27]: Okay, so you mentioned something super interesting to me, which was, is it useful to us? Nathan Pitchaikani [00:39:27]: Yeah. KB [00:39:33]: So what does that mean for you specifically? What do you find useful? Nathan Pitchaikani [00:39:37]: Yeah, so I think like there's 2 aspects, right? There's like what's happening in the industry, right? Like in the industry right now, again, we're seeing supply chains being really, really prolific. We're seeing a lot of ClickFix stuff. We're seeing threat actors be a bit more aggressive, I'd say, third-party compromises. So if we see threat intelligence related to those things or we see threat intelligence related to threat actors that we're actually monitoring, Those are things we generally care about, and we want to make sure that if anything new happens, we're on top of it. KB [00:40:03]: Okay, so anything new that happens, obviously these things can happen super quickly. How do you then become on top of it then to countermeasure potentially what's happening quickly? Nathan Pitchaikani [00:40:13]: Absolutely. Like, so I think that leads into like what I was talking about with like our AI thing. One of our engineers on our team, Orlando, he built something really cool, which basically, like I said, it sort of grabs like Twitter feeds, it grabs accounts, goes to certain like websites. You can give it like Deepa Report and other sources, and it's constantly scraping that. KB [00:40:36]: Like OSINT capability? Nathan Pitchaikani [00:40:37]: Yeah, sort of like an OSINT capability, right? Except it's really focused on threat data and not necessarily like looking up like some user's information. So it's grabbing like IP addresses and file hashes, and then it can sort of focus on certain things we care about because it's AI. You can sort of give it like sort of like game relevance. Maybe there's articles that are related to stuff in the gaming industry, like maybe like the Discord or maybe some other gaming company. If they had something security related, it'll catch that. And the idea that it's constantly running, right, and constantly grabbing this data makes our lives so much easier. KB [00:40:37]: Right. Nathan Pitchaikani [00:41:07]: Now there's a second aspect to this that I think is important to talk about. It's grabbing the data, right? But the next level of that is it needs to scan our environment, right? So it's grabbing that data, it's then going out to our tools via MCPs and actually checking to see, hey, these IOCs, these IP addresses, are they actually existing in our environment? And then based off that, it'll either alert us or it'll just sort of let us know that, hey, these articles exist, might be interesting to check them out. Orlando, again, a member of my team, did a really, really fantastic job building this, and it's something that's been very useful for us. KB [00:41:39]: Gathering all this information, like IP addresses, etc. Do you think that can help as a preventative mechanism? Because now I'm starting to hear that come through my interviews around like preventing these things because we've got the intelligence to do it. And then I ask someone, well, why don't we do that before? We didn't have it. Would you say now we've got it or do you think we've got better tooling to capture that? Nathan Pitchaikani [00:42:00]: I think before we could definitely do it, but I think it would be a lot more, a lot harder of an engineering capability to do a lot more like coding and software engineering, right? But now with honestly like AI, there's still that aspect of like coding and saying, hey, go look at these things, but the AI can actually analyze the different datasets because articles are all in different formats, right? Like it's all natural language, like something's gonna have to understand that. If you tried to manually code that, that's going to be really tricky, right? Like you could probably do some like regex and try and grab like IP addresses or domains. KB [00:42:00]: Yeah. Nathan Pitchaikani [00:42:32]: But there could be totally false positives here. And now with AI, you're sort of able to really get that context and have that confidence. And I think that's where we're at, right? We're seeing like Claude and Anthropic do a really good job and a lot of people are being able to build things that they realistically weren't able to build before. KB [00:42:47]: Fair. So talking about false positives, that's something historically we've spoken a lot in the industry when it comes to like alerts, etc. So do you think with AI it reduces a lot of that noise then? or do you still think there are some false positives? Nathan Pitchaikani [00:43:02]: That's a great question. So let's talk about it in terms of threat intelligence before we go into general things. I think that it's a fun one otherwise, but I think from what we've seen so far, I haven't seen too many false positives. And what I'll also say is like, I'd much rather something be a false positive than not have access to the data, right? As long as I'm not getting blasted with false positives, right? There's always an acceptable limit of like, you know, you have a few false positives and you're okay with it. KB [00:43:28]: But you're saying that's rare, yeah? Nathan Pitchaikani [00:43:30]: Yeah, yeah, yeah. For us, generally, that's rare and we haven't really seen any issues. So with what Orlando's built, we've seen a lot of actual production-based value. KB [00:43:41]: Okay, so when you say production-based value, tell me more, like, what's your definition? Nathan Pitchaikani [00:43:46]: So what I mean by that is more of like that we're confident that this is not something that is sort of just in development, it's something that is actually working in our environment. It's actually grabbing articles that are relevant to us or just articles in general. It's filtering the IOCs, it's checking our environment for them, and it's letting us know, right? Generally, like our dev to production is sort of like something's in development, it's been there for a bit, it's, you know, we're testing it out, we're seeing if it works well. And if it does, then we sort of tend to move into production, into our alert pipeline. We haven't fully moved it into our alert pipeline yet where we like actively triage it, but it's still, it's like about there. KB [00:43:46]: Mm-hmm. Nathan Pitchaikani [00:44:22]: And so far, like looking at it, been pretty nice. KB [00:44:25]: Okay, so staying on this talk track, do you remember when people were talking a lot about like alert fatigue? Nathan Pitchaikani [00:44:25]: Yeah. KB [00:44:30]: Because there were so many alerts and then they had to look through them, sift through them, trawl through them, all the things. But now with AI and what you're saying here, Nathan, it's reducing it. So does that mean the alert fatigue will dissipate or do you still think it'll be there regardless? Nathan Pitchaikani [00:44:45]: Yeah, I think it's a great question. So I'm gonna answer this in 2 parts. So I think there's an aspect of it will— AI can help reduce false positives, right? Because it'll, you know, note that it's something that's benign. But I think AI itself will also create false positives. KB [00:44:45]: Okay. Nathan Pitchaikani [00:45:04]: And what I mean by that is what we've sort of started seeing in the industry. I've talked to quite a few other companies about this, is a lot of people are vibe coding these days and they want to build things, but they don't really understand what's actually happening under the hood. We've had situations where, you know, someone was vibecoding and it basically built a credential exporter for AWS creds. And obviously detections fired off of that because someone's building something to export AWS credentials. And, you know, it's an interesting case because when you look at that, you're like, you get the alert and you're like, wow, this is really weird. KB [00:45:04]: Yeah. Nathan Pitchaikani [00:45:34]: And then you look at the process and you see like Claude and you're like, okay, AI, sure, someone's probably vibecoding. But then you ask yourself, the person vibecoding, or is this maybe endpoint compromise and a threat actor is abusing AI to do malicious actions? And that's the thing where now I think we're going to have a lot more of a human element where we're going to be having to reach out to people. Like, we'll have to make sure that that user maybe on Slack, whatever communication method— sorry, English is hard— is not compromised, right? And then sort of be like, okay, he's not compromised. Let me reach out and verify with the user and sort of have that trust. Like, hey, were you vibecoding? Does this look familiar? And the person's like, well, in one case, the person was like, I don't remember doing that. And then he's like, let me look at my code. Oh yeah, that is actually what it did, right? And didn't even know it did that, right? And so there's that interesting thing where you have people who are, you know, doing technical things, but they're not aware of what's happening. KB [00:46:28]: Okay, so what's coming up in my mind is we closed one door with AI to reduce alert fatigue, but then we've potentially opened another door because of people doing vibe coding, which then it looks like a perceived threat. Nathan Pitchaikani [00:46:43]: Exactly. KB [00:46:43]: So which one's better or worse? Nathan Pitchaikani [00:46:46]: It's a great question. I think I would much— I would— I think this is better. And the reason why I'm saying that is because I feel like we're able to shift left more to be able to be more proactive with threat intelligence and catch things faster and block things hopefully faster. I don't think we're fully there yet, but I think in the next 6 months, year, it'll be a much more proactive threat intelligence, grabbing data, proactively checking environments, proactively making detections, making sure you're secure. KB [00:46:46]: Mm-hmm. Nathan Pitchaikani [00:47:14]: I think companies are still working out the kinks, like figuring out is this exactly how everyone should do it? I think we're still in that test phase, right? Like where we're experimenting with it. And that's that proactive aspect, right? Then there's sort of the reactive aspect. We have these detections coming in, right? Right, which are false positives and coming from AI. And that's what I'm sort of saying. I'd rather us be proactive, right? And like catch stuff and prevent stuff early. If detections misfire, that's okay. But at least now I'm checking telemetry early, right? And I'm not missing something that could happen. KB [00:47:45]: So staying with this, would you say, you mentioned before the next 6 to 12 months. So would you say that we're sort of front-loading some of the risk and then in the next 6 to 12 it'll start to sort of die down then? Nathan Pitchaikani [00:47:57]: Oh, it's a good question. I think it'll depend. I think with like, again, threat intelligence, 100%. I think with like the vibe coding aspect and like people vibe coding and then detections getting firing and whatnot, I think we'll definitely see. I think companies will also sort of like mature on how they use AI in general. I'm sure you've heard like multiple companies have like blown AI budgets out the window because there was no control, right? Yes, with the tokens. Yeah, exactly, right? Like, so I think there'll be a more maturity stance about AI. I think a lot of people are in this like experiment phase. Nathan Pitchaikani [00:48:31]: It's like, hey, here's AI, you go figure it out. You go figure out how you should use it. You go figure out how it should benefit you. And because of that, there isn't really the best security guardrails around it, right? KB [00:48:31]: Right. Nathan Pitchaikani [00:48:41]: Which then affects detection engineering and detections because since there's not good guardrails, you're gonna see it sort of go haywire and touch things that it shouldn't. KB [00:48:47]: Okay, so going back, blowing the budget. So do you think because people have blown the budget, like I was reading like one company blew the budget in a month for the whole year. Nathan Pitchaikani [00:48:47]: Yup, yup. KB [00:48:57]: So does that mean they're like, well, we've exceeded the budget. Does that mean that the pendulum will start to fall in the middle now because there's no more budget to be used? Like it's gone. Nathan Pitchaikani [00:49:09]: Yeah, I think it'll really depend on each company and what, you know, their stance is. I think the biggest thing is like, they're really, when you think about AI training and like how people are using AI to vibe code or do things, I don't think that training really exists very well. I think like, personally, I haven't seen anything. It's very much been like, go figure it out yourself, right? And a lot of the learning I've done is through talking to peers and mentors and figuring out what works for them and how they optimize things. KB [00:49:09]: Right. Nathan Pitchaikani [00:49:36]: And like what I've noticed is like if you optimize things really well, you can keep costs really down. But if you don't optimize things and you sort of are not, You're giving huge, massive paragraphs to analyze, big context, huge markdown files, right? And you're sort of just blowing basically token usage, then yeah, it's going to get expensive, right? But I think there's that learning aspect to that. We're just not, as people using AI, just aren't there yet. KB [00:50:02]: So there's been conversations in the industry around the whole ROI around AI. But then I've been hearing as well, because we're still in this experiment phase, it'll take a while before people start to get it. And then when they get it, to your point, they'll optimise it. Once they optimise it, it'll then start to reduce it. Yeah, but like, how long? I mean, people can't keep saying like every month we keep blowing the budget. Nathan Pitchaikani [00:50:02]: Yeah. KB [00:50:23]: So like, what's happening now? Nathan Pitchaikani [00:50:23]: I mean, I feel like in my opinion, it's again, I think every company is different. I'm not too sure about our company, but like how I feel like is if you're again doing things correctly, you're properly developing, I think it makes sense. My honestly biggest concern is that I don't, to my knowledge, I don't think AI companies are profitable right now, right? Like they, what that means is I think AI models are gonna keep increasing in price. Like we've already seen that with a few of the newer models and that prices are going up. So my concern is if they keep going up, right? KB [00:50:23]: Mm-hmm. Nathan Pitchaikani [00:50:56]: Then like, what's the value now? If it goes up to a point where it's more expensive than a headcount, like why use AI, right? That's another thing to think about, right? And I think that's something that a lot of technologists have started thinking about a bit too, is like, yeah, this is like super hype. It's like amazing. It's really great. But if we can't optimize our usage and the price goes up so much, like, we like— this is the cost of like 2 headcount. Like, does this make sense? KB [00:51:23]: Do you think it'll get to that stage then? Nathan Pitchaikani [00:51:25]: I'm really curious. I don't know. KB [00:51:25]: 2 headcount? Nathan Pitchaikani [00:51:28]: I mean, it depends. I mean, it just depends how much it keeps going up in price, right? Right? Like, I think right now, again, like, you know, AI models, what I've seen, like the newer ones, they're like 4x what it was before or 2x before. And if it keeps scaling like that, you know. KB [00:51:43]: So will that AI bubble burst then? Nathan Pitchaikani [00:51:45]: Maybe. Who knows? I don't have an answer. And I think that the biggest thing right now is between AI companies like Anthropic, OpenAI, they're— everyone's competing, right? Right. And they're also trying to be who's the best one, right? And that's where that huge thing is happening. KB [00:51:45]: Of course. Nathan Pitchaikani [00:52:00]: And I think like eventually maybe there'll be a winner, right? And then maybe there'll be some interesting costs and we'll see what happens with that. But maybe we'll know in a year, who knows? I think it's like really honestly a roller coaster. And I think because it's a roller coaster, I think that's where it's this thing where like people don't really know what best practices are. People are sort of just vibe coding, trying things out, and that leads to interesting security stuff being affected. KB [00:52:20]: So on that note, because of how these AI companies are so competitive, one minute one's a top dog, then they're not. Nathan Pitchaikani [00:52:20]: Yeah. KB [00:52:27]: So do you see the volatility continuing to go the way it's been going in terms of someone's on top, then they're not, then they're down the bottom, then no one cares, now someone cares again? Nathan Pitchaikani [00:52:37]: Yeah, that's a good question. I haven't messed with too many different models from different companies recently. I think there is, you know, certain companies or models are better at certain things, and I think it's very based off your use cases and whatnot. So, but I do think it's gonna— it's pretty competitive, I think, the market. I think there's, again, certain things where certain things are better in my opinion, but everyone also interacts with AI, I think, differently and has different preferences on how they expect development to work. KB [00:53:03]: So what's something that the security industry still gets wrong about it? So even well-resourced companies like Riot, for example. Nathan Pitchaikani [00:53:11]: Yeah, that's a good question. I think like one of the big things I'll talk about is sort of like the threat landscape is huge these days, right? With AI and sort of people like maybe making auto-crafted phishing emails, right? OSINT is like a thing. You can find out something about somebody, use AI to grab a bunch of information, make a really convincing email about all their hobbies, and send them an email, and then maybe get them to click on a link. And, you know, that's scary. And I think the thing is defenders these days are really focused in this aspect of let's react. Let's like, oh, it's threat intel. Let's make sure that we're on top of it, right? Let's make sure that we are, you know, getting the threat intel, checking our environment, making detections. Oh, this new technology came out. Nathan Pitchaikani [00:53:52]: We need to make sure we're watching this. There's a new vulnerability. And you're sort of in this thing of like, sort of like a Band-Aid, right? Like new things keep happening and you're constantly trying to fix it. KB [00:53:52]: Right. Nathan Pitchaikani [00:54:02]: But I think one of the things that also really matters is like, just take a step back, right? And actually understand what is our threat landscape? Like, what are we concerned about? What are our crown jewels? What are our things that are really important to Riot Games or any company, right? And like, do we have coverage for that? Like, yes, there's an aspect of like, you know, everything else that's going on in the world, right? But like, what's important to us? And do we have detections for that? Do we have coverage for that? KB [00:54:25]: Wouldn't you say most companies sort of know that? Like big enterprises like Riot, for example? Nathan Pitchaikani [00:54:29]: Yeah, I would hope like, you know, they have an idea of, you know, like what generally would you say like what endpoints are most important or like what infrastructure is the most important? Like what are their crown jewels is sort of a great way of looking at it. KB [00:54:42]: But would you also say if you zoom out at the landscape, it changes day to day? Like new things are happening, so perhaps one priority was good that day, that then it changes. The velocity now with AI and friends, it's changing faster than before. Nathan Pitchaikani [00:54:55]: Yeah, I think realistically, yeah, crown jewels can definitely change. I think at a high level though, they will still be based off what the company is for and what it's about, right? So not gonna go too much into Riot Games crown jewels per se, right? But for instance, you can sort of just think about it, like if any company, they probably care about the data that they have. Right, that's like maybe their customer data. Probably pretty important. They probably don't want that getting leaked, right? Or maybe internal source code, probably don't want that getting leaked. KB [00:54:55]: Facts. Nathan Pitchaikani [00:55:25]: So, you know, general companies will have crown jewels, and I think those, if you keep it at a very high level, I think that stuff won't change. Sure, maybe different departments will get spun up to work on different projects and whatnot, and you might have to keep up with that, which is tricky, 'cause development's a lot faster. But the general high-level thing I think will stay the same. KB [00:55:41]: You've also described threat intelligence as a critical part of modern response, but with AI agents now gathering, processing, and enriching intelligence at scale, how do you sort of see that changing forward? I know you sort of touched on it a little bit, but maybe let's go into this a bit more. Nathan Pitchaikani [00:55:57]: It's a good question. I think, again, it's sort of like what I alluded to a little bit earlier, right? Probably, I think we'll know in about 6 months to a year to actually know if something, if this works, right? KB [00:55:57]: Right. Nathan Pitchaikani [00:56:08]: I think like this idea that we're using threat, like we're using AI agents to sort of scrape and grab threat intel, make sure that it actually like is valuable to us checking our environment. If we see that like we're not necessarily getting true positives, but we're seeing that the data that it's scraping is accurate and helpful, then over a 6-month period we keep seeing that, then we know that we're in a good place, right? We know that this is something that can work and that we can keep using. KB [00:56:34]: Okay, so to check to see if it's accurate, do you use an agent to get the information and then use another agent to check it? Nathan Pitchaikani [00:56:42]: Yeah, so we can definitely do that, but also there's an aspect of manual review, right? So right now obviously we can automate everything, but I think what we can do is when we get that data, we can have it sent somewhere, we can review it ourselves and be like, hey, like, did the agent do a good job going to the different sources, grabbing the data we care about? Oh, it did. Okay, so sort of like a Q&A type of task. KB [00:57:00]: And then it will learn and know, and it just, so it iterates. Nathan Pitchaikani [00:57:00]: Yeah, exactly. KB [00:57:04]: Then your manual time will be reduced, hopefully over a longer period of time. Nathan Pitchaikani [00:57:04]: Exactly. KB [00:57:12]: So then do you envision that whole human in the loop, on the loop, will that still be a thing? Nathan Pitchaikani [00:57:17]: So I think the human in the loop will always be a thing. I think the matter that will be gone is this aspect of like going and grabbing data. And then like having it check our environments. I think once that's done, there'll be like a little report, right? What the human now is able to review. KB [00:57:17]: Right. Nathan Pitchaikani [00:57:31]: So it's sort of like the way I like to think about this, right, is sort of now you are like a manager in an essence, and you've got your little AI agent going doing a little task for you, right? Like your IC, and it's going doing things. Another way I like to think about it too, to be honest, like I'm an IC, right? So I do a bunch of projects. I now can do multiple things at once, right? I can work on something and I don't have to worry about like looking at threat feeds anymore. I can let that, like, let the AI thing handle it. I can work on building some new detections. I can work on some other stuff, and I can multitask. And like, as a person, I love multitasking, right? So for me, it sort of empowers me even more. KB [00:58:06]: Would you say that with multitasking, perhaps one aspect gets lost or gets more attention, and then there's a gap? Or, I mean, I'm just playing devil's advocate. Nathan Pitchaikani [00:58:14]: Yeah, no, 100%. I think like when you're multitasking, you have to be very careful, right? And I think that's why, before you do anything, you should always have peer reviews. Like, before we ever move anything into production, we ever like decide to actually make detection rules, we have a GitHub repo that we use for our detections. Same thing for like threat intel, and we have someone else look it over. So just because I think it's good, someone else should look at it, just because I might miss something, right? I'm human, right? KB [00:58:14]: Right. Nathan Pitchaikani [00:58:38]: And the last thing we want is something getting pushed, which then is like a mistake, and then that starts like crippling our environment. KB [00:58:40]: Well, it almost sounds like history repeats itself because that's what we used to do manually back in the day. And now we're using agents, but we're still sort of doing the whole peer review process anyway. Nathan Pitchaikani [00:58:52]: Yeah, it's just that we're not grabbing the information ourselves or manually checking our tools. And that, that process, from like my POV, would take me a bit of time, right? Like checking for news feeds and like going out and checking like my EDR tool, my SIEM, you know, my network data, my logs. That would take probably around 30 minutes, right? Not too long. If you're really fast, maybe 15 minutes. But 15 minutes every day. KB [00:58:52]: Repetitive though. Nathan Pitchaikani [00:59:15]: Yeah, and it's repetitive too. And to be honest, I personally don't like repetitive tasks. I get bored. So if I can automate things like that, and then I can use that time to honestly even just relax for a little bit, or think about some other project or work on something else, I think that's valuable time. KB [00:59:28]: And would you say, and I know people have spoken about this in the industry, that having the repetitive tasks, doing the same thing over and over again, that's where mistakes Mistakes start to be introduced. You're doing the same thing over and over, something's bound to happen because you're on autopilot a little bit. Nathan Pitchaikani [00:59:43]: Exactly. And that's sort of where, like, again, automation is there, right? And we sort of talked about it earlier where, like, realistically, we could have built an automation to do this. You could easily engineer something, some code that would go to sources and then sort of scrape it, grab the IOCs. But, like, how accurate would that be versus what we can maybe do now with natural language processing? KB [01:00:00]: Okay, so then, okay, so focusing then on speed, would you say that there's a risk we become so focused on speed that we start sacrificing like quality? Nathan Pitchaikani [01:00:10]: Yeah, I think it's a good question, and I think the answer I would say is like, I hope we don't get to that point, right? I think quality is always more important than quantity. I think that's like, in my opinion, in security that is very important. You don't want like 100 detections right, that are just like, eh. You want 10 detections that are very good. And that's something like I firmly believe on, right? And yes, AI might help, you know, us do more quantity, but again, part of that peer review and stuff aspect is where we can actually try and get that quality. KB [01:00:10]: Right. Nathan Pitchaikani [01:00:43]: For instance, like, you know, we use a CLI tool also to help generate threat hunting reports, and it gives suggestions for detections and, you know, scans our environments, and it's not bad, but when I look at it, I'm like, this detection could be a bit better. Like, it's not considering this and this and this, right? That's why we still need that sort of human experience, right? From someone who's like been in the scene, understands what it's looking at, to sort of make the detection higher fidelity and also have more context for other things it can correlate against. KB [01:01:10]: So you said before, I hope we don't get to that point, but would you say we are at that point? And I ask this question because now if you look at like a business, generally speaking, they're moving faster than ever before to adopt these tools. Like back in the day, In an enterprise, you have to go through all these people, peer reviews, people signing stuff off, go through 50 people before they'd purchase, procure a tool. Now it's becoming faster. And the reason for that, as you'd know, is to stay competitive. So would you say that people are just trying to get there faster then? Nathan Pitchaikani [01:01:41]: I think, yeah, people are. But like, I think the biggest thing is like, do you, does your company or does your, you know, have the policies or the tools to make sure that you're still having proper verification checks? And as long as you're having those checks, those guidelines to making sure things are getting reviewed before they go to production, people aren't just skipping a step. I recently heard a story about AI was doing something and it killed a production database, right? Oh my gosh. And it literally crippled a customer's environment, right? That's obviously not good, right? So that's why, again, peer reviewing and stuff like that, it's imperative. People might want to ship fast, and I think it's great they want to ship fast, But there has to be that verification aspect, and I hope that does not go away. KB [01:02:20]: Because even the last sort of 12 months, looking back retrospectively in a lot of my interviews, people saying like, ship faster, we've got to ship faster, stay competitive and stay on top of things. But again, is it because we're in this infancy stage of AI where it's like we're going to go gung-ho, but now we're going to pull it back a little bit? We've blown all our budget for the year. The pendulum will start to come back in the middle like now because the things that you're saying here today, Nathan, I'm starting to hear that trickle through in a lot of my interviews now. So are we at that point now in this AI journey where people are starting to perhaps, hate to say it, slow down? Nathan Pitchaikani [01:02:55]: I think so, right? I think before, like, everyone was like, oh my God, AI is amazing, let me use it to quickly build things. Like, startups were like, I mean, they're still sort of blowing up. You see startups constantly have new AI being used in different ways, right? But I think eventually, right, it'll be like, how much more can you do? Right, how many new things can you come up with, right? Like, yes, people are trying to go fast to stay competitive and work with other things, other companies, right? But again, I think that's a great way to like, for a new product or a new thing. But once you already have that set up, then it's like the quality, right? And that's where I think like it's gonna be a bit more slowing down a little bit, making sure the actual roots are strong. KB [01:02:55]: Mm-hmm. Nathan Pitchaikani [01:03:30]: Because when you go fast, the roots may not be strong, right? You're just, you're just sort of growing it really fast, maybe using some special fertilizer or something. I don't know. KB [01:03:30]: True. Nathan Pitchaikani [01:03:38]: Maybe not healthy. KB [01:03:38]: I wanna flip over now to the attacker's side. So as we know, attackers are now using AI to automate reconnaissance, which then accelerates exploitation and then move from access to impact in minutes really, rather than days, weeks, et cetera. So does that fundamentally change what defenders need to prioritize then day to day? Nathan Pitchaikani [01:03:38]: Yeah. KB [01:04:01]: And again, that sort of ties back into the speed. I know it's a little bit of a hard question to answer. Nathan Pitchaikani [01:04:01]: Yeah. KB [01:04:06]: But I think it's, it's a needed one. Nathan Pitchaikani [01:04:09]: Yeah, 100%. So I think like to your question, right, asking about like how fast attackers are able to move and defenders sort of have to play this catch-up game, I think it's a tricky thing, right? We're seeing so many things happening with like Mythos or just like people finding vulnerabilities so much faster, scanning stuff, finding public exploits, and abusing it and having like actual security incidents. And I do think it's a genuine concern, right? And I think to allude back to the threat intelligence where we started about, that's why that threat intelligence piece is so important, right? KB [01:04:09]: Mm. Nathan Pitchaikani [01:04:41]: 'Cause yes, zero days can happen more, vulnerabilities can happen more, but if we're able to have sort of this raw threat feed, we're able to get threat intelligence faster, then we're in sort of in a better place. KB [01:04:52]: So then would you say that we, because we've got all of this capability and intel, we as an industry can get ahead of the attackers? Nathan Pitchaikani [01:04:59]: I would hope so. I think it's hard to deal with zero days, right? You can't really do much, right? If like no one knows about it and all of a sudden an attacker hits you from a zero day and that's where the issue occurs. But hopefully, you know, through disclosures and people talking about it, then, you know, other companies can stay safe. But my biggest concerns to your point is like zero days where literally people are finding vulnerabilities and abusing them. KB [01:05:21]: So what do you think happens now with that space? Because obviously there's a lot of stuff going on there, people don't have answers. I mean, it's not an easy one to answer, but yeah, we have to do something. Nathan Pitchaikani [01:05:31]: Yeah, I think the answer like is sort of again like make sure that we care, we protect what our core components are, right? Like we should legitimately think about like what are our crown jewels in our company, and let's make sure that we have detections and stuff to like catch that. So if something does happen, right, we can then go from there. Something else that's pretty big is deception technology, right? Or honeypots. KB [01:05:31]: Yep. Nathan Pitchaikani [01:05:56]: Right. Assume breach compromise, where you are assuming that something happens, right? And if something does happen, you have ways of detecting. So I think that's also something that honestly might be a pretty hot topic for dealing with zero days. I think there's a few different ways, like I said, but I think that really comes down to like really understanding what you're trying to protect and making sure you have coverage for that. So if something does happen, you're already ready. What you really don't want happening is something happens and you have no idea it happened because you don't have detections because you're too worried about like other random companies' zero days or anything like that. KB [01:06:28]: So given the climate that we're in with the zero days, would you say that's the catalyst now for companies trying to get answers specific to their company, like forgetting about what's happening in the market, et cetera? Nathan Pitchaikani [01:06:40]: I think again, companies are sort of dealing with zero days and they're thinking about how they want to approach it. I think the best way to, again, look at it is sort of understand your crown jewels, and based off that, make sure you've got proper coverage for those. I think also, again, like we mentioned earlier, deception technology is a great way to ensure that if something does happen, you detect it early. Yeah. KB [01:06:57]: So would you say that now that's becoming a very big focus in terms of like honeypots? We've spoken a lot about getting ahead of it because we've got some of this intel to hopefully, even if it's 1 millimeter ahead, we're still ahead. Nathan Pitchaikani [01:06:57]: Yeah. KB [01:07:08]: So do you think that that gap will get wider now as things start to mature, the way in which people are optimizing these tools and the way they're approaching a lot of these things, that gap will widen? Nathan Pitchaikani [01:07:20]: Yeah, I think I'm hoping it necessarily doesn't, right? I think that, you know, like I said, there's always that case of zero days. And I think it's important that we have this understanding that like what happens when we do get compromised, how fast can we detect an attacker? Right. We talked earlier about how like AI is leveraging attackers to be able to be more like creative with phishing emails or other ways of getting in. And I think that's like a lot of how like threat actors are getting into, you know, companies is through phishing emails or social engineering. And I think, you know, if that happens, how fast can we detect that? And I think that's something that's something that companies generally need to be thinking about is like, what's that initial access sort of changing? How is AI affecting that initial access? And do you have something to sort of cover that? KB [01:08:09]: So Nathan, given everything that we've spoken about today, what do you sort of see happening, like unfolding now for the rest of the year? I know things are compounding like week to week, things are getting better, things are changing day to day, but what's your sort of view? Nathan Pitchaikani [01:08:23]: Yeah, I think what I'd like to sort of see is sort of, are we still going to keep seeing supply chain attacks constantly happening, right? Are incidents gonna keep being crazy? Are we gonna see just like a bunch of, you know, Chrome extensions and IDE extensions, fake ChatGPT installers, ClickFix? Like this stuff has been pretty predominant, I think in the last like 6 months. Are we gonna see sort of this landscape change or stay the same? Are we gonna see fixes coming out of the industry to try and make it easier? Granted, it is generally hard though. KB [01:08:23]: Yeah. Nathan Pitchaikani [01:08:56]: And sort of like, how does that change, right? This is like the main attack vector we're seeing from threat actors. Do they just keep abusing this? And will sort of the response from defenders sort of be, okay, this is how it's gonna be for the next year. Let's just make sure we have, we're aware of what can happen. Let's make sure we can triage this. Let's make sure we can detect it as fast as possible and remediate it. KB [01:08:56]: Mm-hmm. Nathan Pitchaikani [01:09:14]: That's what I'm personally really curious about, 'cause it's been like this for at least 4 months. And I think it's probably one of the long, not the longest type of sort of scenarios we've seen, But it's definitely been quite a bit. KB [01:09:26]: So then with supply chain attacks, do you envision that's going to probably get worse before it gets better? Nathan Pitchaikani [01:09:29]: I think, yeah, it's definitely gonna get worse before it gets better. I think in the last month or 2 months, it's definitely gotten worse to the point where like we're seeing them like daily. Like it's gotten to the point where I literally can't even keep track of them anymore. Like I used to like be able to track and be like, oh cool, this thing happened. Okay, time to go do a thing. Right? But now, like, it's just, it's impossible for me to do that. And that's where, again, I'm like, I'm happy AI can help scrape the sources, grab the information, and then do the initial check for us because there's no way I can constantly be on Twitter. No way I can constantly look at all these news articles. Nathan Pitchaikani [01:10:04]: It's just not possible. KB [01:10:05]: So why do you hypothesize it's getting worse then? Nathan Pitchaikani [01:10:08]: I think it's getting worse because people are generally realizing that people are getting compromised, right? It's working. A lot of companies are not sort of ready for how to respond to this, don't have the resources to be able to check all their environments. They're not at that maturity aspect, right? And I think there's this drastic shift now to make sure that you have visibility into like what libraries people are using, and I think that's very important. But again, maybe in 6 months companies are a bit more prepared, they have visibility, they understand, you know, these are the things that we need to make sure matter, and then they can go from there. KB [01:10:37]: So do you think companies are going to be more stringent now with their suppliers? Because it's not just like the supplier that they outsource to, it's second, third, 4th party. Nathan Pitchaikani [01:10:37]: Yeah. KB [01:10:47]: So I've even heard up to 6. So how do you— okay, how do you start managing that? And then how do you manage your suppliers' suppliers? And then it just keeps going on. What are people doing? Yeah. What are you doing? Nathan Pitchaikani [01:11:01]: Yeah, I think it's a great question. Like, I think like for like internal repositories, right? I think it's like maybe for like libraries, let's make sure that we start having internal repositories. Don't let people do whatever they want and grab whatever libraries they want. Have it all sort of locally or privately stored, and then do updates yourself and manage it. That way everyone pulls from there. I think that's the mature way of approaching it. I think it's hard to get to that place, but I think because it's a lot of standardization, developers aren't used to like being able to be like, oh, I'm gonna go here to grab my packages. KB [01:11:30]: They just grab it. Nathan Pitchaikani [01:11:30]: Yeah, they just go to like internet. But if you sort of set up like, you know, block policies and set up a new governance around it, I think that's one way of approaching it. KB [01:11:36]: What about SBOM then? Would that help with some of that in terms of auditing it to make sure this isn't suspect? Nathan Pitchaikani [01:11:46]: Possibly. I'm not too sure on that topic. Maybe. Something else that's also interesting is third-party data. We talk about third-party supply chain with packages, but there's also just how your data is being used. Like, is another company using your data? And then is some other company using that data? Like, are you working with vendors, contractors, right? Is that secure? Or what's their security policies, right? We're seeing a lot of not just like— we're seeing supply chain in not just like packages, but we're also seeing it like third parties getting compromised and contractor companies getting compromised, and they have access to your data, and that's, you know, can be an issue too. KB [01:12:22]: So then would you say that if people are spending so much time auditing policies for their suppliers, they're then not worrying about their security internally? Nathan Pitchaikani [01:12:22]: No, I think it's a matter of both, right? I think you should ideally have security teams that are focusing on different things, right? So you have some people who are managing, you know, your libraries, your code base and stuff. You've got people who are working managing all your third-party security risk, right? And verifying vendors are, you know, following the correct protocols that they should be when you onboard them and, you know, making sure they're using like MFA or You know, they have an antivirus tool and stuff like that, or a phishing tool. KB [01:12:22]: So sort of to wrap today, Nathan, we've spoken a lot about threat intelligence being distributed function at Riot Games more specifically, rather than something owned by a single team. Nathan Pitchaikani [01:12:22]: Yeah. KB [01:12:32]: Walk me through that, and what does that sort of operating model internally look like then for you? Nathan Pitchaikani [01:12:38]: Yeah, so generally how it sort of works, threat intelligence for us, right, is like I said, we have a few different ways of processing it. We have the automated way we've talked a lot about. We have the manual way too, right? Where we have like something called MISP, it's open source. We are able to go in there and make create events and then we're able to add attributes. This then works where it actually, we have a scheduled job where it'll go to MISP, grab an event, grab IOCs, and then send it to our different tools like our EDR, our network telemetry, and then look to block or alert if any hits happen. Likewise, we also send it to Datadog, our SIEM. And then in our SIEM, this actually gives us more scope, right? 'Cause now threats aren't just happening in EDR space, in network space. I mean, I think we also gotta start thinking about SaaS applications. KB [01:12:38]: Yeah. Nathan Pitchaikani [01:13:23]: And some SaaS applications we have the viz in network and EDR, but some we don't, right? So being able to actually get that enriched data that's coming from MISP and actually add it to our logs on ingestion with Datadog Cloud SIEM, that's sort of been really useful. And that gives us more access to like some of our identity logs or our like maybe like GitHub audit logs or any other type of SaaS audit logs. KB [01:13:44]: And so as a result of that, it's saved you a lot of time? Nathan Pitchaikani [01:13:47]: Oh, absolutely, right? It saves a lot of time. It also makes us feel more covered from an aspect, right? Now we're not just looking at threat intel and being blocked in our firewalls or network and EDR. We're also sort of looking now at our SaaS applications and identity applications. So now, like, we have a wider scope of sort of understanding and actually using our threat intel. KB [01:14:05]: So when you say covered, you mean ingestion then of the sources as well as looking at the logs in more fidelity? Nathan Pitchaikani [01:14:13]: Yes, in a way, right? So like basically whatever gets ingested into our Datadog Cloud SIEM, we send everything there, right? So we send all our identity logs, we send like our cloud logs, we send EDR telemetry logs, we send, you know, firewall-related data or network telemetry, and again SaaS apps. And right now we have ways, you know, most network, you know, telemetry tools support basically threat intel, same thing with EDRs, but most SaaS apps don't support like giving them like an IP block list or, you know, like domains or whatnot that you're worried about. They don't do that. But now that we have those logs coming into the Datadog Cloud SIEM, we're able to actually like look at and see like, hey, this IP address was maybe used in maybe GitHub. That's a malicious IP we saw on our threat intel. KB [01:14:13]: Mm-hmm. Nathan Pitchaikani [01:14:59]: And now we can alert on that and be like, have a little more visibility. KB [01:15:02]: So in terms of any closing comments, final thoughts, anything you'd like to say today? Nathan Pitchaikani [01:15:07]: Yeah, I think like for closing comments, I think the biggest thing I'd say is AI is definitely making the security space way more interesting. It's making a defender's life harder but also easier at the same time in different ways that honestly we wouldn't really expect, right? Easier in the way of, cool, you can now utilize AI to summarize a bunch of information easier for your alerts. You're able to sort of scrape threat intel sources and grab a bunch of stuff and check your environment. But at the same time, you have AI which is doing the opposite, which is actually giving you more alerts. KB [01:15:07]: Right. Nathan Pitchaikani [01:15:38]: From people who are sort of, you know, vibecoding or doing things that they didn't know they're doing, and it's creating alerts. So it's a little bit of both, which honestly, we would— it's a very interesting like thing, right? We wouldn't really expect that. And I think it's honestly on security teams to sort of be a bit more proactive and sort of like up to date on like what's actually happening. 'Cause if security teams aren't like gonna decide to sort of use AI to help them enrich data or look at everything or sort of start scanning threat intel, I think that security teams are gonna sort of slow down, right? They won't be able to deal with everything that's happening in a manual aspect. It's just not possible, right? A single human can't function 24/7 anymore. It's just not possible. So yeah. KB [01:16:17]: Thank you so much for your time. Nathan Pitchaikani [01:16:18]: Yeah, thank you. Thank you for having me. KB [01:16:21]: And there you have it. Stay tuned for more.

Other Episodes