Episode Transcript
[00:00:00] Michael Pastor: if lawsuits occur and we're dealing with the question of did someone, you know, operate with a reasonable standard of care, and it, it is revealed in discovery that there were shortcuts. The defense to that, it's gonna be really hard to defend that, to say, "But I trusted how amazing this technology was," right? I, I believe that the, the technology was so, so great. That defense is not gonna hold up.
KB: From KBI Media, I'm Karissa Breen, and this is KBKast. My guest today is Michael Pastor, Dean for Technology Law Programs at New York Law School and former General Counsel of New York City Cyber Command. We talk about why, " I trusted the technology," won't hold up in a lawsuit, his problem with the phrase human in the loop, and what reasonable security looks like now that AI keeps raising the bar. [00:01:00]
[00:01:01] VO: If you find these conversations useful, hit follow. It's the single best way to make sure the next one lands right into your feed, and it helps other execs find the show.
[00:01:11] KB: All righty. Let's get into it. So Michael, thanks for joining. I want to start with Mythos or Mythos or however people sort of say it in the market, depending on who you ask. But the tools make vulnerability discovery cheap and continuous. We're seeing that across a couple of folks that have been speaking to you like yourself. Now, we're going to look at today's angle a little bit differently, but the legal standard for reasonable security rise for everyone, given your background and your pedigree
[00:01:42] Michael Pastor: I think what you're gonna wanna look for is not the cost and not the fact that the technology runs continuously, but the fact that the AI tool has improved the effectiveness of the vulnerability scanning of the organization above and beyond what the human vulnerability team would've [00:02:00] already been doing with whatever technology they could have deployed.
[00:02:03] Michael Pastor: There may be a speed component to that, too, right? Where it's like you now have more... You have a different view of what's reasonable given how quick the AI tools can do it. You know, I wouldn't be looking at cheap and continuous. Many cyber tools run continuously now. Some are inexpensive, some are expensive.
[00:02:21] Michael Pastor: Really, what you're looking for is effectiveness and, of course, documentation of effectiveness from a counsel perspective. Ultimately, the lawyer will want to converse with the security people and see what the tool found, and hopefully the security people will be able to explain to counsel what has happened.
[00:02:41] Michael Pastor: I think that AI does create this new wrinkle in all of this, where there seems to be instances where the technologists aren't quite sure what the tool has done, even if the vulnerability identified is a true one and one that you can validate. But there's that fear there where you're identifying a vulnerability, but [00:03:00] you can't quite be sure how the vulnerability was identified.
[00:03:03] Michael Pastor: Again, if it's validated and the AI tool helps you be more effective, getting rid of false positives, focusing in on the ones that are real, that is a good thing. But what is reasonable is really gonna come down to what the humans can validate at the second level after the AI tool has done whatever it has done.
[00:03:23] KB: So this is interesting. So when you say reasonable, and I know that given your background, I know in like contract law, there's, there can be clauses that say company does reasonable endeavors to correct the situation if there was a problem, stuff like that. So do you think this whole reasonable word people will misconstrue of the meaning of it?
[00:03:42] KB: Or it'll be like, "Hey, Michael thinks X, but Karissa thinks Y"?
[00:03:46] Michael Pastor: Sure. So I will specify, since we might be speaking to a global audience here, that reasonableness as a standard may apply or may not apply depending upon the jurisdiction. The US does have, [00:04:00] depending upon the claim and the lawsuit involved, a standard where we look for what we think the reasonable person would've done in the situation that is presented.
[00:04:11] Michael Pastor: That is meant, to answer your question, to draw away from the subjective view and to create a somewhat ephemeral, artificial standard for what, for what is reasonable. But it is a word that is used. I'll, I'll cite one, one example. The New York State Department of Financial Services, which has extensive cybersecurity regulations that apply to, uh, the financial institutions that operate here in the state of New York, where I am, refers to reasonable safeguards as the, you know, as do various laws.
[00:04:42] Michael Pastor: Courts are gonna interpret what it means for those reasonable safeguards to, to look like in an AI world. I will say one thing. I think that AI is going to complicate things because of the speed of transformation and the speed of adoption. Sort [00:05:00] of goes back to your first question in a way. There was a time where you could kinda have a glimpse of it and at least have some sense that that standard is gonna be in place for some period of time.
[00:05:12] Michael Pastor: I do think we're in a situation now, and I may be jumping a little bit off your, your question, where the different AI companies have some degree of interest in announcing that their model has done something wildly exciting and crazy and hard to believe. I can't remember a time where that was so true.
[00:05:32] Michael Pastor: There, there might have been times where in previous eras a, a company would say, "We now can put this on the cloud in a way that we, we never could." But right now we feel like in a particular moment, and I, I feel like I'm about to use a word that's overused but I'll use it anyway, this, this hype moment where we are.
[00:05:49] Michael Pastor: The first iteration of the news is that this thing presents a type of security threat that we've never seen before. [00:06:00] And I think we as cybersecurity lawyers are the ones that need to bring a healthy dose of skepticism to the work that we're doing. But as I tell my students all the time, we also can't be blockers, right?
[00:06:14] Michael Pastor: It's not skepticism such that you don't believe anything. I think these AI tools are going to bring about revolutionary change for all who work in technology in any operation, right? Infrastructure, privacy, data management writ large, security, and that could be security of the enterprise or could be security at the endpoint, you name it.
[00:06:34] Michael Pastor: I do think that there's a chance here for, for lack of a better way to put it, an arms race fr- between the enemies and the people trying to protect networks and systems.
[00:06:45] KB: So on that note, do you believe where you are now, that this sort of eradicates that bias in terms of going back to reasonable endeavors, two companies arguing about something.
[00:06:56] KB: I could say, "Hey, but I did these reasonable endeavors." The other party could say, "Well, we [00:07:00] don't think you did." But perhaps if there is AI in the middle that can look at it from a, I don't know, mediation perspective, maybe they can look at both sides and look at the facts rather than the emotion and all those sort of things.
[00:07:12] KB: So do you think there's still people sort of worried about will that be, be reasonable in terms of their definition? But from what I'm hearing from what you're saying, it looks like it will still be fairly weighted though, so it's not, it's not gonna have that bias that I might have or you may have, example, as human beings.
[00:07:29] KB: So how do you think that's then gonna play out as we move forward?
[00:07:33] Michael Pastor: Well, in its best iteration, it will be that. It won't be weighted, and it will be as objective as possible, given that we're not talking about scientific objectivity here. We're talking about ob- an objective legal standard, which will ultimately be interpreted, at least for the time being, by a human adjudicator to the extent a dis- a dispute goes there.
[00:07:53] Michael Pastor: I will say I think you raise a fascinating question underlying your question to me. So the, the question is [00:08:00] this: Can AI be utilized to improve the strength of the trust between counterparties? Because on either side of a transaction or a contractual relationship- There can be a tendency for people to dig in, and they will say, "My technology is telling me this."
[00:08:18] Michael Pastor: And the counterparties say, "Well, my technologists are telling me this." Then the lawyers posture up, and suddenly you're at each other's throats in a way that frankly is just not productive in the contractual world anyway, unless you want to litigate, in which case, go fight it out in court and someone will decide.
[00:08:31] Michael Pastor: That's a really interesting question. I have a lot of positive view towards AI validation, and that goes well beyond cybersecurity. And people ask me the question all the time, "How, how do you see AI being deployed in the practice of law?" Including for people who practice cybersecurity law. I see huge upside potential in AI validating premises that we come up with based upon our own legal reasoning [00:09:00] and judgment.
[00:09:00] Michael Pastor: And, and why I think it's important to say that is I think that the best law work here will involve a great amount of human work on the front end, and then the technical back and forth with the A tool... AI tools on the back end for the purpose of validation. So what do I mean by that? I see huge risk in shortcuts.
[00:09:23] Michael Pastor: This could be CISO shortcuts, it can be lawyer shortcuts. There's just this tendency to say, "If I just pull up this really powerful tool and give it three bullets by way of background, it will give me the validated and correct answer that I otherwise independently would've gotten to as a technologist or as a lawyer."
[00:09:44] Michael Pastor: That, to me, is ... That shortcut concern is a big problem. And by the way, to the extent you were interested in asking me about liability in that front, well, if lawsuits occur and we're dealing with the question of did someone, you know, [00:10:00] operate with a reasonable standard of care, and it, it is revealed in discovery that there were shortcuts. The defense to that, it's gonna be really hard to defend that, to say, "But I trusted how amazing this technology was," right?
[00:10:17] Michael Pastor: I, I believe that the, the technology was so, so great. That defense is not gonna hold up, which is why the technology people are gonna need to do their homework when technology comes in the enterprise door.
[00:10:28] KB: And I think that's important, because where my mind was going as you were speaking there, Michael, was around, look at all the breaches that we're having.
[00:10:35] KB: And then there's a breach that happens, major breach, the company comes out and says, "We're sorry. We really care about your data." But again, if we go and look into it, potentially people would say, "Well, actually, reasonable endeavors were not taken. That's why there was a breach." So do you think in your legal experience there's gonna be a breach, then we're gonna leverage AI to go through it, sift through, trawl through all the [00:11:00] stuff to then discover reasonable endeavors were not taken, so therefore the whole tune of trying to repair the trust in the market around the breach will change fundamentally?
[00:11:12] Michael Pastor: Right. I think that what you see with AI, and this has been true for decades with cybersecurity, is that the sexy tech headlines are gonna be focused on the amazing things that the tech can do. Not the amazing things that the tech can do to protect a system, but rather the things that they can do to generate new revenues and new business and to grow enterprises, because we sort of operate in a business world setting where that's generally gonna be the focus, and the news will often be about discoveries and developments until the bad day occurs.
[00:11:47] Michael Pastor: And then when the bad day occurs, there's recrimination, there is finger-pointing, someone is hung out to dry as the person to blame. All I can say about that is I have some [00:12:00] optimism, based upon my experience in cyber for many years, that as advances occur from the threat actors, there will be advances that occur on the defensive side.
[00:12:14] Michael Pastor: But one of the things that we need to look for is strong government and regulatory work in that space as well. I mean, to give one example, New York State has a chief cyber officer who's now the lead for both security of all kinds in New York State, and he's a former client of mine at the New York City Cyber Command where I was GC.
[00:12:36] Michael Pastor: He, from the government perspective, is looking at the threat to the population of the state of New York, which I think is an excellent model, right? People, when they live in a state, they look to their state to consider their protection. But I can guarantee without having spoken with anyone over in that office, that the other thing they're thinking about is the coordination with the private sector and the companies that own [00:13:00] critical infrastructure that are not publicly owned.
[00:13:02] Michael Pastor: Having a coherent approach will be necessary. I just refuse to believe that the bad guys are gonna be necessarily better than us at AI, provided that we spend the time and the work to do what we need to do, and provided that lawyers, like I used to be, support that work and think about that work in our transactions with vendors.
[00:13:25] KB: There was an outage caused flow-on effects. Is it gonna be hard then from a defense point of view for, for people to say, "Well, actually you just were careless. You, you missed the boat here, and that's why we had a, an issue"? And then as a result of that, will then government legislation, all this sort of stuff, regulators come in and I think you use this term here in the United States, throw the book at them?
[00:13:47] KB: So therefore, will we start to see more of that, and really perhaps not absolve these companies from what they've done in terms of the breaches? 'Cause the reason why I say that, I'd like to caveat [00:14:00] it by saying people are becoming very desensitized to these breaches. Yeah. Doesn't mean the impact really goes away.
[00:14:05] KB: So it's more so I would... I'm curious to understand that forward planning on what that looks like now, because people are starting to get fed up with these businesses. Now, whether they didn't mean it or they meant it, it's the result of what's happened. So I'm, I'm curious to get your thoughts here.
[00:14:19] Michael Pastor: Many thoughts on this question.
[00:14:20] Michael Pastor: I'll, I'll start by saying I wouldn't use the phrase throw the book in this particular setting, because throw the book here connotes a criminal prosecution or maybe a regulatory investigation where the company's in hot water. Which by the way, is gonna happen, so I'm not saying that throwing the book at them will not happen.
[00:14:37] Michael Pastor: I'm just saying as it relates to the potential for good legislative work and regulatory work, what you're looking for is two things. One, you're looking for regulators and lawmakers to clarify the state of play for all the technologists in a jurisdiction such that there is less doubt about what is [00:15:00] expected of organizations.
[00:15:01] Michael Pastor: But you also want legislation and regulation that is not so heavy-handed that it gets in the way of us innovating and being on the forefront of the technological work that we need to do. Having been in government, I will tell you that most of the regulators I've known and the regulators that I know that are out there, they do operate in good faith and want to promulgate regulations that are understandable and reasonable, and that can be adhered to.
[00:15:34] Michael Pastor: Doesn't always mean that they do that. Lawmaking can be a very, very messy process, but I do think that there's space here for regulators to work. And I'll say this, if they don't operate in that space, there will be a reckoning of one kind or another, right? They... It's not-- When people get hurt, let's say a major enterprise gets hurt by a hack, they're not just [00:16:00] gonna walk away and say, "Oh, it's, this happens in the day."
[00:16:03] Michael Pastor: They're going to seek redress most likely. And some might say, "Well, if, had there been a regulatory overlay, this harm would not have happened." I mean, I'll just go back to my lodestar for you. We care about cybersecurity because we want people in organizations to not get hurt. That's the point of why we do this.
[00:16:23] Michael Pastor: And do I think that AI in the hands of a bad actor can result in a cyber attack that causes direct harm to people or companies? Yes, I do, and I, I think we're naive not to think about how to address that.
[00:16:43] KB: Then my other question would be, you said before finger-pointing, so one of the thing that's been coming up a lot in my interviews is like, well, who owns the thing that happened?
[00:16:52] KB: So even everyone's used the Air Canada situation. Do you believe that companies potentially will start to [00:17:00] say, 'cause we've seen over the years, "Oh, it was the vendor, it was this, it was some third party supplier that no one's ever heard of that caused the accident." Do you think that people are gonna start to say, "Oh, it was the AI machine."
[00:17:10] KB: And now the conversation is, well, who owns that? Is it the vendor that's running the AI? But is it still, still linked to the business though, but is it their problem because they outsource it to another company? It's quite a conundrum. So curious to also get your thinking here.
[00:17:24] Michael Pastor: Right. So this is something that I, I think about all the time, and it's actually front of mind today because in a week our semester opens up and one of the classes I teach is drafting contracts for technology transactions, and that class is all about trying to identify and mitigate risks when a party goes out into the world to procure a technology good or services, which now is going to include an AI component in almost every contract. I'll say again, and I, and I'm gonna sound somewhat traditionalist here, I don't think we [00:18:00] lack the mental and intellectual ability to tackle that question just because AI is so novel and different.
[00:18:08] Michael Pastor: To the question I think you raised in, in the middle of your question, when someone argues, "Well, this was not my fault Because AI did it and not me. That's gonna sound really similar to, "This is not my fault because whatever technology I deployed to do the thing you wanted me to do had a flaw and it resulted in a problem for us."
[00:18:36] Michael Pastor: So I think a lot of work is gonna go in and must go in on the front end, the contracting work upfront, where a good contract is gonna get meeting, meeting of the minds about the different things that can go wrong and who is at risk and who is going to be liable for the things that, that go wrong. These are conversations that happen all the time.
[00:18:56] Michael Pastor: Again, one wildcard here is do [00:19:00] the parties know what went wrong? That could be different i- than, than prior times maybe. Although before AI became all the rage, there were plenty of instances where two parties would disagree about what happened from a cybersecurity perspective. Was a, a system breached or not?
[00:19:19] Michael Pastor: Could data be accessed? Was it stolen? And one party will say, "My folks are telling me it was not stolen." And another party says, "It was stolen." So those disputes always happened. I'm curious to see how AI could maybe be used to resolve the disputes. And then one wide open question, which may not be the subject of your questions for me today, is what will courts do when cyber disputes end up in adjudication?
[00:19:49] Michael Pastor: Will courts think to apply AI tools to help them adjudicate the dispute? That to me feels like the Wild Wild West, but I doubt it will be the Wild Wild West for long [00:20:00] because I think that courts are gonna need some ability to wrestle with these highly technical questions, but we'll have to see.
[00:20:00] KB: We'll come back to that in a moment after a quick word from our sponsor. If you’re working in AI, machine learning or data science, you’re likely already handling sensitive information. Proving your security and compliance posture? That’s where Vanta comes in. Vanta helps AI-driven teams fast-track compliance. Think SOC 2, ISO 27001, GDPR, with minimal disruption to development. Visit vanta.com/kbkast. That’s V-A-N-T-A dot com slash KBKast to learn more.
[00:20:47] KB: That's interesting. So okay, before I get onto that question, I want to go back a step to contract, writing contracts upfront. So do you envision now companies or especially big enterprises are in [00:21:00] the process of rearchitecting what those contracts look like to make sure if something were to happen, it's very clear who is responsible rather than it was this person, 'cause then they're gonna have people like me calling 'em up saying, "Hey, I heard there was a breach.
[00:21:11] KB: What's happening?" And then what happens when there's misconstrued information, it obviously creates distrust and we know how it all goes. So do you think businesses are now putting, "Well, what does our contracts looks like with our suppliers and our vendors and whoever else we use, contractors?" To see where the responsibility lands, 'cause if something were to go wrong, we know very clearly where those lines are.
[00:21:34] KB: And then potentially if there was a external vendor was an issue , we can potentially blame them in their eyes rather than- Definitely ... it was us.
[00:21:43] Michael Pastor: Definitely. So I will say this: speaking in very plain terms, there are two different ways to do contract management as a general counsel, as the head of procurement, and other transactional lawyers at an organization.
[00:21:57] Michael Pastor: One way you do it is every single [00:22:00] year and month and day, do we know our portfolio of contracts, what they say, when they expire, who they're with? And are we confident that those terms say what we want them to say given the business needs in the present moment? And if they don't, which I think is most responsive to your question, we deploy our own human lawyers to analyze the terms, and those human lawyers are undoubtedly using AI to help them analyze those, those terms and to think about revisions.
[00:22:31] Michael Pastor: And I think most enterprises would not be surprised if a counterparty comes to them and says, "Hey, we just looked at clause 49 of this contract we signed three years ago, and it might have worked then, but it doesn't work now. Can we talk about renegotiation?" And if there's good trust and a good relationship between the parties, they will.
[00:22:49] Michael Pastor: The other way contract work gets done, which is not the way I recommend, is that contract lawyers and general counsels have too much confidence in the contracts [00:23:00] they've negotiated and executed years ago such that they become complacent and don't do the proactive work. We talk about productivity all the time in our GC training here.
[00:23:10] Michael Pastor: Don't do the proactive work to look at what the contracts say. I will say that we are in an age now where there are countless number of tools that are offered to general counsels that help in that process, that really make it almost inexcusable that you wouldn't do that. But I'm going from but to but, but it happens.
[00:23:35] Michael Pastor: But you still need to validate what those tools tell you about what the contracts say. You still need a human being to read the term in question, that that human understands the business, and then you decide what you need to revise. But I think to answer your question, it's a wholehearted yes. They have to be looking at what their contracts say, and do they need to, need to say different things based upon the world that we're living in.[00:24:00]
[00:24:00] KB: And then flipping back over to case goes to sit in front of a judge, what does that look like? What do you think is realistically going to happen? Now, I also was thinking about as you were talking through that, there may be technical jargon that a legal professional doesn't understand. It changes all of the time.
[00:24:20] KB: You have to be really-- Even in my field of work, interviewing people like yourself day in, day out, it's even hard for me to keep up, let alone someone who's doing law stuff. So do you think it will help on that part to understand, well, what does this term mean, this cyber term that this, this dude's talking about?
[00:24:36] KB: Do you think that could be something which helps the outcome overall? It's not necessarily a zero-sum game, so do you see that as being a, a benefit, or do you still think people will not like the presence of, you know, AI assisted attorneys, for example?
[00:24:53] Michael Pastor: I think the jury's still out on that a little bit. I will tell you that when I was general [00:25:00] counsel of the Cyber Command and, and had a great team of lawyers working with me for our client, which was, you know, the chief information security officer of the city of New York and all the technologists that work for that person, I never really encountered a moment where if I had the time to converse with the technologists, I felt that I didn't understand the technology sufficiently well such that I could give them legal advice about that technology or advise them in the contract front.
[00:25:27] Michael Pastor: When I say the jury's out, especially because I'm not practicing as a cyber lawyer, lawyer currently, is can AI facilitate that conversation between the technologist and the lawyer? I have some doubt about whether it will, but I really don't wanna make a judgment on it, because maybe there are instances where the technologist is really struggling to put the technology in plain English, and the AI tool can help.
[00:25:57] Michael Pastor: So open question for me on that [00:26:00] one.
[00:26:00] KB: 'Cause that leads me to my next question around human in the loop, which I wanna understand probably from you is if a human is approving something, so it's like, "Hey, I don't understand this thing," you're the human in the loop, you have to approve it. But what if they, the human, don't actually understand in any fidelity what they're then approving?
[00:26:22] KB: Because I've been speaking to people recently that saying, "Well, people are becoming complacent now, KB, so they're just sort of approving whatever's coming through because we're conditioned that way." Right. So now I'm hearing the opposite of people saying, "I don't believe in human in the loop," because people are just gonna say yes.
[00:26:36] KB: So I'm curious then, that responsibility still goes back to the human because the, the ultimately the buck stops with them.
[00:26:43] Michael Pastor: So I, I wanna be careful that I don't go on too long on this topic 'cause I don't wanna occupy all our time on what I think to be a hugely important point for conversation. So I don't like the phrase human in the loop, because it implies a world [00:27:00] where technology does 99% of the work, and at one point in time in the circle, there is a 1% input from a human doing something that humans do.
[00:27:11] Michael Pastor: A much better paradigm for me is something like the human's in charge, or the human's overseeing, or the human's controlling whatever it is that the organization is trying to do, and using technology to do the thing that they're trying to do. So for me, what's the problem with human in the loop? Well, to your point, if you're just in the loop but you have no sense of the context of how the product was developed or how the application was developed, you have no sense of the business objective of the organization in either buying that product or building it, and someone comes to you and says, "Are you good?
[00:27:53] Michael Pastor: Are you good, lawyer?" And the lawyer says, "I'm good," or, "I'm not good." Well, the question I pose is, on what basis are you [00:28:00] saying you're good or you're, you're not good if you had no sense of the context for the question, right? When we ask a lawyer, which in this case is the human we're talking about, we're asking them to do the thing that lawyers have done for centuries, which is understand the laws that apply to the question, understand the legal and business risks that arise based upon the question, and give counsel to the technologists and the business people about what they are trying to do.
[00:28:34] Michael Pastor: So I think it's just more about human oversight, and I don't want to be misread here to say that you need humans at every step. I'm not trying to swing the pendulum all the other way, where it's AI in the loop, humans doing everything, and you put AI in the loop. I don't, I don't think that's right. But AI in the loop, for me frames it in a way that the human component is hugely diminished.
[00:28:59] Michael Pastor: [00:29:00] And I'll go back to a point I made, I think a couple questions earlier. If something goes wrong, nobody is going to blame the AI. It, it just won't work that way. Someone's gonna say, "Well, a human was in the loop, and they approved it." And one of the things the human might say as an act of desperation there is, "Well, I've been completely cut out of everything, and so I'm left with no real facts or information upon which to base my approval."
[00:29:28] Michael Pastor: But you don't wanna be in that situation. You wanna be in a place where the lawyers are empowered to do what they really need to do.
[00:29:36] KB: That is so true, and I can definitely see an instance where someone's like, "Well, I've just been hitting approve the last one year, and then 1% of things have gone wrong."
[00:29:43] Michael Pastor: Listen, and I'll tell you this, I mean, counsel get very popular when they are known as easy approvals, right? Because from the business side, they think, "If I go to that person, I'm gonna get my approval in a day, and I'm gonna feel good about it." But that's just not good in-house counsel [00:30:00] work. We can be partners with the business, we can be responsive, we can be efficient in terms of the speed with which we give our answers.
[00:30:09] Michael Pastor: But we're not just in the loop. I, I remember my best moments as a technology lawyer where would be the client would call and say, "Hey Michael, I'm about a month from proposing an idea, and this is what we're trying to do, and I don't even know what we're gonna buy and I don't even know what we're gonna do.
[00:30:29] Michael Pastor: But based upon that, can you start to think about how you would approach this from a legal analysis, how you would think about contract terms that would apply?" I would go and noodle, sort of whiteboard in our own space with the team before it had even gotten started. And then when it gets started, you start to iterate together.
[00:30:49] Michael Pastor: Sometimes the technology folks would convey, "Michael, we don't need you now. We're deep in the weeds on the sandbox, the product development. No real legal questions presented at this point. No privacy, no data questions, [00:31:00] so hang tight." But midway point, "Hey, before we proceed too much farther here, what do you foresee being an issue?"
[00:31:06] Michael Pastor: I look at it, think about the laws, think about the company policy. I say, "I think we're fine. Keep proceeding. Keep iterating." And you have that partnership which is really in-house work at its best. I, I think human in the loop really diminishes what a good partnership with counsel looks like, and I don't see anything in the law that supports that structure, really. I, I, and I'll give you one counter example.
[00:31:35] Michael Pastor: A hot topic in the space right now is this notion of privacy by design and security by design. That, to me, is a really interesting one because what you're saying is we are going to start from scratch with privacy and security implications in mind. I don't really have a problem with that, but does the lawyer get cut out because it's all been done by design?
[00:31:58] Michael Pastor: Hopefully not.
[00:31:59] KB: Do you think the [00:32:00] other thing is, like, context as well? Like, if I said, "Hey, go and do this thing," but I don't tell you why or there's context behind it, you might be like, "Nah, I don't really wanna do that. Sucks. It's a bad idea." But if I'm like, "Hey, this is the reason why," you might be more inclined.
[00:32:13] KB: So to go back to the we're not just in the loop, there is that oversight of understanding, yes, the facts, what should happen, the process, the policy, context as well. There's always gonna be something that is an anomaly that perhaps maybe does need to be spoken to other peers or colleagues to understand or do we need to approve this thing or not?
[00:32:33] KB: Is it allowed? There needs to be context there, so would you say that some of that's missing? And I say that because, you know, nowadays people don't really understand the mechanics of full-blown coding, so there's probably gonna be more nuanced things that people miss, so it's hard for them to perhaps in any full fidelity adjudicate something because they're only sort of seeing the last 10%.
[00:32:55] Michael Pastor: Sure. Context is hugely important for all the work that [00:33:00] in-house lawyers do as it relates to technology projects and the things that the technology group is trying to do. I wanna add a point to that, though, which I think sometimes gets lost. In most in-house organizations, you have lawyers who've worked in that role for years and years and years.
[00:33:19] Michael Pastor: They don't just bring context for the project. They bring all of that historical data and understanding for where the company had been, in my instance, where the agency had been, and they know that. So when you say human in the loop, are you saying you don't want them to draw upon all those years of contextual knowledge?
[00:33:45] Michael Pastor: Which by the way- might help them get to yes quicker than not, provided they're allowed to apply the knowledge to the particular thing, right? If you have a human in the loop where it says, "Please click A three times if you approve," [00:34:00] and the lawyer feels constrained to think about all the context that person has from prior deals, prior regulatory actions, from prior litigations, from prior, you know, revenue calls where we're talking about different things we've done to make more money or do our job more effectively, why would you wanna constrain that, that person?
[00:34:20] Michael Pastor: Now, I will say, and I feel like in all AI conversations that you tend to veer back and forth on things, it's the nature of it. I do think that AI tools will advance to help in-house lawyers draw upon all of that human knowledge and potentially centralize that human knowledge in a closed loop database where the different lawyers can kind of pull from the collective knowledge of the organization in a way that could be really, really productive, and may even increase efficiencies because you don't have to go through the same thing again and again.
[00:34:54] Michael Pastor: You might say, "How did we deal with this indemnity clause in a situation like this [00:35:00] provided that the counterparty was similar?" might pull from that, and you might build the knowledge and improve as you go. But I think your point about context is great. Lawyers do their best job when they understand what the client wants.
[00:35:13] Michael Pastor: We're here to help, but we can't help if we don't know what you're trying to do.
[00:35:16] KB: Yeah, that's so true, and I think that sometimes... I think, I think there were tests out there, like personality tests, and it would ask you a question. You're like, "Well, that depends. Like, is it this or that, or is it this and that?"
[00:35:27] KB: But it's like, "No, answer the question. Is it yes or no?" But I'm like, there's so much context there, and as a journalist it's like I wanna have-- I wanna know all of these answers. So, I think that that's really important, and it's important because we're obviously now these are the questions that are being asked, so I think it was really important for you to come on here today.
[00:35:45] KB: But in terms of looking forward, I think another really good question for you, Michael, would be as we're now moving into this next era of how things are being done, we've touched on it a little bit before around contracts. But what do you [00:36:00] believe general counsel should demand or ask politely in contracts with, like, AI vendors?
[00:36:07] KB: Or what does that then look like now? Because you're probably the first person that's really touched on this and talking through what happens now.
[00:36:16] Michael Pastor: Sure. So I think what general counsel will need to be demanding, perhaps with more force than they ever have in prior instances, is transparency into what's going on with their data and what the counterparty is up to from a technological perspective.
[00:36:34] Michael Pastor: So you're gonna want enforceable audit rights where you can audit What happened to your data? You're going to want cooperation agreements where the counterparty has to actually help you understand based upon their technical expertise and has that obligation to operate in good faith. You're going to want notifications, you know, enforceable, you know, notifications [00:37:00] on a timeline that are useful to you if there's an issue with the data you have, and that can be a security issue or it could be an integrity issue, could be any kind of issue, but you're going to want enforceable timelines, when you should be notified, what the notifications say, right?
[00:37:12] Michael Pastor: Not just when you get it, but what it says. Other big one is to demand that you have cross-portfolio transparency as to what the vendor learns on the aggregate. Now, not asking the vendor to disclose any confidences about its relationship with any other counterparty. But you have this anonymized, aggregated data that may indicate threats that comes in from one vendor.
[00:37:40] Michael Pastor: Well, if your vendor knows about a threat based upon one other contract it has or a cohort of contracts it has, you want your contract to say, "Alert me of that without revealing any confidence," because I know that that can be done, "but alert me of what you've learned." And all of this comes down to, and I'm [00:38:00] glad I hit this word before we close out, trust. Good contractual relationships depend upon trust.
[00:38:07] Michael Pastor: Trust at the inception, trust through- throughout the contract, and trust when the, the contract is terminated, should it be terminated. Transparency is going to help with trust. When you're negotiating a contract with a vendor and they push back hard on transparency for whatever reasons they'll proffer, you know, trade secrets, we can't tell you this, this is the crown jewels, I can't talk about that.
[00:38:29] Michael Pastor: At some point in time, you are-- you're paying money to them, where you give your valuable data to them, and they possess it and can do stuff with it, and if something goes wrong, you deserve to know. Now, the best vendor lawyers out there, and the counsel that work for them from an outside per- perspective, are gonna probably push on that 'cause it might be expensive, it might be burdensome, might be hard to customize the notifications.
[00:38:54] Michael Pastor: They're gonna say, "We can't do this for you if we do it for you, if we do it for everybody." These are the conversations that tech transaction lawyers have all the [00:39:00] time. But if I'm a, a good GC dealing with a tool where AI is front and center, those are the things I'm gonna demand.
[00:39:07] KB: And then Michael, final question for you would be, because now of the terrain that we're sort of walking through at the moment, a lot of we haven't seen things before, it's new.
[00:39:17] KB: Do you envision that some of these contracts of how they were historically have been implemented, do you think they'll be terminated because maybe someone didn't disclose something, or maybe there isn't trust, or, or, "No, I don't wanna share that information with you." So will we start to see perhaps a bit of churn here?
[00:39:33] KB: Will it become a little bit volatile in this market? What are your thoughts?
[00:39:35] Michael Pastor: I don't think that termination is gonna pop just because of the environment that we find ourselves in. I think termination tends to occur for a variety of reasons that can frequently have nothing to do with technological advances.
[00:39:50] Michael Pastor: It, it can have to do with changes in business platforms, interoperability questions that have arisen post-contract. So I don't think termination is likely to [00:40:00] occur. But I will say at renewal time when there's a question about- Yes, you're gonna terminate because if you don't renew, the contract's gonna end.
[00:40:10] Michael Pastor: I think a lot of contractual conversations might get harder because it was just easier back then to do that. I will say that we always say that, right? We always say four years ago it was easier to negotiate the contract, and if you teleport back in time four years ago, the contract lawyers were saying, "This is impossible.
[00:40:27] Michael Pastor: I can't negotiate this because it's too complicated." That would've happened with the cloud, it would've happened with the internet, it would've happened with mobile devices. All the advances that we've had, there would've been moments where people said, "I can't do a contract with this vendor because they're gonna do business on their mobile devices.
[00:40:40] Michael Pastor: How can we do that?" And now it seems silly that we wouldn't do contract with a vendor because they have mobile devices deployed to do our work. You wouldn't say that. And I think we're gonna get there with AI as well, where it's gonna get to a point where we no longer have to ask sort of frontier-like questions about this.
[00:40:55] Michael Pastor: So I think renewal time is gonna be really interesting, but I'll end kind of where I [00:41:00] started. I think this is work that good lawyers and good technologists working together can, can get done. I'm optimistic about what, what this will look like. There are always gonna be bad faith actors. You gotta prepare for them and defend against them.
[00:41:13] Michael Pastor: But generally speaking, I have a hopeful view of where this is all gonna turn out.
[00:41:13] KB: That was Michael Pastor, everybody. I keep thinking about his point that nobody is going to blame the AI. How when something breaks, a person who signed off on it and, "I trusted the tool," won't help them. If you're a leader listening to this, I'd be asking your general counsel when your biggest AI vendor contract was last reviewed, and whether you could enforce its audit rights and notification timelines if you had to. I read every reply. If you've got some thoughts on this one, send me a message on LinkedIn.
[00:41:56] VO: KBKast - Cyber for the C-Suite [00:42:00]
