Next in Tech | Ep. 213: AI and Privacy

View Full Transcript

Eric Hanselman: Welcome to Next in Tech, an S&P Global Market Intelligence podcast where the world of emerging tech lives. I'm your host, Eric Hanselman, Chief Analyst for Industry and Company Data at S&P Global Market Intelligence. And today, we're going to be discussing the thorny topic of privacy in the world of AI. And joining me to discuss it is Paige Bartley, Paige, welcome back to the podcast. 
 

Paige Bartley: Great to be here, as always. 
 

Eric Hanselman: Great to have you on because there are so many different angles to this and so many different perspectives, but we've got a set of data that has recently come back out of a set of studies that we can dive into that give us some interesting perspectives. So, I kick us off. What are you seeing on the -- I guess, the privacy ops study and some of the more corporate perspectives on AI and privacy? 
 

Paige Bartley: Yes, absolutely. And I think it is important to make that distinction that there are 2 halves to the story. You have the organizational, the corporate side of things, what are their motivators, what are they doing? What are the pressures to improve data privacy operations or as we like to call it, privacy ops. But then there's the consumer side of the story where you have consumer behavior and motivations as well and how they're interacting with businesses, particularly in the B2C space and how that's moving the needle on privacy. But I think I'd like to start on the organizational side, the corporate side. What we have seen in our most recent voice of the enterprise survey on privacy ops is that these motivations for improving data privacy practices are shifting over time. So, it used to be that year-over-year, we would see regulatory compliance as a very top motivation pretty much consistently for organizations on why they want to improve data privacy. But that's starting to change in the sense that now you have Gen AI, you have AI-enabled software even for enterprise use cases and obviously not consumer-facing, but things like data integration, CRM, all of this is becoming enhanced with AI. Today, almost 2/3 of organizations recognize regulatory and legal compliance as a top motivator for data privacy. But when you look out 2 years on that time horizon, that's where things get interesting. And on the 2-year time horizon, regulatory compliance is dropping as a key motivator for improving these practices. Of course, that raises the question, what actually is taking its place, what was moving organizations forward in this field? And where we saw the biggest jump was the ability to accelerate those opportunities to leverage AI and Gen AI. So, this is the argument -- yes, this is the argument that privacy ops, data governance, especially as a foundation, are really providing what I like to call guardrails, not speed bumps for these activities. They are helping organizations move faster with these technologies without creating undue risk. So pretty much providing the scaffolding for appropriate practices. And that can actually help the business innovate more quickly. 
 

Eric Hanselman: So was this something where -- I mean, regulatory and compliance back when things like GDPR, the U.K. privacy regulation was new, people were really trying to figure it out. Are we actually getting to a point where there is a maturing of expectation and an understanding that you can actually do good things for the company with this as well? 
 

Paige Bartley: I would believe so. If you look at our market monitor market sizing, which includes privacy ops as a category within data management, that is still the fastest-growing segment for compound annual growth rate over the next several years. Part of that's because it's a younger segment of vendors. A lot of these vendors just sort of came out of nowhere in 2016 when GDPR was first announced. Now GDPR wasn't implemented until May of 2018. And there's been a maturation of these expectations because especially if you're a mid-market company or even a smaller company, that's typically not who the regulators go after unless there have been egregious sort of violations. I think what happened was a lot of companies saw that in that stick and carrot model, the regulatory enforcement is a stick that can only really sell products for so long, especially if a company hasn't been subject to that sort of punishment. -- and they get a little bit complacent in the punishment regard, right? So, they're looking for opportunities to turn privacy practices to turn data governance practices into more -- less of a cost center, more of a value add to the business, and that's where a lot of the innovation is. So, there's a broader narrative now of how organizations are trying to incorporate these practices into the broader business operations rather than just having a bunch of compliance or legal people stuck in the back room. 
 

Eric Hanselman: So, there's opportunity to now deal with this. They basically tackle the regulatory pieces, and the expectations are that now they want to figure out how to put data to work. And so certainly, that promise in the age of AI is that we can feed all of this into the AI and life will be better. 
 

Paige Bartley: How do we get more AI out of what we're doing? How do we adopt more AI? How do we build more models appropriately without having that elevated risk of running a fall of regulations. The U.S., in particular, doesn't really have a lot of framework around AI, right, in terms of legislation. So, we're looking to the EU. We're looking to other regions that are developing these regulations. Organizations have to keep that in mind clearly because there's a lot of uncertainty in the geopolitical sense for what's going to happen in terms of AI regulation. So, there aren't really prescriptive rules for organizations that primarily operate in the U.S. on what to do. So, they're looking for the opportunity side of things. How do you build engagement with your customers, build those trusted lasting relationships that lead to a longer lifetime value? Or how do you make more data within the organization appropriately available to those end users that need to use it for analysis, for building your AI models, how do you help them work more efficiently? 
 

Eric Hanselman: How do you actually leverage a lot of these capabilities to make it happen? Well, we've talked a lot about data management and what that process needs to be. It sounds like actually organizations are at least heading in the right direction in terms of their level of sophistication in terms of how they're handling it. 
 

Paige Bartley: Yes. I would think so on the aggregate level. But of course, once you start to dive into the data, that's where things get a little bit more interesting. And you'll see that organizations that self-identify as being more advanced on that digital transformation maturity curve broadly have more affirmative responses to privacy-oriented questions about how they practice and implement. And likewise, for organizations that self-identify as being earlier adopters of technology, you tend to also see higher responses there for indicating your privacy practices. 
 

Eric Hanselman: So, in fact, there is some good correlation there. And I guess the challenge is to ensure that the rest of the market starts to catch up in the same way. Now what do you see in terms of some of that transition? You talked about some of the priorities for where organizations are. What do you think about what they're looking at in terms of working with their end customers and how that translates into the expectations of what they have to have. And I think on the other side, what consumers are thinking about. 
 

Paige Bartley: So, I think when businesses are trying to engage with their customers, especially if they're in the business-to-consumer space or B2C space, which is a significant factor that's often overlooked in that your consumers, broadly speaking, are very motivated today by data security and data privacy. That doesn't mean they know what to do about it. So, in our voice of the connected user landscape research, which is U.S. population representative, we see that year-over-year, concern about data security and separately data privacy because we do define those differently, remains extraordinarily high. So, the vast majority of consumers in the U.S. are indicating that they are concerned about those practices. Now what does it mean, right? When we ask our consumers, what do they currently do to protect the privacy and security of personal data online, it's all bad news for businesses, right? The first thing they do will limit amount of information they share online. So, think about as a business, you're trying to conduct deeper analysis about those behaviors, you're trying to craft that sort of catered personalized customer experience, but they're withholding information because they do not trust the organization they're working with. The second thing they do is they simply avoid organizations they don't trust. So, if you're an online retailer or another online service oriented towards consumers and they don't already have an established trusted relationship with you, there's a very good chance if they don't trust you, they're going to go to your competitor who may have a better posture or at least positioning around data privacy and data security practices. So, on the business side, this is where data privacy practices really translate to ROI for the B2C space because consumers change their behavior when their trust is eroded. 
 

Eric Hanselman: If they see there's been loss, if they -- if the policies or practices are uncertain, they're going to take their business elsewhere. 
 

Paige Bartley: Or if there's a perceived deficiency in those practices, right? It doesn't have to be reality because consumers respond to what they think is necessarily true. So, there's 2 sides to that coin. Now as an organization, you need to have robust privacy and data governance and data security practices and you need to have a public-facing image of what that means, but it cannot just be a PR campaign because ultimately, that will become unearth at some point, right? If you have a massive data breach or some other major privacy violation, then you're just back in the headlines. And that's where you lose that customer trust. 
 

Eric Hanselman: And presumably, those are events that if -- once that customer trust has been lost, gaining it back is all the much more complicated and difficult. You've got to reestablish that brand image because, of course, if there was some level of trust to begin with. The fact that it got tarnished in whatever manner means that now you've got to build that back from a position in which that the trust has been broken as opposed to simply having to win trust in an unknown position to start off with. 
 

Paige Bartley: Absolutely. And that's what we see typically in our research is that especially for those greenfield customers, the net new customers that don't have that established relationship, they have ultimate mobility in where they go for those services or those products. Now with existing customers, it can be a bit stickier, but you have some explaining to do, right? There may be reduced upsell, cross-sell opportunities. But when a consumer is embedded or dedicated to using a service, let's say, it's financial services, there's some friction associated with changing products or services. 
 

Eric Hanselman: And it seems like this is one of those things that you run into, particularly in things like payment methods. And hey, convenience goes a long way and a lot of things like buy now, pay later, those consumers come into that environment relatively uninformed about the security postures of the particular payment methods. And so that's -- it's on the provider to ensure that they've actually got -- can establish a reputation, can not only provide the convenience of whatever the payment method happens to be, but also ensure that customer data is -- or that customers are confident that their data is going to be handled well as part of that transaction. 
 

Paige Bartley: Absolutely. And it's a loyalty issue as well, right? So, we ask the U.S.-based consumers what data privacy features, if any, would make them more loyal to the company or the brand that offered them. These are all fundamental rights associated with several privacy regulations worldwide. But the #1 response there is that they want the ability to permanently delete the information the company has about them and a close second is going to be ability to deny access to that information. Now that's especially relevant in the U.S. market because there is no federal level industry-agnostic privacy law at this point. We have certain laws like HIPAA, they're industry specific. -- which weren't even necessarily written originally as privacy laws. They're more about data portability or other tangential requirements. But when we ask consumers today, what would make you more loyal to this brand, it's that level of trust that they know that, that company, that organization, that brand will do what they say they're going to do with that data. So, if I make a request to delete my data, do I have the assurance that they're actually going to do it effectively? Do I have the assurance that I have the ability to deny access, especially in the U.S. where there is no upfront law, not even in California, where they can deny that collection of data at the onset, right? So, in California, for anyone familiar with the California Consumer Privacy Act or the law that modified California Privacy Rights Act, that is all about denial of sharing data with third parties downstream. So, it's a very U.S. American law in the sense that we assume data will be collected. 
 

Eric Hanselman: First-party data is there no matter what. So... 
 

Paige Bartley: Yes. 
 

Eric Hanselman: Well, but it's interesting because GDPR and EU, you've got a right to be forgotten. But there is that aspect, again, given that we're literally in the wild west here on this part of the planet, there is that -- still that consumer concern. I guess what's your thought about how businesses can address that? And I guess, is that basic policy? Is that transparency? What are ways in which they can communicate that, in fact, they are going to respect what happens with that first-party data, even if we do have requirements that are talking about distribution to third parties and such. Are there particular things that they can do that help to communicate that? Or is this really a consumer-specific kind of engagement? 
 

Paige Bartley: It's not consumer specific. And the reason I suspect that is because when we look into our survey data around enterprise data privacy practices, it's often the businesses that have a mixed model of B2B and B2C that have some of the highest levels of maturity across data privacy functions. This is because, obviously, they're dealing with the nature of consumer regulations, but they're also having to deal with working with other businesses where some of those expectations are more structured and legally negotiated. -- right? So just because there's not a regulation, when you're doing business with another business, a lot of it comes down to contractual terms and SLAs. And there's not really as much wiggle room to hide what you're doing necessarily. So, I think for businesses, the biggest opportunity is obviously not necessarily to meet your regulatory requirements. That's a clear half of the equation. Those are just, in some cases, check boxes to make sure you're doing the right thing. But how do you operationalize it and have it support your business outcomes rather than just being something that's going to be overhead or a cost center for the organization. And that's where we see a lot of the movement forward. 
 

Eric Hanselman: It's interesting that there -- the combined B2B and B2C businesses are those that have combined B2B and B2C businesses as part of their organization. Because it does seem like you start to get into a lot of those issues that are coming -- that we're starting to see on the data security side and the overall security side in financial services regulations. So, things like the UK's DORA regulations for resiliency has a duty to understand the third parties that you work with and how robust, secure and potentially resilient they happen to be. It sounds like maybe those are the kind of things that are coming into play on the data privacy and security side as well. 
 

Paige Bartley: I think it's absolutely true that organizations are trying to function more cohesively in more of a holistic way, like I mentioned, almost joked earlier that you had your compliance, your legal people in the backroom dealing with their specialized tooling for data governance, data privacy, data compliance and all of those related functions. This is not something that can be siloed any longer. And I think this is something we're seeing not just for privacy, but for other business functions as well because everything is moving so much more rapidly in terms of the evolution of technology available to businesses. If you look, for example, within our data, AI and analytics research, we have to publish a whole report every month on just what has happened in Gen AI. So, if you're a business just keeping up with the available technology, the art of the possible, you have to have your data estate in order to be able to appropriately conduct things like retrieval augmented generation, right? It all goes back to garbage in, garbage out, the most overused cliche in data management, but it sticks around because it's true, right? It's not just about security and privacy of this data, it's about the relevancy, about the integrity of it, about the quality of it and how that can contribute to more scalable, repeatable outcomes in initiatives such as AI. But not exclusively, you still have business intelligence. You still have self-service data access. How do you enable all of that safely without having to tinker around with the controls 24/7 to make sure everyone has access to the right thing. 
 

Eric Hanselman: It sounds like this is really driving the nature of the business, the business value is driving a lot of this -- the maturation process for a lot of these capabilities, which is something we don't often see in the tech side. 
 

Paige Bartley: Yes. I will say the flip side of this story is that the people responsible both for decision-making and for the actual execution of common privacy functions for data privacy. This is often still falling on IT. in almost every case. If you look at the NIST privacy framework, which has 5 core functions in each one of those functions, our survey data would suggest that IT is responsible not only for the decision-making, so think about product selection, but also for the execution for the boots on the ground. And there's an imbalance there in who is really directing this effort in selecting relevant technologies because we all know that IT are some of the best firefighters on earth. That's almost their full-time job in many cases. Is it really appropriate or optimal for them to be having these additional responsibilities when they may not be as aligned with the value-added business outcomes. 
 

Eric Hanselman: Because the business outcomes are going to be strategic, and the IT approach is often looking for expediency. It may not necessarily be the -- or maybe the strategic path going forward. Is that where you're headed? 
 

Paige Bartley: I think it's a longer-term assessment, right? IT is very concerned with efficiency, with cost optimization. This is a bigger problem than efficiency and cost optimization. It's about how do you create that greater lifetime value for your customers? Or how do you accelerate your ability to utilize emergent technologies that may not be regulated yet. It's much more of a strategic decision being made. 
 

Eric Hanselman: Well, and to your point, there -- I think we can reasonably expect that somewhere down the road, maybe not in the immediate future, but somewhere down the road, there are going to be stricter regulatory requirements and that organizations are going to be well served if, in fact, they have data management capabilities in place already that not only are going to help on the business side, but will hopefully manage their regulatory and compliance requirements as those evolve as well. 
 

Paige Bartley: Absolutely. Regulations worldwide tend to not decrease over time. That's the overall trend, right? They're sort of multiplicative. But the biggest consideration for organizations in choosing the right practices, the right technologies, the right policies I would say is consistency, right? So, in a vacuum of prescriptive regulatory direction or rules for what to do, you have these business motivators, like, let's say, you want to increase the utilization of AI. You need that internal consistency and structure because once the regulations inevitably do touch your business, you want to be able to prove that you're making good faith efforts along the way to do the right thing. And I've always written in my research that half of the battle of regulation is not becoming compliant. It's proving that you are compliant all along. It's literally half of the equation. Just being compliant is not good enough. You have to be able to clearly demonstrate that you are making those efforts to do so, and consistency is a big part of that when there isn't something prescriptive telling you exactly what to do. 
 

Eric Hanselman: If you don't know precisely what the bar is, at least as long as you can ensure that you are meeting a bar, which should hopefully get you far enough down the road. 
 

Paige Bartley: Absolutely. And creating that internal sort of set of standards is important, not only for future regulation, but to achieve these outcomes we've already discussed, like the acceleration of AI adoption and data access for your workers. 
 

Eric Hanselman: And that should be something that the internal business motivations are really what you're doing is all for. I just need to make sure you do them in a sufficiently compliant fashion to be able to make it all work. Absolutely. A lot of things that need to get done, but interesting to see the good news that it does seem like a lot of this is getting done. So interesting. Well, I will point our listeners at both of these research studies and some of the data associated with them. We'll point to them in the show notes around this. But Pig has been great. So many other questions we could dig into, but we are at time for this episode. But thank you very much. This has been great. 
 

Paige Bartley: Always love being here. 
 

Eric Hanselman: Great to have you back on and hopefully it will be a short time until you're back because there is so much more to talk about but that is it for this episode of Next in Tech. Thanks to our audience for staying with us. And thanks to our production team, Sophie Carr, Feranmi Adeoshun, Kyra Smith, and Adam Kovalsky on the marketing and events teams. I hope you'll join us for our next episode where we're going to be talking about what was going on at the QCon conference actually some of us are just back from the conference in London. Interesting perspectives on what's happening in cloud native, and some fascinating developments on the horizon in terms of where that's going. I hope you'll join us then because there is always something Next in Tech.