The Consent Gap Hiding Inside Enterprise AI

Enterprise AI conversations tend to start with data. Does the company have enough customer information to train a model, personalize an experience or support an automated decision?

The question that gets missed is whether the company actually has permission to use that data for the specific purpose it has in mind. 

Customer data can be accurate, accessible and valuable while still carrying restrictions. Someone may agree to receive personalized recommendations without agreeing to have their information used to train an AI model. Someone else may want email updates once a month but reject daily text messages.

That is why I have started describing consent and preference management as permissions governance. It answers a simple question: Who, or what, has permission to access this data and use it for a particular purpose?

As AI becomes connected to more customer experiences, companies need to answer that question before the system acts. Documented AI incidents increased from 233 in 2024 to 362 in 2025 — a 55% jump in a single year, and a sign of how much is riding on getting this right.

Available Data Is Not Authorized Data

Most large organizations already have technology for collecting, storing and moving customer information. They may know where the data lives, which customer it belongs to and how to make it available to another application.

Permissions governance answers a different set of questions:

• Why was the information collected?
• What did the customer agree to?
• Which uses does that agreement cover?
• Has the customer changed their mind?
• Which systems need to receive that update?

These questions become especially important when data collected for one purpose is introduced into a new AI use case. Information sitting in a CRM or customer profile is not automatically appropriate for every model, campaign or automated decision.

Consent is one lawful basis for processing personal data, but it is not the only one. Requirements depend on the jurisdiction, data and use case. The broader point is that availability alone does not prove authorization.Regulators are already examining what happens when AI models are developed using unlawfully processed personal data. Questions about how information was obtained do not disappear once that information enters a model.

AI Needs More Than “Opt In” or “Opt Out”

Traditional consent systems often reduce the customer’s decision to two choices: opt in or opt out. That does not reflect how people actually want to interact with companies.

A customer may want product recommendations but reject unrelated advertising. They may accept personalization while declining the use of their information for model training. They may still like a brand’s emails but want to hear from it once a month instead of every day.

A review of 22 consumer-facing AI systems found wide variation in how products request permission and manage personalization, with no consistent standard across the systems studied.

Giving customers the ability to opt down can be a huge unlock. Without that option, someone who becomes frustrated by too many messages may unsubscribe completely. The company loses a willing customer because it never gave that person a more reasonable choice.

The experience becomes invasive the moment that exchange breaks down — when the customer does not remember asking for it or cannot understand why the company is acting on certain information. Businesses can avoid much of that confusion by being direct: Here is the information we want to collect, here is how we want to use it and here is what you will receive in return.

A Customer’s Choice Has to Reach Every System

Capturing a preference on a website is only the beginning. That choice must reach every system using the customer’s information. A customer might update a preference while an email platform, service application or AI environment continues acting on an older answer, and different business units may end up holding conflicting records as a result. A customer may also move from anonymous browsing to an authenticated account without the company ever connecting their earlier choices to the new profile.

This is where consent can no longer sit in a privacy silo that the rest of the business barely knows exists. It has to connect with marketing, customer experience, data governance and the systems deploying AI. A functioning permission layer should maintain the latest record, connect choices across customer profiles, associate each permission with a specific purpose and send changes to every relevant system.

If the central record shows a customer withdrew permission but downstream systems keep using the data anyway, recording the choice hasn’t actually honored it. That problem becomes much larger with AI. One stale permission can influence thousands of recommendations, messages or automated actions before anyone notices.

Privacy Cannot Afford to Be the “Office of No”

None of this works if privacy is only involved after the fact. Privacy teams have earned a reputation in some companies as the office of no, and in many cases that happens because privacy is brought into a project after the important decisions have already been made.

A marketing team develops a campaign, a data team builds a new model or leadership decides it needs to move quickly on AI. Privacy receives the plan near the end and identifies a problem. At that point, the only realistic answer may be no.

Bringing privacy into the process earlier changes the conversation. The team can help the business keep moving while checking the right boxes along the way. Privacy by design is much easier than trying to unwind a system after customer data has already moved through it.

This matters because privacy teams are often small, even inside the world’s largest companies. These small but mighty teams cannot manually follow every customer choice across every application. They need systems that are flexible, easy to use and capable of carrying permissions throughout the business.

Privacy professionals also need the confidence to explain their work in business terms. They sit between the company and the customer and can help both sides get more value from the relationship.

The Audit Trail Has to Reach the AI Decision

When an AI-driven decision is questioned, the company should be able to reconstruct what happened. That includes identifying which customer data was used, what the customer had agreed to, which notice they received and whether the latest choice had reached the system before it acted. Established AI guidance emphasizes documenting data, system requirements and decisions throughout the AI lifecycle.

This evidence supports regulatory inquiries, but it also has practical value. Privacy, marketing, customer experience and data teams need to understand why something went wrong, identify which systems fell out of sync and correct the process.

Capturing consent is not the same as proving it later. A record becomes much harder to reconstruct after data has moved through multiple systems and influenced an automated action.

Trust Is Built or Broken One Interaction at a Time

Trust can sound like a buzzword, but it is the foundation of every customer relationship. Every interaction makes that relationship a little better or a little worse, whether the company realizes it or not.

As adoption grows, companies have more opportunities to create value, but also more opportunities for small governance gaps to affect customers. Before scaling a customer-facing AI application, leaders should understand what data it will use, which permissions apply and how quickly a customer’s updated choice will reach the system.

A recommendation should not rely on information the customer did not expect to be used. A message should not arrive through a channel the customer rejected. An automated system should not act simply because the data is available.

Customers are already telling organizations what they want. The next step is making sure the AI systems acting on their data are capable of listening.