AI Can’t Fix Bad Soil: How Companies Can Ready Their Internal Ecosystem for Successful AI Deployment

Though business leaders are often stereotyped as being all about the bottom line, a recent study has demonstrated that over 80% of companies are not tracking ROI on their AI spending. Conversely, those who are tracking ROI are finding it’s not living up to the hype with just a quarter of global CEOs reporting their AI investments are meeting ROI expectations. 

But as the adage goes, “a poor carpenter blames his tools” – in other words, for many, ROI is disappointing because AI deployments have been set up to fail. If we look at a business as a garden, in order for productivity and profits to grow, there are certain steps that have to be taken before deploying a tool like AI in order for it to have the greatest measurable impact. 

Step 1: Identify where humans are essential

Maybe due to the over-promise of capabilities inherent in the marketing of LLM products, there is a common misconception that AI is a plug-and-play affair. In reality, the best AI deployments start by identifying where human oversight is non-negotiable.

For example, when working with a legal services firm, my team and I were tasked with implementing an AI system capable of processing massive volumes of legal documents –  classifying them, extracting key facts, and deciding whether to retain, redact, or delete the files.

While AI handled the heavy-lifting by scanning documents for relevance, tagging sensitive data, and summarizing answers, the results were then passed to human attorneys who could review the work, confirm the legal judgements, and override classifications when necessary.

Not only did this help shield the firm from potential risk, but isolating the cost of automation from the cost of oversight will also make for cleaner ROI audits down the road. 

Step 2: Pinpoint how AI can best augment your people

To maximize the ROI on AI, you need to be selective about where it can best serve your organization. Ideal processes to offload include repetitive or rule-based tasks (ie basic customer service triage or invoice coding), knowledge heavy look-ups like contract clauses, and error-prone data entry, among others.

It’s then equally important that AI models are strategically configured to compliment, rather than disrupt workflow. To do this, map employee workflows into tasks and then label those tasks under one of three process categories: generate, select, or judge. Generative tasks can be handed over to AI, tasks requiring judgement stay with human employees, and tasks that call for selection can be a collaborative process where AI suggests next steps and humans determine the best path forward.

In the legal services example above, AI was handling initial triage by classifying documents (generative), flagging sensitive content (generative), and surfacing likely answers (selective). In this way, the role of human employees shifted from digging through the fine details of documents to verifying results (judgement) – turning work that used to take days into a matter of hours. 

As for ROI, this frees up more time to be spent on the exceptions to the rules, which are where profits hide. 

Step 3: Standardize your training data

Fine tuning LLMs with your enterprise data can unlock competitive advantages, but for AI to be fruitful, it needs nutrient-rich soil, which means good, clean data. Bad or noisy data will poison results and amplify bias. In short, your data discipline dictates output reliability. 

So what does this entail? A large volume and variety of data is important, but it’s equally important that it’s high quality. Inconsistencies across data formats and naming conventions or missing/incomplete fields will negatively affect the quality of raw inputs. Similarly, duplicate or unstructured data pipelines will bloat storage bills and slowdown model performance. 

Therefore it’s imperative that data inputs have quality controls and strong governance – meaning access control and regulatory compliance. Without these filters, you’re not investing in AI, you’re just burning cash on cleanup loops. 

With all the AI hype, it’s understandable that leaders may feel pressured to dive into an implementation as quickly as possible, but taking the time to deploy a model strategically, or fertilizing the ground before planting seeds, will lead to far greater success and returns on investment.