Thought Leaders
AI Meets Analytics Engineering: AI Maturity for Process
Companies in all kinds of industries and specialties are feeling the need to dive headfirst into the world of AI, and that includes analytics engineering. The opportunities are real, and they’re exciting, but organizations looking to make the most of them should implement their process structure in a thoughtful and realistic way, based on their AI maturity. Let’s explore how.
AI Maturity Levels
When describing the complexity of the AI you’re using, AI maturity levels are helpful as a simple, clarifying framework.
Level 1: Assisted Intelligence (Automation): Basic automation of repetitive tasks and workflows. Examples: chatbots with fixed replies, website crawlers, internal search tools.
Level 2: Augmented Intelligence (Guided Analysis): You tell the AI the method/model and what to analyze, and it does the rest. Examples: ad bidding algorithms, content summarizers.
Level 3: Autonomous Intelligence (Self-Learning AI): AI picks out methods, finds pattern, and offers recommendations. Examples: self-driving cars, autonomous stock trading robots.
Pressure to “AI”
Of course, today there is so much pressure to “be at 3.” But that largely depends on your company’s resources, assets, core capabilities, knowledge and people. The best approach is to start where it makes sense, even if it’s at Level 1.
Right now, there are millions of articles online that bombard you with ideas for AI. But ideas don’t bring home the bacon. According to Adobe, only 12% of companies currently have working AI solutions that demonstrate clear ROI. Most of these are still in pilot phases, evaluating effectiveness, or facing challenges in scaling AI initiatives. Many teams struggle to identify where AI can be meaningfully integrated into their workflows and even when a use case is identified, many organizations lack the internal capabilities to build custom AI tools or find reliable external solutions.
Let’s stick to realistic applications. There are great ways to develop gradual AI usage into realistic elements of work for leaders who are ready to scale up responsibly.
What can be automated when you build analytics for an ecommerce/SaaS product
In my view, there are two common processes within successful analytics engineering teams that are great candidates to build AI maturity:
- AI for Knowledge Management & Onboarding
- AI for QA & Audit Automation
1. AI for Knowledge Management & Onboarding”
Documentation is a key tool to keep track of complex systems. As per the Process Framework, SDR (Solution Design Reference) needs to document every one of the five steps. It could look like this:
- Process #1: Documenting expected outcomes.
- Process #2: Keeping track of key data collection stories being regularly checked.
- Process #3: Noting the history of 3-rd party Tech Stakeholders data requirements.
- Process #4: Detailing the data layer across all apps and surfaces.
- Process #5: Describing and detailing the engineering architecture with diagrams, hierarchies and requirements.
Now, let’s add some AI to this.
Level 1
At level 1, you can start using internal AI chatbots for document retrieval.
Many companies now have internal chatbots that can be trained on proprietary documentation. If you don’t have an internal chatbot, you can use incognito mode, or redact your documentation before feeding it to the bot.
Feed your bot your SDRs, QA manuals, naming conventions and implementation standards.
After a successful release, feed your AI project notes or implementation plans, to add to the documentation. Then, ask the AI the following questions:
- “What is the best method to implement X if I want to use the same logic as Y?”
- “What data is required to be collected for tracking purchases?”
- “What tags are missing from this new product page?”
The result of this process is that you spend less time skimming docs or pinging coworkers, new team members can self-serve answers and tribal knowledge becomes scalable.
There are some caveats here. This method really works if you do a good job of keeping up with your documentation, and it only becomes scalable if you train and require your people to use the tool.
Level 2
If this works for your department, think about scaling up to connecting your chatbot directly to your tech. You can automate the automation.
Level 3
I’m sure the sky’s the limit here. The direction I would pursue would be building a proactive AI that flags inconsistencies and offers improvements. The reality is that few companies are reaching this level, and I’m writing this article for the bulk of us, who are still learning the ropes.
2. AI for QA & Audit Automation
Regularly auditing your data collection methods is one of the best practices for the Process Framework. Very often, the auditor will either be a QA team, or alternatively an auditing tool can be used. For example, ObservePoint is a decked out and highly customizable tool, that lets you build highly intricate audit flows. Even with a robot, you could always use some more AI, right?
Level 1
Let’s start by automating the technicalities. Building journeys in robot audit tools like ObservePoint is often technical. And requires a lot of support. To automate some of these repetitive technical tasks, while building audit journeys, you can once again ask for the help of an AI chatbot. Ask the AI the following questions:
“Give me the CSS selector for the “Next Step” button.”
“Write custom code that automatically opts into all cookies.”
The result of this should be greater ease of use when working with highly technical tools, faster triage and troubleshooting and less dependency on support and frontend developers.
Some caveats here are that if you don’t use a robot crawler for your data audits, you may be using a QA team. The QA team can look into adopting automations for common steps. Start small, ramp up once ready; your next steps only become clearer once you take the first step.
Level 2
For Level 2 AI usage, look into integrating your Chatbot, with the tool directly, while avoiding manual prompting of a chatbot.
Level 3
Lastly, for Level 3 usage, the sky is again the limit. Discover how you can make your automations more proactive in spotting improvements and recommending solutions. Only go this route if you feel you are comfortably navigating Level 2.
What Not to Automate (Yet)
Let’s consider Best Practice #3: Collaborate with 3rd-party tech stakeholders. This is something humans still do best. You can use AI to prep for vendor calls, summarize contracts or draft integration outlines. But for now, relationship building stays human.
Final Thoughts
Even if you don’t have a budget for custom AI development, you can start with the tools you already have. A good process and a good chatbot go a long way.
Start simple, with Level 1 or 2, and let your team get comfortable. Once you see where AI is saving time and boosting consistency, you’ll know where to invest in more advanced tools. The hardest part of AI adoption is often figuring out where you need it in the first place. Once you’ve established that baseline, try moving things up a level with your team, and see how much smoother AI-powered analytics engineering can be.
Want to learn more about AI in analytics engineering, and more specifically, best processes for clean data collection? Check out my article about creating a Clean Ecommerce Data Framework.
