Daniel Ciolek, Head of Research and Development at InvGate – Interview Series

Daniel is a passionate IT professional with more than 15 years of experience in the industry. He has a PhD. in Computer Science and a long career in technology research. His interests fall in multiple areas, such as Artificial Intelligence, Software Engineering, and High Performance Computing.

Daniel is the Head of Research and Development at InvGate, where he leads the R&D initiatives. He works along with the Product and Business Development teams to design, implement, and monitor the company's R&D strategy. When he's not researching, he's teaching.

InvGate empowers organizations by providing the tools to deliver seamless service across departments, from IT to Facilities.

When and how did you initially become interested in computer science?

My interest in computer science dates back to my early childhood. I was always fascinated by electronic devices, often finding myself exploring and trying to understand how they worked. As I grew older, this curiosity led me to coding. I still remember the fun I had writing my first programs. From that moment on, there was no doubt in my mind that I wanted to pursue a career in computer science.

You are currently leading R&D initiatives and implementing novel generative AI applications. Can you discuss some of your work?

Absolutely. In our R&D department, we tackle complex problems that can be challenging to represent and solve efficiently. Our work isn't confined to generative AI applications, but the recent advancements in this field have created a wealth of opportunities we're keen to exploit.

One of our main objectives at InvGate has always been to optimize the usability of our software. We do this by monitoring how it's used, identifying bottlenecks, and diligently working towards removing them. One such bottleneck we've encountered often is related to the understanding and utilization of natural language. This was a particularly difficult issue to address without the use of Large Language Models (LLMs).

However, with the recent emergence of cost-effective LLMs, we've been able to streamline these use cases. Our capabilities now include providing writing recommendations, automatically drafting knowledge base articles, and summarizing extensive pieces of text, among many other language-based features.

At InvGate, your team applies a strategy that is called “agnostic AI”. Could you define what this means and why it is important?

Agnostic AI is fundamentally about flexibility and adaptability. Essentially, it's about not committing to a single AI model or provider. Instead, we aim to keep our options open, leveraging the best each AI provider offers, while avoiding the risk of being locked into one system.

You can think of it like this: should we use OpenAI's GPT, Google's Gemini, or Meta's Llama-2 for our generative AI features? Should we opt for a pay-as-you-go cloud deployment, a managed instance, or a self-hosted deployment? These aren't trivial decisions, and they may even change over time as new models are released and new providers enter the market.

The Agnostic AI approach ensures that our system is always ready to adapt. Our implementation has three key components: an interface, a router, and the AI models themselves. The interface abstracts away the implementation details of the AI system, making it easier for other parts of our software to interact with it. The router decides where to send each request based on various factors such as the type of request and the capabilities of the available AI models. Finally, the models perform the actual AI tasks, which may require custom data pre-processing and result formatting processes.

Can you describe the methodological aspects that guide your decision-making process when selecting the most suitable AI models and providers for specific tasks?

For each new feature we develop, we begin by creating an evaluation benchmark. This benchmark is designed to assess the efficiency of different AI models in solving the task at hand. But we don't just focus on performance, we also consider the speed and cost of each model. This gives us a holistic view of each model's value, allowing us to choose the most cost-effective option for routing requests.

However, our process doesn't end there. In the fast-evolving field of AI, new models are constantly being released and existing ones are regularly updated. So, whenever a new or updated model becomes available, we rerun our evaluation benchmark. This lets us compare the performance of the new or updated model with that of our current selection. If a new model outperforms the current one, we then update our router module to reflect this change.

What are some of the challenges of seamlessly switching between various AI models and providers?

Seamlessly switching between various AI models and providers indeed presents a set of unique challenges.

Firstly, each AI provider requires inputs formatted in specific ways, and the AI models can react differently to the same requests. This means we need to optimize individually for each model, which can be quite complex given the variety of options.

Secondly, AI models have different capabilities. For example, some models can generate output in JSON format, a feature that proves useful in many of our implementations. Others can process large amounts of text, enabling us to use a more comprehensive context for some tasks. Managing these capabilities to maximize the potential of each model is an essential part of our work.

Finally, we need to ensure that AI-generated responses are safe to use. Generative AI models can sometimes produce “hallucinations”, or generate responses that are false, out of context, or even potentially harmful. To mitigate this, we implement rigorous post-processing sanitization filters to detect and filter out inappropriate responses.

How is the interface designed within your agnostic AI system to ensure it effectively abstracts the complexities of the underlying AI technologies for user-friendly interactions?

The design of our interface is a collaborative effort between R&D and the engineering teams. We work on a feature-by-feature basis, defining the requirements and available data for each feature. Then, we design an API that seamlessly integrates with the product, implementing it in our internal AI-Service. This allows the engineering teams to focus on the business logic, while our AI-Service handles the complexities of dealing with different AI providers.

This process doesn’t rely on cutting-edge research, but instead on the application of proven software engineering practices.

Considering global operations, how does InvGate handle the challenge of regional availability and compliance with local data regulations?

Ensuring regional availability and compliance with local data regulations is a crucial part of our operations at InvGate. We carefully select AI providers that can not only operate at scale, but also uphold top security standards and comply with regional regulations.

For instance, we only consider providers that adhere to regulations such as the General Data Protection Regulation (GDPR) in the EU. This ensures that we can safely deploy our services in different regions, with the confidence that we are operating within the local legal framework.

Major cloud providers such as AWS, Azure, and Google Cloud satisfy these requirements and offer a broad range of AI functionalities, making them suitable partners for our global operations. Furthermore, we continuously monitor changes in local data regulations to ensure ongoing compliance, adjusting our practices as needed.

How has InvGate's approach to developing IT solutions evolved over the last decade, particularly with the integration of Generative AI?

Over the last decade, InvGate's approach to developing IT solutions has evolved significantly. We've expanded our feature base with advanced capabilities like automated workflows, device discovery, and Configuration Management Database (CMDB). These features have greatly simplified IT operations for our users.

Recently, we've started integrating GenAI into our products. This has been made possible thanks to the recent advancements in LLM providers, who have started offering cost-effective solutions. The integration of GenAI has allowed us to enhance our products with AI-powered support, making our solutions more efficient and user-friendly.

While it's still early days, we predict that AI will become a ubiquitous tool in IT operations. As such, we plan to continue evolving our products by further integrating AI technologies.

Can you explain how the generative AI within the AI Hub enhances the speed and quality of responses to common IT incidents?

The generative AI within our AI Hub significantly enhances both the speed and quality of responses to common IT incidents. It does this through a multi-step process:

Initial Contact: When a user encounters a problem, he or she can open a chat with our AI-powered Virtual Agent (VA) and describe the issue. The VA autonomously searches through the company's Knowledge Base (KB) and a public database of IT troubleshooting guides, providing guidance in a conversational manner. This often resolves the problem quickly and efficiently.

Ticket Creation: If the issue is more complex, the VA can create a ticket, automatically extracting relevant information from the conversation.

Ticket Assignment: The system assigns the ticket to a support agent based on the ticket's category, priority, and the agent's experience with similar issues.

Agent Interaction: The agent can contact the user for additional information or to notify them that the issue has been resolved. The interaction is enhanced with AI, providing writing recommendations to improve communication.

Escalation: If the issue requires escalation, automatic summarization features help managers quickly understand the problem.

Postmortem Analysis: After the ticket is closed, the AI performs a root cause analysis, aiding in postmortem analysis and reports. The agent can also use the AI to draft a knowledge base article, facilitating the resolution of similar issues in the future.

While we've already implemented most of these features, we're continually working on further enhancements and improvements.

With upcoming features like the smarter MS Teams Virtual Agent, what are the expected enhancements in conversational support experiences?

One promising path forward is to extend the conversational experience into a “copilot”, not only capable of replying to questions and taking simple actions, but also taking more complex actions on behalf of the users. This could be useful to improve users’ self-service capabilities, as well as to provide additional powerful tools to agents. Eventually, these powerful conversational interfaces will make AI an ubiquitous companion.

Thank you for the great interview, readers who wish to learn more should visit InvGate.