Report Review: Appen’s Annual State of AI Report

Appen Limited, a global AI leader in providing data sourcing, data preparation, and model evaluation by humans at scale, has released its highly-anticipated annual “State of AI and Machine Learning Report.” 

The State of AI and Machine Learning Report is an annual report focused on the strategies implemented by all sized companies across industries as they further their AI maturity. The latest edition is the eighth released by Appen, and it highlights top approaches to data management and security, responsible AI, and external data providers and their role in advancing progress. 

Main Findings of the Report

The report’s main takeaways involved sourcing, quality, evaluation, adoption, and ethics. 

One of the report’s main findings was that 51% of participants agree that data accuracy is critical to their AI use case. It’s well known that accurate and high-quality data is crucial to the success of AI models, but many business leaders have a significant gap in ideal vs. reality in achieving data accuracy, according to the report. 

Another key takeaway was that companies are increasingly shifting their focus to responsible AI and maturing their strategies. An increasing number of business leaders and technologists are working to improve the data quality that drives AI projects, which promotes inclusive datasets and unbiased models. The report found that 80% of respondents believe data diversity is “extremely important” or “very important.” It also found that 95% of respondents agree that synthetic data will be a key player in creating inclusive datasets. 

Mark Brayan is CEO at Appen. 

“This year’s State of AI report finds that 93% of respondents believe responsible AI is the foundation of all AI projects,” Brayan said. “The problem is, many are facing the challenges of trying to build great AI with poor datasets, and it’s creating a significant roadblock to reaching their goals.” 

Here are some of the other key takeaways from the report: 

• Sourcing: 42% of technologists say the data sourcing stage of the AI lifecycle is very challenging, and business leaders were less likely to report data sourcing as very challenging (24%).

• Quality: More than half of respondents say data accuracy is critical to the success of AI, but only 6% reported achieving data accuracy higher than 90%.

• Evaluation: There’s a strong consensus around the importance of human-in-the-loop machine learning with 81% stating its very or extremely important. 97% reported human-in-the-loop evaluation is important for accurate model performance.

• Adoption: Technologists are split on whether their organization is ahead or even with others in their industry. US respondents are more likely to say their organizations are ahead of others in their industry at adopting AI when compared to European respondents.

• Ethics: 93% of respondents agree that responsible AI is a foundation for all AI projects within their organization. 

Sujatha Sagiraju is Chief Product Officer at Appen. 

“The majority of AI efforts are spent managing data for the AI lifecycle, which means it is an incredible undertaking for AI leads to handle alone – and is the area many are struggling with,” Sagiraju said. “Sourcing high-quality data is critical to the success of AI solutions, and we are seeing organizations emphasize the importance of data accuracy.” 

Wilson Pang is CTO at Appen. 

“Data accuracy is critical to the success of AI and ML models as qualitatively rich data yields better model outputs and consistent processing and decision-making,” Pang said. “For good results, datasets must be accurate, comprehensive, and scalable.” 

You can find the full State of AI and Machine Learning Report here.