AI Models Struggle To Predict People’s Irregular Behavior During Covid-19 Pandemic

Retail and service companies around the world make use of AI algorithms to predict customer behaviors, take stock of inventory, estimate marketing impacts, and detect possible instances of fraud. The machine learning models used to make these predictions are trained on patterns derived from the normal, everyday activity of people. Unfortunately, our day-to-day activity has changed during the coronavirus pandemic, and as MIT Technology Review reported current machine learning models are being thrown off as a result. The severity of the problem differs from company to company, but many models have been negatively impacted by the sudden change in people’s behavior over the course of the past few weeks.

When the coronavirus pandemic occurred, the purchasing habits of people shifted dramatically. Prior to the onset of the pandemic, the most commonly purchased objects were things like phone cases, phone chargers, headphones, kitchenware. After the start of the pandemic, Amazon’s top 10 search terms became things like Clorox wipes, Lysol spray, paper towels, hand sanitizer, face masks, and toilet paper. Over the course of the last week of February, the top Amazon searches all became related to products people required to shelter themselves from Covid-19. The correlation of Covid-19 related product searches/purchases and the spread of the disease is so reliable that it can be used to track the spread of the pandemic across different geographical regions. Yet machine learning models break down when the model’s input data is too different from the data used to train the model.

The volatility of the situation has made automation of supply chains and inventories difficult. Rael Cline, the CEO of London-based consultancy Nozzle,  explained that companies are trying to optimize for the demand of toiler paper one week ago, while “this week everyone wants to buy puzzles or gym equipment.”

Other companies have their own share of problems. One company provides investment recommendations based on the sentiment of various news articles, but because the sentiment of news articles at the moment is often more pessimistic than usual, the investing advice could be heavily skewed toward the negative. Meanwhile, a streaming video company utilized recommendation algorithms to suggest content to viewers, but as many people suddenly subscribed to the service their recommendations started to fall from the mark. Yet another company responsible for supplying retailers in India with condiments and sauces discovered bulk orders broke their predictive models.

Different companies are handling the problems caused by pandemic behavior patterns in different ways. Some companies are simply revising their estimates downward. People still continue to subscribe to Netflix and purchase products on Amazon, but they have cut back on luxury spending, postponing purchases on big-ticket items. In a sense, people’s spending behaviors can be conceived of as a contraction of their usual behavior.

Other companies have had to get more hand-on with their models and have engineers make important tweaks to the model and it’s training data. For example, Phrasee is an AI firm that utilizes natural language processing and generation models to create copy and advertisements for a variety of clients. Phrasee always has engineers check what text the model generates, and the company has begun manually filtering out certain phrases in its copy. Phrasee has decided to ban the generation of phrases that might encourage dangerous activities during a time of social distancing, phrases like “party wear”. They have also decided to restrict terms that could lead to anxiety, like “brace yourself”, “buckle up”, or “stock up”.

The Covid-19 crisis has demonstrated that freak events can throw off even highly-trained models that are typically reliable, as things can get much worse than the worst-case scenarios that are typically included within training data. Rajeev Sharma, CEO of AI consultancy Pactera Edge, explained to MIT Technology Review that machine learning models could be made more reliable by being trained on freak events like the Covid-19 pandemic and the Great Depression, in addition to the usual upwards and downwards fluctuations.