Is Traditional Machine Learning Still Relevant?

In recent years, Generative AI has shown promising results in solving complex AI tasks. Modern AI models like ChatGPT, Bard, LLaMA, DALL-E.3, and SAM have showcased remarkable capabilities in solving multidisciplinary problems like visual question answering, segmentation, reasoning, and content generation.

Moreover, Multimodal AI techniques have emerged, capable of processing multiple data modalities, i.e., text, images, audio, and videos simultaneously. With these advancements, it’s natural to wonder: Are we approaching the end of traditional machine learning (ML)?

In this article, we’ll look at the state of the traditional machine learning landscape concerning modern generative AI innovations.

What is Traditional Machine Learning? – What are its Limitations?

Traditional machine learning is a broad term that covers a wide variety of algorithms primarily driven by statistics. The two main types of traditional ML algorithms are supervised and unsupervised. These algorithms are designed to develop models from structured datasets.

Standard traditional machine learning algorithms include:

• Regression algorithms such as linear, lasso, and ridge.
• K-means Clustering.
• Principal Component Analysis (PCA).
• Support Vector Machines (SVM).
• Tree-based algorithms like decision trees and random forest.
• Boosting models such as gradient boosting and XGBoost.

Limitations of Traditional Machine Learning

Traditional ML has the following limitations:

1. Limited Scalability: These models often need help to scale with large and diverse datasets.
2. Data Preprocessing and Feature Engineering: Traditional ML requires extensive preprocessing to transform datasets as per model requirements. Also, feature engineering can be time-consuming and requires multiple iterations to capture complex relationships between data features.
3. High-Dimensional and Unstructured Data: Traditional ML struggles with complex data types like images, audio, videos, and documents.
4. Adaptability to Unseen Data: These models may not adapt well to real-world data that wasn’t part of their training data.

Neural Network: Moving from Machine Learning to Deep Learning & Beyond

Neural network (NN) models are far more complicated than traditional Machine Learning models. The simplest NN – Multi-layer perceptron (MLP) consists of several neurons connected together to understand information and perform tasks, similar to how a human brain functions.

Advances in neural network techniques have formed the basis for transitioning from machine learning to deep learning. For instance, NN used for computer vision tasks (object detection and image segmentation) are called convolutional neural networks (CNNs), such as AlexNet, ResNet, and YOLO.

Today, generative AI technology is taking neural network techniques one step further, allowing it to excel in various AI domains. For instance, neural networks used for natural language processing tasks (like text summarization, question answering, and translation) are known as transformers. Prominent transformer models include BERT, GPT-4, and T5. These models are creating an impact on industries ranging from healthcare, retail, marketing, finance, etc.

Do We Still Need Traditional Machine Learning Algorithms?

While neural networks and their modern variants like transformers have received much attention, traditional ML methods remain crucial. Let us look at why they are still relevant.

1. Simpler Data Requirements

Neural networks demand large datasets for training, whereas ML models can achieve significant results with smaller and simpler datasets. Thus, ML is favored over deep learning for smaller structured datasets and vice versa.

2. Simplicity and Interpretability

Traditional machine learning models are built on top of simpler statistical and probability models. For example, a best-fit line in linear regression establishes the input-output relationship using the least squares method, a statistical operation.

Similarly, decision trees make use of probabilistic principles for classifying data. The use of such principles offers interpretability and makes it easier for AI practitioners to understand the workings of ML algorithms.

Modern NN architectures like transformer and diffusion models (typically used for image generation like Stable Diffusion or Midjourney) have a complex multi-layered network structure. Understanding such networks requires an understanding of advanced mathematical concepts. That’s why they are also referred to as ‘Black Boxes.’

3. Resource Efficiency

Modern neural networks like Large Language Models (LLMs) are trained on clusters of expensive GPUs per their computational requirements. For example, GPT4 was reportedly trained on 25000 Nvidia GPUs for 90 to 100 days.

However, expensive hardware and lengthy training time are not feasible for every practitioner or AI team. On the other hand, the computational efficiency of traditional machine learning algorithms allows practitioners to achieve meaningful results even with constrained resources.

4. Not All Problems Need Deep Learning

Deep Learning is not the absolute solution for all problems. Certain scenarios exist where ML outperforms deep learning.

For instance, in medical diagnosis and prognosis with limited data, an ML algorithm for anomaly detection like REMED delivers better results than deep learning. Similarly, traditional machine learning is significant in scenarios with low computational capacity as a flexible and efficient solution.

Primarily, the selection of the best model for any problem depends on the needs of the organization or practitioner and the nature of the problem at hand.

Machine Learning in 2023

Image Generated Using Leonardo AI

In 2023, traditional machine learning continues to evolve and is competing with deep learning and generative AI. It has several uses in the industry, particularly when dealing with structured datasets.

For instance, many Fast-Moving Consumer Goods (FMCG) companies deal with bulks of tabular data relying on ML algorithms for critical tasks like personalized product recommendations, price optimization, inventory management, and supply chain optimization.

Further, many vision and language models are still based on traditional techniques, offering solutions in hybrid approaches and emerging applications. For example, a recent study titled “Do We Really Need Deep Learning Models for Time Series Forecasting?” has discussed how gradient-boosting regression trees (GBRTs) are more efficient for time series forecasting than deep neural networks.

ML's interpretability remains highly valuable with techniques like SHAP (Shapley Additive Explanations) and LIME (Local Interpretable Model-agnostic Explanations). These techniques explain complex ML models and provide insights about their predictions, thus helping ML practitioners understand their models even better.

Finally, traditional machine learning remains a robust solution for diverse industries addressing scalability, data complexity, and resource constraints. These algorithms are irreplaceable for data analysis and predictive modeling and will continue to be a part of a data scientist's arsenal.

If topics like this intrigue you, explore Unite AI for further insights.