Synthetic sampling as a preprocessing to increase the efficiency of classification models on imbalanced datasets

Abstract

This study proposed a comprehensive strategy to assist in selecting synthetic sampling models that improve the performance of pattern recognition models with imbalanced class samples. Three imbalanced Quantitative Structure-Activity Relationship (SAR) datasets were used to demonstrate this approach's effectiveness. The strategy involved evaluating the performance metrics of various pattern recognition models through a well-organized experimental design. The factors were analyzed individually for each performance metric and simultaneously using the Derringer-Suich desirability function. The main objective was to gain insights into the impact of synthetic sampling on the performance metrics of pattern recognition models, enabling the development of a selection strategy that promotes globally enhanced predictions while discarding approaches that may harm the models. Synthetic sampling preprocessing was used to balance the datasets before building the models. Among the synthetic sampling preprocessings, three undersampling methods (regular undersampling, cluster-based undersampling, and Tomek-Links) were evaluated, along with three oversampling methods (regular oversampling, Synthetic Minority Oversampling Technique - SMOTE, and Adaptive Synthetic Sampling Approach for Imbalanced Learning - ADASYN), and two hybrid methods (Synthetic Minority Oversampling Technique Tomek-Links - SMOTE-TL and SPIDER). Given the non-Gaussian characteristics of the datasets, confirmed by the multivariate Shapiro-Wilk test, classification models were built using models such as Support Vector Machine with radial basis function, C5.0, Artificial Neural Networks, Extreme Gradient Boosting, and Random Forest. The results revealed that oversampling methods improved sensitivity and precision, while undersampling methods increased precision and specificity. Hybrid methods showed improvements in all performance metrics. However, achieving an adequate balance between classes proved more challenging, especially with a limited number of sample variables. Comparison with the original manuscript data results demonstrated that proper sampling preprocessing can enhance performance metrics for imbalanced datasets.