A Review of Dimension Reduction and Feature Selection: A New Perspective

Abstract

With the rapid growth of high-dimensional data, dimensionality reduction has become essential for improving computational efficiency and model performance. This paper presents a comprehensive review of two primary approaches to dimensionality reduction: feature selection (FS) and feature extraction (FE), with particular emphasis on the Chi-Square-based feature selection technique. While Chi-Square has been widely adopted due to its simplicity and effectiveness, its dependence on document frequency introduces notable limitations. This review highlights key developments in feature selection strategies, identifies gaps in existing Chi-Square-based methods, and explores recent improvements proposed in the literature. Finally, the paper offers a new perspective on enhancing Chi-Square by incorporating contextual relevance and hybrid scoring mechanisms, setting the stage for future research.