Feature Selection Techniques for High-Dimensional Data Analysis: Applications, Challenges, and Future Directions
DOI:
https://doi.org/10.63746/njtd.v22i1.2943Keywords:
Application area, Feature selection techniques, Future directions, High-dimensional data, Machine learning modelsAbstract
The curse of dimensionality is a major concern in high-dimensional data. As the challenge persists, researchers have adapted the feature selection process to reduce the dimensionality by selecting the important features. This research aims to conduct a systematic literature review on feature selection techniques in high-dimensional data. The goal is to examine trends in feature selection techniques between 2015 and 2024 to help guide researchers in conducting future research. The study examined five (5) research questions on feature selection techniques in high-dimensional datasets to discover prominently used feature selection techniques between 2015 and 2024. The study also examined machine learning models adapted over the years and feature selection issues identified from other studies suggested solutions to these issues, identified the application for the feature selection techniques and carried out potential future research directions. Five internet-based databases were used to select 40 primary studies to carry out the comprehensive study. The findings reviewed 50 feature selection techniques with the chi-square technique as the most prominent technique used by researchers. A total of 37 machine learning models were identified with support vector machine as the most prominent model used by researchers. This review can help researchers to enhance the efficacy of their techniques and models in future investigations.
References
Aksu, D.; S. Üstebay, M.A. Aydin & T. Atmaca. (2018). Intrusion Detection with Comparative Analysis of Supervised Learning Techniques and Fisher Score Feature Selection Algorithm. 24th IFIP World Computer Congress, WCC, Poznan, Poland, 141–149. https://doi.org/10.1007/978-3-030-00840-6_16
Al-shalif, S. A.; N. Senan, F. Saeed, W. Ghaban, N. Ibrahim, M. Aamir & W. Sharif. (2024). A systematic literature review on meta-heuristic based feature selection techniques for text classification. PeerJ Computer Science, 10, e2084. https://doi.org/10.7717/peerj-cs.2084
Alejandre, F.V., Cortes, N.C. & Anaya, E.A. (2017). Feature selection to detect botnets using machine learning algorithms. International Conference on Electronics, Communications and Computers (CONIELECOMP) (pp. 1–7), Cholula, Mexico. https://doi.org/10.1109/CONIELECOMP.2017.7891834.
Alhassan, A.M. & Wan Zainon, W.M.N. (2021). Review of Feature Selection, Dimensionality Reduction and Classification for Chronic Disease Diagnosis., IEEE Access, 9, 87310–87317. https://doi.org/10.1109/ACCESS.2021.3088613.
Ali, A.; S. Shaukat, M. Tayyab, M.A. Khan, J.S. Khan, Arshad, & J. Ahmad. (2020). Network Intrusion Detection Leveraging Machine Learning and Feature Selection. IEEE 17th International Conference on Smart Communities: Improving Quality of Life Using ICT, IoT and AI (HONET), 49–53, Charlotte, NC, USA. https://doi.org/10.1109/HONET50430.2020.9322813
Arora, N. & Kaur, P.D. (2020). A Bolasso based consistent feature selection enabled random forest classification algorithm: An application to credit risk assessment. Applied Soft Computing, 86(1), 105936. https://doi.org/10.1016/j.asoc.2019.105936.
Ay, Ş.; E. Ekinci & Z. Garip. (2023). A comparative analysis of meta-heuristic optimization algorithms for feature selection on ML-based classification of heart-related diseases. The Journal of Supercomputing,79(11), 11797–11826. https://doi.org/10.1007/s11227-023-05132-3
Bahmaninezhad, F. & Hansen, J. H. L. (2016). Generalized Discriminant Analysis (GDA) for Improved i-Vector Based Speaker Recognition. Interspeech 2016, 3643–3647. https://doi.org/10.21437/Interspeech.2016-1523
Begum, S.; D. Chakraborty & R. Sarkar. (2015). Data Classification Using Feature Selection and kNN Machine Learning Approach. 2015 International Conference on Computational Intelligence and Communication Networks (CICN), 811–814. https://doi.org/10.1109/CICN.2015.165
Bouchlaghem, Y.; Y. Akhiat & S. Amjad. (2022). Feature Selection: A Review and Comparative Study. E3S Web of Conferences, 351, 01046. https://doi.org/10.1051/e3sconf/202235101046
Edlund, C.; T. R. Jackson, N. Khalid, N. Bevan, T. Dale, A. Dengel, S. Ahmed, J. Trygg & R. Sjögren. (2021). LIVECell—A large-scale dataset for label-free live cell segmentation. Nature Methods, 18(9), 1038–1045. https://doi.org/10.1038/s41592-021-01249-6
Eke, C.I., Norman, A. A., Shuib, L. & Nweke, H. F. (2019). A Survey of User Profiling: State-of-the-Art, Challenges, and Solutions. IEEE Access, 7(3), 144907–144924. https://doi.org/10.1109/ACCESS.2019.2944243.
Eke, C. I.; A. A. Norman & M. Mulenga. (2023). Machine learning approach for detecting and combating bring your own device (BYOD) security threats and attacks: a systematic mapping review. Artificial Intelligence Review, 56(8), 8815–8858. https://doi.org/10.1007/s10462-022-10382-3
Eke, C.I., Norman, A.A. & Shuib, L. (2021). Multi-feature fusion framework for sarcasm identification on twitter data: A machine learning based approach. PLOS ONE, 16(6), 0252918. https://doi.org/10.1371/journal.pone.0252918.
Emmanuel, I.; Y. Sun & Z. Wang. (2024). A machine learning-based credit risk prediction engine system using a stacked classifier and a filter-based feature selection method. Journal of Big Data, 11(2), 23. https://doi.org/10.1186/s40537-024-00882-0
Garcia-Chimeno, Y.; B. Garcia-Zapirain; M. Gomez-Beldarrain; B. Fernandez-Ruanova & J. C. Garcia-Monco. (2017). Automatic migraine classification via feature selection committee and machine learning techniques over imaging and questionnaire data. BMC Healthcare Informatics and Decision Making, 17(1), 38-40. https://doi.org/10.1186/s12911-017-0434-4
Hashemi, A.; M. Joodaki; N.Z. Joodaki and M.B. Dowlatshahi. (2022). Ant colony optimization equipped with an ensemble of heuristics through multi-criteria decision making: A case study in ensemble feature selection. Applied Soft Computing, 124, 109046. https://doi.org/10.1016/j.asoc.2022.109046
Higgins JPT & Green S. (2008). Cochrane Handbook for Systematic Reviews of Interventions (J. P. Higgins & S. Green (eds.)). Wiley. https://doi.org/10.1002/9780470712184
Hopf, K. & Reifenrath, S. (2021). Filter Methods for Feature Selection in Supervised Machine Learning Applications - Review and Benchmark. arXiv. https://doi.org/10.48550/arXiv.2111.12140
Sun, J., Passi, K., & Jain, C. K. (2016). Improved microarray data analysis using feature selection methods with machine learning methods. In 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 1527–1534, Shenzhen. https://doi.org/10.1109/BIBM.2016.7822748
Karimi, F.; M.B. Dowlatshahi & A. Hashemi. (2023). SemiACO: A semi-supervised feature selection based on ant colony optimization. Expert Systems with Applications, 214, 119130. https://doi.org/10.1016/j.eswa.2022.119130
Khan, F.; I. Tarimer; H.S. Alwageed; B.C. KaradaÄŸ; M. Fayaz; A.B. Abdusalomov & Y.I. Cho. (2022). Effect of Feature Selection on the Accuracy of Music Popularity Classification Using Machine Learning Algorithms. Electronics, 11(21), 3518. https://doi.org/10.3390/electronics11213518
Khourdifi, Y. & Bahaj, M. (2019). Heart Disease Prediction and Classification Using Machine Learning Algorithms Optimized by Particle Swarm Optimization and Ant Colony Optimization. International Journal of Intelligent Engineering and Systems, 12(1), 242–252. https://doi.org/10.22266/ijies2019.0228.24
Kitchenham, B. & Brereton, P. (2013). A systematic review of systematic review process research in software engineering. Information and Software Technology, 55(12), 2049–2075. https://doi.org/10.1016/j.infsof.2013.07.010
Kitchenham B, C. S. (2007). Guidelines for performing Systematic Literature Reviews in Software Engineering. EBSE Technical Report EBSE-2007-01.
Lv, G.; Y. Xia; Z. Qi; Z. Zhao; L. Tang; C. Chen; S.Yang; Q. Wang & L. Gu. (2023). LncRNA–protein interaction prediction with reweighted feature selection. BMC Bioinformatics, 24(1), 410. https://doi.org/10.1186/s12859-023-05536-1
Mahesh, B. (2020). Machine Learning Algorithms - A Review. International Journal of Science and Research(IJSR), 9(1), 381–386. https://doi.org/10.21275/ART20203995
Mahindru, A. & Sangal, A. L. (2021). FSDroid:- A feature selection technique to detect malware from Android using Machine Learning Techniques. Multimedia Tools and Applications, 80(9), 13271–13323. https://doi.org/10.1007/s11042-020-10367-w
Masoudi-Sobhanzadeh, Y.; H. Motieghader & A. Masoudi-Nejad. (2019). FeatureSelect: a software for feature selection based on machine learning approaches. BMC Bioinformatics, 20(1), 170. https://doi.org/10.1186/s12859-019-2754-0
Mohamed Shaffril, H. A.; S. F. Samsuddin & A. Abu Samah. (2021). The ABC of systematic literature review: the basic methodological guidance for beginners. Quality & Quantity, 55(4), 1319–1346. https://doi.org/10.1007/s11135-020-01059-6
Mohy-eddine, M.; A. Guezzaz, S. Benkirane and M. Azrour. (2023). An efficient network intrusion detection model for IoT security using K-NN classifier and feature selection. Multimedia Tools and Applications, 82(15), 23615–23633. https://doi.org/10.1007/s11042-023-14795-2
Muhammad, M. U.; R. Jiadong; N. S. Muhammad; M. Hussain & I. Muhammad. (2019). Principal Component Analysis of Categorized Polytomous Variable-Based Classification of Diabetes and Other Chronic Diseases. International Journal of Environmental Research and Public Health, 16(19), 3593. https://doi.org/10.3390/ijerph16193593
Naik, N., & Mohan, B. R. (2019). Optimal Feature Selection of Technical Indicator and Stock Prediction Using Machine Learning Technique. International Conference on Emerging Technologies in Computer Engineering 985(1) 261–268. https://doi.org/10.1007/978-981-13-8300-7_22
Narisetty, N. N. (2020). Bayesian model selection for high-dimensional data Handbook of Statistics 43(1). 207–248. https://doi.org/10.1016/bs.host.2019.08.001
Noroozi, Z.; A. Orooji & L. Erfannia. (2023). Analyzing the impact of feature selection methods on machine learning algorithms for heart disease prediction. Scientific Reports, 13(1), 22588. https://doi.org/10.1038/s41598-023-49962-w
Oreski, D.; S. Oreski & B. Klicek. (2017). Effects of dataset characteristics on the performance of feature selection techniques. Applied Soft Computing, 52, 109–119. https://doi.org/10.1016/j.asoc.2016.12.023
Pathak, A.; U. Barman, and T. S. Kumar. (2024). Machine learning approach to detect android malware using feature-selection based on feature importance score. Journal of Engineering Research. https://doi.org/10.1016/j.jer.2024.04.008
Petersen, K.; R. Feldt, S. Mujtaba & M. Mattsson. (2008). Systematic Mapping Studies in Software Engineering. Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, 68-77, Bari, Italy.
Pudjihartono, N.; T. Fadason; A.W. Kempa-Liehr & J. M. O’Sullivan. (2022). A Review of Feature Selection Methods for Machine Learning-Based Disease Risk Prediction. Frontiers in Bioinformatics, 2. https://doi.org/10.3389/fbinf.2022.927312
Ramya, R. S. & Kumaresan, S. (2015). Analysis of feature selection techniques in credit risk assessment. 2015 International Conference on Advanced Computing and Communication Systems,1–6. https://doi.org/10.1109/ICACCS.2015.7324139
Rasitha, G. (2016). Predicting Thyroid Disease using Linear Discriminant Analysis (LDA) Data Mining Technique. Communications on Applied Electronics, 4(1), 4–6. https://doi.org/10.5120/cae2016651990
Rostami, M.; K. Berahmand; E. Nasiri & S. Forouzandeh. (2021). Review of swarm intelligence-based feature selection methods. Engineering Applications of Artificial Intelligence, 100, 104210. https://doi.org/10.1016/j.engappai.2021.104210
Sarker, I. H. (2021). Machine Learning: Algorithms, Real-World Applications and Research Directions. SN Computer Science, 2(3), 160. https://doi.org/10.1007/s42979-021-00592-x
Semwal, V. B.; J. Singha; P. K. Sharma; A. Chauhan & B. Behera. (2017). An optimized feature selection technique based on incremental feature analysis for bio-metric gait data classification. Multimedia Tools and Applications, 76(22), 24457–24475. https://doi.org/10.1007/s11042-016-4110-y
Shafiq, M.; X. Yu; A. K. Bashir; H. N. Chaudhry & D. Wang. (2018). A machine learning approach for feature selection traffic classification using security analysis. The Journal of Supercomputing, 74(10), 4867–4892. https://doi.org/10.1007/s11227-018-2263-3
Sivaranjani, S.; S. Ananya; J. Aravinth & R. Karthika. (2021). Diabetes Prediction using Machine Learning Algorithms with Feature Selection and Dimensionality Reduction. 7th International Conference on Advanced Computing and Communication Systems (ICACCS), 141–146. https://doi.org/10.1109/ICACCS51430.2021.9441935
Spencer, R., F. Thabtah, N. Abdelhamid & M. Thompson. (2020). Exploring feature selection and classification methods for predicting heart disease. Digital Health, 6, 205520762091477. https://doi.org/10.1177/2055207620914777
Srinivasa Rao, P. C.; A. J. Sravan Kumar, Q. Niyaz, P. Sidike & V. K. Devabhaktuni (2021). Binary chemical reaction optimization based feature selection techniques for machine learning classification problems. Expert Systems with Applications, 167, 114169. https://doi.org/10.1016/j.eswa.2020.114169
Tang, H.; Z.-D. Su; Wei, H.-H..W. Chen, & H. Lin. (2016). Prediction of cell-penetrating peptides with feature selection techniques. Biochemical and Biophysical Research Communications, 477(1), 150–154. https://doi.org/10.1016/j.bbrc.2016.06.035
Theng, D. & Bhoyar, K. K. (2024). Feature selection techniques for machine learning: a survey of more than two decades of research. Knowledge and Information Systems, 66(3), 1575–1637. https://doi.org/10.1007/s10115-023-02010-5
Thirumoorthy, K. & Muneeswaran, K. (2022). Feature Selection for Text Classification Using Machine Learning Approaches. National Academy Science Letters, 45(1), 51–56. https://doi.org/10.1007/s40009-021-01043-0
Toğaçar, M.; B. Ergen, Z. Cömert & F. Özyurt. (2020). A Deep Feature Learning Model for Pneumonia Detection Applying a Combination of mRMR Feature Selection and Machine Learning Models. IRBM, 41(4), 212–222. https://doi.org/10.1016/j.irbm.2019.10.006
Trivedi, S. K. (2020). A study on credit scoring modeling with different feature selection and machine learning approaches. Technology in Society, 63, 101413. https://doi.org/10.1016/j.techsoc.2020.101413
Tuba, E.; I. Strumberger, T. Bezdan, N. Bacanin & M. Tuba (2019). Classification and Feature Selection Method for Healthcare Datasets by Brain Storm Optimization Algorithm and Support Vector Machine. Procedia Computer Science, 162, 307–315. https://doi.org/10.1016/j.procs.2019.11.289
Uçar, M. K. (2020). Classification Performance-Based Feature Selection Algorithm for Machine Learning: P-Score. IRBM, 41(4), 229–239. https://doi.org/10.1016/j.irbm.2020.01.006
Upadhyay, D.; J. Manero, M. Zaman and S. Sampalli. (2021). Gradient Boosting Feature Selection With Machine Learning Classifiers for Intrusion Detection on Power Grids. IEEE Transactions on Network and Service Management, 18(1), 1104–1116. https://doi.org/10.1109/TNSM.2020.3032618
Uten E.P., Eke, C.I. and Moses, T. (2025). A feature selection and scoring scheme for dimensionality reduction in a machine learning task. J. Nig. Soc. Phys. Sci, 7, 2273. https://doi.org/10.46481/jnsps.2025.2273.
Xu, Y.; H. Huang; A.A. Heidari; W. Gui; X. Ye; Y. Chen; H. Chen & Z. Pan. (2021). MFeature: Towards high performance evolutionary tools for feature selection. Expert Systems with Applications,186, 115655. https://doi.org/10.1016/j.eswa.2021.115655
Yang, L.; H. Zhang, H. Shen, X. Huang, X. Zhou, G. Rong & D. Shao. (2021). Quality Assessment in Systematic Literature Reviews: A Software Engineering Perspective. Information and Software Technology, 130, 106397. https://doi.org/10.1016/j.infsof.2020.106397
Zaini, N. A. M., & Awang, M. K. (2023). Hybrid Feature Selection Algorithm and Ensemble Stacking for Heart Disease Prediction. International Journal of Advanced Computer Science and Applications, 14(2). https://doi.org/10.14569/IJACSA.2023.0140220
Zhang, P. (2019). A novel feature selection method based on global sensitivity analysis with application in machine learning-based prediction model. Applied Soft Computing, 85, 105859. https://doi.org/10.1016/j.asoc.2019.105859
Zhao, Z.; R. Anand & M. Wang. (2019). Maximum Relevance and Minimum Redundancy Feature Selection Methods for a Marketing Machine Learning Platform. 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA), 442–452. https://doi.org/10.1109/DSAA.2019.00059
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Nigerian Journal of Technological Development
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
In accordance with the Copyright Act of 1976, which became effective January 1, 1978, the following statement signed by each author must accompany the manuscript submitted: "I, the undersigned author, transfer all copyright ownership of the manuscript referenced above to the Nigerian Journal of Technological Development, in the event the work is published. I warrant that the article is original, does not infringe upon any copyright or other proprietary right of any third party, is not under consideration by another journal, and has not been published previously. I have reviewed and approve the submitted version of the manuscript and agree to its publication in the Nigerian Journal of Technological Development." A copyright transfer form should be downloaded from the NJTD Website ( http://njtd.com.ng/index.php/njtd). Author(s) will be consulted, whenever possible, regarding republication of material. All authors must have access to the data presented and the authors and sponsor (if applicable) must agree to share original data with the editor if requested.
