Predicting Student Success in Higher Education: Understanding the Impact of Ensemble Learning Architecture on Model Performance

Abstract

Predicting student success in higher education has become increasingly vital, and machine learning (ML) algorithms offer a promising avenue for achieving this goal. This abstract explores the application of supervised ML algorithms, including Random Forest, Support Vector Machines (SVMs), Gradient Boosting, Extreme Gradient Boosting (XGBoost), and Deep Learning, to forecast student outcomes. Notably, the study delves into the specific use of XGBoost, a robust ensemble learning algorithm, which has shown exceptional accuracy in predicting academic performance. A synthesis of relevant literature, including studies by Ojajuni et al. (2021), Yakubu and Abubakar (2022), and McKinsey (2022), establishes the foundation for understanding ML's impact on student success prediction. The abstract then focuses on studies by Beaulac and Rosenthal (2019), Alyahyan et al. (2020), and Alqurashi (2021), showcasing how Random Forest, SVMs, and XGBoost have been effectively employed in higher education contexts. The primary objective of this study is to scrutinize the influence of ensemble learning architecture, particularly XGBoost, on model performance. It aims to investigate the interplay between prediction accuracy, efficiency, and trade-offs with ensemble size and individual learner complexity. Additionally, the study proposes modifications to enhance XGBoost's performance, acknowledging the intricate relationship between ensemble size and individual learner complexity. The abstract concludes by addressing the ongoing debate surrounding XGBoost and its comparison with LightGBM. Insights into their impact on prediction accuracy and efficiency, considering computational resources, are discussed. The trade-off analysis between ensemble size and learner complexity is emphasized, providing guidelines for striking the optimal balance. Finally, novel modifications to XGBoost are proposed, highlighting adaptive boosting, ensemble-aware regularization, and meta-learning for hyperparameter tuning to further improve its performance across diverse educational applications.

Keywords: Ensemble Learning, Higher Education, Machine Learning, Student Success, XGBoost