RISK-AWARE ENSEMBLE LEARNING FOR CREDIT DEFAULT PREDICTION UNDER ECONOMIC UNCERTAINTY
Keywords:
Credit scoring, Ensemble learning, Economic uncertainty, Distribution shift, Conditional value-at-risk, Probability of defaultAbstract
Credit scoring models are estimated on past loan vintages but deployed into economic conditions that are unknown at origination. When the economy enters a downturn, ensembles fitted to average historical performance produce probability-of-default (PD) estimates that deteriorate precisely when accurate risk measurement matters most. This paper proposes TRACE (Tail-Risk-Aware Calibrated Ensemble), a stacking ensemble whose aggregation layer is explicitly risk-aware with respect to economic regimes. TRACE (i) builds a rolling-origin prediction cube in which every base-learner prediction is a genuine out-of-time forecast; (ii) derives two epistemic margins from the cube—cross-learner and cross-regime dispersion—that enter the PD as non-negative conservatism terms; (iii) fits the aggregator by minimising the conditional value-at-risk of vintage-level log-losses around a recency-anchored nominal regime distribution, which is equivalent to a distributionally robust problem over regime mixtures with bounded likelihood ratio; and (iv) learns the length of the regime memory from data through one-step-ahead validation. The resulting convex program is solved exactly. Experiments on two public loan data sets spanning the Great Recession—90,000 U.S. Small Business Administration (SBA) loans approved in 1993–2010 and 39,786 Lending Club loans issued in 2007–2011—compare TRACE with six base learners and eight ensemble, drift-adaptive and dynamic-selection benchmarks over 51 out-of-time test periods. On the SBA data, TRACE attains the lowest average log-loss, Brier score, calibration error and misclassification cost, reduces the worst-year log-loss by 11.0% relative to classical stacking, and cuts the PD under-estimation during the 2005–2008 stress vintages from 12.7 to 1.5 percentage points. On the Lending Club data, where drift is mild, it performs on par with the strongest ensembles. Ablation, sensitivity and label-delay analyses identify the adaptive regime memory as the main source of improvement and the CVaR objective as an effective control of tail risk.References
[1] Bellotti T, Crook J. Forecasting and stress testing credit card default using dynamic models. International Journal of Forecasting, 2013, 29: 563-574.
[2] Leow M, Crook J. The stability of survival model parameter estimates for predicting the probability of default: Empirical evidence over the credit crisis. European Journal of Operational Research, 2016, 249: 457-464. DOI: 10.1016/j.ejor.2014.09.005.
[3] Bloom N. Fluctuations in uncertainty. Journal of Economic Perspectives, 2014, 28: 153-176. DOI: 10.1257/jep.28.2.153.
[4] Jurado K, Ludvigson S C, Ng S. Measuring uncertainty. American Economic Review, 2015, 105: 1177-1216. DOI: 10.1257/aer.20131193.
[5] Baker S R, Bloom N, Davis S J. Measuring economic policy uncertainty. The Quarterly Journal of Economics, 2016, 131: 1593-1636. DOI: 10.1093/qje/qjw024.
[6] Lessmann S, Baesens B, Seow H-V, et al. Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research. European Journal of Operational Research, 2015, 247: 124-136. DOI: 10.1016/j.ejor.2015.05.030.
[7] Grinsztajn L, Oyallon E, Varoquaux G. Why do tree-based models still outperform deep learning on typical tabular data? In: Advances in neural information processing systems 35 (NeurIPS 2022), datasets and benchmarks track, 2022.
[8] Xia Y, Liu C, Da B, et al. A novel heterogeneous ensemble credit scoring model based on bstacking approach. Expert Systems with Applications, 2018, 93: 182-199. DOI: 10.1016/j.eswa.2017.10.022.
[9] Abellán J, Castellano J G. A comparative study on base classifiers in ensemble methods for credit scoring. Expert Systems with Applications, 2017, 73: 1-10.
[10] Gunnarsson B R, Broucke S vanden, Baesens B, et al. Deep learning for credit scoring: Do or don’t? European Journal of Operational Research, 2021, 295: 292-305.
[11] Dumitrescu E, Hué S, Hurlin C, et al. Machine learning for credit scoring: Improving logistic regression with non-linear decision-tree effects. European Journal of Operational Research, 2022, 297: 1178-1192. DOI: 10.1016/j.ejor.2021.06.053.
[12] Rockafellar R T, Uryasev S. Optimization of conditional value-at-risk. Journal of Risk, 2000, 2: 21-41.
[13] Duchi J C, Namkoong H. Learning models with uniform performance via distributionally robust optimization. The Annals of Statistics, 2021, 49: 1378-1406. DOI: 10.1214/20-AOS2004.
[14] Ala’raj M, Abbod M F. Classifiers consensus system approach for credit scoring. Knowledge-Based Systems, 2016, 104: 89-105.
[15] Xia Y, Zhao J, He L, et al. A novel tree-based dynamic heterogeneous ensemble method for credit scoring. Expert Systems with Applications, 2020, 159: 113615. DOI: 10.1016/j.eswa.2020.113615.
[16] Chen T, Guestrin C. XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, 2016: 785-794.
[17] Ke G, Meng Q, Finley T, et al. LightGBM: A highly efficient gradient boosting decision tree. In: Advances in neural information processing systems 30 (NIPS 2017), 2017: 3146-3154.
[18] Prokhorenkova L, Gusev G, Vorobev A, et al. CatBoost: Unbiased boosting with categorical features. In: Advances in neural information processing systems 31 (NeurIPS 2018), 2018.
[19] Malekipirbazari M, Aksakalli V. Risk assessment in social lending via random forests. Expert Systems with Applications, 2015, 42: 4621-4631. DOI: 10.1016/j.eswa.2015.02.001.
[20] Emekter R, Tu Y, Jirasakuldech B, et al. Evaluating credit risk and loan performance in online peer-to-peer (P2P) lending. Applied Economics, 2015, 47: 54-70. DOI: 10.1080/00036846.2014.962222.
[21] Jagtiani J, Lemieux C. The roles of alternative data and machine learning in fintech lending: Evidence from the LendingClub consumer platform. Financial Management, 2019, 48: 1009-1029.
[22] Brown I, Mues C. An experimental comparison of classification algorithms for imbalanced credit scoring data sets. Expert Systems with Applications, 2012, 39: 3446-3453.
[23] Fitzpatrick T, Mues C. An empirical comparison of classification algorithms for mortgage default prediction: Evidence from a distressed mortgage market. European Journal of Operational Research, 2016, 249: 427-439. DOI: 10.1016/j.ejor.2015.09.014.
[24] Bellotti T, Crook J. Retail credit stress testing using a discrete hazard model with macroeconomic factors. Journal of the Operational Research Society, 2014, 65: 340-350. DOI: 10.1057/jors.2013.91.
[25] Sousa M R, Gama J, Brandão E. A new dynamic modeling framework for credit risk assessment. Expert Systems with Applications, 2016, 45: 341-351. DOI: 10.1016/j.eswa.2015.09.055.
[26] Gama J, Žliobaitė I, Bifet A, et al. A survey on concept drift adaptation. ACM Computing Surveys, 2014, 46: 44:1-44:37. DOI: 10.1145/2523813.
[27] Wang H, Fan W, Yu P S, et al. Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the 9th ACM SIGKDD international conference on knowledge discovery and data mining, 2003: 226-235.
[28] Ko A H R, Sabourin R, Britto Jr. A S. From dynamic classifier selection to dynamic ensemble selection. Pattern Recognition, 2008, 41: 1718-1731.
[29] Cruz R M O, Sabourin R, Cavalcanti G D C. Dynamic classifier selection: Recent advances and perspectives. Information Fusion, 2018, 41: 195-216.
[30] Sagawa S, Koh P W, Hashimoto T B, et al. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. In: International conference on learning representations (ICLR), 2020.
[31] Lakshminarayanan B, Pritzel A, Blundell C. Simple and scalable predictive uncertainty estimation using deep ensembles. In: Advances in neural information processing systems 30 (NIPS 2017), 2017: 6402-6413.
[32] Malinin A, Prokhorenkova L, Ustimenko A. Uncertainty in gradient boosting via ensembles. In: International conference on learning representations (ICLR), 2021.
[33] Ovadia Y, Fertig E, Ren J, et al. Can you trust your model’s uncertainty? Evaluating predictive uncertainty under dataset shift. In: Advances in neural information processing systems 32 (NeurIPS 2019), 2019.
[34] Guo C, Pleiss G, Sun Y, et al. On calibration of modern neural networks. In: Proceedings of the 34th international conference on machine learning (ICML), 2017: 1321-1330.
[35] Verbraken T, Bravo C, Weber R, et al. Development and application of consumer credit scoring models using profit-based classification measures. European Journal of Operational Research, 2014, 238: 505-513. DOI: 10.1016/j.ejor.2014.04.001.
[36] Serrano-Cinca C, Gutiérrez-Nieto B. The use of profit scoring as an alternative to credit scoring systems in peer-to-peer (P2P) lending. Decision Support Systems, 2016, 89: 113-122. DOI: 10.1016/j.dss.2016.06.014.
[37] Wolpert D H. Stacked generalization. Neural Networks, 1992, 5: 241-259.
[38] Li M, Mickel A, Taylor S. “Should This Loan be Approved or Denied?”: A large dataset with class assignment guidelines. Journal of Statistics Education, 2018, 26: 55-66. DOI: 10.1080/10691898.2018.1434342.
[39] Caruana R, Niculescu-Mizil A, Crew G, et al. Ensemble selection from libraries of models. In: Proceedings of the 21st international conference on machine learning (ICML), 2004.
[40] Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 2006, 7: 1-30.