Volume 20, Issue 4 (3-2024)
JSDP 2024, 20(4): 45-66 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
moradi B, Khalaj M, Taghizadeh Harat A. Improved Ensemble Learning Model by Swarm Intelligence for Mobile Subscribers’ Churn Prediction. JSDP 2024; 20 (4) : 4
URL: http://jsdp.rcisp.ac.ir/article-1-1344-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1344-en.html
Abstract: (376 Views)
In today’s competitive world, companies need to analyze, identify and predict the behavior of their customers and respond to their demands earlier than their competitors. Moreover, in many industries such as mobile telecommunications, the cost of maintaining existing customers (customer retention) is much lower than the cost of attracting a new customer. Therefore, the problem of identifying customers who are going to leave the company, so-called Customer Churn Prediction (CCP), and preventing them by offering Incentives is essential in these industries. In this direction, researchers have presented competent methods using data mining and artificial intelligence tools to identify potential churners. Machine learning (ML) methods are one of the most powerful and widely used techniques to deal with the CCP problem, since they can properly extract and learn complex relationships between the customers’ attributes and their churn intention. Artificial Neural Networks (ANNs), Support Vector Machines (SVMs), Decision Trees (DTs), Logistic Regression (LR), and Naïve Bayes (NB) are among the well-known ML models utilized in numerous studies to tackle the CCP problem. Also, ensemble learning techniques such as Adaboost, Gradient Boost, and Extreme Gradient Boost (XG_boost) have been widely used to solve the CCP problem since they can aggregate the capabilities of multiple ML models. Hence, in order to improve the process of predicting customer churn, in this paper we propose a novel ensemble learning based approach, which is designed based on the two-level stacking technique. We employ six prominent ML models in each level of our proposed ensemble model including MLP, SVM-RBF, DT, NB, KNN, and LR. We also benefit from the Gray Wolf Optimization (GWO) algorithm as an efficient swarm intelligence based search algorithm to select the most effective features and also adjust the hyper-parameters in the proposed model. We have implemented our proposed model using Python and simulated it on two well-known customer churn datasets in the telecom market (IBM_Telco and Duke_Cell2Cell) to evaluate its performance. In this direction, we first demonstrated the optimal features’ subset and the parameter values obtained from applying the GWO algorithm on each dataset. Next, we compared the performance of the proposed ensemble model with each of the base learners using common evaluation criteria including accuracy, precision, recall, F1 score and AUC. The results show that the proposed ensemble model can collect the capabilities of all the base learners and it works better than each of the basic ML models. Afterward, we compared the obtained results from the suggested model with the common ensemble models (Adaboost, Gradient_boost, XG_boost, and Cat_boost) The experimental results show the superiority of the proposed method over other evaluated ensemble models in all the evaluation criteria. Eventually, our method is compared with two recent CCP approaches introduced in the literature. This analysis reveals that, except for the recall criterion in the Duke_Cell2Cell dataset, our introduced method achieves superior results compared to the considered approaches in both datasets.
Article number: 4
Keywords: Customer Churn Prediction, Mobile Telecommunication, Ensemble Learning, Swarm Intelligence, Gray Wolf Optimization Algorithm
Type of Study: Research |
Subject:
Paper
Received: 2022/10/25 | Accepted: 2022/12/26 | Published: 2024/04/25 | ePublished: 2024/04/25
Received: 2022/10/25 | Accepted: 2022/12/26 | Published: 2024/04/25 | ePublished: 2024/04/25
References
1. [1] W. Jianxun, "A study on customer acquisition cost and customer retention cost: Review and outlook," INNOVATION AND MANAGEMENT, 2012.
2. [2] A. Bilal Zorić, "Predicting customer churn in banking industry using neural networks," Interdisciplinary Description of Complex Systems: INDECS, vol. 14, no. 2, pp. 116-124, 2016. [DOI:10.7906/indecs.14.2.1]
3. [3] K. G. M. Karvana, S. Yazid, A. Syalim, and P. Mursanto, "Customer churn analysis and prediction using data mining models in banking industry," in 2019 International Workshop on Big Data and Information Security (IWBIS), 2019, pp. 33-38: IEEE. [DOI:10.1109/IWBIS.2019.8935884]
4. [4] A. Keramati, H. Ghaneei, and S. M. Mirmohammadi, "Developing a prediction model for customer churn from electronic banking services using data mining," Financial Innovation, vol. 2, no. 1, pp. 1-13, 2016. [DOI:10.1186/s40854-016-0029-6]
5. [5] J. Kaur, V. Arora, and S. Bali, "Influence of technological advances and change in marketing strategies using analytics in retail industry," International journal of system assurance engineering and management, vol. 11, no. 5, pp. 953-961, 2020. [DOI:10.1007/s13198-020-01023-5]
6. [6] A. Dingli, V. Marmara, and N. S. Fournier, "Comparison of deep learning algorithms to predict customer churn within a local retail industry," International journal of machine learning and computing, vol. 7, no. 5, pp. 128-132, 2017. [DOI:10.18178/ijmlc.2017.7.5.634]
7. [7] A. Idris and A. Khan, "Churn prediction system for telecom using filter-wrapper and ensemble classification," The Computer Journal, vol. 60, no. 3, pp. 410-430, 2017.
8. [8] T. Vafeiadis, K. I. Diamantaras, G. Sarigiannidis, and K. C. Chatzisavvas, "A comparison of machine learning techniques for customer churn prediction," Simulation Modelling Practice and Theory, vol. 55, pp. 1-9, 2015. [DOI:10.1016/j.simpat.2015.03.003]
9. [9] "IBM Telco customer churn," https://www.kaggle.com/datasets/blastchar/telco-customer-churn, 2018.
10. [10] J. Burez and D. Van den Poel, "Handling class imbalance in customer churn prediction," Expert Systems with Applications, vol. 36, no. 3, pp. 4626-4636, 2009. [DOI:10.1016/j.eswa.2008.05.027]
11. [11] G. Bonaccorso, Machine learning algorithms. Packt Publishing Ltd, 2017.
12. [12] D. W. Hosmer Jr, S. Lemeshow, and R. X. Sturdivant, Applied logistic regression. John Wiley & Sons, 2013. [DOI:10.1002/9781118548387]
13. [13] J. R. Quinlan, "Induction of decision trees," Machine learning, vol. 1, no. 1, pp. 81-106, 1986. [DOI:10.1007/BF00116251]
14. [14] I. Rish, "An empirical study of the naive Bayes classifier," in IJCAI 2001 workshop on empirical methods in artificial intelligence, 2001, vol. 3, no. 22, pp. 41-46.
15. [15] Z. Pawlak, "Rough sets," International journal of computer & information sciences, vol. 11, no. 5, pp. 341-356, 1982. [DOI:10.1007/BF01001956]
16. [16] C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273-297, 1995. [DOI:10.1007/BF00994018]
17. [17] M. H. Hassoun, Fundamentals of artificial neural networks. MIT press, 1995. [DOI:10.1109/JPROC.1996.503146]
18. [18] A. Idris, A. Khan, and Y. S. Lee, "Genetic programming and adaboosting based churn prediction for telecom," in 2012 IEEE international conference on Systems, Man, and Cybernetics (SMC), 2012, pp. 1328-1332: IEEE. [DOI:10.1109/ICSMC.2012.6377917]
19. [19] Y. Beeharry and R. Tsokizep Fokone, "Hybrid approach using machine learning algorithms for customers' churn prediction in the telecommunications industry," Concurrency and Computation: Practice and Experience, vol. 34, no. 4, p. e6627, 2022. [DOI:10.1002/cpe.6627]
20. [20] P. Lalwani, M. K. Mishra, J. S. Chadha, and P. Sethi, "Customer churn prediction system: a machine learning approach," Computing, vol. 104, no. 2, pp. 271-294, 2022. [DOI:10.1007/s00607-021-00908-y]
21. [21] S. Mirjalili, S. M. Mirjalili, and A. Lewis, "Grey wolf optimizer," Advances in engineering software, vol. 69, pp. 46-61, 2014. [DOI:10.1016/j.advengsoft.2013.12.007]
22. [22] M. C. Mozer, R. Wolniewicz, D. B. Grimes, E. Johnson, and H. Kaushansky, "Predicting subscriber dissatisfaction and improving retention in the wireless telecommunications industry," IEEE Transactions on neural networks, vol. 11, no. 3, pp. 690-696, 2000. [DOI:10.1109/72.846740] [PMID]
23. [23] J. Hadden, A. Tiwari, R. Roy, and D. Ruta, "Computer assisted customer churn management: State-of-the-art and future trends," Computers & Operations Research, vol. 34, no. 10, pp. 2902-2917, 2007. [DOI:10.1016/j.cor.2005.11.007]
24. [24] K. Coussement and D. Van den Poel, "Churn prediction in subscription services: An application of support vector machines while comparing two parameter-selection techniques," Expert systems with applications, vol. 34, no. 1, pp. 313-327, 2008. [DOI:10.1016/j.eswa.2006.09.038]
25. [25] P. C. Pendharkar, "Genetic algorithm based neural network approaches for predicting churn in cellular wireless network services," Expert Systems with Applications, vol. 36, no. 3, pp. 6714-6720, 2009. [DOI:10.1016/j.eswa.2008.08.050]
26. [26] R. Yu, X. An, B. Jin, J. Shi, O. A. Move, and Y. Liu, "Particle classification optimization-based BP network for telecommunication customer churn prediction," Neural Computing and Applications, vol. 29, no. 3, pp. 707-720, 2018. [DOI:10.1007/s00521-016-2477-3]
27. [27] M. Imani, "Customer Churn Prediction in Telecommunication Using Machine Learning: A Comparison Study," AUT Journal of Modeling and Simulation, vol. 52, no. 2, pp. 8-8, 2020.
28. [28] S. Wu, W.-C. Yau, T.-S. Ong, and S.-C. Chong, "Integrated churn prediction and customer segmentation framework for telco business," IEEE Access, vol. 9, pp. 62118-62136, 2021. [DOI:10.1109/ACCESS.2021.3073776]
29. [29] "telecom churn (cell2cell)," https://www.kaggle.com/datasets/jpacse/datasets-for-churn-telecom, 2018.
30. [30] E. Hanif, "Applications of data mining techniques for churn prediction and cross-selling in the telecommunications industry," Dublin Business School, 2019.
31. [31] J. Pamina, B. Raja, S. SathyaBama, M. Sruthi, and A. VJ, "An effective classifier for predicting churn in telecommunication," Jour of Adv Research in Dynamical & Control Systems, vol. 11, 2019.
32. [32] N. I. Mohammad, S. A. Ismail, M. N. Kama, O. M. Yusop, and A. Azmi, "Customer churn prediction in telecommunication industry using machine learning classifiers," in Proceedings of the 3rd international conference on vision, image and signal processing, 2019, pp. 1-7. [DOI:10.1145/3387168.3387219]
33. [33] S. Agrawal, A. Das, A. Gaikwad, and S. Dhage, "Customer churn prediction modelling based on behavioural patterns analysis using deep learning," in 2018 International conference on smart computing and electronic enterprise (ICSCEE), 2018, pp. 1-6: IEEE. [DOI:10.1109/ICSCEE.2018.8538420]
34. [34] A. Amin, F. Al-Obeidat, B. Shah, A. Adnan, J. Loo, and S. Anwar, "Customer churn prediction in telecommunication industry using data certainty," Journal of Business Research, vol. 94, pp. 290-301, 2019. [DOI:10.1016/j.jbusres.2018.03.003]
35. [35] S. Momin, T. Bohra, and P. Raut, "Prediction of customer churn using machine learning," in EAI International Conference on Big Data Innovation for Sustainable Cognitive Computing, 2020, pp. 203-212: Springer. [DOI:10.1007/978-3-030-19562-5_20]
36. [36] S. Wael Fujo, S. Subramanian, and M. Ahmad Khder, "Customer Churn Prediction in Telecommunication Industry Using Deep Learning," Information Sciences Letters, vol. 11, no. 1, p. 24, 2022. [DOI:10.18576/isl/110120]
37. [37] I. V. Pustokhina, D. A. Pustokhin, P. T. Nguyen, M. Elhoseny, and K. Shankar, "Multi-objective rain optimization algorithm with WELM model for customer churn prediction in telecommunication sector," Complex & Intelligent Systems, pp. 1-13, 2021. [DOI:10.1007/s40747-021-00353-6]
38. [38] A. De Caigny, K. Coussement, and K. W. De Bock, "A new hybrid classification algorithm for customer churn prediction based on logistic regression and decision trees," European Journal of Operational Research, vol. 269, no. 2, pp. 760-772, 2018. [DOI:10.1016/j.ejor.2018.02.009]
39. [39] V. Umayaparvathi and K. Iyakutti, "Automated feature selection and churn prediction using deep learning models," International Research Journal of Engineering and Technology (IRJET), vol. 4, no. 3, pp. 1846-1854, 2017.
40. [40] U. Ahmed, A. Khan, S. H. Khan, A. Basit, I. U. Haq, and Y. S. Lee, "Transfer learning and meta classification based deep churn prediction system for telecom industry," arXiv preprint arXiv:1901.06091, 2019.
41. [41] A. Idris, A. Khan, and Y. S. Lee, "Intelligent churn prediction in telecom: employing mRMR feature selection and RotBoost based ensemble classification," Applied intelligence, vol. 39, no. 3, pp. 659-672, 2013. [DOI:10.1007/s10489-013-0440-x]
42. [42] W. Verbeke, K. Dejaeger, D. Martens, J. Hur, and B. Baesens, "New insights into churn prediction in the telecommunication sector: A profit driven data mining approach," European journal of operational research, vol. 218, no. 1, pp. 211-229, 2012. [DOI:10.1016/j.ejor.2011.09.031]
43. [43] Y. Xie, X. Li, E. Ngai, and W. Ying, "Customer churn prediction using improved balanced random forests," Expert Systems with Applications, vol. 36, no. 3, pp. 5445-5449, 2009. [DOI:10.1016/j.eswa.2008.06.121]
44. [44] S. A. Qureshi, A. S. Rehman, A. M. Qamar, A. Kamal, and A. Rehman, "Telecommunication subscribers' churn prediction model using machine learning," in Eighth international conference on digital information management (ICDIM 2013), 2013, pp. 131-136: IEEE. [DOI:10.1109/ICDIM.2013.6693977]
45. [45] N. V. Chawla, "Data mining for imbalanced datasets: An overview," Data mining and knowledge discovery handbook, pp. 875-886, 2009. [DOI:10.1007/978-0-387-09823-4_45]
46. [46] Q. Gu, Z. Li, and J. Han, "Generalized fisher score for feature selection," arXiv preprint arXiv:1202.3725, 2012.
47. [47] H. Emami, "Stock exchange trading optimization algorithm: a human-inspired method for global optimization," The Journal of Supercomputing, vol. 78, no. 2, pp. 2125-2174, 2022. [DOI:10.1007/s11227-021-03943-w] [PMID] []
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
