Volume 19, Issue 1 (5-2022)                   JSDP 2022, 19(1): 39-42 | Back to browse issues page


XML Persian Abstract Print


Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Pourali H, Omranpour H. Ensemble Kernel Learning Model for Prediction of Time Series Based on the Support Vector Regression and Meta Heuristic Search. JSDP 2022; 19 (1) : 3
URL: http://jsdp.rcisp.ac.ir/article-1-1162-en.html
Babol Noshirvani University of Technology
Abstract:   (1447 Views)
In this paper, a method is presented for predicting time series. Time series prediction is a process which predicted future system values based on information obtained from past and present data points. Time series prediction models are widely used in various fields of engineering, economics, etc. The main purpose of using different models for time series prediction is to make the forecast with the greatest accuracy. The model presented in this paper is based on the combination of kernels and support vector regression. Support vector regression is highly capable of solving function estimation problems by using its kernels, but kernels’ parameters need to be adjusted. First we have preprocessing phase which includes normalizing data and separating data for testing and training. In proposed model, ten different kernels were used. Five kernels were selected as the best kernels by trial and error and these kernels are applied to data. There probably is only a few of the kernels that are useful for the problem, and we are not aware of which kernels are useful for our problem so kernel outputs aggregate by applying a coefficient. This combination creates a new secondary space. The output is given to support vector regression to construct a model that predicts values exactly ɛ accurate, which means the predicted values do not deviate more than ɛ from the original data. This model predicts values by using a leave one out model. Each kernel has parameters that need to be set to optimum values in order to get the best results. Hence in the proposed model, the kernel parameters and their weights are learned by the Gray Wolf Optimizer. This optimizer has been able to provide appropriate answers to many problems, especially challenging problems and has a superior ability to solve the high-dimension problems. By running program in consecutive iterations and examining the different values of the parameters, the optimizer learns the best of them which prediction error has been reduced, and finally returns their best value. The proposed model is implemented on five standard time series and compared to other method, test based on the RMSE criterion for DJ time series, improved by 1.58 point, Radio time series, improved by 0.178 point, and Sunspot time series, improved by 1.709 point. Finally, we analyzed the results, Statistical evaluation by Wilcoxon Signed-Rank Test where the p value is very low compared to the proposed method and CNN-FCM, AR_ model per scale, Multiresolution AR model and ANN methods, slightly lower for Wavelet-HFCM and ANFIS methods and slightly lower than one for SAE-FCM method and at the end provide a relation to find the window size in the model by obtaining the average of peak differences, valley differences, and consecutive peak, and valley differences for the actual values of the training data in exchange for their sequence number in time series.
Article number: 3
Full-Text [PDF 1569 kb]   (370 Downloads)    
Type of Study: Research | Subject: Paper
Received: 2020/08/9 | Accepted: 2021/01/10 | Published: 2022/06/22 | ePublished: 2022/06/22

References
1. [1] E. Kayacan, B. Ulutas and O. Kaynak, "Grey system theory-based models in time series prediction," Expert Systems with Applications, vol. 37, no. 2, pp. 1784-1789, 2010. [DOI:10.1016/j.eswa.2009.07.064]
2. [2] B. Paaben, C. Gopfert and B. Hammer, "Time Series Prediction for Graph in Kernel andDissimilarity Spaces," Neural Processing Letters, no. 48, pp. 669-689, 2018. [DOI:10.1007/s11063-017-9684-5]
3. [3] S. C. Nayak, B. B. Misra and H. S. Behera, "Efficient financial time series prediction with evolutionary virtual data position exploration," Neural Computing and Application, no. 31, pp. 1053-1074, 2019. [DOI:10.1007/s00521-017-3061-1]
4. [4] M. A. Villegas, D. J. Pedregal and J. R. Trapero, "A support vector machine for model selection in demand forecasting application," Computers & Industrial Engineering, vol. 121, pp. 1-7, 2018. [DOI:10.1016/j.cie.2018.04.042]
5. [5] P. Liu, J. Liu and K. Wu, "CNN-FCM: System modeling promotes stability of deep learning in time series prediction," Knowledge-Based Systems, vol. 203, 2020. [DOI:10.1016/j.knosys.2020.106081]
6. [6] J. H. Sadaei, P. Cândido de Lima e Silva, F. Gadelha Guimarães and M. Hisyam Lee, "Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series," Energy, vol. 175, pp. 365-377, 2019. [DOI:10.1016/j.energy.2019.03.081]
7. [7] J. Hu, X. Wang, Y. Zhang, D. Zhang, M. Zhang and J. Xue, "Time Series Prediction Method Based on Variant LSTM Recurrent Neural Network," Neural Processing Letters, 2020. [DOI:10.1007/s11063-020-10319-3]
8. [8] K. Yuan, J. Liu, S. Yang, K. Wu and F. Shen, "Time series forecasting based on kernel mapping and high-order fuzzy cognitive maps," Knowledge-Based Systems, vol. 206, 2020. [DOI:10.1016/j.knosys.2020.106359]
9. [9] J. Wang, Z. Peng, X. Wang, C. Li and J. Wu, "Deep Fuzzy Cognitive Maps for Interpretable Multivariate Time Series Prediction," IEEE Transactions on Fuzzy Systems, pp. 1-1, 2020.
10. [10] S. Yang and J. Liu, "Time Series Forecasting based on High-Order Fuzzy Cognitive Maps and Wavelet Transform," IEEE Transactions on Fuzzy System, vol. 26, no. 6, pp. 3391-3402, 2018. [DOI:10.1109/TFUZZ.2018.2831640]
11. [11] Q. Xiao, "Time series prediction using bayesian filtering model and fuzzy neural networks," Optik - International Journal for Light and Electron Optics, vol. 140, pp. 104-113, 2017. [DOI:10.1016/j.ijleo.2017.03.096]
12. [12] H. Omranpour, F. Azadian, "Presenting a Fuzzy Approach to Optimize Predicting High Order Time series," Signal and Data Processing vol. 15, no. 2, pp. 3-16, 2018. [DOI:10.29252/jsdp.15.2.3]
13. [13] C. Bergmeir, R. J. Hyndman and B. Koo, "A note on the validity of cross-validation for evaluating autoregressive time series prediction," Computational Statistics and Data Analysis, vol. 120, pp. 70-83, 2018. [DOI:10.1016/j.csda.2017.11.003]
14. [14] W. Xu, H. Peng, X. Zeng, F. Zhou, X. Tian and X. Peng, "Deep belief network-based AR model for nonlinear time series forecasting," Applied Soft Computing Journal, vol. 77, pp. 605-621, 2019. [DOI:10.1016/j.asoc.2019.02.006]
15. [15] L. Bianchi, M. Dorigo, L. M. Gambardella and W. J. Gutjahr, "A survey on metaheuristics for stochastic combinatorial optimization," Natural Computing, vol. 8, pp. 239-287, 2009. [DOI:10.1007/s11047-008-9098-4]
16. [16] G. Cornuéjols , "Valid inequalities for mixed integer linear programs," Mathematical Programming, vol. 112, pp. 3-44, 2008. [DOI:10.1007/s10107-006-0086-0]
17. [17] M. Avriel, Nonlinear Programming: Analysis and Methods, New York: Dover Publications, 2003.
18. [18] A. H. Land and A. G. Doig, "An automatic method for solving discrete programming problems," 50 Years of Integer Programming 1958-2008, pp. 105-132, 2010. [DOI:10.1007/978-3-540-68279-0_5] [PMID] [PMCID]
19. [19] A. R. Simpson, G. C. Dancy and L. J. Murphy, "Genetic algorithms compared to other techniques for pipe optimization," Journal of water resources planning and management, vol. 120, no. 4, pp. 423-443, 1994. [DOI:10.1061/(ASCE)0733-9496(1994)120:4(423)]
20. [20] S. Mirjalili, "The Ant Lion Optimizer," Advances in Engineering Software, vol. 83, pp. 80-98, 2015. [DOI:10.1016/j.advengsoft.2015.01.010]
21. [21] B. T. Ojemakinde, Support Vector Regression for Non-Stationary Time Series, Knoxville: University of Tennessee, 2006.
22. [22] S. Lin, S. zhang, J. Qiao, H. Liu and G. Yu, "A Parameter Choosing Method of SVR for Time Series Prediction," in The 9th International Conference for Young Computer Scientists, Liaoning, China, 2008. [DOI:10.1109/ICYCS.2008.393]
23. [23] C. Hsin, J. M. Ho and D. T. Lee, "Travel-Time Prediction With Support Vector Regression," IEEE Transactions on Intelligent Transportation Systems, vol. 5, no. 4, pp. 276-281, 2004. [DOI:10.1109/TITS.2004.837813]
24. [24] X. Ma, Y. Zhang, H. Cao, S. Zhang and Y. Zhou, "Nonlinear Regression with High-Dimensional Space Mapping for Blood Component Spectral Quantitative Analysis," Journal of Spectroscopy, 2018. [DOI:10.1155/2018/2689750]
25. [25] T. Hofmann, B. Schölkopf and A. J. Smola, "Kernel Methods in Machine Learning," Institute of Mathematical Statistics, vol. 36, no. 3, pp. 1171-1220, 2008. [DOI:10.1214/009053607000000677]
26. [26] J. Xie, "Time Series Prediction Based on Recurrent LS-SVM with Mixed Kernel," in Asia-Pacific Conference on Information Processing, Shenzhen, China, 2009. [DOI:10.1109/APCIP.2009.37] [PMCID]
27. [27] S. Mirjalili, S. M. Mirjalili and A. Lewis, "Grey Wolf Optimizer," Advances in Engineering Software, vol. 69, pp. 46-61, 2014. [DOI:10.1016/j.advengsoft.2013.12.007]
28. [28] M. R. Mosavi, M. Khishe and A. Ghamgosar, "CLASSIFICATION OF SONAR DATA SET USING NEURAL NETWORK TRAINED BY GRAY WOLF OPTIMIZATION," Neural Network World, vol. 4, pp. 393-415, 2016. [DOI:10.14311/NNW.2016.26.023]
29. [29] J. Heinermann and O. Kramer, "Precise Wind Power Prediction with SVM Ensemble Regression," in International Conference on Artificial Neural Networks, Hamburg, Germany, 2014. [DOI:10.1007/978-3-319-11179-7_100]
30. [30] K. Wu, J. Liu, P. Liu and S. Yang, "Time Series Prediction Using Sparse Autoencoder and High-order Fuzzy Cognitive Maps," IEEE Transactions on Fuzzy Systems, 2019. [DOI:10.1109/TFUZZ.2019.2956904]
31. [31] J. Shing and R. Jang, "ANFIS: adaptive-network-based fuzzy inference system," IEEE Transactions on Systems, Man, and Cybernetics, vol. 23, no. 3, pp. 665-685, 1993. [DOI:10.1109/21.256541]
32. [32] G. Zheng, J. Starck, J. Campbell and F. Murtagh, "Multiscale transforms for filtering financial data streams," Journal of Computational Intelligence in Finance, vol. 7, no. 18-35, 1999.
33. [33] O. Renaud, J. L. Starck and F. Murtagh, "Wavelet-Based Combined Signal Filtering and Prediction," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 35, no. 6, pp. 1241-1251, 2005. [DOI:10.1109/TSMCB.2005.850182] [PMID]
34. [34] A. B. Geva, "ScaleNet-multiscale neural-network architecture for time series prediction," IEEE Transactions on Neural Networks, vol. 9, no. 6, pp. 1471-1482, 1998. [DOI:10.1109/72.728396] [PMID]
35. [35] J. Derrac, S. García, D. Molina and F. Herrera, "A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms," Swarm and Evolutionary Computation, vol. 1, no. 1, pp. 3-18, 2011. [DOI:10.1016/j.swevo.2011.02.002]

Add your comments about this article : Your username or Email:
CAPTCHA

Send email to the article author


Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

© 2015 All Rights Reserved | Signal and Data Processing