Volume 20, Issue 3 (12-2023)
JSDP 2023, 20(3): 127-140 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
derogarmoghadam A, Karami Molaei M R, Hassanzadeh M. Design of a filter bank-based convolutional neural network for handwritten digit images classification. JSDP 2023; 20 (3) : 9
URL: http://jsdp.rcisp.ac.ir/article-1-1320-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1320-en.html
Babol Noshirvani University of Technology
Abstract: (660 Views)
Background: In recent years, convolutional neural networks (CNNs) have been increasingly used in various applications of machine vision. CNNs simulate the function of the brain's visual cortex and have a powerful structure for analyzing visual images. However, the diversity of digital images, their content, and their features necessitate that CNN networks are specially designed, and their parameters are carefully adjusted to achieve higher efficiency in any classification problem. In this regard, in many previous studies, researchers have attempted to increase the efficiency of the CNNs by setting their adjustable parameters more accurately.
New method: New method: In this study, we presented a novel initializing method for the kernels of the first convolutional layer of the CNN networks. We designed a filter bank with specialized kernels and used them in the first convolution layer of the proposed models. These kernels, compared to the random kernels in traditional CNNs, extract more effective features from the input images without increasing the computational cost of the network, and improve the classification accuracy by covering all the important characteristics.
Results: The dataset used in this paper was the MNIST database of handwritten digits. We examined the performance of CNN networks when three different types of kernels were used in their first convolution layer. The first group of kernels had constant coefficients; the second group had random coefficients, and finally, the kernels of the third group were specially designed to extract a wide range of image features. Our experiments on a single-layer CNN network with three types of kernels (constant numbers, random numbers, and filter-bank) showed the average classification accuracy of MNIST images in 50 times of network training to be 74.94%, 86.47%, and 91.89%, respectively, and for a three-layer CNN network, 88.82%, 96.16%, and 99.14%, respectively.
Comparison with existing methods: Compared to the kernels with randomized coefficients, the use of specialized kernels in the first convolution layer of the CNN networks has several important advantages: 1) They can be designed to extract all important features of the input images, 2) They can be designed more effectively based on the problem in hand, 3) They cause the training to start from a more appropriate point, and in this way, the speed of training and the classification accuracy of the network increase.
Conclusion: This study provides a novel method for initializing kernels in convolution layers of CNN networks to enhance their performance in image classification works. Our results show that compared to random kernels, the kernels used in the proposed models extract more effective features from the images at different frequencies and increase the classification accuracy by starting the training algorithm from a more appropriate point, without increasing the computational cost. Therefore, it can be concluded that the initial coefficients of the convolution layer kernels are effective on the classification accuracy of CNN networks, and by using more effective kernels in the convolution layers, these networks can be made specific to the problem and, in this way, increase the efficiency of the network.
New method: New method: In this study, we presented a novel initializing method for the kernels of the first convolutional layer of the CNN networks. We designed a filter bank with specialized kernels and used them in the first convolution layer of the proposed models. These kernels, compared to the random kernels in traditional CNNs, extract more effective features from the input images without increasing the computational cost of the network, and improve the classification accuracy by covering all the important characteristics.
Results: The dataset used in this paper was the MNIST database of handwritten digits. We examined the performance of CNN networks when three different types of kernels were used in their first convolution layer. The first group of kernels had constant coefficients; the second group had random coefficients, and finally, the kernels of the third group were specially designed to extract a wide range of image features. Our experiments on a single-layer CNN network with three types of kernels (constant numbers, random numbers, and filter-bank) showed the average classification accuracy of MNIST images in 50 times of network training to be 74.94%, 86.47%, and 91.89%, respectively, and for a three-layer CNN network, 88.82%, 96.16%, and 99.14%, respectively.
Comparison with existing methods: Compared to the kernels with randomized coefficients, the use of specialized kernels in the first convolution layer of the CNN networks has several important advantages: 1) They can be designed to extract all important features of the input images, 2) They can be designed more effectively based on the problem in hand, 3) They cause the training to start from a more appropriate point, and in this way, the speed of training and the classification accuracy of the network increase.
Conclusion: This study provides a novel method for initializing kernels in convolution layers of CNN networks to enhance their performance in image classification works. Our results show that compared to random kernels, the kernels used in the proposed models extract more effective features from the images at different frequencies and increase the classification accuracy by starting the training algorithm from a more appropriate point, without increasing the computational cost. Therefore, it can be concluded that the initial coefficients of the convolution layer kernels are effective on the classification accuracy of CNN networks, and by using more effective kernels in the convolution layers, these networks can be made specific to the problem and, in this way, increase the efficiency of the network.
Article number: 9
Type of Study: Research |
Subject:
Paper
Received: 2022/06/28 | Accepted: 2023/07/18 | Published: 2024/01/14 | ePublished: 2024/01/14
Received: 2022/06/28 | Accepted: 2023/07/18 | Published: 2024/01/14 | ePublished: 2024/01/14
References
1. [1] Javidi, B., "Image recognition and classification: algorithms, systems, and applications". 2002: CRC press.
2. [2] Lu D, Weng Q. "A survey of image classification methods and techniques for improving classification performance". International journal of Remote sensing. 2007;28(5):823-70. [DOI:10.1080/01431160600746456]
3. [3] Nath, S.S., et al. "A survey of image classification methods and techniques". International conference on control, instrumentation, communication and computational technologies (ICCICCT). 2014, IEEE.
4. [4] Dat, P.K.K.O., "Image Classification Difficulties".
5. [5] Jain, G. and J. Ko, "Handwritten digits recognition". Multimedia Systems, Project Report, University of Toronto, 2008: p. 1-3.
6. [6] Jain, A.K., R.P.W. Duin, and J. Mao, "Statistical pattern recognition: A review". IEEE Transactions on pattern analysis and machine intelligence, 2000. 22(1): p. 4-37. [DOI:10.1109/34.824819]
7. [7] Egmont-Petersen, M., D. de Ridder, and H. Handels, "Image processing with neural networks: a review". Pattern recognition, 2002. 35(10): p. 2279-2301. [DOI:10.1016/S0031-3203(01)00178-9]
8. [8] Ghosh, M.M.A. and A.Y. Maghari. "A comparative study on handwriting digit recognition using neural networks". in 2017 international conference on promising electronic technologies (ICPET). 2017. IEEE.
9. [9] Bala, R. and D. Kumar, "Classification using ANN: A review". Int. J. Comput. Intell. Res, 2017. 13(7): p. 1811-1820.
10. [10] Bhatnagar, S., D. Ghosal, and M.H. Kolekar. "Classification of fashion article images using convolutional neural networks". in 2017 Fourth International Conference on Image Information Processing (ICIIP). 2017 IEEE.
11. [11] Chen, F., et al., "Assessing four neural networks on handwritten digit recognition dataset (MNIST)". arXiv preprint arXiv:1811.08278, 2018.
12. [12] Hubel, D.H. and T.N. Wiesel, "Receptive fields, binocular interaction and functional architecture in the cat's visual cortex". The Journal of physiology, 1962. 160(1): p. 106.
13. [13] Fukushima, K., "Neural network model for selective attention in visual pattern recognition and associative recall". Applied Optics, 1987. 26(23): p. 4985-4992. [DOI:10.1364/AO.26.004985] [PMID]
14. [14] Momeny M, Sarram M A, Latif A, Sheikhpour R. A Convolutional Neural Network based on Adaptive Pooling for Classification of Noisy Images. JSDP 2021; 17 (4) :139-154. [DOI:10.29252/jsdp.17.4.139]
15. [15] Traore, B.B., B. Kamsu-Foguem, and F. Tangara, "Deep convolution neural network for image recognition". Ecological Informatics, 2018. 48: p. 257-268. [DOI:10.1016/j.ecoinf.2018.10.002]
16. [16] Duan, M., G. Wang, and C. Niu, "Method of small sample size image recognition based on convolution neural network". Computer Engineering and Design, 2018. 39(1): p. 224-229.
17. [17] Zhou, X., et al. "Application of deep learning in object detection". in 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS). 2017. IEEE.
18. [18] Ren, S., et al., "Faster r-cnn: Towards real-time object detection with region proposal networks". Advances in neural information processing systems, 2015. 28.
19. [19] Liu, Z., et al., "Salient object detection for RGB-D image by single stream recurrent convolution neural network". Neurocomputing, 2019. 363: p. 46-57.
https://doi.org/10.1016/j.neucom.2019.07.012 [DOI:10.1016/j.neucom.2019.01.085]
20. [20] Lang, R., L. Zhao, and K. Jia. "Brain tumor image segmentation based on convolution neural network". in 2016 9th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI). 2016. IEEE.
21. [21] Sultana, F., A. Sufian, and P. Dutta, "Evolution of image segmentation using deep convolutional neural network: a survey". Knowledge-Based Systems, 2020. 201: p. 106062.
22. [22] Al-Saffar, A.A.M., H. Tao, and M.A. Talab. "Review of deep convolution neural network in image classification". in 2017 International conference on radar, antenna, microwave, electronics, and telecommunications (ICRAMET). 2017. IEEE.
23. [23] LeCun, Y., et al., "Gradient-based learning applied to document recognition". Proceedings of the IEEE, 1998. 86(11): p. 2278-2324. [DOI:10.1109/5.726791]
24. [24] Simard, P.Y., D. Steinkraus, and J.C. Platt. "Best practices for convolutional neural networks applied to visual document analysis". in Icdar. 2003.
25. [25] Nielsen, M.A., "Neural networks and deep learning". Vol. 25. 2015: Determination press San Francisco, CA, USA.
26. [26] Wan, L., et al. "Regularization of neural networks using dropconnect". in International conference on machine learning. 2013. PMLR.
27. [27] Tabik, S., et al., "A snapshot of image pre-processing for convolutional neural networks: case study of MNIST". 2017.
28. [28] Ahlawat, S., et al., "Improved handwritten digit recognition using convolutional neural networks (CNN)". Sensors, 2020. 20(12): p. 3344.
29. [29] Ahlawat, S. and A. Choudhary, "Hybrid CNN-SVM classifier for handwritten digit recognition". Procedia Computer Science, 2020. 167: p.2554-2560. [DOI:10.1016/j.procs.2020.03.309]
30. [30] Ali, S., et al., "An effective and improved CNN-ELM classifier for handwritten digits recognition and classification". Symmetry, 2020. 12(10): p. 1742.
31. [31] Calderon, A., S. Roa, and J. Victorino, "Handwritten digit recognition using convolutional neural networks and gabor filters". Proc. Int. Congr. Comput. Intell, 2003: p. 1-9.
32. [32] Le, Q.V., et al. "On optimization methods for deep learning". in ICML. 2011.
33. [33] LeCun, Y., C. Cortes, and C.J. Burges, "The MNIST database of handwritten digits". Website. 1998.
34. [34] Yamashita, R., et al., "Convolutional neural networks: an overview and application in radiology". Insights into imaging, 2018. 9(4): p. 611-629. [DOI:10.1007/s13244-018-0639-9] [PMID] []
35. [35] Cui, H. and J. Bai, "A new hyperparameters optimization method for convolutional neural networks". Pattern Recognition Letters, 2019. 125: p. 828-834. [DOI:10.1016/j.patrec.2019.02.009]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
