Volume 21, Issue 3 (12-2024)
JSDP 2024, 21(3): 69-84 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Jahani S A, Mohebbi K, Zamani Boroujeni F. Improving Scene Recognition in Remote Sensing Using Deep Learning and Feature Selector. JSDP 2024; 21 (3) : 3
URL: http://jsdp.rcisp.ac.ir/article-1-1398-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1398-en.html
Islamic Azad university
Abstract: (1137 Views)
Remote sensing images as a valuable data source in Earth observation can help in measuring and observing detailed structures on the Earth's surface. Scene detection in remote sensing images has many applications in various fields such as urban planning, natural hazard detection, environmental monitoring, vegetation mapping, and geographic object detection. One of the key problems in the interpretation of remote sensing images is the scene classification of remote sensing images. Feature extraction is very important in scene detection and classification. Convolutional neural networks are one of the deep learning methods that have significantly increased the performance of tasks such as object recognition and scene classification, but their performance is highly dependent on the number of labeled images available, which are not available enough especially in the field of remote sensing. Recently, transfer learning, especially for the fine-tuning of pre-trained convolutional neural networks, has attracted more attention from researchers as a practical strategy for scene classification in remote sensing. However, the lack of use of local features and global deep model that is trained on the target data set is one of the limitations of current methods. Also, if these networks are not deep enough and the images do not pass through multiple filters, they cannot extract more semantic information, and the extracted features do not have high discrimination power, and as a result, scene recognition is not performed well. On the other hand, the features extracted through local features are very large, and not using feature selector methods reduces the accuracy of the model. In this research, to solve the mentioned limitations, a hybrid approach of feature extraction has been proposed in which three types of features including two types of deep local and global features and one type of manual local feature are combined with each other. To extract deep features, pre-trained convolutional networks have been used. The pre-trained networks used are: ResNet, InceptionNet, GoogleNet and EfficientNet_b0. In order to extract as much information as possible from the images, a convolutional network with 20 fully connected layers is proposed. Also, a combined feature selection stage consisting of two categories of filtering and packing algorithms is included in this model. Finally, scene detection is performed using several different classification algorithms. The different structure of pre-trained convolutional networks and their appropriate depth can be effective in improving the extraction of deep features. In addition, the combination of three categories of different features can provide a more comprehensive knowledge of images. The evaluation of the proposed solution on the UCM, AID, RSSCN7 and NWPU-RESISC45 datasets has obtained the accuracy of 99.27%, 97.91%, 99.09% and 93.09% respectively in identifying and classifying images. As a result, this solution has shown a better performance compared to the models that used the manual extraction of features, as well as the methods that use normal convolutional models.
Article number: 3
Type of Study: Research |
Subject:
Paper
Received: 2023/09/9 | Accepted: 2024/08/21 | Published: 2025/01/17 | ePublished: 2025/01/17
Received: 2023/09/9 | Accepted: 2024/08/21 | Published: 2025/01/17 | ePublished: 2025/01/17
References
1. Q. Hu et al., "Exploring the use of Google Earth imagery and object-based methods in land use/cover mapping," Remote Sensing, vol. 5, no. 11, pp. 6026-6042, 2013. [DOI:10.3390/rs5116026]
2. L. Gómez-Chova, D. Tuia, G. Moser, and G. Camps-Valls, "Multimodal classification of remote sensing images: A review and future directions," Proceedings of the IEEE, vol. 103, no. 9, pp. 1560-1584, 2015. [DOI:10.1109/JPROC.2015.2449668]
3. A. Tayyebi, B. C. Pijanowski, and A. H. Tayyebi, "An urban growth boundary model using neural networks, GIS and radial parameterization: An application to Tehran, Iran," Landscape and Urban Planning, vol. 100, no. 1-2, pp. 35-44, 2011. [DOI:10.1016/j.landurbplan.2010.10.007]
4. Z. Y. Lv, W. Shi, X. Zhang, and J. A. Benediktsson, "Landslide inventory mapping from bitemporal high-resolution remote sensing images using change detection and multiscale segmentation," IEEE journal of selected topics in applied earth observations and remote sensing, vol. 11, no. 5, pp. 1520-1532, 2018. [DOI:10.1109/JSTARS.2018.2803784]
5. T. R. Martha, N. Kerle, C. J. Van Westen, V. Jetten, and K. V. Kumar, "Segment optimization and data-driven thresholding for knowledge-based landslide detection by object-based image analysis," IEEE transactions on geoscience and remote sensing, vol. 49, no. 12, pp. 4928-4943, 2011. [DOI:10.1109/TGRS.2011.2151866]
6. F. Ghazouani, I. R. Farah, and B. Solaiman, "A multi-level semantic scene interpretation strategy for change interpretation in remote sensing imagery," IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 11, pp. 8775-8795, 2019. [DOI:10.1109/TGRS.2019.2922908]
7. Y. Li, Y. Zhang, X. Huang, and A. L. Yuille, "Deep networks under scene-level supervision for multi-class geospatial object detection from remote sensing images," ISPRS journal of photogrammetry and remote sensing, vol. 146, pp. 182-196, 2018. [DOI:10.1016/j.isprsjprs.2018.09.014]
8. Y. Gu, Y. Wang, and Y. Li, "A survey on deep learning-driven remote sensing image scene understanding: Scene classification, scene retrieval and scene-guided object detection," Applied Sciences, vol. 9, no. 10, p. 2110, 2019. [DOI:10.3390/app9102110]
9. G. Cheng, X. Xie, J. Han, L. Guo, and G.-S. Xia, "Remote sensing image scene classification meets deep learning: Challenges, methods, benchmarks, and opportunities," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp. 3735-3756, 2020. [DOI:10.1109/JSTARS.2020.3005403]
10. W. Wang, Y. Chen, and P. Ghamisi, "Transferring CNN With Adaptive Learning for Remote Sensing Scene Classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-18, 2022. [DOI:10.1109/TGRS.2022.3190934]
11. K. Xu, H. Huang, P. Deng, and Y. Li, "Deep feature aggregation framework driven by graph convolutional network for scene classification in remote sensing," IEEE Transactions on Neural Networks and Learning Systems, 2021. [DOI:10.1109/TNNLS.2021.3071369] [PMID]
12. S. Aggarwal, "Principles of remote sensing," Satellite remote sensing and GIS applications in agricultural meteorology, vol. 23, no. 2, pp. 23-28, 2004.
13. M. a. Ahmadi and R. Dianat, "Introducing a method for extracting features from facial images based on applying transformations to features obtained from convolutional neural networks," (in eng), Signal and Data Processing, Research vol. 17, no. 3, pp. 141-156, 2020. [DOI:10.29252/jsdp.17.3.141]
14. A. Khan, A. Sohail, U. Zahoora, and A. S. Qureshi, "A survey of the recent architectures of deep convolutional neural networks," Artificial intelligence review, vol. 53, no. 8, pp. 5455-5516, 2020. [DOI:10.1007/s10462-020-09825-6]
15. M. A. Zare Chahooki and z. khalifeh zadeh, "A General Investigation on the Combination of Local and Global Feature Selection Methods for Request Identification on Telegram," (in eng), Signal and Data Processing, Applicable vol. 19, no. 2, pp. 175-196, 2022. [DOI:10.52547/jsdp.19.2.175]
16. M. Imani and H. Ghassemian, "Supervised Feature Extraction of Face Images for Improvement of Recognition Accuracy," (in eng), Signal and Data Processing, Research vol. 16, no. 1, pp. 158-172, 2019. [DOI:10.29252/jsdp.16.1.158]
17. J. Zhang, C. Lu, X. Li, H.-J. Kim, and J. Wang, "A full convolutional network based on DenseNet for remote sensing scene classification," Mathematical Biosciences and Engineering, vol. 16, no. 5, pp. 3345-3367, 2019. [DOI:10.3934/mbe.2019167] [PMID]
18. H. Sun, S. Li, X. Zheng, and X. Lu, "Remote sensing scene classification by gated bidirectional network," IEEE Transactions on Geoscience and Remote Sensing, vol. 58, no. 1, pp. 82-96, 2019. [DOI:10.1109/TGRS.2019.2931801]
19. J. Xie, N. He, L. Fang, and A. Plaza, "Scale-free convolutional neural network for remote sensing scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 9, pp. 6916-6928, 2019. [DOI:10.1109/TGRS.2019.2909695]
20. C. Shi, T. Wang, and L. Wang, "Branch feature fusion convolution network for remote sensing scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, pp 5194-5210, 2020. [DOI:10.1109/JSTARS.2020.3018307]
21. S.-C. Hung, H.-C. Wu, and M.-H. Tseng, "Remote sensing scene classification and explanation using RSSCNet and LIME," Applied Sciences, vol. 10, no. 18, p. 6151, 2020. [DOI:10.3390/app10186151]
22. X. Tang, Q. Ma, X. Zhang, F. Liu, J. Ma, and L. Jiao, "Attention consistent network for remote sensing scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 2030-2045, 2021. [DOI:10.1109/JSTARS.2021.3051569]
23. R. M. Anwer, F. S. Khan, and J. Laaksonen, "Compact deep color features for remote sensing scene classification," Neural Processing Letters, vol. 53, no. 2, pp. 1523-1544, 2021. [DOI:10.1007/s11063-021-10463-4]
24. T. Tian, L. Li, W. Chen, and H. Zhou, "SEMSDNet: A multiscale dense network with attention for remote sensing scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 5501-5514, 2021. [DOI:10.1109/JSTARS.2021.3074508]
25. Q. Bi, H. Zhang, and K. Qin, "Multi-scale stacking attention pooling for remote sensing scene classification," Neurocomputing, vol. 436, pp. 147-161. 2021. [DOI:10.1016/j.neucom.2021.01.038]
26. S. Mei, K. Yan, M. Ma, X. Chen, S. Zhang, and Q. Du, "Remote sensing scene classification using sparse representation-based framework with deep feature fusion," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 5867-5878, 2021. [DOI:10.1109/JSTARS.2021.3084441]
27. J. Shen, T. Zhang, Y. Wang, R. Wang, Q. Wang, and M. Qi, "A Dual-Model Architecture with Grouping-Attention-Fusion for Remote Sensing Scene Classification," Remote Sensing, vol. 13, no. 3, p. 433, 2021. [DOI:10.3390/rs13030433]
28. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in 2009 IEEE conference on computer vision and pattern recognition, 2009, pp. 248-255: Ieee. [DOI:10.1109/CVPR.2009.5206848]
29. B.-D. Liu, J. Meng, W.-Y. Xie, S. Shao, Y. Li, and Y. Wang, "Weighted spatial pyramid matching collaborative representation for remote-sensing-image scene classification," Remote Sensing, vol. 11, no. 5, p. 518, 2019. [DOI:10.3390/rs11050518]
30. B.-D. Liu, W.-Y. Xie, J. Meng, Y. Li, and Y. Wang, "Hybrid collaborative representation for remote-sensing image scene classification," Remote Sensing, vol. 10, no. 12, p. 1934, 2018. [DOI:10.3390/rs10121934]
31. Y. Liu, Y. Liu, and L. Ding, "Scene classification based on two-stage deep feature fusion," IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 183-186, 2017. [DOI:10.1109/LGRS.2017.2779469]
32. T. Gong, X. Zheng, and X. Lu, "Remote Sensing Scene Classification with Multi-task Learning," in Proceedings of the 7th China High Resolution Earth Observation Conference (CHREOC 2020), 2022, pp. 403-418: Springer. [DOI:10.1007/978-981-16-5735-1_30]
33. R. Cao, L. Fang, T. Lu, and N. He, "Self-attention-based deep feature fusion for remote sensing scene classification," IEEE Geoscience and Remote Sensing Letters, vol. 18, no. 1, pp. 43-47, 2020. [DOI:10.1109/LGRS.2020.2968550]
34. Q. Bi, K. Qin, H. Zhang, Z. Li, and K. Xu, "RADC-Net: A residual attention based convolution network for aerial scene classification," Neurocomputing, vol. 377, pp. 345-359, 2020. [DOI:10.1016/j.neucom.2019.11.068]
35. K. Xu, H. Huang, Y. Li, and G. Shi, "Multilayer feature fusion network for scene classification in remote sensing," IEEE Geoscience and Remote Sensing Letters, vol. 17, no. 11, pp. 1894-1898, 2020. [DOI:10.1109/LGRS.2019.2960026]
36. W. Zhang, P. Tang, and L. Zhao, "Remote sensing image scene classification using CNN-CapsNet," Remote Sensing, vol. 11, no. 5, p. 494, 2019. [DOI:10.3390/rs11050494]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
