Volume 16, Issue 2 (9-2019)                   JSDP 2019, 16(2): 61-76 | Back to browse issues page


XML Persian Abstract Print


Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Torkian A, Moallem P. Multi-frame Super Resolution for Improving Vehicle Licence Plate Recognition. JSDP 2019; 16 (2) :61-76
URL: http://jsdp.rcisp.ac.ir/article-1-642-en.html
University of Isfahan
Abstract:   (4044 Views)

License plate recognition (LPR) by digital image processing, which is widely used in traffic monitor and control, is one of the most important goals in Intelligent Transportation System (ITS). In real ITS, the resolution of input images are not very high since technology challenges and cost of high resolution cameras. However, when the license plate image is taken at low resolution, the license plate cannot be readable; hence, the recognition algorithm could not work well. There are many reasons resulting in the degradation of captured license plate images, such as downsampling, blurring, warping, noising, and distance of car from camera. Many researchers try to enhance the quality of input images by image restoration algorithms to improve the LPR final accuracy.
Recently, super-resolution (SR) techniques are widely used to construct a high-resolution (HR) image from several observed low-resolution (LR) images, thereby removing the degradations caused by the imaging of a low resolution camera. As mentioned, in real ITS, the resolution of input image is not high, but there are successive frames from a target, therefore multi-frame SR methods can be used to overcome the ITS resolution challenges.
In this paper, an SR technique based on POCS (Projection onto Convex Sets) is used to reconstruct an HR license plate image from a set of registered LR images. The normalized convolution (NC) framework is used in POCS, in which the local signal is approximated through a projection onto a subspace. However, the window function of adaptive NC is adapted to local linear structures. This results in more samples of the same modality being fused for the reconstruction, which in turn reduces diffusion across discontinuities, that is very important factor in improving LPR accuracy.
The first step in multi-frame SR is image registration which is necessary to improve quality of the reconstructed HR image, especially in LPR when the quality of the reconstructed edges of characters is very important. For simplicity, it is often supposed simple motions (usually translation) between successive frames in multi-frame SR, but changes in scale, rotation and translation in license plate successive images may happened. It means that the registration is one of the main challenges in SR used for LPR. This paper proposes use of a two-step image matching algorithm to improve the quality of registration stage. In the first step, Fourier-Mellin image matching is used for registration which overcomes the scale and rotation challenge, but the accuracy of registration is not suitable. After matching of the successive input images by Fourier-Mellin algorithm, the Keren or Vandewalle image matching is used to improve the quality of final registration. For real LR images, Fourier-Mellin plus Keren shows higher performance while for simulated LR images, Fourier-Mellin plus Vandewalle shows higher performance.
In order to compare the results of two proposed SR algorithms for LPR application with the other methods, we prepare three real datasets of successive frames for Persian LPR, the first and the second one are captured HR and LR successive frames, respectively, while the third one is a downsampled LR version of HR frames. The output HR image of all compared methods is feed to a demo version of a Persian LPR software (www.farsiocr.ir), and the accuracy of each character and the accuracy each license are reported. Five SR methods are compared including: cubic interpolation, ASDS-AR (Adaptive Sparse Domain Selection and Adaptive Regularization), standard POCS, our first and second proposed SR method which both of them firstly use Fourier-Mellin registration, while the first one uses Keren, and the second one uses Vandewalle image matching for a fine registration. Moreover, to present the effectiveness of using SR methods before LPR, the LR images are also directly feed to LPR software.
The results represent when the length of license is less than 50 pixels, using SR methods before LPR improves the recognition accuracy. Moreover, when the license plate length is less 35 pixels, SR methods could not improve the performances. Our investigations show that for LR downsampled images from HR ones, our proposed SR method with Fourier-Mellin plus Keren registration reaches to the highest performance, while for real LR images, which are captured by a low resolution camera, our proposed SR method with Fourier-Mellin plus Vandewalle registration reaches to the highest performance. On the other hand, since some Persian numerical characters, like 2 (2) and 3 (3) are very similar to each other, all of the compared methods may confuse between them in LPR step, therefore, the accuracy per license of all compared methods are not high. Among all previous compared methods, for LR images with length between 35 to 50 pixels, the standard PCOS shows the best results, while our proposed SR methods improve the accuracy per character around 25%, with respect to PCOS method.
 

Full-Text [PDF 5779 kb]   (1178 Downloads)    
Type of Study: Research | Subject: Paper
Received: 2017/02/12 | Accepted: 2018/05/16 | Published: 2019/09/17 | ePublished: 2019/09/17

References
1. [1] G.A. Montazer, M. Shayestehfar, "Iranian License Plate identification with fuzzy support vector machine", Journal of Signal and Data Pro-cessing (JSDP), vol. 12, no. 1, pp. 47-56, 2015.
2. [2] A. Rahimi, P. Moallem, K. Shahtalebi, M. Momeni, "Using Kalman filter in the frequency domain for multi-frame scalable super reso-lution", Signal Processing, vol. 155, pp. 108-129, 2019. [DOI:10.1016/j.sigpro.2018.09.012]
3. [3] Z. Li, G. Han, S. Xiao, and X. Chen, "MAP-based single-frame super-resolution image reconstruction for license plate recognition", International Conference on Computational Intelligence and Software Engineering (CiSE), 2009. [DOI:10.1109/CISE.2009.5366989]
4. [4] W. Dong, L. Zhang, G. Shi, and X. Wu, "Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regula-rization", IEEE Transactions on Image Pro-cessing, vol. 20, no. 7, pp. 1838-1857, 2011. [DOI:10.1109/TIP.2011.2108306] [PMID]
5. [5] S. Aliyan, and A. Broumandnia, "A New Machine Learning Approach to Deblurring License Plate Using K-Means Clustering Method", International Journal of Advanced Research in Artificial Intelligence, vol. 1, no. 2, 2011. [DOI:10.14569/IJARAI.2012.010202]
6. [6] M. Tanaka, and M. Okutomi, "A fast MAP-based super-resolution algorithm for general motion", Proceedings SPIE 6065, Computational Imaging IV, 60651B, 2006. [DOI:10.1117/12.642152]
7. [7] J. Yuan, S.-d. Du, and X. Zhu, "Fast super-resolution for license plate image recons-truction", 19th International Conference on Pa-ttern Recognition (ICPR), 2008. [DOI:10.1109/ICPR.2008.4760990]
8. [8] S. S. Panda, M. Prasad, G. Jena, "POCS based super-resolution image reconstruction using an adaptive regularization parameter", Inter-national Journal of Computer Science Issues, vol. 8, no. 2, pp. 155-158, 2011.
9. [9] L. Tang, "Blind Super-Resolution Image Reconstruction Based on Weighted POCS", International Journal of Multimedia and Ubiqui-tous Engineering, vol. 11, no. 5, pp. 367-376, 2016. [DOI:10.14257/ijmue.2016.11.5.34]
10. [10] S.-C. Lin, and C.-T. Chen, "Reconstructing vehicle license plate image from low resolution images using nonuniform interpolation method", International Journal of Image Processing (IJIP), vol. 1, no. 2, pp. 21-28, 2008.
11. [11] T. Ahmad, and X.M. Li, "An Integrated Interpolation-based Super Resolution Re-construction Algorithm for Video Survei-llance", Journal of Communications (JCM), vol. 7, no. 6, 464-472, 2012. [DOI:10.4304/jcm.7.6.464-472]
12. [12] K. Z. Zefreh, W. van Aarle, K. Batenburg, and J. Sijbers, "Super-resolution of license plate images using algebraic reconstruction technique", Journal of Image and Graphics, vol. 1, no. 2, 94-98, 2013. [DOI:10.12720/joig.1.2.94-98]
13. [13] X. Yan, Q. Shen, and X. Liu, "Super-resolution reconstruction for license plate image in video surveillance system", 10th International Con-ference on Communications and Networking in China (ChinaCom), 2015.
14. [14] D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using sub-pixel displa-cements", International Conference on Computer Vision and Pattern Recognition (CVPR), 1988.
15. [15] T. Q. Pham, L. J. Van Vliet, and K. Schutte, "Robust fusion of irregularly sampled data using adaptive normalized convolution", EURASIP Journal on Applied Signal Processing, vol. 2006, pp. 1-12, 2006. [DOI:10.1155/ASP/2006/83268]
16. [16] P. Vandewalle, S. Susstrunk, and M. Vetterli, "A frequency domain approach to registration of aliased images with application to super-resolution", EURASIP Journal on Advances in Signal Processing, 2006:071459, 2006. [DOI:10.1155/ASP/2006/71459]
17. [17] J. Wang, Z. Xu, and J. Zhang, "Image Registration with Hyperspectral Data Based on Fourier-Mellin Transform", International Jour-nal of Signal Processing Systems, vol. 1, no. 1, pp. 107-110, 2013. [DOI:10.12720/ijsps.1.1.107-110]
18. [18] F. Xu, H. Wang, L. Xu, and C. Huang, "", 2nd International Conference on Computer and Automation Engineering (ICCAE), 2010.
19. [19] R. Berthilsson, "Affine correlation", 14th Inter-national Conference on Pattern Recognition (ICPR), vol. 2, 1458-1460, 1998.

Add your comments about this article : Your username or Email:
CAPTCHA

Send email to the article author


Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

© 2015 All Rights Reserved | Signal and Data Processing