Volume 16, Issue 1 (5-2019)
JSDP 2019, 16(1): 57-74 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Haji-Esmaeili M M, Montazer G. Automatic Colorization of Grayscale Images Using Generative Adversarial Networks. JSDP 2019; 16 (1) :57-74
URL: http://jsdp.rcisp.ac.ir/article-1-789-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-789-en.html
Tarbiat Modares University
Abstract: (4246 Views)
Automatic colorization of gray scale images poses a unique challenge in Information Retrieval. The goal of this field is to colorize images which have lost some color channels (such as the RGB channels or the AB channels in the LAB color space) while only having the brightness channel available, which is usually the case in a vast array of old photos and portraits. Having the ability to colorize such images would give us a multitude of possibilities ranging from colorizing old and historic images to providing alternate colorizations for real images or artistic creations. Be that as it may, the progress in this field is trivial compared to what the professionals are able to do using special-purpose applications such as Photoshop or GIMP. On the other hand, losing the information stored in color channels and having only access to the primary brightness channel, makes this problem a unique challenge, since the main aim of automatic colorization is not to find the image’s “real” color but to colorize it in such a way that makes it “seem real” as the original color information is lost forever and the only way to colorize it, is to provide a somewhat “proper” estimation. In this research we propose a model to automatically colorize gray human portraits. We start by reviewing the methods used for the task of image colorization and provide an explanation as to why most of them collapse to a situation known as “Averaging”. To counteract this effect, we design our end-to-end model with two separate deep neural networks forming a Generative Adversarial Network (GAN), one to colorize the images and the other to evaluate the colorization of the first network and guide it towards the proper distribution. The results show improvements over other proposed methods in this field especially in the case of colorizing human portraits along faster train times. This method not only works on real human portraits but also on non-human and artistic portraits that can be leveraged to colorize hand-drawn images some of which may take minutes up to hours by hand.
Keywords: colorization, gray scale images, deep neural networks, generative adversarial networks, image information retrieval
Type of Study: Applicable |
Subject:
Paper
Received: 2018/02/19 | Accepted: 2019/01/9 | Published: 2019/06/10 | ePublished: 2019/06/10
Received: 2018/02/19 | Accepted: 2019/01/9 | Published: 2019/06/10 | ePublished: 2019/06/10
References
1. [1] J. J. Lloyd, "The Complexity of Recolouring Photos," 2017. [Online]. Available: https-://www.fxguide.com/featured/the-complexity-of-re-colouring-photos/.
2. [2] "r/colorizationrequests." [Online]. Available: https://www.reddit.com/r/colorizationrequests.
3. [3] P. Whitt, Pro Photo Colorizing with GIMP. Apress, 2016. [DOI:10.1007/978-1-4842-1949-2]
4. [4] S. Koo, "Automatic Colorization with Deep Convolutional Generative Adversarial Networks," 2016. [Online]. Available: http://cs231n.stan-ford.edu/reports2016/224_Report.pdf.
5. [5] Aleju, "Aleju Torch Colorizer," 2016. [Online]. Available: https://github.com/aleju/colorizer.
6. [6] R. Zhang, P. Isola, and A. A. Efros, "Colorful Image Colorization," Eccv, pp. 1-25, 2016. [DOI:10.1007/978-3-319-46487-9_40]
7. [7] R. Dahl, "Automatic Colorization," 2016. [Online]. Available: http://tinyclouds.org/colo-rize/.
8. [8] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Networks," Jun. 2014.
9. [9] A. Radford, L. Metz, and S. Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks," arXiv, pp. 1-15, 2015.
10. [10] S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative Adversarial Text to Image Synthesis," Icml, pp. 1060-1069, 2016.
11. [11] M. Arjovsky, S. Chintala, and L. Bottou, "Wasserstein GAN," Jan. 2017.
12. [12] M. Mirza and S. Osindero, "Conditional Generative Adversarial Nets," CoRR, pp. 1-7, 2014.
13. [13] A. Levin, D. Lischinski, and Y. Weiss, "Colorization using optimization," ACM Trans. Graph., vol. 23, no. 3, p. 689, 2004. [DOI:10.1145/1015706.1015780]
14. [14] Y.-C. Huang, Y.-S. Tung, J.-C. Chen, S.-W. Wang, and J.-L. Wu, "An adaptive edge detection based colorization algorithm and its applications," Proc. 13th Annu. ACM Int. Conf. Multimed. - Multimed. '05, no. January, p. 351, 2005. [DOI:10.1145/1101149.1101223]
15. [15] L. Yatziv and G. Sapiro, "Fast image and video colorization using chrominance blending," IEEE Trans. Image Process., vol. 15, no. 5, pp. 1120-1129, 2006. [DOI:10.1109/TIP.2005.864231] [PMID]
16. [16] T. Welsh, M. Ashikhmin, and K. Mueller, "Transferring color to greyscale images," ACM Trans. Graph., vol. 21, no. 3, pp. 277-280, 2002. [DOI:10.1145/566654.566576]
17. [17] R. Gupta, A. Chia, and D. Rajan, "Image colorization using similar images," Proc. 20th …, pp. 369-378, 2012. [DOI:10.1145/2393347.2393402]
18. [18] Z. Cheng, Q. Yang, and B. Sheng, "Deep colorization," Proc. IEEE Int. Conf. Comput. Vis., vol. 11-18-Dece, pp. 415-423, 2016.
19. [19] A. Deshpande, J. Rock, and D. Forsyth, "Learning large-scale automatic image colorization," Proc. IEEE Int. Conf. Comput. Vis., vol. 11-18-Dece, pp. 567-575, 2016.
20. [20] G. Charpiat, M. Hofmann, and B. Schölkopf, "Automatic image colorization via multimodal predictions," Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 5304 LNCS, no. PART 3, pp. 126-139, 2008. [DOI:10.1007/978-3-540-88690-7_10]
21. [21] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," Int. J. Comput. Vis., vol. 115, no. 3, pp. 211-252, 2015. [DOI:10.1007/s11263-015-0816-y]
22. [22] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," Iclr, vol. 96, no. 2, pp. 1-14, 2015.
23. [23] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," Arxiv.Org, vol. 7, no. 3, pp. 171-180, 2015.
24. [24] S. Iizuka, Edgar Simo-Serra, and H. Ishikawa, "Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification," Siggraph '16, vol. 35, no. 4, pp. 1-11, 2016. [DOI:10.1145/2897824.2925974]
25. [25] A. Krizhevsky, Ii. Sulskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," in Nips, 2012, pp. 1-9.
26. [26] A. Odena, V. Dumoulin, and C. Olah, "Deconvolution and Checkerboard Artifacts," Drill, pp. 1-14, 2016. [DOI:10.23915/distill.00003]
27. [27] Z. Liu, P. Luo, X. Wang, and X. Tang, "Deep learning face attributes in the wild," Proc. IEEE Int. Conf. Comput. Vis., vol. 11-18-Dece, pp. 3730-3738, 2016.
28. [28] Y. Guo, L. Zhang, Y. Hu, X. He, and J. Gao, "MS-Celeb-1M : Challenge of Recognizing One Million Celebrities in the Real World," Eur. Conf. Comput. Vis., pp. 87-102, 2016. [DOI:10.2352/ISSN.2470-1173.2016.11.IMAWM-463]
29. [29] M. D. Zeiler, "ADADELTA: An Adaptive Learning Rate Method," arXiv, p. 6, 2012.
30. [30] D. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," Int. Conf. Learn. Represent., 2014.
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
