Volume 16, Issue 4 (3-2020)
JSDP 2020, 16(4): 59-72 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Banitalebi-Dehkordi M, Ebrahimi-moghadam A, Khademi M, Hadizadeh H. Compressed-Sampling-Based Image Saliency Detection in the Wavelet Domain. JSDP 2020; 16 (4) :59-72
URL: http://jsdp.rcisp.ac.ir/article-1-874-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-874-en.html
Abstract: (3124 Views)
When watching natural scenes, an overwhelming amount of information is delivered to the Human Visual System (HVS). The optic nerve is estimated to receive around 108 bits of information a second. This large amount of information can’t be processed right away through our neural system. Visual attention mechanism enables HVS to spend neural resources efficiently, only on the selected parts of the scene at order. This results in a better and faster perception of events.
In order to perform saliency measurement on visual data, subjective eye-tracking experiments may be carried out. These experiments involve using devices to track eye movements of a number of subjects while they watch images or videos on a screen.
That being said, such devices are not very suitable in practice due to hardship involved with carrying out experiments, such as need to have restricted test environment, being time consuming as well as expensive. Instead, researchers developed Computational Visual Attention Models (VAMs) in attempts to mimic the HVS saliency prediction process.
Visual Attention Modelling has widely been used in various areas of image processing and understanding. Computational models of visual attention aim to predict the most interesting areas of an image to the observers. To this end, these models produce saliency maps, in which each pixel is assigned a likelihood value of being looked at. In other words, saliency maps highlight where the most likely for viewers to look at in an image is. Knowing the Regions of Interests (ROIs) can be helpful in applications such as image and video compression, object recognition and detection, visual search, retargeting, retrieval, image matching, and segmentation.
Saliency prediction is generally done in a bottom-up, top-down, or hybrid fashion. Bottom-up approaches exploit low-level attributes such as brightness, color, edges, texture, etc. Top-down approaches focus on context-dependent information from the scene such as appearance of humans, animals, text, etc. Hybrid methods combine the two streams.
This paper proposes a new method of saliency prediction using sparse wavelet coefficients selected from low-level bottom-up saliency features. Wavelet based image methods are used widely in image processing algorithms as they are especially powerful in decomposing images into several scales of resolutions. In our method, first random compressive sampling is performed on wavelet coefficients in the Lab color space. Random sampling enables a reduction in computational complexity and provides a sparse representation of the coefficients. The number of decomposition levels is chosen based on the information diffusion property of the signal. In the proposed method, the sampling can be done at a rate different than the Nyquist rate, and based on the sparsity degree of the signal. It is shown that having the basis vectors of a sparse representation of the signal, can result in an accurate signal reconstruction. In this work, the sparsity degree and thus the sampling rate is computed empirically. Next, local and global saliency maps are generated from these random samples to account for small-scale and large-scale (scene-wide) saliency attributes. These maps are then combined to form an overall saliency map. The overall saliency map therefore includes both local, and global saliency attributes. The main contribution of this paper is the use of compressive sampling in creating a novel wavelet domain representation for image saliency prediction.
Extensive performance evaluations show that the proposed method provides a promising saliency prediction performance while the computation complexity remains reasonable, thanks to the dimensionality reduction of compressive sampling. In particular, the proposed method demonstrated favorable precision, recall, and F-measure, when compared to state-of-the-art saliency detection methods, over large-scale datasets. We hope the proposed approach brings ideas to the saliency analysis research community.
In order to perform saliency measurement on visual data, subjective eye-tracking experiments may be carried out. These experiments involve using devices to track eye movements of a number of subjects while they watch images or videos on a screen.
That being said, such devices are not very suitable in practice due to hardship involved with carrying out experiments, such as need to have restricted test environment, being time consuming as well as expensive. Instead, researchers developed Computational Visual Attention Models (VAMs) in attempts to mimic the HVS saliency prediction process.
Visual Attention Modelling has widely been used in various areas of image processing and understanding. Computational models of visual attention aim to predict the most interesting areas of an image to the observers. To this end, these models produce saliency maps, in which each pixel is assigned a likelihood value of being looked at. In other words, saliency maps highlight where the most likely for viewers to look at in an image is. Knowing the Regions of Interests (ROIs) can be helpful in applications such as image and video compression, object recognition and detection, visual search, retargeting, retrieval, image matching, and segmentation.
Saliency prediction is generally done in a bottom-up, top-down, or hybrid fashion. Bottom-up approaches exploit low-level attributes such as brightness, color, edges, texture, etc. Top-down approaches focus on context-dependent information from the scene such as appearance of humans, animals, text, etc. Hybrid methods combine the two streams.
This paper proposes a new method of saliency prediction using sparse wavelet coefficients selected from low-level bottom-up saliency features. Wavelet based image methods are used widely in image processing algorithms as they are especially powerful in decomposing images into several scales of resolutions. In our method, first random compressive sampling is performed on wavelet coefficients in the Lab color space. Random sampling enables a reduction in computational complexity and provides a sparse representation of the coefficients. The number of decomposition levels is chosen based on the information diffusion property of the signal. In the proposed method, the sampling can be done at a rate different than the Nyquist rate, and based on the sparsity degree of the signal. It is shown that having the basis vectors of a sparse representation of the signal, can result in an accurate signal reconstruction. In this work, the sparsity degree and thus the sampling rate is computed empirically. Next, local and global saliency maps are generated from these random samples to account for small-scale and large-scale (scene-wide) saliency attributes. These maps are then combined to form an overall saliency map. The overall saliency map therefore includes both local, and global saliency attributes. The main contribution of this paper is the use of compressive sampling in creating a novel wavelet domain representation for image saliency prediction.
Extensive performance evaluations show that the proposed method provides a promising saliency prediction performance while the computation complexity remains reasonable, thanks to the dimensionality reduction of compressive sampling. In particular, the proposed method demonstrated favorable precision, recall, and F-measure, when compared to state-of-the-art saliency detection methods, over large-scale datasets. We hope the proposed approach brings ideas to the saliency analysis research community.
Type of Study: Research |
Subject:
Paper
Received: 2018/06/13 | Accepted: 2019/02/13 | Published: 2020/04/20 | ePublished: 2020/04/20
Received: 2018/06/13 | Accepted: 2019/02/13 | Published: 2020/04/20 | ePublished: 2020/04/20
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
