Volume 19, Issue 1 (5-2022)
JSDP 2022, 19(1): 125-136 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Zabihzadeh D, Zahedi S, Monsefi R. An Effective Approach for Robust Metric Learning in the Presence of Label Noise. JSDP 2022; 19 (1) : 10
URL: http://jsdp.rcisp.ac.ir/article-1-1096-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1096-en.html
Sabzevar University of New Technology
Abstract: (1456 Views)
Many algorithms in machine learning, pattern recognition, and data mining are based on a similarity/distance measure. For instance, the kNN classifier and clustering algorithms such as k-means require a similarity/distance function. Also, in Content-Based Information Retrieval (CBIR) systems, we need to rank the retrieved objects based on the similarity to the query. As generic measures like Euclidean and cosine similarity are not appropriate in many applications, metric learning algorithms have been developed with the aim of learning an optimal distance function from data. These methods often need training data in the form of pair or triplet sets. Nowadays, this training data is popularly obtained via crowdsourcing from the Internet. Therefore, this information may be contaminated with label noise resulting in the poor performance of the learned metric. In some datasets, even it is possible that the learned metrics perform worse than the general ones such as Euclidean. To address this emerging challenge, we present a new robust metric learning algorithm that can identify outliers and label noise simultaneously from training side information. For this purpose, we model the probability distribution of label noise based on information in the training data. The proposed distribution function efficiently assigns the high probability to the data points contaminated with label noise. On the other hand, its value on the normal instances is near zero. Afterward, we weight the training instances according to these probabilities in our metric learning optimization problem. The proposed optimization problem can be solved using available SVM libraries such as LibSVM efficiently. Note that the proposed approach for identifying data with label noise is general and can easily be applied to any existing metric learning algorithms. After the metric learning phase, we utilized both the weights and the learned metric to enhance the accuracy of the metric-based classifier such as kNN. Several experiments are conducted on both real and synthetic datasets. The results confirm that the proposed algorithm enhances the performance of the learned metric in the presence of label noise and considerably outperforms state-of-the-art peer methods at different noise levels.
Article number: 10
Type of Study: Research |
Subject:
Paper
Received: 2019/12/5 | Accepted: 2020/08/18 | Published: 2022/06/22 | ePublished: 2022/06/22
Received: 2019/12/5 | Accepted: 2020/08/18 | Published: 2022/06/22 | ePublished: 2022/06/22
References
1. [1] D. Zabihzadeh, R. Monsefi, and H. S. Yazdi, "Sparse Bayesian similarity learning based on posterior distribution of data," Engineering Applications of Artificial Intelligence, vol. 67, pp. 173-186, 2018. [DOI:10.1016/j.engappai.2017.09.023]
2. [2] L. Lin, G. Wang, W. Zuo, X. Feng, and L. Zhang, "Cross-domain visual matching via generalized similarity measure and feature learning," IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 6, pp. 1089-1102, 2017. [DOI:10.1109/TPAMI.2016.2567386] [PMID]
3. [3] J. Lu, X. Zhou, Y.-P. Tan, Y. Shang, and J. Zhou, "Neighborhood repulsed metric learning for kinship verification," IEEE transactions on pattern analysis and machine intelligence, vol. 36, no. 2, pp. 331-345, 2014. [DOI:10.1109/TPAMI.2013.134] [PMID]
4. [4] S. Bak and P. Carr, "One-Shot Metric Learning for Person Re-identification," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 2990-2999. [DOI:10.1109/CVPR.2017.171]
5. [5] N. Jiang, W. Liu, and Y. Wu, "Order determination and sparsity-regularized metric learning adaptive visual tracking," in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, 2012: IEEE, pp. 1956-1963. [DOI:10.1109/CVPR.2012.6247897]
6. [6] M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid, "Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation," in Computer Vision, 2009 IEEE 12th International Conference on, 2009: IEEE, pp. 309-316. [DOI:10.1109/ICCV.2009.5459266]
7. [7] G. Chechik, V. Sharma, U. Shalit, and S. Bengio, "Large Scale Online Learning of Image Similarity Through Ranking," J. Mach. Learn. Res., vol. 11, pp. 1109-1135, 2010. [DOI:10.1007/978-3-642-02172-5_2]
8. [8] X. Hao, S. C. H. Hoi, J. Rong, and Z. Peilin, "Online Multiple Kernel Similarity Learning for Visual Search," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 36, no. 3, pp. 536-549, 2014, doi: 10.1109/TPAMI-.2013.149. [DOI:10.1109/TPAMI.2013.149] [PMID]
9. [9] P. Wu, S. C. H. Hoi, P. Zhao, C. Miao, and Z. Y. Liu, "Online Multi-Modal Distance Metric Learning with Application to Image Retrieval," IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 2, pp. 454-467, 2016, doi: 10.1109/TKDE.2015.2477296. [DOI:10.1109/TKDE.2015.2477296]
10. [10] J. Li, C. Xu, W. Yang, C. Sun, and D. Tao, "Discriminative Multi-View Interactive Image Re-Ranking," IEEE Transactions on Image Processing, 2017. [DOI:10.1109/TIP.2017.2651379] [PMID]
11. [11] A. Bellet, A. Habrard, and M. Sebban, "A Survey on Metric Learning for Feature Vectors and Structured Data," Technical report, 2014.
12. [12] B. Frénay and M. Verleysen, "Classification in the presence of label noise: a survey," IEEE transactions on neural networks and learning systems, vol. 25, no. 5, pp. 845-869, 2013. [DOI:10.1109/TNNLS.2013.2292894] [PMID]
13. [13] T. Yang, R. Jin, and A. K. Jain, "Learning from noisy side information by generalized maximum entropy model," in Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010: Citeseer, pp. 1199-1206.
14. [14] K. Huang, R. Jin, Z. Xu, and C.-L. Liu, "Robust metric learning by smooth optimiza-tion," arXiv preprint arXiv:1203.3461, 2012.
15. [15] Y. Nesterov, "Smooth minimization of non-smooth functions," Mathematical programm-ing, vol. 103, no. 1, pp. 127-152, 2005. [DOI:10.1007/s10107-004-0552-5]
16. [16] D. Wang and X. Tan, "Robust Distance Metric Learning in the Presence of Label Noise," in AAAI, 2014, pp. 1321-1327. [DOI:10.1609/aaai.v28i1.8903]
17. [17] H. Wang, F. Nie, and H. Huang, "Robust Distance Metric Learning via Simultaneous L1-Norm Minimization and Maximization," in Proceedings of the 31st International Conference on Machine Learning (ICML-14), T. Jebara and E. P. Xing, Eds., 2014, [Formatter not found: ResolvePDF]: JMLR Workshop and Conference Proceedings, pp. 1836-1844. [Online]. Available: http://jml-r.org/proceedings/papers/v32/wangj14.pdf. [Online]. Available: http://jmlr.org/proceed-ings/papers/v32/wangj14.pdf
18. [18] S. Xiang, F. Nie, and C. Zhang, "Learning a Mahalanobis distance metric for data clustering and classification," Pattern Recogn., vol. 41, no. 12, pp. 3600-3612, 2008, doi: 10.1016/j.patcog.2008.05.018. [DOI:10.1016/j.patcog.2008.05.018]
19. [19] D. Wang and X. Tan, "Robust Distance Metric Learning via Bayesian Inference," IEEE Transactions on Image Processing, vol. 27, no. 3, pp. 1542-1553, 2018. [DOI:10.1109/TIP.2017.2782366] [PMID]
20. [20] D. Zabihzadeh, R. Monsefi, and H. S. Yazdi, "Sparse Bayesian approach for metric learning in latent space," Knowledge-Based Systems, vol. 178, pp. 11-24, 2019. [DOI:10.1016/j.knosys.2019.04.009]
21. [21] K. Q. Weinberger and L. K. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification," J. Mach. Learn. Res., vol. 10, pp. 207-244, 2009.
22. [22] S. Al-Obaidi, D. Zabihzadeh, A. S. Rasheed, and R. Monsefi, "Robust Metric Learning based on the Rescaled Hinge Loss," arXiv preprint arXiv:1904.11711, 2019. [DOI:10.1007/s13042-020-01137-z]
23. [23] F. Wang, W. Zuo, L. Zhang, D. Meng, and D. Zhang, "A kernel classification framework for metric learning," IEEE transactions on neural networks and learning systems, vol. 26, no. 9, pp. 1950-1962, 2015. [DOI:10.1109/TNNLS.2014.2361142] [PMID]
24. [24] C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," ACM transactions on intelligent systems and technology (TIST), vol. 2, no. 3, p. 27, 2011. [DOI:10.1145/1961189.1961199]
25. [25] L. v. d. Maaten and G. Hinton, "Visualizing data using t-SNE," Journal of machine learning research, vol. 9, no. Nov, pp. 2579-2605, 2008.
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
