Volume 18, Issue 2 (10-2021)
JSDP 2021, 18(2): 29-44 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ghaffari H R, Jalali Mojahed A. Feature extraction based on the more resolution of the classes using auxiliary classifiers. JSDP 2021; 18 (2) :29-44
URL: http://jsdp.rcisp.ac.ir/article-1-986-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-986-en.html
Department of Computer Engineering,Department of Computer Engineering, University of Ferdows, Ferdows, Iran
Abstract: (1728 Views)
Classification is a machine learning method used to predict a particular sample’s label with the least error. The present study was conducted using label prediction ability with the help of a classifier to create a new feature. Today, there are several feature-extraction methods like principal component analysis (PCA) and independent component analysis (ICA) that are widely used in different fields; however, they all suffer from the high cost of transferring to another space. The purpose of the proposed method was to create a higher distinction between various classes using the new feature in a way that, make the data in the classes closer to each other. As a result, for increasing the efficiency of classifiers, more differentiation is created between the data of various classes. Firstly, the suggested labels for the primary data set were determined using one or more classifiers and added to the primary data set as a new feature. The model was created using a new data set. The new feature for training and testing data sets was provided separately. The tests were performed on 20 standard data sets and the results of the proposed method were compared with those of the two methods described in the related studies. The outputs indicated that the proposed method has significantly improved the classification accuracy. In the second part of the tests, the resolution of the new feature was examined according to two criteria, namely Information Gain and Gini Index, for examining the effectiveness of the proposed method. The results showed that the feature obtained in the proposed method has higher Information Gain and lower Gini Index in most cases, as it has less irregularity. To prevent the increase in data dimensions, the feature with the least Information Gain was replaced with the feature extracted with the most Information Gain. The results of this step showed an increase in efficiency as well.
Type of Study: Research |
Subject:
Paper
Received: 2019/03/14 | Accepted: 2020/08/18 | Published: 2021/10/8 | ePublished: 2021/10/8
Received: 2019/03/14 | Accepted: 2020/08/18 | Published: 2021/10/8 | ePublished: 2021/10/8
References
1. [1] S. J.Russell and P. Norvig, Artificial intelligence: a modern approach. Malaysia, Pearson Education Limited, 2016.
2. [2] R. P. Duin and D. M. J. Tax, "Statistical pattern recognition", In Handbook of Pattern Recognition and Computer Vision, pp. 3-24, 2005. [DOI:10.1142/9789812775320_0001]
3. [3] Guyon, S. Gunn, M.Nikravesh, L. A. Zadeh, and editors, Feature extraction: foundations and applications, Vol. 207. Springer, 2008.
4. [4] T. Cover and P. Hart, "Nearest neighbor pattern classification," IEEE transactions on information theory, vol. 13, no. 1, pp. 21-27, 1967. [DOI:10.1109/TIT.1967.1053964]
5. [5] C. Cortes and V. Vapnik, "Support-vector networks," Machine learning, vol. 20, no. 3, pp. 273-297, 1995. [DOI:10.1007/BF00994018]
6. [6] J. Showe-Taylor and N. Christianini, Support vector machines and other kernel-based learning methods, 2000. [DOI:10.1017/CBO9780511801389] [PMCID]
7. [7] J. R. Quinlan, C4. 5: programs for machine learning. Elsevier, 2014.
8. [8] B. W. Silverman and M. C. Jones, "An important contribution to nonparametric discriminant analysis and density estimation," International statistical review/revue Internationale de statistique, pp. 233-238, 1989. [DOI:10.2307/1403796]
9. [9] R. C.Barros, M. P. Basgalupp, A. C. De Carvalho, and A. A. Freitas, "A survey of evolutionary algorithms for decision-tree induction," IEEETransactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no 3, pp. 291-312, 2012. [DOI:10.1109/TSMCC.2011.2157494]
10. [10] L. Breiman, "Random forests," Machine learning, vol. 45, no. 1, pp. 5-32, 2001. [DOI:10.1023/A:1010933404324]
11. [11] M.Woźniak, M.GrañaandE. Corchado, "A survey of multiple classifier systems as hybrid systems," Information Fusion, vol. 16, pp. 3-17, 2014. [DOI:10.1016/j.inffus.2013.04.006]
12. [12] H. Hotelling, "Analysis of a Complex of Statistical Variables into Principal Components," Journal of Educational Psychology, vol. 24, no. 6, pp. 417-441, 1933. [DOI:10.1037/h0071325]
13. [13] P. Comon, "Independent component analysis, a new concept?" Signal Processing, vo. 36, no. 3, pp. 287-314, 1994. [DOI:10.1016/0165-1684(94)90029-9]
14. [14] K. Fukunaga, "Introduction to Statistical Pattern Recognition," San Diego: Academic Press Inc, 1990. [DOI:10.1016/B978-0-08-047865-4.50007-7] [PMID]
15. [15] C. F. Tsai and C. Y Lin, "A triangle area based nearest neighbors approach to intrusion detection," Pattern recognition. vol. 43, no. 1, pp. 222-229, 2010. [DOI:10.1016/j.patcog.2009.05.017]
16. [16] W. C.Lin, S. W. Ke and C. F. Tsai, "CANN: An intrusion detection system based on combining cluster centers and nearest neighbors", Knowledge-based systems, no. 78, pp. 13-21, 2015. [DOI:10.1016/j.knosys.2015.01.009]
17. [17] X. Wang, C. Zhang and K. Zheng, "Intrusion detection algorithm based on density, cluster centers, and nearest neighbors", China Communications, vol. 13, no. 7, pp. 24-31, 2016. [DOI:10.1109/CC.2016.7559072]
18. [18] A. Asuncion and D. J. Newman, UCI Machine Learning Repository, University of California, 2007. https://archive.ics.uci.edu/ml/index.php
19. [19] C. W. Hsua and C. J. Lin, "A comparison of methods for multiclass support vector machines", IEEE transactions on Neural Networks, vol. 13, no. 2, pp. 415-425, 2002. [DOI:10.1109/72.991427] [PMID]
20. [20] T. T. Wong, "Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation", Pattern Recognition, vol. 48, no. 9, pp. 2839-2846, 2015. [DOI:10.1016/j.patcog.2015.03.009]
21. [21] J. T. Townsend, "Theoretical analysis of an alphabetic confusion matrix", Perception & Psychophysics, vol. 9, no. 1, pp. 40-50, 1971. [DOI:10.3758/BF03213026]
22. [22] M. Dash and H. Liu, "Consistency-based search in feature selection", Artificial intelligence, vol. 151, no. 1-2, pp. 155-176, 2003. [DOI:10.1016/S0004-3702(03)00079-1]
23. [23] J. R. Quinlan, "Induction of decision trees", Machine learning, vol. 1, no. 1, pp. 81-106, 1986. [DOI:10.1007/BF00116251]
24. [24] L. Breiman, "Classification and regression trees".Routledge, 2017. [DOI:10.1201/9781315139470]
25. [25] L. E. Raileanu and K. Stoffel, "Theoretical comparison between the gini index and information gain criteria", Annals of Mathematics and Artificial Intelligence, vol. 41, no. 1, pp.77-93, 2004. [DOI:10.1023/B:AMAI.0000018580.96245.c6]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
