Volume 19, Issue 2 (9-2022)
JSDP 2022, 19(2): 161-174 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Tohidi N, Hasheminejad S M H. Optimizing question answering systems by Accelerated Particle Swarm Optimization (APSO). JSDP 2022; 19 (2) : 11
URL: http://jsdp.rcisp.ac.ir/article-1-1098-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1098-en.html
Alzahra University
Abstract: (1149 Views)
One of the most important research areas in natural language processing is Question Answering Systems (QASs). Existing search engines, with Google at the top, have many remarkable capabilities. However, there is a basic limitation; search engines do not have deduction capability which a QAS is expected to have. In this perspective, a search engine may be viewed as a semi-mechanized QAS. Upgrading a search engine such to a QAS is a task whose complexity is hard to exaggerate. To achieve success, new concepts and ideas are needed to address difficult problems which arise when knowledge has to be dealt with in an environment of imprecision, uncertainty and partial truth.
QASs are search engines that have the ability to provide a brief and accurate answer to each question in natural language for instance, the question that a search engine answers with a set of documents, a QAS answers with a paragraph, sentence or etc. In this paper, a solution is proposed to optimize the performance and speed of web-based QASs for answering English questions.
As evolutionary algorithms are suitable for issues with large search space, in this approach we have used an evolutionary algorithm to optimize QASs. In this regard, we have chosen APSO which is a simplified version of PSO. The proposed method consists of five main stages: question analysis, pre-process, retrieval, extraction and ranking. We have tried to provide a method that would be more accurate in choosing the most probable answer from the documents that have been retrieved by the standard search engine and at the same time, be faster than similar methods. In ranking process, various attributes can be extracted from the text that are used in APSO. For this purpose, in addition to selecting a sentence from the text and examining its attributes, different cut parts of the sentence are selected each time by changing the beginning and end points of the cut part. The attributes which have been used in this study are: 1. Number of unigrams similar to the question words, 2. Number of bigrams similar to the question words, 3. Number of unigrams similar to the question words in the cut part, 4. Number of bigrams similar to the question words in the cut part, 5. Number of synonyms with the question words and 6. Number of synonyms with the question words in the cut part. The fitness function is the weighted sum of these attributes.
Top-1 accuracy and MRR are the most valid metrics for measuring the performance of QASs. The proposed method has achieved the accuracy (top-1 accuracy) of 0.527 with respect to the standard dataset and the MRR of it, is 0.711. Both of these results are improved compared to most similar systems. In addition, the time taken to answer the input question in the proposed method, has been significantly reduced compared to similar methods. In general, the accuracy and MRR in this paper have progressed and the system needs less time to find the answer, in comparison with existing QASs.
QASs are search engines that have the ability to provide a brief and accurate answer to each question in natural language for instance, the question that a search engine answers with a set of documents, a QAS answers with a paragraph, sentence or etc. In this paper, a solution is proposed to optimize the performance and speed of web-based QASs for answering English questions.
As evolutionary algorithms are suitable for issues with large search space, in this approach we have used an evolutionary algorithm to optimize QASs. In this regard, we have chosen APSO which is a simplified version of PSO. The proposed method consists of five main stages: question analysis, pre-process, retrieval, extraction and ranking. We have tried to provide a method that would be more accurate in choosing the most probable answer from the documents that have been retrieved by the standard search engine and at the same time, be faster than similar methods. In ranking process, various attributes can be extracted from the text that are used in APSO. For this purpose, in addition to selecting a sentence from the text and examining its attributes, different cut parts of the sentence are selected each time by changing the beginning and end points of the cut part. The attributes which have been used in this study are: 1. Number of unigrams similar to the question words, 2. Number of bigrams similar to the question words, 3. Number of unigrams similar to the question words in the cut part, 4. Number of bigrams similar to the question words in the cut part, 5. Number of synonyms with the question words and 6. Number of synonyms with the question words in the cut part. The fitness function is the weighted sum of these attributes.
Top-1 accuracy and MRR are the most valid metrics for measuring the performance of QASs. The proposed method has achieved the accuracy (top-1 accuracy) of 0.527 with respect to the standard dataset and the MRR of it, is 0.711. Both of these results are improved compared to most similar systems. In addition, the time taken to answer the input question in the proposed method, has been significantly reduced compared to similar methods. In general, the accuracy and MRR in this paper have progressed and the system needs less time to find the answer, in comparison with existing QASs.
Article number: 11
Keywords: question answering system, natural language processing, accelerated particle swarm optimization (APSO)
Type of Study: Research |
Subject:
Paper
Received: 2019/12/14 | Accepted: 2021/01/24 | Published: 2022/09/30 | ePublished: 2022/09/30
Received: 2019/12/14 | Accepted: 2021/01/24 | Published: 2022/09/30 | ePublished: 2022/09/30
References
1. [1] F. Sebastiani, "Machine Learning in in Automated Text Categorization," ACM Computing Surveys (CSUR), vol. 34, pp. 1-47, 2002. [DOI:10.1145/505282.505283]
2. [2] H. Sundblad, "Question Classification in Question Answering Systems," 2007.
3. [3] M. Bazrafshan, S. Heidari, E. Dorodi, A. Rahnama, Z. Sarabi, E. Sherkat, M.Fahordi, M. Lali, M. Homan, SMH. Hashemienjad, "Question Answering for Quran," ITRC, 2012.
4. [4] Tohidi, Nasim; Rustamov, Rustam B., "Short Overview of Advanced Metaheuristic Methods," International Journal on Technical and Physical Problems of Engineering (IJTPE), vol. 14, pp. 84-97, 2022.
5. [5] D. Graff, "The AQUAINT Corpus of English News Text LDC2002T31," Web Download. Philadelphia: Linguistic Data Consortium, 2002.
6. [6] I.Khodadi, M. Saniee Abadeh, "A Memetic-Based Approach for Web-Based Question Answering," Information Technology and Computer Science, vol. 9, pp. 39-45, 2014. [DOI:10.5815/ijitcs.2014.09.05]
7. [7] J. Kennedy and R. C. Eberhart, "Particle swarm optimization," in in Proceedings of the IEEE International Conference on Neural Networks, 1995.
8. [8] A. L. Ballardini, "A tutorial on Particle Swarm Optimization Clustering," ArXiv, vol. abs/1809.01942, 2018.
9. [9] Y. XS, Nature-Inspired Metaheuristic Algorithms., Luniver Press, 2008.
10. [10] N. Tohidi, Ch. Dadkhah, "Improving the performance of video Collaborative Filtering Recommender Systems using Optimization Algorithm," International Journal of Nonlinear Analysis and Applications, vol. 11, no. 1, pp. 283-295, 2020.
11. [11] A. Figueroa, G. Neumann, "Genetic Algorithms for Data-DrivenWeb Question Answering, Massachusetts Institute of Technology," Evolutionary Computation, 2008. [DOI:10.1162/evco.2008.16.1.89] [PMID]
12. [12] A. Mishra and S. Kumar Jain, "A survey on question answering systems with classification," Elsevier, Computer and Information Sciences, vol. 28, pp. 345-361, 2015. [DOI:10.1016/j.jksuci.2014.10.007]
13. [13] N. Tohidi, Ch. Dadkhah, B. Rustamov, "Optimizing the performance of Persian multi-objective question answering system," in 16th International Conference on Technical and Physical Problems of Electrical Engineering, Istanbul, 2020.
14. [14] N. Tohidi, Ch. Dadkhah, B. Rustamov, "Optimizing Persian Multi-objective Question Answering System," International Journal on Technical and Physical Problems of Engineering (IJTPE), vol. 13, pp. 62-69, 2021.
15. [15] H.Yu, D. Kaufman, "A cognitive evaluation of four online Search Engines for Answering Definitional Questioned posed by Physicians," in Pacific Symposium on Biocomputing, 2007.
16. [16] N. Tohidi, S.M.H. Hasheminejad, "MOQAS: Multi-objective question answering system," Journal of Intelligent & Fuzzy Systems, vol. 36, no. 4, pp. 3495-3512, 2019. [DOI:10.3233/JIFS-181364]
17. [17] N. Smith, A. Heilman, M. Hwa, "Question Generation as a Competitive Undergraduate Course Project," in In Proceedings of the NSF Workshop on the Question Generation Shared Task and Evaluation Challenge, Arlington, VA, 2008.
18. [18] I. Khodadi, M. Saniee Abadeh, "Genetic programming-based feature learning for question answering," Elsevier, Information Processing and Management, vol. 40, 2015.
19. [19] A.Severyn, A.Moschitti, "Automatic Feature Engineering for Answer Selection and Extraction," in EMNLP Conference, 2013.
20. [20] A.Severyn, M.Nicosia, A. Moschitti, "Learning adaptable patterns for passage reranking," in CoNLL Conference, 2013.
21. [21] A.Severyn, A.Moschitti, "Structural relationships for large-scale learning of answer re-ranking," in ACM, In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, 2012. [DOI:10.1145/2348283.2348383]
22. [22] A. Moschitti, S. Quarteroni, "Linguistic kernels for answer re-ranking in question answering systems", Elsevier, Information Processing and Management, vol. 47, p. 825-842, 2011. [DOI:10.1016/j.ipm.2010.06.002]
23. [23] M. H. Heie, E. W. D.Whittaker, S. Furui, "Question answering using statistical language modelling," Computer Speech and Language, vol. 26, no. 3, pp. 193-209, 2012. [DOI:10.1016/j.csl.2011.11.001]
24. [24] P. Moreda, H. Llorens, E. Saquete, M. Palomar, "Combining semantic information in question answering systems," Information Processing & Management, vol. 47, no. 6, pp. 870-885, 2011. [DOI:10.1016/j.ipm.2010.03.008]
25. [25] S. Yoon, A. Jatowt, K. Tanaka, "Detecting Intent of Web Queries Using Questions and Answers in CQA Corpus," in 2011 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology, Lyon, 2011. [DOI:10.1109/WI-IAT.2011.41]
26. [26] S. Kandasamy, A.Cherukuri, "Information retrieval for Question Answering System using Knowledge based Query Reconstruction by adapted LESK and LATENT Semantic analysis," International Journal of Computer Science & Applications, vol. 14, no. 2, 2017.
27. [27] D. Croce, A. Moschitti, R. Basili, "Structured lexical similarity via convolution kernels on dependency trees," in Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2011. [DOI:10.1145/2063576.2063878]
28. [28] H.Toba, Z. Ming, Y. Adriani, M. Chua, "Discovering high quality answers in community question answering archives using a hierarchy of classifiers," Elzevier, Information Sciences, vol. 261, pp. 101-115, 2014. [DOI:10.1016/j.ins.2013.10.030]
29. [29] Z. Yu, L. Su, L. Zhao, Q. Mao, C. Guo, "Question classification based on co-training style semi-supervised learning," Elzevier, Pattern Recognition Letters, vol. 31, no. 13, pp. 1975-1980, 2010. [DOI:10.1016/j.patrec.2010.06.010]
30. [30] A. Ulysse Côté, Kh. Richard; Lamontagne, Luc; Bergeron, Jonathan; Laviolette, François; Bergeron-Guyard, Alexandre;, "Optimizing Question-Answering Systems Using Genetic Algorithms," in Proceedings of the Twenty-Eighth International Florida Artificial Intelligence Research Society Conference, 2015.
31. [31] K. Karpagam, A. Saradha, "A Hybrid Optimization Technique for Effective Document Clustering in Question Answering System." ICTACT Journal on Soft Computing, vol. 7, no. 3, pp. 1447-1451, 2017. [DOI:10.21917/ijsc.2017.0200]
32. [32] Ojokoh, Bolanle; Adebisi, Emmanue, "A Review of Question Answering Systems," Journal of Web Engineering, vol. 17-8, pp. 717-758, 2019. [DOI:10.13052/jwe1540-9589.1785]
33. [33] S.M.H. Hasheminejad, "An Evolutionary Approach to Identify Logical Components," Journal of Systems and Software, vol. 96, pp. 24-50, 2014. [DOI:10.1016/j.jss.2014.05.033]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
