Volume 21, Issue 3 (12-2024)
JSDP 2024, 21(3): 97-110 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Naghavi M, Hassani Ahangar M R, Amiri Jezeh A. Identify the Named Entities Using Deep Learning and Reinforcement Approach. JSDP 2024; 21 (3) : 5
URL: http://jsdp.rcisp.ac.ir/article-1-1232-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1232-en.html
Assistant Professor, Faculty of Computer, Network, and Communications, Imam Hossein University, Tehran, Iran
Abstract: (729 Views)
Named Entity Recognition (NER) has emerged as a critical and highly applicable task in the field of Natural Language Processing (NLP). Its significance stems from its essential role in numerous NLP applications, such as machine translation, question answering, text summarization, and information extraction. Recent studies highlight the substantial impact of advancements in Artificial Intelligence (AI), particularly Deep Neural Networks (DNNs), on improving the performance of NER systems.
Deep Neural Networks, with their ability to learn complex patterns and extract rich features, have opened new horizons in addressing NLP challenges. These methods leverage advanced language models like BERT and GPT to enable deeper comprehension of linguistic structures and semantic relationships. One of their prominent capabilities is to capture long-term dependencies in complex sentences while reducing the reliance on manually engineered features.
This research introduces a novel hybrid approach for Named Entity Recognition in both Persian and English languages, based on deep neural networks and semantic language models. To address the dependency on large datasets, the proposed method employs an iterative logic mechanism that facilitates effective learning with limited data. The proposed system was evaluated on three datasets: The CoNLL 2003 dataset for English, Two Persian datasets, Arman and Peyma.
Deep Neural Networks, with their ability to learn complex patterns and extract rich features, have opened new horizons in addressing NLP challenges. These methods leverage advanced language models like BERT and GPT to enable deeper comprehension of linguistic structures and semantic relationships. One of their prominent capabilities is to capture long-term dependencies in complex sentences while reducing the reliance on manually engineered features.
This research introduces a novel hybrid approach for Named Entity Recognition in both Persian and English languages, based on deep neural networks and semantic language models. To address the dependency on large datasets, the proposed method employs an iterative logic mechanism that facilitates effective learning with limited data. The proposed system was evaluated on three datasets: The CoNLL 2003 dataset for English, Two Persian datasets, Arman and Peyma.
Experimental results demonstrate that the proposed method achieves F1-scores of 95.3, 96.32, and 94.72 on the CoNLL, Arman, and Peyma datasets, respectively. These scores reflect significant improvements over previous methods.
The findings of this study suggest that combining advanced language models with deep neural networks can significantly enhance the accuracy and efficiency of NER systems. These achievements pave the way for developing effective NLP tools for low-resource languages, particularly Persian, and enable the application of this technology in both industrial and research contexts.
Article number: 5
Type of Study: Research |
Subject:
Paper
Received: 2021/05/15 | Accepted: 2024/02/25 | Published: 2025/01/17 | ePublished: 2025/01/17
Received: 2021/05/15 | Accepted: 2024/02/25 | Published: 2025/01/17 | ePublished: 2025/01/17
References
1. D. Nadeau and S. Sekine, "A survey of named entity recognition and classification," Lingvisticae Investigationes, vol. 30, no. 1, pp. 3-26, 2007. [DOI:10.1075/li.30.1.03nad]
2. J. Guo, G. Xu, X. Cheng, and H. Li, "Named entity recognition in query," in Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, 2009, pp. 267-274. [DOI:10.1145/1571941.1571989] []
3. D. Petkova and W. B. Croft, "Proximity-based document representation for named entity retrieval," in Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, 2007, pp. 731-740. [DOI:10.1145/1321440.1321542]
4. B. Larsen, "A trainable summarizer with knowledge acquired from robust NLP techniques," Advances in automatic text summarization, vol. 71, 1999.
5. D. Mollá, M. Van Zaanen, and D. Smith, "Named entity recognition for question answering," 2006.
6. B. Babych and A. Hartley, "Improving machine translation quality with automatic named entity recognition," in Proceedings of the 7th International EAMT workshop on MT and other language technology tools, Improving MT through other language technology tools, Resource and tools for building MT at EACL 2003, 2003. [DOI:10.3115/1609822.1609823]
7. S. Scholar."Semantic Scholar." https://www.semanticscholar.org (accessed 2024).
8. I. Keraghel, S. Morbieu, and M. Nadif, "A survey on recent advances in named entity recognition," arXiv preprint, arXiv:2401.10825, 2024.
9. J. Yang, T. Zhang, C.-Y. Tsai, Y. Lu, and L. Yao, "Evolution and emerging trends of named entity recognition: Bibliometric analysis from 2000 to 2023," Heliyon, 2024. [DOI:10.1016/j.heliyon.2024.e30053] [PMID] []
10. Z. Hu, W. Hou, and X. Liu, "Deep learning for named entity recognition: A survey," Neural Computing and Applications, vol. 36, no. 16, pp. 8995-9022, 2024. [DOI:10.1007/s00521-024-09646-6]
11. Y. Park, G. Son, and M. Rho, "Biomedical flat and nested named entity recognition: Methods, challenges, and advances," Applied Sciences, vol. 14, no. 20, 2024. [DOI:10.3390/app14209302]
12. J. Liu, M. Sun, W. Zhang, G. Xie, Y. Jing, X. Li, and Z. Shi, "Dae-ner: Dual-channel attention enhancement for Chinese named entity recognition," Computer Speech & Language, vol. 85, p. 101581, 2024. [DOI:10.1016/j.csl.2023.101581]
13. P. Deshmukh, N. Kulkarni, S. Kulkarni, K. Manghani, P. A. Khadkikar, and R. Joshi, "Named entity recognition for Indic languages: A comprehensive survey," in 2024 1st International Conference on Trends in Engineering Systems and Technologies (ICTEST), pp. 1-6, IEEE, 2024. [DOI:10.1109/ICTEST60614.2024.10576183]
14. R. Grishman and B. M. Sundheim, "Message understanding conference-6: A brief history," in COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics, 1996. [DOI:10.3115/992628.992709]
15. E. F. Sang and F. De Meulder, "Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition," arXiv preprint cs/0306050, 2003.
16. G. R. Doddington, A. Mitchell, M. A. Przybocki, L. A. Ramshaw, S. M. Strassel, and R. M. Weischedel, "The automatic content extraction (ace) program-tasks, data, and evaluation," in Lrec, 2004, vol. 2, no. 1: Lisbon, pp. 837-840.
17. G. Demartini, T. Iofciu, and A. P. De Vries, "Overview of the INEX 2009 entity ranking track," in International Workshop of the Initiative for the Evaluation of XML Retrieval, 2009: Springer, pp. 254-264. [DOI:10.1007/978-3-642-14556-8_26]
18. K. Balog, P. Serdyukov, and A. P. de Vries, "Overview of the TREC 2011 Entity Track," in TREC, 2011, vol. 2011, p. 11. [DOI:10.6028/NIST.SP.500-296.entity-overview]
19. M. Marrero, J. Urbano, S. Sánchez-Cuadrado, J. Morato, and J. M. Gómez-Berbís, "Named entity recognition: fallacies, challenges and opportunities," Computer Standards & Interfaces, vol. 35, no. 5, pp. 482-489, 2013. [DOI:10.1016/j.csi.2012.09.004]
20. M. L. Patawar and M. Potey, "Approaches to named entity recognition: a survey," International Journal of Innovative Research in Computer and Communication Engineering, vol. 3, no. 12, pp. 12201-12208, 2015.
21. H. Pu and W. Chen, "Review of multimodal named entity recognition studies," Data Analysis and Knowledge Discovery, vol. 8, no. 4, pp. 50-63, 2024.
22. Momtazi S, Torabi F. Named Entity Recognition in Persian Text using Deep Learning. JSDP 2020; 16 (4) :93-112 [DOI:10.29252/jsdp.16.4.93]
23. O. Khade, S. Jagdale, G. Takalikar, M. Inamdar, R. Joshi, and A. S. Ghotkar, "Enhancing code-mixing in named entity recognition: A comprehensive survey of deep learning models," in 2024 Second International Conference on Emerging Trends in Information Technology and Engineering (ICETITE), pp. 1-6, IEEE, 2024. [DOI:10.1109/ic-ETITE58242.2024.10493709]
24. H. Pu and W. Chen, "Review of multimodal named entity recognition studies," Data Analysis and Knowledge Discovery, vol. 8, no. 4, pp. 50-63, 2024.
25. H. Wang, X. Xu, T. Wang, and B. Jing, "Research progress of multimodal named entity recognition," Journal of Zhengzhou University: Engineering Science, vol. 45, no. 2, 2024.
26. M. Zali and M. Firoozbakht, "Named Entities Recognition and Classification System for Persian Texts Based on Neural Network," Iranian Research Institute for Information Science and Technology, vol. 34, pp. 473-486, 2018.
27. Shahshahani M S, Mohseni M, Shakery A, Faili H. PAYMA: A Tagged Corpus of Persian Named Entities. JSDP 2019; 16 (1) :91-110 [DOI:10.29252/jsdp.16.1.91]
28. M. Mohseni and A. Tebbifakhr, "MorphoBERT: a Persian NER system with BERT and morphological analysis," in Proceedings of The First International Workshop on NLP Solutions for Under Resourced Languages (NSURL 2019) co-located with ICNLSP 2019-Short Papers, 2019, pp. 23-30.
29. E. Taher, S. A. Hoseini, and M. Shamsfard, "Beheshti-NER: Persian named entity recognition Using BERT," arXiv preprint arXiv:2003.08875, 2020.
30. M. H. Bokaei and M. Mahmoudi, "Improved deep persian named entity recognition," in 2018 9th International Symposium on Telecommunications (IST), 2018: IEEE, pp. 381-386. [DOI:10.1109/ISTEL.2018.8661067]
31. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
32. K. Cho et al., "Learning phrase representations using RNN encoder-decoder for statistical machine translation," arXiv preprint arXiv:1406.1078, 2014. [DOI:10.3115/v1/D14-1179]
33. I. Sutskever, O. Vinyals, and Q. V. Le, "Sequence to sequence learning with neural networks," arXiv preprint arXiv:1409.3215, 2014.
34. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in International conference on machine learning, 2015: PMLR, pp. 448-456.
35. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929-1958, 2014.
36. M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," IEEE transactions on Signal Processing, vol. 45, no. 11, pp. 2673-2681, 1997. [DOI:10.1109/78.650093]
37. [G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708. [DOI:10.1109/CVPR.2017.243] []
38. M. W. Browne, "Cross-validation methods," Journal of mathematical psychology, vol. 44, no. 1, pp. 108-132, 2000. [DOI:10.1006/jmps.1999.1279] [PMID]
39. I. Yamada, A. Asai, H. Shindo, H. Takeda, and Y. Matsumoto, "LUKE: deep contextualized entity representations with entity-aware self-attention," arXiv preprint arXiv:2010.01057, 2020. [DOI:10.18653/v1/2020.emnlp-main.523]
40. A. Baevski, S. Edunov, Y. Liu, L. Zettlemoyer, and M. Auli, "Cloze-driven pretraining of self-attention networks," arXiv preprint arXiv:1903.07785, 2019. [DOI:10.18653/v1/D19-1539]
41. J. Yu, B. Bohnet, and M. Poesio, "Named entity recognition as dependency parsing," arXiv preprint arXiv:2005.07150, 2020. [DOI:10.18653/v1/2020.acl-main.577]
42. Y. Jiang, C. Hu, T. Xiao, C. Zhang, and J. Zhu, "Improved differentiable architecture search for language modeling and named entity recognition," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2pp. 3576-3581, 2019. [DOI:10.18653/v1/D19-1367] [PMID]
43. Z. Wang, J. Shang, L. Liu, L. Lu, J. Liu, and J. Han, "Crossweigh: Training named entity tagger from imperfect annotations," arXiv preprint arXiv:1909.01441, 2019. [DOI:10.18653/v1/D19-1519] [PMID] []
44. J. Straková, M. Straka, and J. Hajič, "Neural architectures for nested NER through linearization," arXiv preprint arXiv:1908.06926, 2019. [DOI:10.18653/v1/P19-1527]
45. Y. Luo, F. Xiao, and H. Zhao, "Hierarchical contextualized representation for named entity recognition," in Proceedings of the AAAI Conference on Artificial Intelligence, 2020, vol. 34, no. 05, pp. 8441-8448. [DOI:10.1609/aaai.v34i05.6363]
46. X. Li, J. Feng, Y. Meng, Q. Han, F. Wu, and J. Li, "A unified mrc framework for named entity recognition," arXiv preprint arXiv:1910.11476, 2019. [DOI:10.18653/v1/2020.acl-main.519]
47. M. Farahani, M. Gharachorloo, M. Farahani, and M. Manthouri, "ParsBERT: Transformer-based Model for Persian Language Understanding," arXiv preprint arXiv:2005.12515, 2020. [DOI:10.1007/s11063-021-10528-4]
48. L. Hafezi and M. Rezaeian, "Neural architecture for persian named entity recognition," in 2018 4th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS), 2018: IEEE, pp. 61-64. [DOI:10.1109/ICSPIS.2018.8700549]
49. H. Poostchi, E. Z. Borzeshi, and M. Piccardi, "Bilstm-crf for persian named-entity recognition armanpersonercorpus: The first entity-annotated persian dataset," in Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), 2018.
50. H. Poostchi, E. Z. Borzeshi, M. Abdous, and M. Piccardi, "PersoNER: Persian named-entity recognition," in COLING 2016-26th International Conference on Computational Linguistics, Proceedings of COLING 2016: Technical Papers, 2016.
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
