1403 21 1 59 142

یک الگوریتم خوشه‌بندی جدید چندگامه مبتنی بر تاخیر تکرار برای بهبود کیفیت سرویس اینترنت اشیاء A new multi-hop clustering algorithm based on iterative delay to enhance QoS for Internet of Things 1 1 اینترنت اشیاء به شبکه‌ای از اشیاء فیزیکی اطلاق می‌شود که در آن اشیاء دارای شناسه‌ی منحصر، قادر هستند با یکدیگر و یا با کاربر نهایی از طریق اینترنت ارتباط برقرار کنند. به دلیل محدود بودن برد رادیویی اشیاء و همچنین کاهش انرژی مصرفی، انتقال اطلاعات از طریق اشیاء واسط انجام می‌شود که لزوم مسیریابی را مشخص می‌کند. یک الگوریتم مسیریابی بطور مستقیم بر قابلیت اطمینان، تاخیر، انرژی مصرفی، گذردهی شبکه، استفاده‌ی موثر از پهنای باند و طول عمر شبکه تاثیر می‌گذارد. در این مقاله یک رویکرد مسیریابی جدید مبتنی بر خوشه‌بندی توزیع‌شده و تاخیر تکراری برای بهبود کیفیت سرویس اینترنت اشیاء پیشنهاد می‌شود که اشیاء شبکه به تعدادی خوشه مجزا از هم تقسیم می‌شود. خوشه‌بندی بر اساس حالت‌های مختلف اشیاء همسایه انجام می‌شود و برای ارسال داده‌های سرخوشه‌ها به ایستگاه پایه نیز از روش تاخیر تکرار استفاده می‌شود. در رویکرد پیشنهادی برای انتخاب سرخوشه‌ی واسط مناسب، تاخیر انتقال از شیء سرخوشه تا شیء واسط، تاخیر انتقال از شیء واسط تا چاهک و انرژی باقیمانده‌ی شیء واسط در نظر گرفته می‌شود. نتایج شبیه‌سازی‌های انجام شده در نرم‌افزار Cooja نشان می‌دهد که رویکرد پیشنهادی بطور متوسط در مقایسه با الگوریتم‌های LEACH، LEACH-E، NCACM، Distributed Clustering از نظر میزان مصرف انرژی 33% ، مرگ اولین گره 14%، تعداد اشیاء مرده 12% و نرخ دریافت صحیح بسته‌ها 9% عملکرد بهتری دارد. 2 In general, Internet of Things (IoT) as a new technology refers to a network of physical things in which objects have a unique identity and are able to communicate with each other or with the end user via the Internet.  The Internet of Things refers to a collection of sensor-embedded devices, processing ability, software, and other technologies that connect and exchange data with other devices and systems over the Internet or other communications networks. Due to the limited radio range of objects and also the reduction of energy consumption, information transmission is carried out through the intermediate objects, which highlights the necessity for routing. Routing algorithms can be classified into static and dynamic techniques as well as source initiated and destination initiated approaches. In general, routing algorithms can be classified into data centric, hierarchical, geographical, and quality of service-based mechanisms. A routing algorithm directly affects reliability, transmission latency, power consumption, network throughput, bandwidth utilization, and network lifetime. This paper proposes a new routing method based on distributed clustering and iterative latency to improve the Quality of Service (QoS) of IoT, which divides network things into a number of separate clusters. The proposed method consists of four stages, i.e. network clustering, steady state, multi-hop transmission based on delay estimation, and investigation of adjacent headers. Clustering is performed based on the different states of neighbors, and the iterative delay mechanism is used between the cluster heads. The simulation results conducted through Cooja tool indicate that the proposed method outperforms the LEACH, LEACH-E, NCACM, and distributed clustering techniques in terms of energy consumption and packet delivery ratio by 33% and 9%. Furthermore, simulation results illustrate that the proposed method outperforms in terms of the first node death time and the number of dead objects in scattered and dense networks by 14% and 12%, respectively. 3 14 2021/10/26 1400/8/4 2023/02/22 1401/12/3 فرناز راسخی Farnaz Rasekhi دانشگاه آزاد اسلامی تبریز ایران farnaz.rasekhi@gmail.com شهرام بابائی Shahram Babaie دانشگاه آزاد اسلامی تبریز ایران sh.babaie@iaut.ac.ir Internet of Things (IoT) Distributed clustering Iterative delay Routing Quality of Service (QoS) اینترنت اشیاء خوشه‌بندی توزیع‌شده تاخیر تکرار مسیریابی کیفیت سرویس خدمات آذری امین، نیک‌نژاد محسن و عباسی محمود "بهبود قابلیت اطمینان و عمر باتری در شبکه‌های ارتباطی اینترنت اشیاء: چالش‌ها و راهبردهای مبتنی بر یادگیری ماشینی"، مجله هوش محاسباتی در مهندسی برق، دوره 12، شماره 2، صفحات 52-41، 1400.##A. Azari, M. NikNejad, and M, Abbasi, "Reliability and Battery Lifetime Improvement for IoT Networks: Challenges and AI-powered solutions" Computational Intelligence in Electrical Engineering, Vol. 12, No. 2, pp. 41-52, 2021.##نیره سعیدی و شهرام بابائی "ارائه یک الگوریتم مسیریابی جدید مبتنی بر الگوریتم‌های ژنتیک و تبرید شبیه‌سازی‌شده برای شبکه‌های موردی بین‌خودرویی"، پردازش علائم و داده‌ها، دوره 19، شماره 2، صفحات 72-61، ۱۴۰1.##N. Saeidi and S. Babaie, " A New Hybrid Routing Algorithm based on Genetic Algorithm and Simulated Annealing for Vehicular Ad hoc Networks" Signal and Data Processing, Vol. 19, No. 2, pp. 61-72, 2022.##ابراهیمی مریم، تدین محمد حسام، صیادحقیقی محمد. "الگوریتم‌های اعتماد در اینترنت اشیا: بررسی، تحلیل و ارائه معیارهای ارزیابی"، پردازش علائم و داده‌ها، دوره 18، شماره 2، صفحات 28-3، ۱۴۰۰.##M. Ebrahimi, M. H. Tadayon, and M. Sayad Haghighi, "Trust Management in Internet of Things: Review, Analysis and Establishment of Evaluation Criteria" Signal and Data Processing, Vol. 18, No. 2, pp. 3-28, 2021.##N. Razzaghi and S. Babaie, " A new selfish thing detection method based on Voronoi diagram for Intrenet of Things," The Journal of Supercomputing, vol. 78, pp. 8389-8408, 2022..##R. H. Weber, "Internet of Things - New security and privacy challenges," Comput. Law Secur. Rev., vol. 26, no. 1, pp. 23-30, 2010.##A. Al-fuqaha, S. Member, M. Guizani, M. Mohammadi, and S. Member, "Internet of Things : A Survey on Enabling," vol. 17, no. 4, pp. 2347-2376, 2015.##S. Li, L. Da Xu, and S. Zhao, "The internet of things: a survey," Inf. Syst. Front., vol. 17, no. 2, pp. 243-259, 2015.##D. Miorandi, S. Sicari, F. De Pellegrini, and I. Chlamtac, "Internet of things: Vision, applications and research challenges," Ad Hoc Networks, vol. 10, no. 7, pp. 1497-1516, 2012.##A. Dhumane, "Routing Challenges in Internet of Things", CSI Communications, no. January, pp. 19-20, 2015.##A. Dhumane, R. Prasad, and J. Prasad, "Routing Issues in Internet of Things : A Survey," Lect. Notes Eng. Comput. Sci. Proc. Int. MultiConference Eng. Comput. Sci., vol. 1, pp. 404-412, 2016.##M. Afrashteh and S. Babaie, "A route segmented broadcast protocol based on RFID for emergency message dissemi-nation in vehicular ad-hoc networks ," IEEE Transactions on Vehicular Technology, vol. 69, pp. 16017-16026, 2020.##M. S. Pan and S. W. Yang, "A lightweight and distributed geographic multicast routing protocol for IoT applications," Comput. Networks, vol. 112, pp. 95-107, 2017.##W. R. Heinzelman, A. Chandrakasan, and H. Balakrishnan, "Energy-Efficient Communication Protocol for Wireless Microsensor Networks P ;,IEEE International Conference on system sciences, 2000, Maui, USA.##P. Sivakumar and M. Radhika, "Science-Direct ScienceDirect Performance Analysis of LEACH-GA over LEACH and LEACH-C in WSN," Procedia Comput. Sci., vol. 125, pp. 248-256, 2018.##H. B. Patel, "E-LEACH : Improving the LEACH Protocol for Privacy Preservation in Secure Data Aggregation in Wireless Sensor Networks.", IEEE, 2014 9th International Conference on Industrial and Information Systems (ICIIS),10.1109/ ICIINFS.2014.703 6607.##V. Loscrì, G. Morabito, and S. Marano, "A Two-Levels Hierarchy for Low-Energy Adaptive Clustering Hierarchy (TL-LEACH)," October, 2005.##M. B. Yassein, A. Al-zou, Y. Khamayseh, and W. Mardini, "Improvement on LEACH protocol of wireless sensor network ( VLEACH ) Improvement on LEACH Protocol of Wireless Sensor Network," 10.4156/vol3January, 2009.##S. B. Shah, C. Zhe, Y. Fuliang, I. U. Khan, and N. Ahmad, "Energy and interoperable aware routing for throughput optimization in clustered IoT-wireless sensor networks," Futur. Gener. Comput. Syst. 10.1016/j.future.2017.09.043, 2017.##J. Y. Chang, "A Distributed Cluster Computing Energy-Efficient Routing Scheme for Internet of Things Systems," Wirel. Pers. Commun., vol. 82, no. 2, pp. 757-776, 2015.##T. Qiu, Y. Lv, F. Xia, N. Chen, J. Wan, and A. Tolba, "ERGID: An efficient routing protocol for emergency response Internet of Things," J. Netw. Comput. Appl., vol. 72, no. August, pp. 104-112, 2016.##آذری امین، نیک‌نژاد محسن و عباسی محمود "بهبود قابلیت اطمینان و عمر باتری در شبکه‌های ارتباطی اینترنت اشیاء: چالش‌ها و راهبردهای مبتنی بر یادگیری ماشینی"، مجله هوش محاسباتی در مهندسی برق، دوره 12، شماره 2، صفحات 52-41، 1400.##A. Azari, M. NikNejad, and M, Abbasi, "Reliability and Battery Lifetime Improvement for IoT Networks: Challenges and AI-powered solutions" Computational Intelligence in Electrical Engineering, Vol. 12, No. 2, pp. 41-52, 2021.##نیره سعیدی و شهرام بابائی "ارائه یک الگوریتم مسیریابی جدید مبتنی بر الگوریتم‌های ژنتیک و تبرید شبیه‌سازی‌شده برای شبکه‌های موردی بین‌خودرویی"، پردازش علائم و داده‌ها، دوره 19، شماره 2، صفحات 72-61، ۱۴۰1.##N. Saeidi and S. Babaie, " A New Hybrid Routing Algorithm based on Genetic Algorithm and Simulated Annealing for Vehicular Ad hoc Networks" Signal and Data Processing, Vol. 19, No. 2, pp. 61-72, 2022.##ابراهیمی مریم، تدین محمد حسام، صیادحقیقی محمد. "الگوریتم‌های اعتماد در اینترنت اشیا: بررسی، تحلیل و ارائه معیارهای ارزیابی"، پردازش علائم و داده‌ها، دوره 18، شماره 2، صفحات 28-3، ۱۴۰۰.##M. Ebrahimi, M. H. Tadayon, and M. Sayad Haghighi, "Trust Management in Internet of Things: Review, Analysis and Establishment of Evaluation Criteria" Signal and Data Processing, Vol. 18, No. 2, pp. 3-28, 2021.##N. Razzaghi and S. Babaie, " A new selfish thing detection method based on Voronoi diagram for Intrenet of Things," The Journal of Supercomputing, vol. 78, pp. 8389-8408, 2022..##R. H. Weber, "Internet of Things - New security and privacy challenges," Comput. Law Secur. Rev., vol. 26, no. 1, pp. 23-30, 2010.##A. Al-fuqaha, S. Member, M. Guizani, M. Mohammadi, and S. Member, "Internet of Things : A Survey on Enabling," vol. 17, no. 4, pp. 2347-2376, 2015.##S. Li, L. Da Xu, and S. Zhao, "The internet of things: a survey," Inf. Syst. Front., vol. 17, no. 2, pp. 243-259, 2015.##D. Miorandi, S. Sicari, F. De Pellegrini, and I. Chlamtac, "Internet of things: Vision, applications and research challenges," Ad Hoc Networks, vol. 10, no. 7, pp. 1497-1516, 2012.##A. Dhumane, "Routing Challenges in Internet of Things", CSI Communications, no. January, pp. 19-20, 2015.##A. Dhumane, R. Prasad, and J. Prasad, "Routing Issues in Internet of Things : A Survey," Lect. Notes Eng. Comput. Sci. Proc. Int. MultiConference Eng. Comput. Sci., vol. 1, pp. 404-412, 2016.##M. Afrashteh and S. Babaie, "A route segmented broadcast protocol based on RFID for emergency message dissemi-nation in vehicular ad-hoc networks ," IEEE Transactions on Vehicular Technology, vol. 69, pp. 16017-16026, 2020.##M. S. Pan and S. W. Yang, "A lightweight and distributed geographic multicast routing protocol for IoT applications," Comput. Networks, vol. 112, pp. 95-107, 2017.##W. R. Heinzelman, A. Chandrakasan, and H. Balakrishnan, "Energy-Efficient Communication Protocol for Wireless Microsensor Networks P ;,IEEE International Conference on system sciences, 2000, Maui, USA.##P. Sivakumar and M. Radhika, "Science-Direct ScienceDirect Performance Analysis of LEACH-GA over LEACH and LEACH-C in WSN," Procedia Comput. Sci., vol. 125, pp. 248-256, 2018.##H. B. Patel, "E-LEACH : Improving the LEACH Protocol for Privacy Preservation in Secure Data Aggregation in Wireless Sensor Networks.", IEEE, 2014 9th International Conference on Industrial and Information Systems (ICIIS),10.1109/ ICIINFS.2014.703 6607.##V. Loscrì, G. Morabito, and S. Marano, "A Two-Levels Hierarchy for Low-Energy Adaptive Clustering Hierarchy (TL-LEACH)," October, 2005.##M. B. Yassein, A. Al-zou, Y. Khamayseh, and W. Mardini, "Improvement on LEACH protocol of wireless sensor network ( VLEACH ) Improvement on LEACH Protocol of Wireless Sensor Network," 10.4156/vol3January, 2009.##S. B. Shah, C. Zhe, Y. Fuliang, I. U. Khan, and N. Ahmad, "Energy and interoperable aware routing for throughput optimization in clustered IoT-wireless sensor networks," Futur. Gener. Comput. Syst. 10.1016/j.future.2017.09.043, 2017.##J. Y. Chang, "A Distributed Cluster Computing Energy-Efficient Routing Scheme for Internet of Things Systems," Wirel. Pers. Commun., vol. 82, no. 2, pp. 757-776, 2015.##T. Qiu, Y. Lv, F. Xia, N. Chen, J. Wan, and A. Tolba, "ERGID: An efficient routing protocol for emergency response Internet of Things," J. Netw. Comput. Appl., vol. 72, no. August, pp. 104-112, 2016.## ##

استخراج و ترکیب ویژگیهای کارآمد از توالی پروتئین به منظور دسته‌بندی پروتئین بر اساس جنگل چرخش Extracting and combination efficient feature from protein sequence for classify protein based on rotation forest 1 1 چکیده پیش‌‎بینی عملکرد پروتئین یکی از چالش‌های اصلی در بیوانفورماتیک می‌باشد که کاربردهای زیادی دارد. در سال­های اخیر در تحقیقات بسیاری از روش‌های یادگیری ماشین در این زمینه استفاده شده‌اند. در این روش‌ها ابتدا باید از توالی پروتئین ویژگی‌های مختلف استخراج شود و بر اساس ویژگی‌های استخراج شده عمل دسته‌بندی انجام شود. غالبا روش‌های استخراج ویژگی بر اساس خصوصیات فیزیکی و شیمایی توالی پروئتین می‌باشد. بنابراین استخراج ویژگی‌هایی مناسب از توالی پروتئین باعث افزایش و بهبود عملکرد روش‌های یادگیری ماشین می‌شود. در این مقاله، یک مجموعه جدید از ویژگی­ها بر اساس روش­های PSSM،  PsePSSM، K-gram ، AAC و روش نوین TFCRF که تا کنون در این کاربرد استفاده نشده برای استخراج ویژگی‌های مناسب پیشنهاد شده است. ویژگی‌های استخراج شده با استفاده از این روش قدرت تمایز کنندگی خوبی بین داده‌ها در دسته‌ها، به مدل‌های یادگیری ماشین می‌دهد. در روش TFCRF وزن‌دهی ویژگی‌ها علاوه بر توجه به چگونگی توزیع آنها در توالی‌های مختلف به چگونگی توزیع آنها در طبقات مختلف نیز توجه می‌شود. در مرحله بعد با استفاده از ویژگی‌های استخراج شده با استفاده از روش جنگل چرخ عمل دسته‌بندی انجام می‌شود. روش پیشنهادی با دسته‌بند‌های مختلف و روش‌های متفاوت مقایسه شده است. نتایج حاصل نشان دهنده کارایی مناسب روش پیشنهادی نسبت به سایر روش‌های نوین در این کاربرد  می‌باشد. 2 Abstract Protein function prediction is one of the main challenges in bioinformatics, which has many applications. In recent years, many researches in this field have been used machine learning methods. In these methods, First, different features should be extracted from the protein sequence and classification should be done based on the extracted features. The feature extraction methods are based on the physical and chemical properties of the protein sequence. Therefore, extracting suitable features from protein sequence increases and improves the performance of machine learning methods. In this paper, usage of a new set of features based on Position-Specific Scoring Matrix (PSSM), Pseudo-Position Specific Scoring Matrix (PsePSSM), K-gram, Amino Acid Composition (AAC) and the new Term Frequency and Category Relevancy Factor (TFCRF) method, which has not been used in this application so far, is proposed to extract suitable features. In the PSSM method for protein BLAST searches, a scoring matrix is used, in which amino acid substitution scores are given separately for each position in a multi-sequence protein alignment. The PsePSSM feature is described by considering different ranking correlation factors along a protein sequenc to preserve information about the amino acid sequence. The normalized occurrence frequency of a certain number of amino acids in the protein is calculated by the ACC method. An K-gram is a set of K successive items in a protein that  include amino acid. In the TFCRF weighting method, in addition to paying attention to how these are distributed in different sequences, how these are distributed in different classes is also paid attention to.The features extracted using this method give machine learning models a good discriminating power between data in classes. In the next step, classification is done using the extracted features using the rotation forest method. This classifier is a successful ensemble method for a wide range of data mining applications. In this method, the feature space is changed through Principal Component Analysis (PCA), which increases the power of this classifier. The proposed method has been compared to different classifiers. The results show that the efficiency of the proposed method is much better than other state-of–the-art methods in this application. 15 26 2021/10/262023/07/1 1402/4/10 2023/02/222024/02/25 1402/12/6 جمشید پیرگزی jamshid pirgazi دانشکده مهندسی برق و کامپیوتر، دانشگاه علم و فناوری مازندران، بهشهر، ایران ایران j.pirgazi@mazust.ac.ir علی قنبری سرخی Ali Ghanbari sorkhi دانشکده مهندسی برق و کامپیوتر، دانشگاه علم و فناوری مازندران، بهشهر، ایران ایران ali.ghanbari@mazust.ac.ir مجید ایرانپور مبارکه majid Iranpour Mobarakeh استادیار، دانشکده مهندسی کامپیوتر و فناوری اطلاعات، دانشگاه پیام نور، تهران، ایران ایران iranpour@pnu.ac.ir Protein sequence feature extraction TFCRF rotation forest relevancy factor توالی پروتئین استخراج ویژگی TFCRF جنگل چرخش فاکتور ارتباط Heider, D., et al., A computational approach for the identification of small GTPases based on preprocessed amino acid sequences. Technology in cancer research & treatment, 2009. 8(5): p. 333-341.##Löchel, H.F., et al., SCOTCH: subtype A coreceptor tropism classification in HIV-1. Bioinformatics, 2018. 34(15): p. 2575-2580.##Heider, D. and D. Hoffmann, Interpol: An R package for preprocessing of protein sequences. BioData mining, 2011. 4(1): p. 1-6.##Armano, G. and A. Giuliani, A two-tiered 2d visual tool for assessing classifier performance. Information Sciences, 2018. 463: p. 323-343.##Yu, X., I. Weber, and R. Harrison. Sparse representation for HIV-1 protease drug resistance prediction. in Proceedings of the 2013 SIAM international conference on data mining. 2013. SIAM.##Spänig, S. and D. Heider, Encodings and models for antimicrobial peptide classification for multi-resistant pathogens. BioData Mining, 2019. 12(1): p. 1-29.##Zhang, J., et al., Variable selection from a feature representing protein sequences: a case of classification on bacterial type IV secreted effectors. BMC bioinformatics, 2020. 21: p. 1-15.##De Santis, E., et al. Dissimilarity space representations and automatic feature selection for protein function prediction. in 2018 International joint conference on neural networks (IJCNN). 2018. IEEE.##Bonetta, R. and G. Valentino, Machine learning techniques for protein function prediction. Proteins: Structure, Function, and Bioinformatics, 2020. 88(3): p. 397-413.##Rizzo, R., et al. Classification experiments of DNA sequences by using a deep neural network and chaos game representation. in Proceedings of the 17th International Conference on Computer Systems and Technologies 2016. 2016.##Qian, W., et al., Feature selection for label distribution learning via feature similarity and label correlation. Information Sciences, 2022. 582: p. 38-59.##Abu Khurma, R., et al., A review of the modification strategies of the nature inspired algorithms for feature selection problem. Mathematics, 2022. 10(3): p. 464.##Törönen, P. and L. Holm, PANNZER-a practical tool for protein function prediction. Protein Science, 2022. 31(1): p. 118-128.##Lv, Z., C. Ao, and Q. Zou, Protein function prediction: from traditional classifier to deep learning. Proteomics, 2019. 19(14): p. 1900119.##Mahood, E.H., L.H. Kruse, and G.D. Moghe, Machine learning: a powerful tool for gene function prediction in plants. Applications in Plant Sciences, 2020. 8(7): p. e11376.##Martino, A., A. Rizzi, and F.M.F. Mascioli. Supervised approaches for protein function prediction by topological data analysis. in 2018 International joint conference on neural networks (IJCNN). 2018. IEEE.##You, R., et al., NetGO: improving large-scale protein function prediction with massive network information. Nucleic acids research, 2019. 47(W1): p. W379-W387.##Lai, B. and J. Xu, Accurate protein function prediction via graph attention networks with predicted structure information. Briefings in Bioinformatics, 2022. 23(1): p. bbab502.##Liu, X., Deep recurrent neural network for protein function prediction from sequence. arXiv preprint arXiv:1701.08318, 2017.##Kulmanov, M. and R. Hoehndorf, DeepGOPlus: improved protein function prediction from sequence. Bioinformatics, 2020. 36(2): p. 422-429.##Sureyya Rifaioglu, A., et al., DEEPred: automated protein function prediction with multi-task feed-forward deep neural networks. Scientific reports, 2019. 9(1): p. 7344.##Li, M., et al., A deep learning framework for predicting protein functions with co-occurrence of GO terms. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022. 20(2): p. 833-842.##Shen, H.-B. and K.-C. Chou, Nuc-PLoc: a new web-server for predicting protein subnuclear localization by fusing PseAA composition and PsePSSM. Protein Engineering, Design & Selection, 2007. 20(11): p. 561-567.##Akbar, S., et al., iHBP-DeepPSSM: Identifying hormone binding proteins using PsePSSM based evolutionary features and deep learning approach. Chemometrics and Intelligent Laboratory Systems, 2020. 204: p. 104103.##Jones, D.T., Protein secondary structure prediction based on position-specific scoring matrices. Journal of molecular biology, 1999. 292(2): p. 195-202.##Altschul, S.F., et al., Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic acids research, 1997. 25(17): p. 3389-3402.##Chou, K.C., Prediction of protein cellular attributes using pseudo‐amino acid composition. Proteins: Structure, Function, and Bioinformatics, 2001. 43(3): p. 246-255.##Bhasin, M. and G.P. Raghava, Classification of nuclear receptors based on amino acid composition and dipeptide composition. Journal of Biological Chemistry, 2004. 279(22): p. 23262-23266.##Sorkhi, A.G., J. Pirgazi, and V. Ghasemi, A hybrid feature extraction scheme for efficient malonylation site prediction. Scientific Reports, 2022. 12(1): p. 5756.##Pirgazi, J., A.R. Khanteymoori, and M. Jalilkhani, GENIRF: An algorithm for gene regulatory network inference using rotation forest. Current Bioinformatics, 2018. 13(4): p. 407-419.##Rhee, S.-Y., et al., Genotypic predictors of human immunodeficiency virus type 1 drug resistance. Proceedings of the National Academy of Sciences, 2006. 103(46): p. 17355-17360.##Heider D, Verheyen J, Hoffmann D. Machine learning on normalized protein sequences. BMC research notes. 2011 Dec;4:1-0.##Hou, T. Zhang, W. Wang, J. Wang, W., Predicting drug resistance of the HIV‐1 protease using molecular interaction energy components. Proteins: Structure, Function, and Bioinformatics. 2009 Mar; 74(4): p. 837-46.##Löchel HF, Eger D, Sperlea T, Heider D. Deep learning on chaos game representation for proteins. Bioinformatics. 2020 Jan 1;36(1): p. 272-9.##Heider, D., et al., A computational approach for the identification of small GTPases based on preprocessed amino acid sequences. Technology in cancer research & treatment, 2009. 8(5): p. 333-341.##Löchel, H.F., et al., SCOTCH: subtype A coreceptor tropism classification in HIV-1. Bioinformatics, 2018. 34(15): p. 2575-2580.##Heider, D. and D. Hoffmann, Interpol: An R package for preprocessing of protein sequences. BioData mining, 2011. 4(1): p. 1-6.##Armano, G. and A. Giuliani, A two-tiered 2d visual tool for assessing classifier performance. Information Sciences, 2018. 463: p. 323-343.##Yu, X., I. Weber, and R. Harrison. Sparse representation for HIV-1 protease drug resistance prediction. in Proceedings of the 2013 SIAM international conference on data mining. 2013. SIAM.##Spänig, S. and D. Heider, Encodings and models for antimicrobial peptide classification for multi-resistant pathogens. BioData Mining, 2019. 12(1): p. 1-29.##Zhang, J., et al., Variable selection from a feature representing protein sequences: a case of classification on bacterial type IV secreted effectors. BMC bioinformatics, 2020. 21: p. 1-15.##De Santis, E., et al. Dissimilarity space representations and automatic feature selection for protein function prediction. in 2018 International joint conference on neural networks (IJCNN). 2018. IEEE.##Bonetta, R. and G. Valentino, Machine learning techniques for protein function prediction. Proteins: Structure, Function, and Bioinformatics, 2020. 88(3): p. 397-413.##Rizzo, R., et al. Classification experiments of DNA sequences by using a deep neural network and chaos game representation. in Proceedings of the 17th International Conference on Computer Systems and Technologies 2016. 2016.##Qian, W., et al., Feature selection for label distribution learning via feature similarity and label correlation. Information Sciences, 2022. 582: p. 38-59.##Abu Khurma, R., et al., A review of the modification strategies of the nature inspired algorithms for feature selection problem. Mathematics, 2022. 10(3): p. 464.##Törönen, P. and L. Holm, PANNZER-a practical tool for protein function prediction. Protein Science, 2022. 31(1): p. 118-128.##Lv, Z., C. Ao, and Q. Zou, Protein function prediction: from traditional classifier to deep learning. Proteomics, 2019. 19(14): p. 1900119.##Mahood, E.H., L.H. Kruse, and G.D. Moghe, Machine learning: a powerful tool for gene function prediction in plants. Applications in Plant Sciences, 2020. 8(7): p. e11376.##Martino, A., A. Rizzi, and F.M.F. Mascioli. Supervised approaches for protein function prediction by topological data analysis. in 2018 International joint conference on neural networks (IJCNN). 2018. IEEE.##You, R., et al., NetGO: improving large-scale protein function prediction with massive network information. Nucleic acids research, 2019. 47(W1): p. W379-W387.##Lai, B. and J. Xu, Accurate protein function prediction via graph attention networks with predicted structure information. Briefings in Bioinformatics, 2022. 23(1): p. bbab502.##Liu, X., Deep recurrent neural network for protein function prediction from sequence. arXiv preprint arXiv:1701.08318, 2017.##Kulmanov, M. and R. Hoehndorf, DeepGOPlus: improved protein function prediction from sequence. Bioinformatics, 2020. 36(2): p. 422-429.##Sureyya Rifaioglu, A., et al., DEEPred: automated protein function prediction with multi-task feed-forward deep neural networks. Scientific reports, 2019. 9(1): p. 7344.##Li, M., et al., A deep learning framework for predicting protein functions with co-occurrence of GO terms. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022. 20(2): p. 833-842.##Shen, H.-B. and K.-C. Chou, Nuc-PLoc: a new web-server for predicting protein subnuclear localization by fusing PseAA composition and PsePSSM. Protein Engineering, Design & Selection, 2007. 20(11): p. 561-567.##Akbar, S., et al., iHBP-DeepPSSM: Identifying hormone binding proteins using PsePSSM based evolutionary features and deep learning approach. Chemometrics and Intelligent Laboratory Systems, 2020. 204: p. 104103.##Jones, D.T., Protein secondary structure prediction based on position-specific scoring matrices. Journal of molecular biology, 1999. 292(2): p. 195-202.##Altschul, S.F., et al., Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic acids research, 1997. 25(17): p. 3389-3402.##Chou, K.C., Prediction of protein cellular attributes using pseudo‐amino acid composition. Proteins: Structure, Function, and Bioinformatics, 2001. 43(3): p. 246-255.##Bhasin, M. and G.P. Raghava, Classification of nuclear receptors based on amino acid composition and dipeptide composition. Journal of Biological Chemistry, 2004. 279(22): p. 23262-23266.##Sorkhi, A.G., J. Pirgazi, and V. Ghasemi, A hybrid feature extraction scheme for efficient malonylation site prediction. Scientific Reports, 2022. 12(1): p. 5756.##Pirgazi, J., A.R. Khanteymoori, and M. Jalilkhani, GENIRF: An algorithm for gene regulatory network inference using rotation forest. Current Bioinformatics, 2018. 13(4): p. 407-419.##Rhee, S.-Y., et al., Genotypic predictors of human immunodeficiency virus type 1 drug resistance. Proceedings of the National Academy of Sciences, 2006. 103(46): p. 17355-17360.##Heider D, Verheyen J, Hoffmann D. Machine learning on normalized protein sequences. BMC research notes. 2011 Dec;4:1-0.##Hou, T. Zhang, W. Wang, J. Wang, W., Predicting drug resistance of the HIV‐1 protease using molecular interaction energy components. Proteins: Structure, Function, and Bioinformatics. 2009 Mar; 74(4): p. 837-46.##Löchel HF, Eger D, Sperlea T, Heider D. Deep learning on chaos game representation for proteins. Bioinformatics. 2020 Jan 1;36(1): p. 272-9.## ##

تشخیص حالت غیر نرمال ماشین های دوار با داده کاوی در پارامترهای حفاظتی Exploring on rotating machines abnormal state with data mining in protective parameters 1 1 برای محافظت از ماشین­ های دوار و جلوگیری از کارکرد آنها در حالت­های غیر عادی به ­صورت سنتی از سیستم­های کنترل حفاظتی و داده­ های فرایندی بهره­ گیری می­ شود. در این مقاله روشی پیشنهاد شده است که بتوان از تاثیرات غیر مستقیم حالت­های کارکرد غیر عادی با استفاده از شیوه­ های داده ­کاوی حالت غیر طبیعی کارکرد ماشین ­های دوار را تشخیص داد. یکی از حالت ­های خطرناک کارکرد غیر عادی در کمپرسورها به­ عنوان یکی از ماشین­ های دوار با اهمیت در صنایع، وضعیت سرج می­ باشد. دراین مقاله، با استفاده از داده های واقعی ذخیره شده در طول سه سال متوالی یک کمپرسور سه مرحله ای واحد سرمایش یک پالایشگاه گاز ارتباط میان وضعیت سرج کمپرسور و میزان لرزش نقاط مختلف آن بررسی شده است. با شیوه ­های داده­ کاوی اثبات شده است که ارتباط مستقیمی بین حالت سرج و میزان لرزش وجود دارد. همچنین نقاط حساس­ تر به لرزش در زمان­های سرج شناسایی شده است و اثبات شده است که از طریق اندازه­ گیری این نقاط می­توان سرج را تشخیص داد. بنابراین  علاوه بر شیوه­ های موجود و سنتی قبلی که از داده­ های فرایندی استفاده می­ کنند، می­توان از میزان لرزش نقاط به­ عنوان یک سیستم حفاظتی افزونه برای تشخیص سرج  بهره گرفت و از این طریق حفاظت بیشتری از کمپرسور در برابر وضعیت سرج  بعمل آورد. در این مطالعه، ارزیابی شیوه ­های مختلف داده­ کاوی نیز صورت گرفته است که نتایج روش نزدیکترین همسایه با تعداد همسایه دو دارای بهترین کارایی بوده است و همچنین اثرات تعداد رکورد موجود در مجموعه داده روی کیفیت و دقت نتایج بررسی شده است. 2 In order to protect rotating machines and prevent their operation in unusual situations, protective control systems and process data are traditionally used. In this article, a method has been proposed to detect the indirect effects of abnormal operating modes using data mining methods. One of the dangerous conditions of abnormal operation in compressors, as one of the important rotating machines in industries, is the surge condition. In this article, the real data stored during three years of a three-stage refrigerant compressor in a gas refinery are used. the relationship between the surge state of the compressor and the amount of vibration in its different parts has been investigated. It has been proven with data mining methods that there is a direct relationship between the state of surge and the amount of vibration. Also, more sensitive points to vibration during the surges have been identified and it has been proven that by measuring these points, surges can be detected. Therefore, in addition to the existing and previous traditional methods that use process data, it is possible to use the amount of vibration of the points as an extension protection system for surge detection. in this way, more protection of the compressor against the state of surge can be achieved. In this study, various data mining methods have been evaluated, and the results of the nearest neighbor method with the number of neighbors of two have the best performance, and the effects of the number of records in the data set on the quality and accuracy of the results have been investigated. 27 38 2021/10/262023/07/12022/11/13 1401/8/22 2023/02/222024/02/252024/02/25 1402/12/6 الهام پروین نیا Elham Parvinnia دانشگاه آزاد اسلامی واحد شیراز ایران eparvinnia@gmail.com محمد صفری mohammad Safari دانشگاه آزاد اسلامی واحد شیراز ایران Safari.md@gmail.com سید علیرضا خیامی seyedalireza khayami دانشگاه آزاد اسلامی واحد شیراز ایران khayami.alr@gmail.com Rotating machine Datamining Surge detection Compressor Protection parameters ماشین های دوار داده کاوی تشخیص سرج کمپرسور پارامترهای حفاظتی پروین نیا، فرداد، "ارائه یک سامانه تصمیم یار جهت پیش بینی خاموشی اضطراری نیروگاه های برق آبی با استفاده از استخراج قوانین انجمنی"، نشریه علمی پژوهشی کیفیت و بهره‌وری صنعت برق ایران، شماره 13، 15- 27، 1397.##L. Chang Zheng, X. Bing, Compressor Surge Detection Based on Online Learning, Third International Conference on Intelligent Human-Machine Systems and Cybernetics, Vol. 2 , PP. 123-126, 2011##A. Chaouni, Benabdellah, A. Benghabrit, I. Bouhaddou, A survey of clustering algorithms for an industrial context, Second International Conference on Intelligent Computing in Data Sciences (ICDS 2018), 2019##A. Contreras-Valdes, JP. Amezquita-Sanchez, D. Granados-Lieberman, M. Valtierra-Rodriguez, Predictive data mining techniques for fault diagnosis of electric equipment: a review. Applied Sciences,2020.##R.H. Cunha, I.N. DaSilva, A. Goedtl, W.F. Godoy, A Comprehensive Evaluation of Intelligent Classifiers for Fault Identification in Three Phases Induction Motors, Electric Power System Research, Vol.127, PP.249-258, 2015##M. Diaz, D. Morinigo Sotelo, O. Duque Perez, Advances in Classifier Evaluation: Novel Insight for an Electric Data Driven Motor Diagnosis, IEEE Access, Vol.4, PP.7028-7038, 2016##M. Gribbestad, MU. Hassan, IA. Hameed, K. Sundli, Health Monitoring of Air Compressors Using Reconstruction-Based Deep Learning for Anomaly Detection with Increased Transparency, Entropy, 2021.##L. Guo, Y. Lei, S. Xing, T. Yan, N. Li, Deep Convolutional Transfer Learning Network: A New Method for Intelligent Fault Diagnosis of Machines with Unlabeled Data, IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2018##A. Kusiak, A. Verma, Analyzing Bearing Fault in Wind Turbines: A Datamining Approach, Renewable Energy, Vol.48, PP.110-116,2012##G.K. McMillan, Centrifugal And Axial Compressor Control, Momentum Press, 2010##M. Orkisz, P. Lipnicki, " International Symposium on Power Electronics, Electrical Drives, Automation and Motion (SPEEDAM)", Electronic ISBN: 978-1-4799-4749-2, 2014##A. Purarjomand Langrudi, A. Ghapanchi, M. Esmalifalak, A Datamining Approach for Fault Diagnosis: An Application of Anomaly Detection Algorithm, Measurement, Vol. 55, PP. 343-352, 2014##A. Sivakumar, S. Vaithiyanathan, Vibration based Data Analysis of Single Acting Compressor through Condition Monitoring and Multilayer Perceptron-A Machine Learning Classifier, InIOP Conference Series: Materials Science and Engineering, 2021.##P.N. Tan, M. Steinbach, V. Kumar, Introduction To Data mining, Pearson,2005##L. Wen, X. Li, L. Gao, Y. Zhang, A New Convolutional Neural Network Based Data-Driven Fault Diagnosis Method, IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2017##A. Contreras-Valdes, JP. Amezquita-Sanchez, D. Granados-Lieberman, M. Valtierra-Rodriguez, Predictive data mining techniques for fault diagnosis of electric equipment: a review, Applied Sciences, 2020.##L. Zhong, Y. Liu, J. Zhao, W. Wang, Deep predictive controller designed for centrifugal compressor system anti-surge, In2020 Chinese Automation Congress (CAC), 2020##پروین نیا، فرداد، "ارائه یک سامانه تصمیم یار جهت پیش بینی خاموشی اضطراری نیروگاه های برق آبی با استفاده از استخراج قوانین انجمنی"، نشریه علمی پژوهشی کیفیت و بهره‌وری صنعت برق ایران، شماره 13، 15- 27، 1397.##L. Chang Zheng, X. Bing, Compressor Surge Detection Based on Online Learning, Third International Conference on Intelligent Human-Machine Systems and Cybernetics, Vol. 2 , PP. 123-126, 2011##A. Chaouni, Benabdellah, A. Benghabrit, I. Bouhaddou, A survey of clustering algorithms for an industrial context, Second International Conference on Intelligent Computing in Data Sciences (ICDS 2018), 2019##A. Contreras-Valdes, JP. Amezquita-Sanchez, D. Granados-Lieberman, M. Valtierra-Rodriguez, Predictive data mining techniques for fault diagnosis of electric equipment: a review. Applied Sciences,2020.##R.H. Cunha, I.N. DaSilva, A. Goedtl, W.F. Godoy, A Comprehensive Evaluation of Intelligent Classifiers for Fault Identification in Three Phases Induction Motors, Electric Power System Research, Vol.127, PP.249-258, 2015##M. Diaz, D. Morinigo Sotelo, O. Duque Perez, Advances in Classifier Evaluation: Novel Insight for an Electric Data Driven Motor Diagnosis, IEEE Access, Vol.4, PP.7028-7038, 2016##M. Gribbestad, MU. Hassan, IA. Hameed, K. Sundli, Health Monitoring of Air Compressors Using Reconstruction-Based Deep Learning for Anomaly Detection with Increased Transparency, Entropy, 2021.##L. Guo, Y. Lei, S. Xing, T. Yan, N. Li, Deep Convolutional Transfer Learning Network: A New Method for Intelligent Fault Diagnosis of Machines with Unlabeled Data, IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2018##A. Kusiak, A. Verma, Analyzing Bearing Fault in Wind Turbines: A Datamining Approach, Renewable Energy, Vol.48, PP.110-116,2012##G.K. McMillan, Centrifugal And Axial Compressor Control, Momentum Press, 2010##M. Orkisz, P. Lipnicki, " International Symposium on Power Electronics, Electrical Drives, Automation and Motion (SPEEDAM)", Electronic ISBN: 978-1-4799-4749-2, 2014##A. Purarjomand Langrudi, A. Ghapanchi, M. Esmalifalak, A Datamining Approach for Fault Diagnosis: An Application of Anomaly Detection Algorithm, Measurement, Vol. 55, PP. 343-352, 2014##A. Sivakumar, S. Vaithiyanathan, Vibration based Data Analysis of Single Acting Compressor through Condition Monitoring and Multilayer Perceptron-A Machine Learning Classifier, InIOP Conference Series: Materials Science and Engineering, 2021.##P.N. Tan, M. Steinbach, V. Kumar, Introduction To Data mining, Pearson,2005##L. Wen, X. Li, L. Gao, Y. Zhang, A New Convolutional Neural Network Based Data-Driven Fault Diagnosis Method, IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, 2017##A. Contreras-Valdes, JP. Amezquita-Sanchez, D. Granados-Lieberman, M. Valtierra-Rodriguez, Predictive data mining techniques for fault diagnosis of electric equipment: a review, Applied Sciences, 2020.##L. Zhong, Y. Liu, J. Zhao, W. Wang, Deep predictive controller designed for centrifugal compressor system anti-surge, In2020 Chinese Automation Congress (CAC), 2020## ##

ارائه روشی جدید برای خوشه بندی داده های مخلوط بر مبنای تعداد ویژگی مشابه Presenting a new method for mixed data clustering based on the number of similar features 1 1 خوشه ­بندی عملیاتی است که در آن مجموعه­ای از نمونه داده‌ها، نسبت به میزان شباهت، دسته­بندی می­شوند. نمونه داده­های خوشه­بندی، عددی یا مخلوطی از عددی و غیرعددی (اسمی) هستند. یافتن میزان شباهت و اندازه‌گیری فاصله، از چالش­های خوشه­بندی داده ­های مخلوط است. در این مقاله سعی شده است در محاسبه میزان شباهت و تعیین فاصله، به پارامتر "تعداد ویژگی‌های مشابه" توجه شود. در نسبت دادن هر نمونه به خوشه در مواردی که فاصله‌ها برابر یا نزدیک باشد، تعداد ویژگی‌های مشترک نمونه‌ها تعیین کننده خوشه مناسب خواهد بود. برای محاسبه فاصله در الگوریتم مورد نظر از تفاضل عددی نرمالسازی شده برای ویژگی‌های عددی و از فاصله همینگ برای ویژگی‌های غیرعددی استفاده شده است. تعیین مرکز خوشه اولیه نیز مانند بسیاری از روش‌ها بصورت تصادفی انجام شده است و در تکرارهای بعدی الگوریتم، نمونه مناسب‌تر به عنوان مرکز خوشه انتخاب می‌شود. الگوریتم مورد نظر با 5 الگوریتم دیگر در 5 مجموعه‌ داده مقایسه شده است. در بررسی نتایج، از سه معیارAccuracy ، RI، F-Measure  استفاده شده است. طبق نتایج آزمایشات، در سه مجموعه‌داده، الگوریتم موردنظر حداقل دو درصد بهتر از دو الگوریتم و یک درصد بهتر از یکی دیگر از الگوریتم‌ها عمل کرده است. در یکی دیگر از مجموعه‌داده‌ها الگوریتم موردنظر نتایج برابر یا نزدیک به یک درصد دقت بهتر نسبت به الگوریتم برتر داشت. در مجموعه‌داده آخر نیز الگوریتم مورد نظر در رتبه دوم از بین 5 الگوریتم قرار داشت. 2 Clustering is an operation in which a set of data samples is categorized according to the degree of similarity. Examples of clustering data are numerical or a mixture of numerical and non-numerical (nominal) data. Finding similarities and measuring distances is one of the challenges of mixed data clustering. In the related works, to detect the degree of similarity and obtain the distance value, only the parameter of the distance value was considered and the cluster was selected based on its value. Clustering in this way, especially for mixed data, has not had very accurate results. In this paper, we have tried to pay attention to the parameter "number of similar features" in calculating the degree of similarity and determining the distance. In assigning each sample to a cluster in cases where the distances are equal or close, the number of common features of the samples will determine the appropriate cluster. That is, we will pay attention to the "number of similar features" in addition to the distance to select the cluster. This idea believes that in cases where the distance of the cluster centers is close to the data object, it is better to choose the cluster center that has more features similar to the data object. Logically and also according to the proposed algorithm, the amount of similarity should be in a larger number of features, not just a few limited features but with high similarity. The parameter of the "number of similar features" has a specific definition and is obtained with a suitable threshold. If the distance value of two features is less than the threshold, those two features are considered as similar features. To calculate the distance in the algorithm, the normalized numerical difference for numerical properties and the Hamming distance for non-numerical properties are used. Determining the initial cluster centers, like many methods, is done randomly, and in subsequent iterations of the algorithm, more appropriate samples are selected as the cluster centers. The algorithm is compared with 5 other algorithms in 5 datasets. In examining the results, three criteria of Accuracy, RI and F-Measure have been used. According to the test results, in the mixed and integer datasets, the algorithm performs at least two percent better than the two algorithms and one percent better than the other algorithm. In another data set, the proposed algorithm had results equal to or close to one percent better accuracy than the superior algorithm. In the last data set, the proposed algorithm was ranked second among 5 algorithms. In general, the proposed algorithm won the top rank in most of the results, and in the rest of the cases, it won the second rank out of the five tested algorithms. 39 52 2021/10/262023/07/12022/11/132022/08/3 1401/5/12 2023/02/222024/02/252024/02/252024/02/25 1402/12/6 حمید رضایی دانشگاه تربیت دبیر شهید رجایی ایران hid.rezaei@nit.ac.ir نگین دانشپور Negin Daneshpour دانشگاه تربیت دبیر شهید رجایی ایران ndaneshpour@sru.ac.ir Clustering Mixed data Distance of values Similarity of values Cluster Center. خوشه‌بندی داده مخلوط فاصله مقادیر تشابه مقادیر مرکز خوشه. Ahmad, Amir, and Shehroz S. Khan. "Survey of state-of-the-art mixed data clustering algorithms." Ieee Access 7 (2019): 31883-31902.##Ahmad, Amir, and Shehroz S. Khan. "initKmix-A novel initial partition generation algorithm for clustering mixed data using k-means-based clustering." Expert Systems with Applications 167 (2021): 114149.##Behzadi, Sahar, et al. "Clustering of mixed-type data considering concept hierarchies: problem specification and algorithm." International Journal of Data Science and Analytics 10.3 (2020): 233-248.##Kumar, Pradeep, and Anita Kanavalli. "A Similarity based K-Means Clustering Technique for Categorical Data in Data Mining Application." International Journal of Intelligent Engineering and Systems 14.2 (2021): 43-51.##Ji, Jinchao, et al. "A Multi-View Clustering Algorithm for Mixed Numeric and Categorical Data." IEEE Access 9 (2021): 24913-24924.##Sangam, Ravi Sankar, and Hari Om. "An equi-biased k-prototypes algorithm for clustering mixed-type data." Sādhanā 43.3 (2018): 1-12.##Yuan, Fang, Youlong Yang, and Tiantian Yuan. "A dissimilarity measure for mixed nominal and ordinal attribute data in k-Modes algorithm." Applied Intelligence 50.5 (2020): 1498-1509##Jia, Ziqi, and Ling Song. "Weighted k-Prototypes Clustering Algorithm Based on the Hybrid Dissimilarity Coefficient." Mathematical Problems in Engineering 2020 (2020).##Jia, Hong, Yiu-ming Cheung, and Jiming Liu. "A new distance metric for unsupervised learning of categorical data." IEEE transactions on neural networks and learning systems 27.5 (2015): 1065-1079.##Ji, Jinchao, et al. "Clustering mixed numeric and categorical data with artificial bee colony strategy." Journal of Intelligent & Fuzzy Systems 36.2 (2019): 1521-1530.##Skabar, Andrew. "Clustering Mixed-Attribute Data using Random Walk." Procedia Computer Science 108 (2017): 988-997.##Du, Mingjing, Shifei Ding, and Yu Xue. "A novel density peaks clustering algorithm for mixed data." Pattern Recognition Letters 97 (2017): 46-53##Qian, Yuhua, et al. "Space structure and clustering of categorical data." IEEE transactions on neural networks and learning systems 27.10 (2015): 2047-2059.##dos Santos, Tiago RL, and Luis E. Zárate. "Categorical data clustering: What similarity measure to recommend?. " Expert Systems with Applications 42.3 (2015): 1247-1260.##Ahmad, Amir, and Sarosh Hashmi. "K-Harmonic means type clustering algorithm for mixed datasets." Applied Soft Computing 48 (2016): 39-49.##Ji, Jinchao, et al. "An initialization method for clustering mixed numeric and categorical data based on the density and distance." International Journal of Pattern Recognition and Artificial Intelligence 29.07 (2015): 1550024.##van de Velden, Michel, Alfonso Iodice D'Enza, and Angelos Markos. "Distance‐based clustering of mixed data." Wiley Interdisciplinary Reviews: Computational Statistics 11.3 (2019): e1456.##Caruso, Giulia, et al. "Cluster analysis: An application to a real mixed-type data set." Models and Theories in Social Systems. Springer, Cham, 2019. 525-533.##Jinyin, Chen, et al. "A novel cluster center fast determination clustering algorithm." Applied Soft Computing 57 (2017): 539-555##Xiong, Jing, and Hong Yu. "An adaptive three-way clustering algorithm for mixed-type data." International Symposium on Methodologies for Intelligent Systems. Springer, Cham, 2018.##Dinh, Duy-Tai, and Van-Nam Huynh. "k-PbC: an improved cluster center initialization for categorical data clustering." Applied Intelligence (2020): 1-23.##Hsu, Chung-Chian, and Yan-Ping Huang. "Incremental clustering of mixed data based on distance hierarchy." Expert systems with applications 35.3 (2008): 1177-1185.##Ahmad, Amir, and Lipika Dey. "A k-mean clustering algorithm for mixed numeric and categorical data." Data & Knowledge Engineering 63.2 (2007): 503-527.##UCI Repository. https://archive.ics.uci.edu/ml/datasets.html. (September 6, 2021).##Ahmad, Amir, and Shehroz S. Khan. "Survey of state-of-the-art mixed data clustering algorithms." Ieee Access 7 (2019): 31883-31902.##Ahmad, Amir, and Shehroz S. Khan. "initKmix-A novel initial partition generation algorithm for clustering mixed data using k-means-based clustering." Expert Systems with Applications 167 (2021): 114149.##Behzadi, Sahar, et al. "Clustering of mixed-type data considering concept hierarchies: problem specification and algorithm." International Journal of Data Science and Analytics 10.3 (2020): 233-248.##Kumar, Pradeep, and Anita Kanavalli. "A Similarity based K-Means Clustering Technique for Categorical Data in Data Mining Application." International Journal of Intelligent Engineering and Systems 14.2 (2021): 43-51.##Ji, Jinchao, et al. "A Multi-View Clustering Algorithm for Mixed Numeric and Categorical Data." IEEE Access 9 (2021): 24913-24924.##Sangam, Ravi Sankar, and Hari Om. "An equi-biased k-prototypes algorithm for clustering mixed-type data." Sādhanā 43.3 (2018): 1-12.##Yuan, Fang, Youlong Yang, and Tiantian Yuan. "A dissimilarity measure for mixed nominal and ordinal attribute data in k-Modes algorithm." Applied Intelligence 50.5 (2020): 1498-1509##Jia, Ziqi, and Ling Song. "Weighted k-Prototypes Clustering Algorithm Based on the Hybrid Dissimilarity Coefficient." Mathematical Problems in Engineering 2020 (2020).##Jia, Hong, Yiu-ming Cheung, and Jiming Liu. "A new distance metric for unsupervised learning of categorical data." IEEE transactions on neural networks and learning systems 27.5 (2015): 1065-1079.##Ji, Jinchao, et al. "Clustering mixed numeric and categorical data with artificial bee colony strategy." Journal of Intelligent & Fuzzy Systems 36.2 (2019): 1521-1530.##Skabar, Andrew. "Clustering Mixed-Attribute Data using Random Walk." Procedia Computer Science 108 (2017): 988-997.##Du, Mingjing, Shifei Ding, and Yu Xue. "A novel density peaks clustering algorithm for mixed data." Pattern Recognition Letters 97 (2017): 46-53##Qian, Yuhua, et al. "Space structure and clustering of categorical data." IEEE transactions on neural networks and learning systems 27.10 (2015): 2047-2059.##dos Santos, Tiago RL, and Luis E. Zárate. "Categorical data clustering: What similarity measure to recommend?. " Expert Systems with Applications 42.3 (2015): 1247-1260.##Ahmad, Amir, and Sarosh Hashmi. "K-Harmonic means type clustering algorithm for mixed datasets." Applied Soft Computing 48 (2016): 39-49.##Ji, Jinchao, et al. "An initialization method for clustering mixed numeric and categorical data based on the density and distance." International Journal of Pattern Recognition and Artificial Intelligence 29.07 (2015): 1550024.##van de Velden, Michel, Alfonso Iodice D'Enza, and Angelos Markos. "Distance‐based clustering of mixed data." Wiley Interdisciplinary Reviews: Computational Statistics 11.3 (2019): e1456.##Caruso, Giulia, et al. "Cluster analysis: An application to a real mixed-type data set." Models and Theories in Social Systems. Springer, Cham, 2019. 525-533.##Jinyin, Chen, et al. "A novel cluster center fast determination clustering algorithm." Applied Soft Computing 57 (2017): 539-555##Xiong, Jing, and Hong Yu. "An adaptive three-way clustering algorithm for mixed-type data." International Symposium on Methodologies for Intelligent Systems. Springer, Cham, 2018.##Dinh, Duy-Tai, and Van-Nam Huynh. "k-PbC: an improved cluster center initialization for categorical data clustering." Applied Intelligence (2020): 1-23.##Hsu, Chung-Chian, and Yan-Ping Huang. "Incremental clustering of mixed data based on distance hierarchy." Expert systems with applications 35.3 (2008): 1177-1185.##Ahmad, Amir, and Lipika Dey. "A k-mean clustering algorithm for mixed numeric and categorical data." Data & Knowledge Engineering 63.2 (2007): 503-527.##UCI Repository. https://archive.ics.uci.edu/ml/datasets.html. (September 6, 2021).## ##

یک چارچوب توزیعی دو مرحله‌ای مبتنی بر خوشه‌بندی برای شناسایی چهره در مقیاس بالا A two-stage clustering-based distributed framework for large-scale face identification 1 1 در زمینه‌ی شناسایی چهره، چالش‌های افت دقت، افزایش نیازمندی به حافظه، و افزایش پیچیدگی زمانی از مشکلات مهمی به شمار می‌آیند. به منظور حل این مسائل، این تحقیق یک رویکرد دومرحله‌ای سه‌واحدی معرفی کرده است: واحد زیرشبکه‌ها، واحد خوشه‌یاب، و واحد تصمیم‌گیر نهایی. در مقابل روش‌های مبتنی بر توزیع تصادفی، روش ارائه شده، از خوشه‌بندی به عنوان روش توزیع مسئله به زیرشبکه‌ها استفاده می‌کند. هر زیرشبکه‌، یک شبکه عصبی عمیق نظارتی است که با داده‌های آموزشی مربوط به دسته‌های خود آموزش می‌بیند. واحد خوشه‌یاب، شباهت بردارهای ویژگی داده‌های آزمون را با میانگین بردارهای ویژگی دسته‌ها مقایسه می‌کند و بهترین خوشه را پیدا می‌کند. در نهایت، واحد تصمیم‌گیر نهایی با ترکیب نتایج دو واحد قبلی، بهترین دسته را انتخاب می‌کند. نتایج نشان می‌دهد که روش پیشنهادی، در مقایسه با روش‌های مشابه، از نظر صحت، بازخوانی، و امتیاز F1 عملکرد بهتری دارد. این روش ضمن سریع‌تر بودن، دارای دقت بالاتری نسبت به روش‌های بدون توزیع می‌باشد و در مقایسه با روش‌های توزیعی تصادفی، سرعتی برابر و دقتی بالاتر دارد. آزمایش‌ها بر روی مجموعه‌دادگان VGGFace2 و  MS-Celeb-1M و Glint360K اجرا شده و نشان می‌دهد که این روش علاوه بر عملکرد بهتر، مقیاس‌پذیری بالاتری را در بازشناسی چهره دارد. 2 Face recognition poses challenges in accuracy, memory efficiency, and computational complexity. This study proposes a two-stage, three-module approach: Subnetwork modules, Cluster-Finder unit, and Final-Decision module. Unlike random distribution methods, our approach employs clustering for distribution. Each subnetwork, a supervised deep neural network, is trained with cluster-specific data. The Cluster-Finder unit compares test data similarity with each subnetwork’s representative. The Final-Decision module selects the best class. Results indicate superior accuracy, recall, and F1 score compared to competitive methods. The approach is faster and more accurate than non-distribution methods, with comparable speed and higher accuracy than random distribution methods. Experiments on VGGFace2, MS-Celeb-1M, and Glint360K datasets confirm both superior performance and scalability. The proposed method, using KMeans for distribution, outperforms Softmax Dissection and Dynamic Active Class Selection. It simplifies training without additional manipulations, offering efficiency over methodologies like Softmax Dissection and ArcFace parallelization. In conclusion, this study focuses on pre-processing and post-processing without added training complexity. A divide-and-conquer approach addresses accuracy and efficiency challenges. In this study, various sources leading to errors in face recognition systems have been examined. These sources include: imprecise features, overfitting, challenging classes, distribution issues, and decision-making complexities. Various classification scenarios are explored, including non-distributed and models with random and intelligent distributions. Inaccurate features uniformly impact all scenarios, with overfitting posing the greatest challenge in non-distributed scenarios. Challenging classes are better distinguished in intelligent distribution scenarios. Inappropriate distribution has less impact in intelligent scenarios, and decision-making challenges exist in both distributions 53 70 2021/10/262023/07/12022/11/132022/08/32023/03/7 1401/12/16 2023/02/222024/02/252024/02/252024/02/252024/02/24 1402/12/5 سید محمد احمدی Sayed Mohammad Ahmadi دانشگاه قم ایران sm.ahmadi@stu.qom.ac.ir روح الله دیانت Rouhollah Dianat دانشگاه قم ایران rdianat@qom.ac.ir face recognition face identification clustering deep learning distributed learning. بازشناسی چهره شناسایی چهره خوشه‌بندی یادگیری عمیق یادگیری توزیعی Sun, Y., et al., Deep learning face representation by joint identification-verification. Advances in neural information processing systems, 2014. 27.##Schroff, F., D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.##Boutros, F., et al., Synthetic data for face recognition: Current state and future prospects. Image and Vision Computing, 2023: p. 104688.##Alansari, M., et al., GhostFaceNets: Lightweight Face Recognition Model From Cheap Operations. IEEE Access, 2023. 11: p. 35429-35446.##Ahmadi, MA., Dianat, R., Introducing a method for extracting features from facial images based on applying transformations to features obtained from convolutional neural networks. Signal and Data Processing, 2020. 17(3): p. 141-156.##احمدی، مرتضیعلی، و دیانت، روح الله. (1399). ارائه یک روش استخراج ویژگی از تصاویر چهره مبتنی بر اعمال تبدیل روی ویژگی های به دست آمده از شبکه های عصبی کانولوشن. پردازش علایم و داده ها، 17(3 (پیاپی 45) )، 141-156.##Yang, S., et al., Faceness-net: Face detection through deep facial part responses. IEEE transactions on pattern analysis and machine intelligence, 2017. 40(8): p. 1845-1859.##Zhang, K., et al., Joint face detection and alignment using multitask cascaded convolutional networks. IEEE signal processing letters, 2016. 23(10): p. 1499-1503.##Hu, P. and D. Ramanan. Finding tiny faces. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.##Mamieva, D., et al., Improved face detection method via learning small faces on hard images based on a deep learning approach. Sensors, 2023. 23(1): p. 502.##Si, T., F. He, and P. Li, Hybrid feature constraint with clustering for unsupervised person re-identification. The Visual Computer, 2022.##Zhao, W., et al., Face recognition: A literature survey. ACM computing surveys (CSUR), 2003. 35(4): p. 399-458.##Campadelli, P., R. Lanzarotti, and C. Savazzi. A feature-based face recognition system. in 12th International Conference on Image Analysis and Processing, 2003. Proceedings. 2003. IEEE.##Bern, M. and D. Eppstein, Mesh generation and optimal triangulation, in Computing in Euclidean geometry. 1995, World Scientific. p. 47-123.##Yim, J., et al. Rotating your face using multi-task deep neural network. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.##Jin, X. and X. Tan, Face alignment in-the-wild: A survey. Computer Vision and Image Understanding, 2017. 162: p. 1-22.##Galbally, J. and S. Marcel. Face anti-spoofing based on general image quality assessment. in 2014 22nd international conference on pattern recognition. 2014. IEEE.##Galbally, J., S. Marcel, and J. Fierrez, Biometric antispoofing methods: A survey in face recognition. IEEE Access, 2014. 2: p. 1530-1552.##Yu, Z., et al. Flexible-modal face anti-spoofing: A benchmark. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.##Zhou, E., Z. Cao, and Q. Yin, Naive-deep face recognition: Touching the limit of LFW benchmark or not? arXiv preprint arXiv:1501.04690, 2015.##Jalal, A.S., D.K. Sharma, and B. Sikander, Suspect face retrieval using visual and linguistic information. The Visual Computer, 2022: p. 1-27.##Nasution, M.I.P., et al. Face recognition login authentication for digital payment solution at COVID-19 pandemic. in 2020 3rd International Conference on Computer and Informatics Engineering (IC2IE). 2020. IEEE.##Talahua, J.S., et al., Facial recognition system for people with and without face mask in times of the covid-19 pandemic. Sustainability, 2021. 13(12): p. 6900.##Wang, F., et al., NormFace: L2 Hypersphere Embedding for Face Verification, in Proceedings of the 25th ACM international conference on Multimedia. 2017, Association for Computing Machinery: Mountain View, California, USA. p. 1041-1049.##Liu, W., et al., Large-margin softmax loss for convolutional neural networks. arXiv preprint arXiv:1612.02295, 2016.##Liu, W., et al. Sphereface: Deep hypersphere embedding for face recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.##Wang, F., et al. Normface: L2 hypersphere embedding for face verification. in Proceedings of the 25th ACM international conference on Multimedia. 2017.##Wang, H., et al. Cosface: Large margin cosine loss for deep face recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.##Deng, J., et al. Arcface: Additive angular margin loss for deep face recognition. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.##He, L., et al. Softmax dissection: Towards understanding intra-and inter-class objective for embedding learning. in Proceedings of the AAAI Conference on Artificial Intelligence. 2020.##Zhang, X., et al. Accelerated training for massive classification via dynamic class selection. in Proceedings of the AAAI Conference on Artificial Intelligence. 2018.##An, X., et al. Partial fc: Training 10 million identities on a single machine. in Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.##Wu, Y., et al. Deep convolutional neural network with independent softmax for large scale face recognition. in Proceedings of the 24th ACM international conference on Multimedia. 2016.##Cao, Q., et al. Vggface2: A dataset for recognising faces across pose and age. in 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). 2018. IEEE.##Guo, Y., et al. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. in Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14. 2016. Springer.##Hadsell, R., S. Chopra, and Y. LeCun. Dimensionality reduction by learning an invariant mapping. in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). 2006. IEEE.##Nguyen, B., C. Morell, and B. De Baets, Distance metric learning for ordinal classification based on triplet constraints. Knowledge-Based Systems, 2018. 142: p. 17-28.##Wen, Y., et al. A discriminative feature learning approach for deep face recognition. in Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VII 14. 2016. Springer.##Wang, K., et al. An efficient training approach for very large scale face recognition. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.##Liu, Y. and Q. Liu. Convolutional neural networks with large-margin softmax loss function for cognitive load recognition. in 2017 36th Chinese control conference (CCC). 2017. IEEE.##Dietterich, T.G. and G. Bakiri, Solving multiclass learning problems via error-correcting output codes. Journal of artificial intelligence research, 1994. 2: p. 263-286.##Zhang, Q., et al. Large scale classification in deep neural network with label mapping. in 2018 IEEE International Conference on Data Mining Workshops (ICDMW). 2018. IEEE.##Xu, Y., et al. High performance large scale face recognition with multi-cognition softmax and feature retrieval. in Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017.##He, K., et al. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.##Tim, E. Face Recognition Using Pytorch. 2022 2023/11/02; Available from: https://github.com/timesler/facenet-pytorch.##Sun, Y., et al., Deep learning face representation by joint identification-verification. Advances in neural information processing systems, 2014. 27.##Schroff, F., D. Kalenichenko, and J. Philbin. Facenet: A unified embedding for face recognition and clustering. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.##Boutros, F., et al., Synthetic data for face recognition: Current state and future prospects. Image and Vision Computing, 2023: p. 104688.##Alansari, M., et al., GhostFaceNets: Lightweight Face Recognition Model From Cheap Operations. IEEE Access, 2023. 11: p. 35429-35446.##Ahmadi, MA., Dianat, R., Introducing a method for extracting features from facial images based on applying transformations to features obtained from convolutional neural networks. Signal and Data Processing, 2020. 17(3): p. 141-156.##احمدی، مرتضیعلی، و دیانت، روح الله. (1399). ارائه یک روش استخراج ویژگی از تصاویر چهره مبتنی بر اعمال تبدیل روی ویژگی های به دست آمده از شبکه های عصبی کانولوشن. پردازش علایم و داده ها، 17(3 (پیاپی 45) )، 141-156.##Yang, S., et al., Faceness-net: Face detection through deep facial part responses. IEEE transactions on pattern analysis and machine intelligence, 2017. 40(8): p. 1845-1859.##Zhang, K., et al., Joint face detection and alignment using multitask cascaded convolutional networks. IEEE signal processing letters, 2016. 23(10): p. 1499-1503.##Hu, P. and D. Ramanan. Finding tiny faces. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.##Mamieva, D., et al., Improved face detection method via learning small faces on hard images based on a deep learning approach. Sensors, 2023. 23(1): p. 502.##Si, T., F. He, and P. Li, Hybrid feature constraint with clustering for unsupervised person re-identification. The Visual Computer, 2022.##Zhao, W., et al., Face recognition: A literature survey. ACM computing surveys (CSUR), 2003. 35(4): p. 399-458.##Campadelli, P., R. Lanzarotti, and C. Savazzi. A feature-based face recognition system. in 12th International Conference on Image Analysis and Processing, 2003. Proceedings. 2003. IEEE.##Bern, M. and D. Eppstein, Mesh generation and optimal triangulation, in Computing in Euclidean geometry. 1995, World Scientific. p. 47-123.##Yim, J., et al. Rotating your face using multi-task deep neural network. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.##Jin, X. and X. Tan, Face alignment in-the-wild: A survey. Computer Vision and Image Understanding, 2017. 162: p. 1-22.##Galbally, J. and S. Marcel. Face anti-spoofing based on general image quality assessment. in 2014 22nd international conference on pattern recognition. 2014. IEEE.##Galbally, J., S. Marcel, and J. Fierrez, Biometric antispoofing methods: A survey in face recognition. IEEE Access, 2014. 2: p. 1530-1552.##Yu, Z., et al. Flexible-modal face anti-spoofing: A benchmark. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.##Zhou, E., Z. Cao, and Q. Yin, Naive-deep face recognition: Touching the limit of LFW benchmark or not? arXiv preprint arXiv:1501.04690, 2015.##Jalal, A.S., D.K. Sharma, and B. Sikander, Suspect face retrieval using visual and linguistic information. The Visual Computer, 2022: p. 1-27.##Nasution, M.I.P., et al. Face recognition login authentication for digital payment solution at COVID-19 pandemic. in 2020 3rd International Conference on Computer and Informatics Engineering (IC2IE). 2020. IEEE.##Talahua, J.S., et al., Facial recognition system for people with and without face mask in times of the covid-19 pandemic. Sustainability, 2021. 13(12): p. 6900.##Wang, F., et al., NormFace: L2 Hypersphere Embedding for Face Verification, in Proceedings of the 25th ACM international conference on Multimedia. 2017, Association for Computing Machinery: Mountain View, California, USA. p. 1041-1049.##Liu, W., et al., Large-margin softmax loss for convolutional neural networks. arXiv preprint arXiv:1612.02295, 2016.##Liu, W., et al. Sphereface: Deep hypersphere embedding for face recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.##Wang, F., et al. Normface: L2 hypersphere embedding for face verification. in Proceedings of the 25th ACM international conference on Multimedia. 2017.##Wang, H., et al. Cosface: Large margin cosine loss for deep face recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.##Deng, J., et al. Arcface: Additive angular margin loss for deep face recognition. in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.##He, L., et al. Softmax dissection: Towards understanding intra-and inter-class objective for embedding learning. in Proceedings of the AAAI Conference on Artificial Intelligence. 2020.##Zhang, X., et al. Accelerated training for massive classification via dynamic class selection. in Proceedings of the AAAI Conference on Artificial Intelligence. 2018.##An, X., et al. Partial fc: Training 10 million identities on a single machine. in Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.##Wu, Y., et al. Deep convolutional neural network with independent softmax for large scale face recognition. in Proceedings of the 24th ACM international conference on Multimedia. 2016.##Cao, Q., et al. Vggface2: A dataset for recognising faces across pose and age. in 2018 13th IEEE international conference on automatic face & gesture recognition (FG 2018). 2018. IEEE.##Guo, Y., et al. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. in Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III 14. 2016. Springer.##Hadsell, R., S. Chopra, and Y. LeCun. Dimensionality reduction by learning an invariant mapping. in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06). 2006. IEEE.##Nguyen, B., C. Morell, and B. De Baets, Distance metric learning for ordinal classification based on triplet constraints. Knowledge-Based Systems, 2018. 142: p. 17-28.##Wen, Y., et al. A discriminative feature learning approach for deep face recognition. in Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part VII 14. 2016. Springer.##Wang, K., et al. An efficient training approach for very large scale face recognition. in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.##Liu, Y. and Q. Liu. Convolutional neural networks with large-margin softmax loss function for cognitive load recognition. in 2017 36th Chinese control conference (CCC). 2017. IEEE.##Dietterich, T.G. and G. Bakiri, Solving multiclass learning problems via error-correcting output codes. Journal of artificial intelligence research, 1994. 2: p. 263-286.##Zhang, Q., et al. Large scale classification in deep neural network with label mapping. in 2018 IEEE International Conference on Data Mining Workshops (ICDMW). 2018. IEEE.##Xu, Y., et al. High performance large scale face recognition with multi-cognition softmax and feature retrieval. in Proceedings of the IEEE International Conference on Computer Vision Workshops. 2017.##He, K., et al. Deep residual learning for image recognition. in Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.##Tim, E. Face Recognition Using Pytorch. 2022 2023/11/02; Available from: https://github.com/timesler/facenet-pytorch.## ##

الگوریتمی مبتنی بر گراف برای خوشه‌بندی سوره‌های قرآن کریم A Graph-based Algorithm for Clustering Qur’anic Surahs 1 1 قرآن کتاب نازل‌شده از طرف خداست و تا به امروز اندیشمندان و پژوهش‌گران مختلفی در جهت شناخت قرآن و فهم آن تلاش نموده‌اند. در دسترس بودن سیستم‌های رایانه‌ای فرصت مغتنمی است که با افزایش سرعت پژوهش‌گران در پیمودن مسیر، آن‌ها را در رسیدن به قله‌های بلندتری یاری کند. خوشه‌بندی یکی از روش‌هایی است که برای فهم ساختار داده به کار می‌رود. در این مقاله به خوشه‌بندی سوره‌های قرآن کریم بر اساس هم‌وقوعی کلمات در آن پرداخته‌ و برای دست‌یابی به این هدف از یک رویکرد موجود مبتنی بر گراف استفاده نموده‌ایم. در پژوهش جاری ابتدا هر سوره را به صورت یک گراف غیرجهت‌دار و وزن‌دار بازنمایی کرده‌، سپس بردار هر سوره را بر اساس گراف سوره‌ تشکیل داده‌ایم و پس از آن سوره‌ها را خوشه‌بندی نموده‌ایم. برای ارزیابی کیفیت خوشه‌بندی از معیار نیم‌رخ استفاده کرده‌ایم. بر اساس این معیار در بهترین خوشه‌بندی در بین اجراهای مختلف مقدار نیم‌رخ ۰/۹۱ به دست‌آمده است. این پژوهش زیرساخت ساختاری مناسبی برای توصیف لایه معنایی سوره‌‌ها و آیات قران پیش روی پژوهش‌گران حوزه زبان‌شناسی محاسباتی در دامنه علوم قرآنی فراهم می‌سازد. 2 The Holy Qur'an is revealed from God Almighty. Up to now many scholars and researchers have tried to understand the Holy Qur'an and comprehend it. The availability of computer systems is a great opportunity to help researchers reach higher peaks by speeding them up in their way. Clustering is one of the methods has been used to understand the structure of the data. In clustering, we want to divide samples of data into groups so that the members of each cluster are similar together and are different from the members of the other clusters. Clustering of Qur'anic surahs has been the subject of some computer studies on the Qur'an. In these studies, different approaches have been considered to vectorizing the surahs. In a study, Thabet formed vectors of each surah by considering some stems of Qur'anic words as features and the normalized probability of their occurrences in the surah as feature values and clustered just 24 surahs due to the sparseness of the obtained data matrix. With a similar approach in vectorizing the surahs, Moisl calculated the minimum surah length threshold per feature in order to solve the problem of shorter surahs by using some concepts of statistical sampling theory, and could cluster more surahs. Instead of using words as features, Sharaf considered 13 features including existence of referring to the story of Adam and Ebliys, number of the phrase «یا أَیُّهَا الَّذینَ آمَنُوا» (O you who believe), and determined the method of measuring each feature. Then, he formed data matrix and clustered the Qur'anic surahs. In another study, Sufi et al. considered the topics identified for each verse in the Tafsir Rahnama as features and constructed a binary data matrix based on the presence or absence of that topic in the Tafsir of that surah and applied clustering. In this article, we have clustered the surahs of the Holy Qur'an based on the co-occurrence of words in it. To achieve this goal, we have used an existing graph-based approach. In the present study, we first represent each surah as a weighted undirected graph. Then we form the vector of each surah by considering closed frequent sub-graphs as features and relative occurrence of them in each surah as feature values, and eventually cluster the surahs. We used the Silhouette score to evaluate the quality of clustering. Based on this criterion, in the best clustering among different runs, the Silhouette score of 0.91 was obtained. This research provide a proper structural infrastructure for specifying the semantic layer of Holy Qur'an surahs for computational linguistics researchers in the domain of Qur'anic studies. 71 88 2021/10/262023/07/12022/11/132022/08/32023/03/72021/04/10 1400/1/21 2023/02/222024/02/252024/02/252024/02/252024/02/242022/09/24 1401/7/2 مریم سادات متقی Maryam Sadat Mottaghi دانشگاه شهیدبهشتی ایران m.motaghi88@chmail.ir بهروز مینایی Behrouz Minaei-Bidgoli دانشگاه علم و صنعت ایران b_minaei@iust.ac.ir Document Clustering Text Graph Frequent subgraph Computational Qur'an mining خوشه‌بندی متن بازنمایی شبکه‌ای متن گراف متن زیرگراف‌ پرتکرار قرآن‌کاوی رایانشی اصلانی، اکرم، اسماعیلی، مهدی. «یافتن الگوهای مکرّر در قرآن کریم به‌‌کمک روش‌‌های متن‌‌کاوی»، پردازش علائم و داده‌ها،۱۵ (۳)، ۸۹-۱۰۰، ۱۳۹۷ https://jsdp.rcisp.ac.ir/. [ دسترسی در ۱۹ اردیبهشت ۱۴۰۳].##A. Aslani A, M. Esmaeili, "Finding Frequent Patterns in Holy Quran UsingText Mining," JSDP, vol. 15, no.3, pp. 89-100, 2018. Available: https://jsdp.rcisp.ac.ir/. [Accessed: May. 10, 2024].##صادق‌زمانی، فهیمه، ضرغامی، محمدحسین، «تعیین ساختار روابط بین اعضای خانواده بر اساس ویژگیهای شخصیتی»، اندازه‌گیری تربیتی، شماره ۳۳، ۲۰۸-۱۸۹، ۱۳۹۷. http://journals.atu.ac.ir. [دسترسی در ۵ اسفند۹۹].##Sadegh-Zamani and M. H. Zarghami, "Determining structure of relations between family members based on personality characteristics," Quarterly of Educational Measurement, no. 33, pp. 189-208, 2018. Available: http://journals.atu.ac.ir. [Accesed: Feb. 24, 2021].##صوفی، محسن، علی‌احمدی، علی‌رضا، علی‌احمدی، حسین، مینایی، بهروز، «خوشه‌بندی سوره‌های قرآن با تکنیک‌های داده‌کاوی»، رهیافت‌هایی در علوم قرآن و حدیث، دوره ۵۰، ۱۲۰-۱۰۳، ۱۳۹۷. https://jquran.um.ac.ir. [دسترسی در ۵ اسفند۹۹].##[M. Sufi, A. R. Ali-Ahmadi, H. Ali-Ahmadi, B. Minaei-Bidgoli, "Clustering of Qur'anic Surahs Using Data Mining Techniques", New Approaches in Quran and Hadith Studies, vol. 50, pp. 103-120, 2018-2019. Available: https://jquran.um.ac.ir. [Accesed: Feb. 24, 2021].##قلی‌زاده، بهروز، ساختمان‌های گسسته، چاپ سی و یکم، تهران، مؤسسه انتشارات علمی دانشگاه شریف، چاپ۳۱، ۱۳۹۱.##B. Gholizadeh, Discrete Mathematics, Tehran: Sharif University Press, 2012-2013.##هاشمی رفسنجانی، علی‌اکبر، و جمعی از محققان مرکز فرهنگ و معارف قرآن، تفسیر راهنما، قم: بوستان کتاب قم، چاپ سوم، ۱۳۷۹.##A. Hashemi-Rafsanjani, and a group of researchers from Quranic Science and Culture Center, Tafsir Rahnama, Qom: Bustan Ketab Publisher, 2000-2001.##E. Aftab and MK. Malik, "eRock at Qur'an QA 2022: contemporary deep neural networks for Qur'an based reading comprehension question answers," In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection 2022. pp. 96-103, 2022. Available: https://aclanthology.org/2022.osact-1.11.pdf [Accessed: May. 10, 2024].##M. E. Aktas, and E. Akbas, "Text classification via network topology: A case study on the Holy Quran," In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 16 Dec 2019, 1557-1562. Available: http://www.math .uco.edu. [Accessed: Feb. 24, 2021].##E. Alpaydin, Introduction to machine learning, 2nd ed. Massachusetts: Massachusetts Institutes of Technology, 2010. Available: https://books.google.com [Accessed: Feb. 24, 2021].##E. Atwell, Habash Nizar, Louw Bill, B. Abu Shawar, T. McEnery, W. Zaghouani, and M. El-Haj, "Understanding the Quran: A new grand challenge for computer science and artificial intelligence," ACM-BCS Visions of Computer Science 2010, 2010. Available: https://eprints.lancs.ac.uk. [Accessed: Feb. 25, 2021].##C. M. Bishop, Pattern Recognition and Machine Learning, New York: Springer-Verlag, 2006. Available: http://users.isr.ist.utl.pt/. [Accessed: Feb. 25, 2021].##K. Dukes, and N. Habash, "Morphological Annotation of Quranic Arabic," Lrec, 2010. Available: http://citeseerx.ist.psu.edu/. [Accessed: Feb. 25, 2021].##K. Dukes, "Quranic Arabic Corpus," May. 1, 2011. [online]. Available: http://corpus.quran.com/. [Accessed: Feb. 24, 2021]##M. H. Elahimanesh, B. Minaei-Bidgoli, M. J. Gholami, and H. Juzi, "An Introduction to Noor Corpus and its Language Model." First International Conference on Persian language Processing(ICPLP), Semnan university, 2012. Available: researchgate.net. [Accessed: Feb. 25, 2021].##H. Veeramani, S. Thapa and U. Naseem , "LowResContextQA at Qur'an QA 2023 Shared Task: Temporal and Sequential Representation Augmented Question Answering Span Detection in Arabic," In Proceedings of ArabicNLP 2023, pp. 708-713, 2023. Available: https://aclanthology.org/2023.arabicnlp-1.78. [Accessed: May. 10, 2024].##A. Lim, "The berth planning problem," perations research letters, vol. 22, pp. 105-110, March 1998. Available: https://www.sciencedirect.com/ [Accessed: Feb. 25, 2021].##R. Malhas and T. Elsayed, "Arabic machine reading comprehension on the Holy Qur'an using CL-AraBERT," Information Processing & Management, Vol. 59, no. 6, 2022. Available: ##https://doi.org/10.1016/j.ipm.2022.103068##G. Mediamer, "Semantic Feature Analysis for Multi-Label Text Classification on Topics of the Al-Quran Verses," Journal of Information Processing Systems, vol. 20, no.1, 2024. Available: https://jips-k.org/digital-library/2024/20/1/1. [Accessed: May. 10, 2024].##Y. Mellah, I. Touahri, Z. Kaddari, Z. Haja, J. Berrich and T. Bouchentouf , "LARSA22 at Qur'an QA 2022: text-to-text transformer for finding answers to questions from Qur'an," In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection, pp. 112-119, 2022. Available: https://aclanthology.org/2022.osact-1.13. [Accessed: May. 10, 2024].##M. Mohammed, S. Amin and MM. Aref , "An english islamic articles dataset (eiad) for developing an islambot question answering chatbot," In 2022 5th International Conference on Computing and Informatics (ICCI), pp. 303-309, 2022. Available: https://ieeexplore.ieee.org/abstract/document/9756122/. [Accessed: May. 10, 2024].##H. Moisl, "Sura Length and Lexical Probability Estimation in Cluster Analysis of the Qur'an," ACM Transactions on Asian Language Information Processing (TALIP), vol. 8, pp. 1-19, 2009. Available: https://eprints.ncl.ac.uk/. [Accessed: Feb. 25, 2021].##A. B. Muhammad, "Annotation of conceptual co-reference and text mining the Qur'an,"Ph.D. dissertation, Dept. school of computing, Leeds Univ., UK, 2012. Available: http://etheses.whiterose.ac.uk/. [Accessed: Feb. 25, 2021].##C. Nicolini, C. Bordier, and A. Bifone, "Community detection in weighted brain connectivity networks beyond the resolution limit," Neuroimage, vol. 146, pp. 28-39, 2017. Available: researchgate.net. [Accessed: Feb. 25, 2021].##F. Rousseau, E. Kiagias, and M. Vazirgiannis, "Text categorization as a graph classification problem," In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 26-31 July 2015, pp. 1702-1712. Available: https://www.aclweb.org/. [Accessed: Feb. 25, 2021].##P. J. Rousseeuw, "Silhouettes: a graphical aid to the interpretation and validation of cluster analysis," Journal of computational and applied mathematics, vol. 20, pp. 53-65, 1987. Available: https://www.sciencedirect.com/. [Accessed: Feb. 25, 2021].##A. B. M. Sharaf, and E. Atwell, "QurAna: Corpus of the Quran annotated with Pronominal Anaphora," LREC, 2012, pp. 130-137. Available: http://citeseerx.ist.psu.edu/. [Accessed: Feb. 25, 2021].##A. B. M. Sharaf, and E. Atwell, "QurSim: A corpus for evaluation of relatedness in short texts," LREC, 2012, pp. 2295-2302. Available: http://textminingthequran.com/. [Accessed: Feb. 25, 2021].##S. A. Shirkhorshidi, S. Aghabozorgi Saeed, and T. Y. Wah, "A comparison study on similarity and dissimilarity measures in clustering continuous data," PloS one, vol. 10, 2015. Available: researchgate.net. [Accessed: Feb. 25, 2021].##M. A. Siddiqui, S. M. Faraz, and S. A. Sattar, "Discovering the thematic structure of the Quran using probabilistic topic model," 2013 In 2013 Taibah University International Conference on Advances in Information Technology for the Holy Quran and Its Sciences, IEEE, 2013, pp. 234-239. Available: researchgate.net. [Accessed: Feb. 25, 2021].##I. Takigawa, H. Mamitsuka, "Efficiently mining δ-tolerance closed frequent subgraphs," Machine Learning, vol. 82, pp. 95-121. Available: https://link.springer.com. [Accessed: Feb. 25, 2021].##P. N. Tan, M. Steinbach, A. Karpatne, and V. Kumar, Introduction to data mining, second edition, New York: Pearson Education, 2018.##N. Thabet, "Understanding the thematic structure of the Qur'an: an exploratory .multivariate approach," In Proceedings of the ACL Student Research Workshop, 2005, pp. 7-12. Available: https://www.aclweb.org/. [Accessed: Feb. 25, 2021].##X. Yan, and J. Han, "gspan: Graph-based substructure pattern mining," 2002 IEEE International Conference on Data Mining,2002, pp. 721-724. Available: https://sites.cs.ucsb.edu/. [Accessed: Feb. 25, 2021].##R. Zafarani, M. A. Abbasi, and H. Liu, Social Media Mining: An Introduction, London: Cambridge University Press, 2014. Available: http://citeseerx.ist.psu.edu/. [Accessed: Feb. 25, 2021].##اصلانی، اکرم، اسماعیلی، مهدی. «یافتن الگوهای مکرّر در قرآن کریم به‌‌کمک روش‌‌های متن‌‌کاوی»، پردازش علائم و داده‌ها،۱۵ (۳)، ۸۹-۱۰۰، ۱۳۹۷ https://jsdp.rcisp.ac.ir/. [ دسترسی در ۱۹ اردیبهشت ۱۴۰۳].##A. Aslani A, M. Esmaeili, "Finding Frequent Patterns in Holy Quran UsingText Mining," JSDP, vol. 15, no.3, pp. 89-100, 2018. Available: https://jsdp.rcisp.ac.ir/. [Accessed: May. 10, 2024].##صادق‌زمانی، فهیمه، ضرغامی، محمدحسین، «تعیین ساختار روابط بین اعضای خانواده بر اساس ویژگیهای شخصیتی»، اندازه‌گیری تربیتی، شماره ۳۳، ۲۰۸-۱۸۹، ۱۳۹۷. http://journals.atu.ac.ir. [دسترسی در ۵ اسفند۹۹].##Sadegh-Zamani and M. H. Zarghami, "Determining structure of relations between family members based on personality characteristics," Quarterly of Educational Measurement, no. 33, pp. 189-208, 2018. Available: http://journals.atu.ac.ir. [Accesed: Feb. 24, 2021].##صوفی، محسن، علی‌احمدی، علی‌رضا، علی‌احمدی، حسین، مینایی، بهروز، «خوشه‌بندی سوره‌های قرآن با تکنیک‌های داده‌کاوی»، رهیافت‌هایی در علوم قرآن و حدیث، دوره ۵۰، ۱۲۰-۱۰۳، ۱۳۹۷. https://jquran.um.ac.ir. [دسترسی در ۵ اسفند۹۹].##[M. Sufi, A. R. Ali-Ahmadi, H. Ali-Ahmadi, B. Minaei-Bidgoli, "Clustering of Qur'anic Surahs Using Data Mining Techniques", New Approaches in Quran and Hadith Studies, vol. 50, pp. 103-120, 2018-2019. Available: https://jquran.um.ac.ir. [Accesed: Feb. 24, 2021].##قلی‌زاده، بهروز، ساختمان‌های گسسته، چاپ سی و یکم، تهران، مؤسسه انتشارات علمی دانشگاه شریف، چاپ۳۱، ۱۳۹۱.##B. Gholizadeh, Discrete Mathematics, Tehran: Sharif University Press, 2012-2013.##هاشمی رفسنجانی، علی‌اکبر، و جمعی از محققان مرکز فرهنگ و معارف قرآن، تفسیر راهنما، قم: بوستان کتاب قم، چاپ سوم، ۱۳۷۹.##A. Hashemi-Rafsanjani, and a group of researchers from Quranic Science and Culture Center, Tafsir Rahnama, Qom: Bustan Ketab Publisher, 2000-2001.##E. Aftab and MK. Malik, "eRock at Qur'an QA 2022: contemporary deep neural networks for Qur'an based reading comprehension question answers," In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection 2022. pp. 96-103, 2022. Available: https://aclanthology.org/2022.osact-1.11.pdf [Accessed: May. 10, 2024].##M. E. Aktas, and E. Akbas, "Text classification via network topology: A case study on the Holy Quran," In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), 16 Dec 2019, 1557-1562. Available: http://www.math .uco.edu. [Accessed: Feb. 24, 2021].##E. Alpaydin, Introduction to machine learning, 2nd ed. Massachusetts: Massachusetts Institutes of Technology, 2010. Available: https://books.google.com [Accessed: Feb. 24, 2021].##E. Atwell, Habash Nizar, Louw Bill, B. Abu Shawar, T. McEnery, W. Zaghouani, and M. El-Haj, "Understanding the Quran: A new grand challenge for computer science and artificial intelligence," ACM-BCS Visions of Computer Science 2010, 2010. Available: https://eprints.lancs.ac.uk. [Accessed: Feb. 25, 2021].##C. M. Bishop, Pattern Recognition and Machine Learning, New York: Springer-Verlag, 2006. Available: http://users.isr.ist.utl.pt/. [Accessed: Feb. 25, 2021].##K. Dukes, and N. Habash, "Morphological Annotation of Quranic Arabic," Lrec, 2010. Available: http://citeseerx.ist.psu.edu/. [Accessed: Feb. 25, 2021].##K. Dukes, "Quranic Arabic Corpus," May. 1, 2011. [online]. Available: http://corpus.quran.com/. [Accessed: Feb. 24, 2021]##M. H. Elahimanesh, B. Minaei-Bidgoli, M. J. Gholami, and H. Juzi, "An Introduction to Noor Corpus and its Language Model." First International Conference on Persian language Processing(ICPLP), Semnan university, 2012. Available: researchgate.net. [Accessed: Feb. 25, 2021].##H. Veeramani, S. Thapa and U. Naseem , "LowResContextQA at Qur'an QA 2023 Shared Task: Temporal and Sequential Representation Augmented Question Answering Span Detection in Arabic," In Proceedings of ArabicNLP 2023, pp. 708-713, 2023. Available: https://aclanthology.org/2023.arabicnlp-1.78. [Accessed: May. 10, 2024].##A. Lim, "The berth planning problem," perations research letters, vol. 22, pp. 105-110, March 1998. Available: https://www.sciencedirect.com/ [Accessed: Feb. 25, 2021].##R. Malhas and T. Elsayed, "Arabic machine reading comprehension on the Holy Qur'an using CL-AraBERT," Information Processing & Management, Vol. 59, no. 6, 2022. Available: ##https://doi.org/10.1016/j.ipm.2022.103068##G. Mediamer, "Semantic Feature Analysis for Multi-Label Text Classification on Topics of the Al-Quran Verses," Journal of Information Processing Systems, vol. 20, no.1, 2024. Available: https://jips-k.org/digital-library/2024/20/1/1. [Accessed: May. 10, 2024].##Y. Mellah, I. Touahri, Z. Kaddari, Z. Haja, J. Berrich and T. Bouchentouf , "LARSA22 at Qur'an QA 2022: text-to-text transformer for finding answers to questions from Qur'an," In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection, pp. 112-119, 2022. Available: https://aclanthology.org/2022.osact-1.13. [Accessed: May. 10, 2024].##M. Mohammed, S. Amin and MM. Aref , "An english islamic articles dataset (eiad) for developing an islambot question answering chatbot," In 2022 5th International Conference on Computing and Informatics (ICCI), pp. 303-309, 2022. Available: https://ieeexplore.ieee.org/abstract/document/9756122/. [Accessed: May. 10, 2024].##H. Moisl, "Sura Length and Lexical Probability Estimation in Cluster Analysis of the Qur'an," ACM Transactions on Asian Language Information Processing (TALIP), vol. 8, pp. 1-19, 2009. Available: https://eprints.ncl.ac.uk/. [Accessed: Feb. 25, 2021].##A. B. Muhammad, "Annotation of conceptual co-reference and text mining the Qur'an,"Ph.D. dissertation, Dept. school of computing, Leeds Univ., UK, 2012. Available: http://etheses.whiterose.ac.uk/. [Accessed: Feb. 25, 2021].##C. Nicolini, C. Bordier, and A. Bifone, "Community detection in weighted brain connectivity networks beyond the resolution limit," Neuroimage, vol. 146, pp. 28-39, 2017. Available: researchgate.net. [Accessed: Feb. 25, 2021].##F. Rousseau, E. Kiagias, and M. Vazirgiannis, "Text categorization as a graph classification problem," In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 26-31 July 2015, pp. 1702-1712. Available: https://www.aclweb.org/. [Accessed: Feb. 25, 2021].##P. J. Rousseeuw, "Silhouettes: a graphical aid to the interpretation and validation of cluster analysis," Journal of computational and applied mathematics, vol. 20, pp. 53-65, 1987. Available: https://www.sciencedirect.com/. [Accessed: Feb. 25, 2021].##A. B. M. Sharaf, and E. Atwell, "QurAna: Corpus of the Quran annotated with Pronominal Anaphora," LREC, 2012, pp. 130-137. Available: http://citeseerx.ist.psu.edu/. [Accessed: Feb. 25, 2021].##A. B. M. Sharaf, and E. Atwell, "QurSim: A corpus for evaluation of relatedness in short texts," LREC, 2012, pp. 2295-2302. Available: http://textminingthequran.com/. [Accessed: Feb. 25, 2021].##S. A. Shirkhorshidi, S. Aghabozorgi Saeed, and T. Y. Wah, "A comparison study on similarity and dissimilarity measures in clustering continuous data," PloS one, vol. 10, 2015. Available: researchgate.net. [Accessed: Feb. 25, 2021].##M. A. Siddiqui, S. M. Faraz, and S. A. Sattar, "Discovering the thematic structure of the Quran using probabilistic topic model," 2013 In 2013 Taibah University International Conference on Advances in Information Technology for the Holy Quran and Its Sciences, IEEE, 2013, pp. 234-239. Available: researchgate.net. [Accessed: Feb. 25, 2021].##I. Takigawa, H. Mamitsuka, "Efficiently mining δ-tolerance closed frequent subgraphs," Machine Learning, vol. 82, pp. 95-121. Available: https://link.springer.com. [Accessed: Feb. 25, 2021].##P. N. Tan, M. Steinbach, A. Karpatne, and V. Kumar, Introduction to data mining, second edition, New York: Pearson Education, 2018.##N. Thabet, "Understanding the thematic structure of the Qur'an: an exploratory .multivariate approach," In Proceedings of the ACL Student Research Workshop, 2005, pp. 7-12. Available: https://www.aclweb.org/. [Accessed: Feb. 25, 2021].##X. Yan, and J. Han, "gspan: Graph-based substructure pattern mining," 2002 IEEE International Conference on Data Mining,2002, pp. 721-724. Available: https://sites.cs.ucsb.edu/. [Accessed: Feb. 25, 2021].##R. Zafarani, M. A. Abbasi, and H. Liu, Social Media Mining: An Introduction, London: Cambridge University Press, 2014. Available: http://citeseerx.ist.psu.edu/. [Accessed: Feb. 25, 2021].## ##

طراحی شبکه بهینه‌ساز خودکار الهام گرفته از الگوریتم بهینه‌سازی BFGS با حافظه محدود Designing L-BFGS inspired automatic optimizer network 1 1 امروزه علی‌رغم توسعه مدل‌های یادگیری ماشین برای استخراج ویژگی‌ها به صورت خودکار، هنوز الگوریتم‌های بهینه‌سازی به صورت دستی طراحی می‌شوند. یکی از اهداف فرایادگیری (meta-learning)، خودکار کردن فرایند بهینه‌سازی است. الگوریتم‌های بهینه‌سازی دستی مبتنی بر بردار گرادیان تنها براساس عملیات ضرب داخلی، ضرب اسکالر و جمع برداری بر روی بردارهای ورودی نوشته می شوند. بنابراین می توان گفت که این الگوریتمها در فضای هیلبرت بعد مساله بهینه‌سازی اجرا می شوند. ما نیز قصد داریم با ایده گرفتن از این مطلب، فضایی برای یادگیری ورودی‌ها ایجاد کنیم که مستقل از ابعاد ورودی باشد. بدین منظور با ایده گرفتن از الگوریتم BFGS با حافظه محدود (L-BFGS) و همچنین سلول LSTM یک ساختار جدید با نام Hilbert LSTM (HLSTM) معرفی می‌کنیم که فرایند یادگیری در آن مستقل از ابعاد ورودی انجام می‌شود. به عبارتی الگوریتم یادگیری در فضای هیلبرت مساله بهینه‌سازی اجرا می‌شود. برای رسیدن به این هدف از لایه ضرایب خطی استفاده می‌کنیم که ترکیب خطی بردارهای ورودی را محاسبه می‌کند و ضرایب این ترکیب خطی، با کمک ضرب داخلی بردارهای ورودی بدست می‌آید. آزمایش‌های ما نشان می‌دهند که نتایج به‌دست آمده توسط بهینه‌ساز ارائه شده، به مراتب بهتر از نتایج الگوریتم‌های بهینه‌سازی دستی است. 2 Nowadays using features learned by machines is common and these types of features have excellent quality in comparison with hand-designed features. While many machine learning models are developed to extract features automatically, however, the optimizing algorithms are still designed manually. In this paper, we propose a method to cast the optimizing algorithm as a machine learning problem. This is a branch of machine learning which is named meta-learning or learning to learn. Gradient-based optimization algorithms (e.g. gradient descent and BFGS) receive the gradient vector in each step and, by using the information of the previous points and gradients, estimate the update vector at the current point. The inputs and outputs of these algorithms are vectors whose dimension is the same as the optimization problem. These algorithms are written solely based on vector addition, scalar-product, and inner-product operations. Therefore, we can say that these algorithms are executed in a Hilbert space whose dimension is determined by the optimization problem. In this paper, we propose a novel method for learning to optimize over a Hilbert space of unknown dimensionality. We introduce a new neural network module named Hilbert LSTM (HLSTM) which is based on a novel LSTM cell whose learning process is independent of the input data dimension. This independency is the result of restricting the network to the operations on a Hilbert space, prohibiting the network to work directly with the entries within a vector. To achieve this goal, we use a linear coefficients layer that linearly combines the input vectors based on coefficients computed by their inner products. Training the network based on the inner product between vectors leads to learning an optimization algorithm that is independent of the data dimension. Our experiments show that the proposed optimizer achieves better results in comparison with hand-designed algorithms. 89 100 2021/10/262023/07/12022/11/132022/08/32023/03/72021/04/102020/05/10 1399/2/21 2023/02/222024/02/252024/02/252024/02/252024/02/242022/09/242024/02/17 1402/11/28 محمد اعتصام Mohammad Etesam دانشگاه فردوسی مشهد ایران etesam@mail.um.ac.ir اشکان صادقی لطف آبادی Ashkan Sadeghi-Lotfabadi دانشگاه فردوسی مشهد ایران sadeghia@mail.um.ac.ir سید کمال الدین غیاثی شیرازی Kamaledin Ghiasi-Shirazi دانشگاه فردوسی مشهد ایران k.ghiasi@um.ac.ir Hilbert LSTM LSTM L-BFGS meta-learning automatic optimization Hilbert LSTM LSTM L-BFGS فرایادگیری بهینه‌سازی خودکار M. Andrychowicz et al., "Learning to learn by gradient descent by gradient descent," in Advances in neural information processing systems, 2016, pp. 3981-3989.##Y. Nesterov, "A method for unconstrained convex minimization problem with the rate of convergence O (1/k^ 2)," in Doklady an ussr, 1983, vol. 269, pp. 543-547.##M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in IEEE international conference on neural networks, 1993: IEEE, pp. 586-591.##J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," Journal of machine learning research, vol. 12, no. Jul, pp. 2121-2159, 2011.##T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude," COURSERA: Neural networks for machine learning, vol. 4, no. 2, pp. 26-31, 2012.##D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.##J. Martens and R. Grosse, "Optimizing neural networks with kronecker-factored approximate curvature," in International conference on machine learning, 2015, pp. 2408-2417.##[D. L. Donoho, "Compressed sensing," IEEE Transactions on information theory, vol. 52, no. 4, pp. 1289-1306, 2006.##[D. H. Wolpert and W. G. Macready, "No free lunch theorems for optimization," IEEE transactions on evolutionary computation, vol. 1, no. 1, pp. 67-82, 1997.##[N. J. W. SJ, "Numerical optimization: Springer Science+ Business Media," ed: LLC, 2006.##S. Thrun and L. Pratt, "Learning to learn: Introduction and overview," in Learning to learn: Springer, 1998, pp. 3-17.##J. Schmidhuber, "Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook," Technische Universität München, 1987.##X. Chen et al., "Symbolic discovery of optimization algorithms," Advances in Neural Information Processing Systems, vol. 36, 2024.##T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, "Meta-learning in neural networks: A survey," IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 9, pp. 5149-5169, 2021.##E. Gärtner, L. Metz, M. Andriluka, C. D. Freeman, and C. Sminchisescu, "Transformer-based learned optimization," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 11970-11979.##L. S. Metz, N. Maheswaranathan, C. D. Freeman, B. Poole, and J. N. Sohl-Dickstein, "Training neural networks using learned optimizers," ed: Google Patents, 2022.##E. D. Cubuk, L. S. Metz, S. S. Schoenholz, and A. A. Merchant, "Optimization using learned neural network optimizers," ed: Google Patents, 2022.##B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, "Building machines that learn and think like people," Behavioral and brain sciences, vol. 40, 2017.##A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap, "Meta-learning with memory-augmented neural networks," in International conference on machine learning, 2016, pp. 1842-1850.##J. Schmidhuber, "A neural network that embeds its own meta-levels," in IEEE International Conference on Neural Networks, 1993: IEEE, pp. 407-412.##J. Schmidhuber, "Learning to control fast-weight memories: An alternative to dynamic recurrent networks," Neural Computation, vol. 4, no. 1, pp. 131-139, 1992.##J. Schmidhuber, J. Zhao, and M. Wiering, "Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement," Machine Learning, vol. 28, no. 1, pp. 105-130, 1997.##C. Daniel, J. Taylor, and S. Nowozin, "Learning step size controllers for robust neural network training," in Thirtieth AAAI Conference on Artificial Intelligence, 2016.##T. P. Runarsson and M. T. Jonsson, "Evolution and design of distributed learning rules," in 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No. 00, 2000: IEEE, pp. 59-63.##Y. Bengio, S. Bengio, and J. Cloutier, "Learning a synaptic learning rule: Université de Montréal," Département d'informatique et de recherche opérationnelle, 1990.##N. E. Cotter and P. R. Conwell, "Fixed-weight networks can learn," in 1990 IJCNN International Joint Conference on Neural Networks, 1990: IEEE, pp. 553-559.##A. S. Younger, P. R. Conwell, and N. E. Cotter, "Fixed-weight on-line learning," IEEE Transactions on Neural Networks, vol. 10, no. 2, pp. 272-283, 1999.##S. Hochreiter, A. S. Younger, and P. R. Conwell, "Learning to learn using gradient descent," in International Conference on Artificial Neural Networks, 2001: Springer, pp. 87-94.##A. S. Younger, S. Hochreiter, and P. R. Conwell, "Meta-learning with backpropagation," in IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222), 2001, vol. 3: IEEE.##S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.##M. Andrychowicz et al., "Learning to learn by gradient descent by gradient descent," in Advances in neural information processing systems, 2016, pp. 3981-3989.##Y. Nesterov, "A method for unconstrained convex minimization problem with the rate of convergence O (1/k^ 2)," in Doklady an ussr, 1983, vol. 269, pp. 543-547.##M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: The RPROP algorithm," in IEEE international conference on neural networks, 1993: IEEE, pp. 586-591.##J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," Journal of machine learning research, vol. 12, no. Jul, pp. 2121-2159, 2011.##T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude," COURSERA: Neural networks for machine learning, vol. 4, no. 2, pp. 26-31, 2012.##D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.##J. Martens and R. Grosse, "Optimizing neural networks with kronecker-factored approximate curvature," in International conference on machine learning, 2015, pp. 2408-2417.##[D. L. Donoho, "Compressed sensing," IEEE Transactions on information theory, vol. 52, no. 4, pp. 1289-1306, 2006.##[D. H. Wolpert and W. G. Macready, "No free lunch theorems for optimization," IEEE transactions on evolutionary computation, vol. 1, no. 1, pp. 67-82, 1997.##[N. J. W. SJ, "Numerical optimization: Springer Science+ Business Media," ed: LLC, 2006.##S. Thrun and L. Pratt, "Learning to learn: Introduction and overview," in Learning to learn: Springer, 1998, pp. 3-17.##J. Schmidhuber, "Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook," Technische Universität München, 1987.##X. Chen et al., "Symbolic discovery of optimization algorithms," Advances in Neural Information Processing Systems, vol. 36, 2024.##T. Hospedales, A. Antoniou, P. Micaelli, and A. Storkey, "Meta-learning in neural networks: A survey," IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 9, pp. 5149-5169, 2021.##E. Gärtner, L. Metz, M. Andriluka, C. D. Freeman, and C. Sminchisescu, "Transformer-based learned optimization," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 11970-11979.##L. S. Metz, N. Maheswaranathan, C. D. Freeman, B. Poole, and J. N. Sohl-Dickstein, "Training neural networks using learned optimizers," ed: Google Patents, 2022.##E. D. Cubuk, L. S. Metz, S. S. Schoenholz, and A. A. Merchant, "Optimization using learned neural network optimizers," ed: Google Patents, 2022.##B. M. Lake, T. D. Ullman, J. B. Tenenbaum, and S. J. Gershman, "Building machines that learn and think like people," Behavioral and brain sciences, vol. 40, 2017.##A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap, "Meta-learning with memory-augmented neural networks," in International conference on machine learning, 2016, pp. 1842-1850.##J. Schmidhuber, "A neural network that embeds its own meta-levels," in IEEE International Conference on Neural Networks, 1993: IEEE, pp. 407-412.##J. Schmidhuber, "Learning to control fast-weight memories: An alternative to dynamic recurrent networks," Neural Computation, vol. 4, no. 1, pp. 131-139, 1992.##J. Schmidhuber, J. Zhao, and M. Wiering, "Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement," Machine Learning, vol. 28, no. 1, pp. 105-130, 1997.##C. Daniel, J. Taylor, and S. Nowozin, "Learning step size controllers for robust neural network training," in Thirtieth AAAI Conference on Artificial Intelligence, 2016.##T. P. Runarsson and M. T. Jonsson, "Evolution and design of distributed learning rules," in 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No. 00, 2000: IEEE, pp. 59-63.##Y. Bengio, S. Bengio, and J. Cloutier, "Learning a synaptic learning rule: Université de Montréal," Département d'informatique et de recherche opérationnelle, 1990.##N. E. Cotter and P. R. Conwell, "Fixed-weight networks can learn," in 1990 IJCNN International Joint Conference on Neural Networks, 1990: IEEE, pp. 553-559.##A. S. Younger, P. R. Conwell, and N. E. Cotter, "Fixed-weight on-line learning," IEEE Transactions on Neural Networks, vol. 10, no. 2, pp. 272-283, 1999.##S. Hochreiter, A. S. Younger, and P. R. Conwell, "Learning to learn using gradient descent," in International Conference on Artificial Neural Networks, 2001: Springer, pp. 87-94.##A. S. Younger, S. Hochreiter, and P. R. Conwell, "Meta-learning with backpropagation," in IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No. 01CH37222), 2001, vol. 3: IEEE.##S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.## ##

نور-استم نسخه 1. یک مجموعه داده معیار برای ارزیابی ریشه‌یاب‌های عربی Noor-stem v.1 A Benchmark Dataset for Evaluating the Arabic Stemmers 1 1 ریشه‌یابی مرحله اصلی چندین فرایند پردازشی مانند متن‌کاوی، بازیابی اطلاعات و پردازش زبان طبیعی است. ابزارهای تشخیص میانوند کلمات عربی با چالش‌های زیادی روبرو هستند که بیشتر ناشی از ماهیت پیچیده کلمات این زبان و سبک­های نوشتاری متفاوت آن­ها است. تا جایی که ما می­دانیم، هیچ مجموعه داده­ی ریشه­یابیِ معیاری وجود ندارد که طیف گسترده­ای از چالش­های ریشه­یابی را پوشش دهد. بنابراین، ما توسعه یک مجموعه داده برای ارزیابی پایداری ریشه‌یاب‌ها را در چنین موقعیت­های چالش برانگیزی ارزشمند می­دانیم. این مقاله، نور-استم، یک مجموعه داده معیار با سبک‌های نوشتاری مختلف را برای ارزیابی ابزارهای تشخیص میانوند (استم) عربی معرفی ‌می‌کند. جهت تایید عملکرد این دادگان، عملکرد سه ریشه‌یاب‌ عربی (نور ۱۰، NLTK و تاشفین) مورد ارزیابی قرار گرفته است. نتایج نشان می­دهد که سنجه­ی اف در ریشه­یاب تاشفین بهتر از سایر ریشه‌یاب‌ها است که این موضوع در تحقیقات مرتبط نیز مشاهده شده است. 2 The main task of the tokenization is to divide the sentences of the text into its constituent units and remove punctuation marks (dots, commas, etc.). Each unit is a continuous lexical or grammatical writing chain that is an independent semantic unit. Tokenization occurs at the word level and the extracted units can be used as input to other components such as stemmer. Stemming is the main step of several processing tasks such as text mining, information retrieval, and natural language processing.Arabic stemmers face many challenges, mostly caused by the complex nature of Arabic words and their different writing styles. To our knowledge, there is no gold stemming dataset, which contains a wide variety of different possible stemming challenges, so that, stemmers face numerous and different possible real-world challenges to stem the words. Thus, we find it valuable to develop a dataset for evaluating the sustainability of stemmers in such a variety of challenging situations. In this paper, we introduce Noor-Stem, a benchmark dataset with various writing styles for the evaluation of Arabic stemmers. We use two thousand Arabic words in this dataset. We choose the words from different sources such as holy Quran as well as the Arabic websites and assign them to two groups of human experts to determine the correct stem for each word. The first chosen collection of words includes non-repetitive words of the Quran according to their morphological structure. This collection, with more than 16,000 words, is completely by its Quranic usage, labeling only the words stems. The necessity of morphological analysis in Quranic texts as an example of the index of classical Arabic texts has given rise to this evaluation. The second word collection includes 10 thousand words from the non-repetitive words of the text data in general classic Arabic texts. Out of more than 2,600,000 non-repetitive words, considering that the dataset is going to be gold and each stem must be labeled/ensured by a couple of experts, 10,000 words are chosen, regarding the comprehensive and unique patterns to fully measure the length. The variety of patterns can face each stemmer with a serious challenge to demonstrate its performance in various processes. We evaluate the performance of three Arabic stemmers (Light 10, NLTK and Tashaphyne) on this dataset. The results show that the F-measure of Tashaphyne is better than the other stemmers, which re-proves the superiority of this stemmer in this type of problem, as well. 101 112 2021/10/262023/07/12022/11/132022/08/32023/03/72021/04/102020/05/102022/11/2 1401/8/11 2023/02/222024/02/252024/02/252024/02/252024/02/242022/09/242024/02/172023/12/11 1402/9/20 ازل العصواد Azal Al-Aswad دانشگاه علم و صنعت ایران ایران azal.alamery2@gmail.com بهروز مینایی بیدگلی Behrouz Minaei-Bidgoli دانشگاه علم و صنعت ایران ایران b_minaei@iust.ac.ir محمدابراهیم شناسا Mohammad-Ebrahim Shenassa دانشگاه آزاد واحد تهران-شمال ایران me.shenasa@iau-tnb.ac.ir سیدعلی حسینی Sayyed-Ali Hossayni دانشگاه علم و صنعت ایران ایران sayyed.hossayni@yandex.com حبیب سریانی Habib Seryani دانشگاه رازی کرمانشاه ایران hseryani@noornet.net Benchmark Dataset Stemmer Noor-Stem Infix Information Retrieval دادگان معیار ریشه‌یاب نور-استم میانوند استخراج اطلاعات سریانی، حبیب؛ هاشمی، سید محسن. استمر «نور»؛ موتور هوشمند تشخیص میانوند واژگان عربی، هوش مصنوعی و علوم اسلامی، دوره 1، 54 صفحه##N. Y. Habash, "Introduction to Arabic natural language processing," Synth. Lect. Hum. Lang. Technol., vol. 3, no. 1, pp. 1-187, 2010.##H. Alshalabi, S. Tiun, N. Omar, F. N. AL-Aswadi, and K. A. Alezabi, "Arabic light-based stemmer using new rules," J. King Saud Univ. Inf. Sci., 2021.##D. H. Abd, W. Khan, K. A. Thamer, and A. J. Hussain, "Arabic Light Stemmer Based on ISRI Stemmer," in International Conference on Intelligent Computing, 2021, pp. 32-45.##M. N. Al-Kabi, S. A. Kazakzeh, B. M. A. Ata, S. A. Al-Rababah, and I. M. Alsmadi, "A novel root based Arabic stemmer," J. King Saud Univ. Inf. Sci., vol. 27, no. 2, pp. 94-103, 2015.##S. Levin, "Toward Proto-Nostratic: A new approach to the comparison of Proto-Indo-European and Proto-Afroasiatic. By Allan R. Bomhard," Diachronica, vol. 2, no. 1, pp. 97-104, 1985.##R. Mohammed, "New Arabic stemming based on Arabic patterns," Iraqi J. Sci., vol. 57, no. 3C, pp. 2324-2330, 2016.##L. S. Larkey, L. Ballesteros, and M. E. Connell, "Light stemming for Arabic information retrieval," in Arabic computational morphology, Springer, 2007, pp. 221-243.##Y. Jaafar, D. Namly, K. Bouzoubaa, and A. Yousfi, "Enhancing Arabic stemming process using resources and benchmarking tools," J. King Saud Univ. Inf. Sci., vol. 29, no. 2, pp. 164-170, 2017.##R. Mamoun and M. Ahmed, "Arabic text stemming: Comparative analysis," in 2016 Conference of Basic Sciences and Engineering Studies (SGCAC), 2016, pp. 88-93.##R. A. Sameer, "Modified light stemming algorithm for Arabic language," Iraqi J. Sci., vol. 57, no. 1B, pp. 507-513, 2016.##A. Abdelali, K. Darwish, N. Durrani, and H. Mubarak, "Farasa: A fast and furious segmenter for arabic," in Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, 2016, pp. 11-16.##A. Ayedh, G. Tan, K. Alwesabi, and H. Rajeh, "The effect of preprocessing on arabic document categorization," Algorithms, vol. 9, no. 2, p. 27, 2016.##M. Mustafa, A. S. Eldeen, S. Bani-Ahmad, and A. O. Elfaki, "A comparative survey on arabic stemming: approaches and challenges," Intell. Inf. Manag., vol. 9, no. 02, p. 39, 2017.##Y. A. Alhaj, J. Xiang, D. Zhao, M. A. A. Al-Qaness, M. Abd Elaziz, and A. Dahou, "A Study of the Effects of Stemming Strategies on Arabic Document Classification," IEEE Access, vol. 7, pp. 32664-32671, 2019, doi: 10.1109/ACCESS.2019.2903331.##Y. A. Al[1] Y. A. Alhaj, M. A. A. Al-qaness, A. Dahou, M. Abd Elaziz, D. Zhao, and J. Xiang, "Effects of Light Stemming on Feature Extraction and Selection for Arabic Documents Classification," in Studies in Computational Intelligence, vol. 874, Springer, 2020, M. A. A. Al-qaness, A. Dahou, M. Abd Elaziz, D. Zhao, and J. Xiang, "Effects of Light Stemming on Feature Extraction and Selection for Arabic Documents Classification," in Studies in Computational Intelligence, vol. 874, Springer, 2020, pp. 59-79.##F. S. Utomo, N. Suryana, and M. S. Azmi, "Stemming Impact Analysis On Indonesian Quran Translation And Their Tafsir Classification For Ontology Instances," IIUM Eng. J., vol. 21, no. 1, pp. 33-50, 2020.##A. Alnaied, M. Elbendak, and A. Bulbul, "An intelligent use of stemmer and morphology analysis for Arabic information retrieval," Egypt. Informatics J., 2020.##M. El-Defrawy, Y. El-Sonbaty, and N. A. Belal, "Cbas: Context based arabic stemmer," arXiv Prepr. arXiv1611.00027, 2015.##M. Nabil, M. Aly, and A. Atiya, "Labr: A large scale arabic sentiment analysis benchmark," arXiv Prepr. arXiv1411.6718, 2014.##"Building an International Corpus of Arabic (ICA): progress of compilation stage," in 7th international conference on language engineering, Cairo, Egypt, 2007, pp. 5-6.##M. Diab, N. Habash, O. Rambow, and R. Roth, "LDC Arabic treebanks and associated corpora: Data divisions manual," arXiv Prepr. arXiv1309.5652, 2013.##R. M. Sallam, H. M. Mousa, and M. Hussein, "Improving Arabic text categorization using normalization and stemming techniques," Int. J. Comput. Appl, vol. 135, no. 2, pp. 38-43, 2016.##M. I. Eldesouki, W. Arafa, and K. Darwish, "Stemming techniques of Arabic language: Comparative study from the information retrieval perspective," Egypt. Comput. J., vol. 36, no. 1, pp. 30-49, 2009.##F. N. Al-Aswadi, H. Y. Chan, and K. H. Gan, "Automatic ontology construction from text: a review from shallow to deep learning trend," Artif. Intell. Rev., vol. 53, no. 6, pp. 3901-3928, 2020.##T. Kanan, O. Sadaqa, A. Almhirat, and E. Kanan, "Arabic light stemming: A comparative study between p-stemmer, khoja stemmer, and light10 stemmer," in 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), 2019, pp. 511-515.##M. Naili, A. H. Chaibi, and H. H. Ben Ghezala, "Comparative study of Arabic stemming algorithms for topic identification," Procedia Comput. Sci., vol. 159, pp. 794-802, 2019.##سریانی، حبیب؛ هاشمی، سید محسن. استمر «نور»؛ موتور هوشمند تشخیص میانوند واژگان عربی، هوش مصنوعی و علوم اسلامی، دوره 1، 54 صفحه##N. Y. Habash, "Introduction to Arabic natural language processing," Synth. Lect. Hum. Lang. Technol., vol. 3, no. 1, pp. 1-187, 2010.##H. Alshalabi, S. Tiun, N. Omar, F. N. AL-Aswadi, and K. A. Alezabi, "Arabic light-based stemmer using new rules," J. King Saud Univ. Inf. Sci., 2021.##D. H. Abd, W. Khan, K. A. Thamer, and A. J. Hussain, "Arabic Light Stemmer Based on ISRI Stemmer," in International Conference on Intelligent Computing, 2021, pp. 32-45.##M. N. Al-Kabi, S. A. Kazakzeh, B. M. A. Ata, S. A. Al-Rababah, and I. M. Alsmadi, "A novel root based Arabic stemmer," J. King Saud Univ. Inf. Sci., vol. 27, no. 2, pp. 94-103, 2015.##S. Levin, "Toward Proto-Nostratic: A new approach to the comparison of Proto-Indo-European and Proto-Afroasiatic. By Allan R. Bomhard," Diachronica, vol. 2, no. 1, pp. 97-104, 1985.##R. Mohammed, "New Arabic stemming based on Arabic patterns," Iraqi J. Sci., vol. 57, no. 3C, pp. 2324-2330, 2016.##L. S. Larkey, L. Ballesteros, and M. E. Connell, "Light stemming for Arabic information retrieval," in Arabic computational morphology, Springer, 2007, pp. 221-243.##Y. Jaafar, D. Namly, K. Bouzoubaa, and A. Yousfi, "Enhancing Arabic stemming process using resources and benchmarking tools," J. King Saud Univ. Inf. Sci., vol. 29, no. 2, pp. 164-170, 2017.##R. Mamoun and M. Ahmed, "Arabic text stemming: Comparative analysis," in 2016 Conference of Basic Sciences and Engineering Studies (SGCAC), 2016, pp. 88-93.##R. A. Sameer, "Modified light stemming algorithm for Arabic language," Iraqi J. Sci., vol. 57, no. 1B, pp. 507-513, 2016.##A. Abdelali, K. Darwish, N. Durrani, and H. Mubarak, "Farasa: A fast and furious segmenter for arabic," in Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Demonstrations, 2016, pp. 11-16.##A. Ayedh, G. Tan, K. Alwesabi, and H. Rajeh, "The effect of preprocessing on arabic document categorization," Algorithms, vol. 9, no. 2, p. 27, 2016.##M. Mustafa, A. S. Eldeen, S. Bani-Ahmad, and A. O. Elfaki, "A comparative survey on arabic stemming: approaches and challenges," Intell. Inf. Manag., vol. 9, no. 02, p. 39, 2017.##Y. A. Alhaj, J. Xiang, D. Zhao, M. A. A. Al-Qaness, M. Abd Elaziz, and A. Dahou, "A Study of the Effects of Stemming Strategies on Arabic Document Classification," IEEE Access, vol. 7, pp. 32664-32671, 2019, doi: 10.1109/ACCESS.2019.2903331.##Y. A. Al[1] Y. A. Alhaj, M. A. A. Al-qaness, A. Dahou, M. Abd Elaziz, D. Zhao, and J. Xiang, "Effects of Light Stemming on Feature Extraction and Selection for Arabic Documents Classification," in Studies in Computational Intelligence, vol. 874, Springer, 2020, M. A. A. Al-qaness, A. Dahou, M. Abd Elaziz, D. Zhao, and J. Xiang, "Effects of Light Stemming on Feature Extraction and Selection for Arabic Documents Classification," in Studies in Computational Intelligence, vol. 874, Springer, 2020, pp. 59-79.##F. S. Utomo, N. Suryana, and M. S. Azmi, "Stemming Impact Analysis On Indonesian Quran Translation And Their Tafsir Classification For Ontology Instances," IIUM Eng. J., vol. 21, no. 1, pp. 33-50, 2020.##A. Alnaied, M. Elbendak, and A. Bulbul, "An intelligent use of stemmer and morphology analysis for Arabic information retrieval," Egypt. Informatics J., 2020.##M. El-Defrawy, Y. El-Sonbaty, and N. A. Belal, "Cbas: Context based arabic stemmer," arXiv Prepr. arXiv1611.00027, 2015.##M. Nabil, M. Aly, and A. Atiya, "Labr: A large scale arabic sentiment analysis benchmark," arXiv Prepr. arXiv1411.6718, 2014.##"Building an International Corpus of Arabic (ICA): progress of compilation stage," in 7th international conference on language engineering, Cairo, Egypt, 2007, pp. 5-6.##M. Diab, N. Habash, O. Rambow, and R. Roth, "LDC Arabic treebanks and associated corpora: Data divisions manual," arXiv Prepr. arXiv1309.5652, 2013.##R. M. Sallam, H. M. Mousa, and M. Hussein, "Improving Arabic text categorization using normalization and stemming techniques," Int. J. Comput. Appl, vol. 135, no. 2, pp. 38-43, 2016.##M. I. Eldesouki, W. Arafa, and K. Darwish, "Stemming techniques of Arabic language: Comparative study from the information retrieval perspective," Egypt. Comput. J., vol. 36, no. 1, pp. 30-49, 2009.##F. N. Al-Aswadi, H. Y. Chan, and K. H. Gan, "Automatic ontology construction from text: a review from shallow to deep learning trend," Artif. Intell. Rev., vol. 53, no. 6, pp. 3901-3928, 2020.##T. Kanan, O. Sadaqa, A. Almhirat, and E. Kanan, "Arabic light stemming: A comparative study between p-stemmer, khoja stemmer, and light10 stemmer," in 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS), 2019, pp. 511-515.##M. Naili, A. H. Chaibi, and H. H. Ben Ghezala, "Comparative study of Arabic stemming algorithms for topic identification," Procedia Comput. Sci., vol. 159, pp. 794-802, 2019.## ##

بهبود پالایش مشارکتی در سیستم‌های توصیه‌گر با کمک خوشه‌بندی فازی C– میانگین مرتب‌شده و الگوریتم ازدحام ذرات تطبیقی – آشوبی Improving Collaborative Recommender Systems by Integrating Fuzzy C-Ordered Means Clustering and Chaotic Self-Adaptive Particle Swarm Optimization Algorithm 1 1 سیستم‌های توصیه‌گر زیرمجموعه‌ای از سیستم‌های هوشمند پالایش اطلاعات هستند که در فضای اینترنت علایق کاربر را شناسایی نموده و توصیه‌های مرتبط با سلیقه‌ی کاربر را ارائه می‌دهند. پالایش مشارکتی مبتنی بر کاربر، از مهم‌ترین انواع سیستم‌های توصیه‌گر است. از مهم‌ترین چالش‌ها در این سیستم‌ها پراکندگی و حجم زیاد داده‌ها است که بر کارایی آن‌ها اثرگذار است. در روش پیشنهادی، برای اولین بار از الگوریتم خوشه‌بندی فازی C-میانگین مرتب‌شده و الگوریتم‌ تکاملی ازدحام ذرات تطبیقی آشوبی برای خوشه‌بندی کاربران استفاده‌ شده ‌است. هدف روش پیشنهادی بهبود میزان خطای پیش‌بینی در مجموعه داده‌های حجیم با پراکندگی زیاد و کاهش تأثیر داده های پرت و نویز است. به منظور ارزیابی و اثبات کارایی روش پیشنهادی، آزمایش‌هایی روی پایگاه داده‌های واقعی اجرا شده‌ است. نتایج آزمایش‌ها نشان‌دهنده‌ی برتری روش پیشنهادی نسبت به روش‌های مرز دانش بر اساس معیارهای میانگین خطای مطلق، جذر میانگین مربعات خطا، نرخ صحت و زمان محاسباتی است.   2 Recommender systems are a subset of intelligent information filtering systems that discovers user interests and provide user-friendly recommendations. User-based collaborative filtering recommender systems is one of the most important types of recommender systems. However, they are faced with voluminous data and sparsity problems that have negative effects on the performance of the systems. In the proposed method, fuzzy C-ordered means clustering algorithm is integrated with a chaotic self-adaptive particle swarm evolutionary algorithm for clustering users. The proposed method aims to improve the rating prediction in large sparse datasets and reduce the negative impact of outliers and noisy data. Experiments have been conducted on real-world datasets to evaluate and prove the efficiency of the proposed method. Experimental results show the superiority of the proposed method that the state-of-the-art methods based on prediction error criteria, accuracy rates, and the computational time. 113 124 2021/10/262023/07/12022/11/132022/08/32023/03/72021/04/102020/05/102022/11/22020/03/30 1399/1/11 2023/02/222024/02/252024/02/252024/02/252024/02/242022/09/242024/02/172023/12/112024/02/17 1402/11/28 جواد حمیدزاده Javad Hamidzadeh دانشگاه صنعتی سجاد ایران J_Hamidzadeh@sadjad.ac.ir منا مرادی Mona Moradi دانشگاه سمنان ایران Mmoradi@semnan.ac.ir Recommender systems Collaborative filtering Fuzzy clustering Evolutionary algorithm Chaotic self-adaptive particle swarm optimization algorithm. سیستم توصیه‌گر پالایش مشارکتی خوشه‌بندی فازی الگوریتم تکاملی الگوریتم ازدحام ذرات تطبیقی آشوبی P. Lops, M. Polignano, C. Musto, A. Silletti, and G. Semeraro, "ClayRS: An end-to-end framework for reproducible knowledge-aware recommender systems," Information Systems, vol. 119, p. 102273, 2023, doi: ##https://doi.org/10.1016/j.is.2023.102273##J.-C. Zhang, A. M. Zain, K.-Q. Zhou, X. Chen, and R.-M. Zhang, "A review of recommender systems based on knowledge graph embedding," Expert Systems with Applications, vol. 250, p. 123876, 2024, doi: ##https://doi.org/10.1016/j.eswa.2024.123876##[A. Ghannadrad, M. Arezoumandan, L. Candela, and D. Castelli, "Recommender systems for science: A basic taxonomy," 2022, pp. 1-8.##K. Saini and A. Singh, "A content-based recommender system using stacked LSTM and an attention-based autoencoder," Measurement: Sensors, p. 100975, 2023, doi: https://doi.org/ 10.1016/ j.measen.2023.100975.##https://doi.org/10.1016/j.measen.2023.100975##[P. B. Thorat, R. Goudar, and S. Barve, "Survey on collaborative filtering, content-based filtering and hybrid recommendation system," International Journal of Computer Applications, vol. 110, no. 4, 2015.##[M. Moradi and J. Hamidzadeh, "Ensemble-based Top-k Recommender System Considering Incomplete Data," Journal of AI and Data Mining, 2019.##[X. Su and T. M. Khoshgoftaar, "A survey of collaborative filtering techniques," Advances in artificial intelligence, vol. 2009, 2009.##C. C. Aggarwal, Recommender systems. Springer, 2016.##p. bahrani, B. Minaei Bidgoli, H. Parvin, M. Mirzarezaee, and A. Keshavarz, "An Ontological Hybrid Recommender System for Dealing with Cold Start Problem," jsdp, vol. 19, no. 1, pp. 1-18, 2022, doi: 10.52547/jsdp.19.1.1.##R. Yera and L. Martinez, "Fuzzy tools in recommender systems: A survey," International Journal of Computational Intelligence Systems, vol. 10, no. 1, pp. 776-803, 2017.##E. G. Muñoz, J. Parraga-Alava, J. Meza, J. J. Proaño Morales, and S. Ventura, "Housing fuzzy recommender system: A systematic literature review," Heliyon, vol. 10, no. 5, p. e26444, 2024, doi: ##https://doi.org/10.1016/j.heliyon.2024.e26444##F. Houshmand-Nanehkaran, S. M. Lajevardi, and M. Mahlouji-Bidgholi, "Optimization of fuzzy similarity by genetic algorithm in user-based collaborative filtering recommender systems," Expert Systems, vol. 39, no. 4, p. e12893, 2022, doi: ##https://doi.org/10.1111/exsy.12893##m. robati anaraki and n. riahi, "An evolutionary approach for automating the selection of optimum Algorithm in Collaborative Filtering Recommender Systems," jsdp, vol. 20, no. 1, pp. 59-78, 2023. [Online]. Available: http://jsdp.rcisp.ac.ir/article-1-1206-en.html.##H. Koohi and K. Kiani, "User based Collaborative Filtering using fuzzy C-means," Measurement, vol. 91, pp. 134-139, 2016.##R. Katarya and O. P. Verma, "An effective web page recommender system with fuzzy c-mean clustering," Multimedia Tools and Applications, vol. 76, no. 20, pp. 21481-21496, 2017.##X. Li and D. Li, "Improved Hybrid Collaborative Filtering Algorithm Based on K-Means," in International Conference on Applications and Techniques in Cyber Security and Intelligence, 2018: Springer, pp. 928-934.##N. K. Manikandan and M. Kavitha, "A content recommendation system for e-learning using enhanced Harris Hawks Optimization, Cuckoo search and DSSM," Journal of Intelligent & Fuzzy Systems, no. Preprint, pp. 1-14, 2023.##B. Alhijawi and Y. Kilani, "A collaborative filtering recommender system using genetic algorithm," Information Processing & Management, vol. 57, no. 6, p. 102310, 2020, doi: ##https://doi.org/10.1016/j.ipm.2020.102310##R. Paulavičius, L. Stripinis, S. Sutavičiūtė, D. Kočegarov, and E. Filatovas, "A novel greedy genetic algorithm-based personalized travel recommendation system," Expert Systems with Applications, vol. 230, p. 120580, 2023.##R. Katarya and O. P. Verma, "A collaborative recommender system enhanced with particle swarm optimization technique," Multimedia Tools and Applications, vol. 75, no. 15, pp. 9225-9239, 2016.##R. Katarya, "Movie recommender system with metaheuristic artificial bee," Neural Computing and Applications, vol. 30, no. 6, pp. 1983-1990, 2018.##R. Katarya and O. P. Verma, "Recommender system with grey wolf optimizer and FCM," Neural Computing and Applications, vol. 30, no. 5, pp. 1679-1687, 2018.##R. Katarya and O. P. Verma, "An effective collaborative movie recommender system with cuckoo search," Egyptian Informatics Journal, vol. 18, no. 2, pp. 105-112, 2017.##S. Yadav and S. Nagpal, "An Improved Collaborative Filtering Based Recommender System using Bat Algorithm," Procedia computer science, vol. 132, pp. 1795-1803, 2018.##V. Vellaichamy and V. Kalimuthu, "Hybrid Collaborative Movie Recommender System Using Clustering and Bat Optimization," International Journal of Intelligent Engineering and Systems, vol. 10, no. 5, pp. 38-47, 2017.##J. M. Leski, "Fuzzy c-ordered-means clustering," Fuzzy Sets and Systems, vol. 286, pp. 114-133, 2016.##J. Kennedy and R. Eberhart, "Particle swarm optimization," in Proceedings of ICNN'95 - International Conference on Neural Networks, 27 Nov.-1 Dec. 1995 1995, vol. 4, pp. 1942-1948 vol.4, doi: 10.1109/ICNN.1995.488968.##Z.-H. Zhan, J. Zhang, Y. Li, and H. S.-H. Chung, "Adaptive particle swarm optimization," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39, no. 6, pp. 1362-1381, 2009.##Y. Shi and R. C. Eberhart, "Fuzzy adaptive particle swarm optimization," in Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No. 01TH8546), 2001, vol. 1: IEEE, pp. 101-106.##R. Sadeghi and J. Hamidzadeh, "Automatic support vector data description," Soft Computing, vol. 22, no. 1, pp. 147-158, 2018.##J. Hamidzadeh, R. Sadeghi, and N. Namaei, "Weighted support vector data description based on chaotic bat algorithm," Applied Soft Computing, vol. 60, pp. 540-551, 2017.##B. Alatas, E. Akin, and A. B. Ozer, "Chaos embedded particle swarm optimization algorithms," Chaos, Solitons & Fractals, vol. 40, no. 4, pp. 1715-1734, 2009.##R. Logesh, V. Subramaniyaswamy, V. Vijayakumar, X.-Z. Gao, and V. Indragandhi, "A hybrid quantum-induced swarm intelligence clustering for the urban trip recommendation in smart city," Future Generation Computer Systems, vol. 83, pp. 653-673, 2018.##P. Lops, M. Polignano, C. Musto, A. Silletti, and G. Semeraro, "ClayRS: An end-to-end framework for reproducible knowledge-aware recommender systems," Information Systems, vol. 119, p. 102273, 2023, doi: ##https://doi.org/10.1016/j.is.2023.102273##J.-C. Zhang, A. M. Zain, K.-Q. Zhou, X. Chen, and R.-M. Zhang, "A review of recommender systems based on knowledge graph embedding," Expert Systems with Applications, vol. 250, p. 123876, 2024, doi: ##https://doi.org/10.1016/j.eswa.2024.123876##[A. Ghannadrad, M. Arezoumandan, L. Candela, and D. Castelli, "Recommender systems for science: A basic taxonomy," 2022, pp. 1-8.##K. Saini and A. Singh, "A content-based recommender system using stacked LSTM and an attention-based autoencoder," Measurement: Sensors, p. 100975, 2023, doi: https://doi.org/ 10.1016/ j.measen.2023.100975.##https://doi.org/10.1016/j.measen.2023.100975##[P. B. Thorat, R. Goudar, and S. Barve, "Survey on collaborative filtering, content-based filtering and hybrid recommendation system," International Journal of Computer Applications, vol. 110, no. 4, 2015.##[M. Moradi and J. Hamidzadeh, "Ensemble-based Top-k Recommender System Considering Incomplete Data," Journal of AI and Data Mining, 2019.##[X. Su and T. M. Khoshgoftaar, "A survey of collaborative filtering techniques," Advances in artificial intelligence, vol. 2009, 2009.##C. C. Aggarwal, Recommender systems. Springer, 2016.##p. bahrani, B. Minaei Bidgoli, H. Parvin, M. Mirzarezaee, and A. Keshavarz, "An Ontological Hybrid Recommender System for Dealing with Cold Start Problem," jsdp, vol. 19, no. 1, pp. 1-18, 2022, doi: 10.52547/jsdp.19.1.1.##R. Yera and L. Martinez, "Fuzzy tools in recommender systems: A survey," International Journal of Computational Intelligence Systems, vol. 10, no. 1, pp. 776-803, 2017.##E. G. Muñoz, J. Parraga-Alava, J. Meza, J. J. Proaño Morales, and S. Ventura, "Housing fuzzy recommender system: A systematic literature review," Heliyon, vol. 10, no. 5, p. e26444, 2024, doi: ##https://doi.org/10.1016/j.heliyon.2024.e26444##F. Houshmand-Nanehkaran, S. M. Lajevardi, and M. Mahlouji-Bidgholi, "Optimization of fuzzy similarity by genetic algorithm in user-based collaborative filtering recommender systems," Expert Systems, vol. 39, no. 4, p. e12893, 2022, doi: ##https://doi.org/10.1111/exsy.12893##m. robati anaraki and n. riahi, "An evolutionary approach for automating the selection of optimum Algorithm in Collaborative Filtering Recommender Systems," jsdp, vol. 20, no. 1, pp. 59-78, 2023. [Online]. Available: http://jsdp.rcisp.ac.ir/article-1-1206-en.html.##H. Koohi and K. Kiani, "User based Collaborative Filtering using fuzzy C-means," Measurement, vol. 91, pp. 134-139, 2016.##R. Katarya and O. P. Verma, "An effective web page recommender system with fuzzy c-mean clustering," Multimedia Tools and Applications, vol. 76, no. 20, pp. 21481-21496, 2017.##X. Li and D. Li, "Improved Hybrid Collaborative Filtering Algorithm Based on K-Means," in International Conference on Applications and Techniques in Cyber Security and Intelligence, 2018: Springer, pp. 928-934.##N. K. Manikandan and M. Kavitha, "A content recommendation system for e-learning using enhanced Harris Hawks Optimization, Cuckoo search and DSSM," Journal of Intelligent & Fuzzy Systems, no. Preprint, pp. 1-14, 2023.##B. Alhijawi and Y. Kilani, "A collaborative filtering recommender system using genetic algorithm," Information Processing & Management, vol. 57, no. 6, p. 102310, 2020, doi: ##https://doi.org/10.1016/j.ipm.2020.102310##R. Paulavičius, L. Stripinis, S. Sutavičiūtė, D. Kočegarov, and E. Filatovas, "A novel greedy genetic algorithm-based personalized travel recommendation system," Expert Systems with Applications, vol. 230, p. 120580, 2023.##R. Katarya and O. P. Verma, "A collaborative recommender system enhanced with particle swarm optimization technique," Multimedia Tools and Applications, vol. 75, no. 15, pp. 9225-9239, 2016.##R. Katarya, "Movie recommender system with metaheuristic artificial bee," Neural Computing and Applications, vol. 30, no. 6, pp. 1983-1990, 2018.##R. Katarya and O. P. Verma, "Recommender system with grey wolf optimizer and FCM," Neural Computing and Applications, vol. 30, no. 5, pp. 1679-1687, 2018.##R. Katarya and O. P. Verma, "An effective collaborative movie recommender system with cuckoo search," Egyptian Informatics Journal, vol. 18, no. 2, pp. 105-112, 2017.##S. Yadav and S. Nagpal, "An Improved Collaborative Filtering Based Recommender System using Bat Algorithm," Procedia computer science, vol. 132, pp. 1795-1803, 2018.##V. Vellaichamy and V. Kalimuthu, "Hybrid Collaborative Movie Recommender System Using Clustering and Bat Optimization," International Journal of Intelligent Engineering and Systems, vol. 10, no. 5, pp. 38-47, 2017.##J. M. Leski, "Fuzzy c-ordered-means clustering," Fuzzy Sets and Systems, vol. 286, pp. 114-133, 2016.##J. Kennedy and R. Eberhart, "Particle swarm optimization," in Proceedings of ICNN'95 - International Conference on Neural Networks, 27 Nov.-1 Dec. 1995 1995, vol. 4, pp. 1942-1948 vol.4, doi: 10.1109/ICNN.1995.488968.##Z.-H. Zhan, J. Zhang, Y. Li, and H. S.-H. Chung, "Adaptive particle swarm optimization," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 39, no. 6, pp. 1362-1381, 2009.##Y. Shi and R. C. Eberhart, "Fuzzy adaptive particle swarm optimization," in Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No. 01TH8546), 2001, vol. 1: IEEE, pp. 101-106.##R. Sadeghi and J. Hamidzadeh, "Automatic support vector data description," Soft Computing, vol. 22, no. 1, pp. 147-158, 2018.##J. Hamidzadeh, R. Sadeghi, and N. Namaei, "Weighted support vector data description based on chaotic bat algorithm," Applied Soft Computing, vol. 60, pp. 540-551, 2017.##B. Alatas, E. Akin, and A. B. Ozer, "Chaos embedded particle swarm optimization algorithms," Chaos, Solitons & Fractals, vol. 40, no. 4, pp. 1715-1734, 2009.##R. Logesh, V. Subramaniyaswamy, V. Vijayakumar, X.-Z. Gao, and V. Indragandhi, "A hybrid quantum-induced swarm intelligence clustering for the urban trip recommendation in smart city," Future Generation Computer Systems, vol. 83, pp. 653-673, 2018.## ##

بهبود قدرت تعمیم مدل‌های تشخیص کلام نفرت‌انگیز مبتنی بر تطبیق دامنه Domain adaptation-based method for improving generalization of hate speech detection models 1 1 امروزه با رشد فعالیت در شبکه‌های اجتماعی شاهد افزایش کلام نفرت‌انگیز به صورت برخط هستیم و به‌همین منظور مسئلۀ تشخیص نفرت در فضای مجازی دارای اهمیت است. همچنین تطبیق دامنه نیز در این مسئله و به‌طورکلی در حوزۀ پردازش زبان طبیعی، یکی از چالش‌های مهم است. در بسیاری از مسائل، ضمن تغییر دامنه با افت عملکرد مواجهیم که این موضوع در مسئلۀ نفرت نیز صادق است. در این پژوهش با استفاده از روش‌های تطبیق دامنه سعی در افزایش قدرت تعمیم‌پذیری مدل‌های تشخیص نفرت خواهیم داشت. برای این منظور روش‌های مبتنی بر ترنسفورمر شامل آموزش خصمانۀ دامنه و ترکیب متخصصان را به کار می‌گیریم و همچنین از آموزش چند منبعی استفاده می‌کنیم. آزمایش‌ها با استفاده از چهار مجموعه‌داده در حوزۀ نفرت انجام می‌شوند. در ابتدا مد‌ل‌ها را به‌صورت درون‌ دامنه‌ای و تک منبعی ارزیابی می‌کنیم. در مرحلۀ بعد با اضافه کردن دامنه‌های دیگر به بخش آموزش، شاهد افت نتایج و انتقال منفی هستیم. سپس آزمایش‌های برون دامنه‌ای را ابتدا به‌صورت تک منبعی با مدل DistilBERT انجام می‌دهیم که با تغییر دامنه نتایج به طور قابل توجهی کاهش می‌یابند. به‌منظور افزایش قدرت تطبیق دامنۀ مدل‌ در بخش برون دامنه‌ای، روی چند منبع آموزش را انجام می‌دهیم که حدوداً در نیمی از موارد سبب بهبود نتایج می‌شود که نتیجۀ معناداری نیست. در ادامه با استفاده از روش‌های مبتنی بر ترنسفورمر شامل آموزش خصمانۀ دامنه و ترکیب متخصصان سعی در افزایش قدرت تطبیق دامنۀ مدل‌ها خواهیم داشت که در 87% از آزمایش‌های برون دامنه‌ای چند منبعی شاهد افزایش عملکرد هستیم. البته این روش‌ها در عملکرد آزمایش‌های درون دامنه‌ای هم مؤثر هستند. مسئلۀ مهمی که گاهی موجب افت‌وخیز چشمگیر نتایج می‌شود، مجموعه‌داده‌ها هستند. شباهت داده‌ها و تشابه توزیع بعضی دامنه‌ها باعث افزایش قدرت تطبیق دامنۀ مدل می‌شوند.   2 Today, with the growth of activity in social media, we see an increase in hate speech online and for this reason, the issue of recognizing hate in cyberspace is important. Also, domain adaptation is one of the important challenges in this task and in general in the field of natural language processing. In many issues, while changing the domain, we face a drop in performance, which is also true in the task hate speech. In this research, we try to increase the generalizability of hate detection models by using domain adaptation methods. For this purpose, we use Transformer-based methods, including domain adversarial training and mixture of experts, and we also use multi-source training. Experiments are conducted using four datasets in the domain of hate. At first, we evaluate the models in an in-domain and single-source manner. In the next step, by adding other domains to the education section, we see a drop in results and a negative transfer. Then we perform the out-of-domain tests first as a single source with the DistilBERT model, which significantly reduces the results by changing the domain. In order to increase the power of domain adaptation of the model in the out-of-domain part, we perform the training on several sources, leads to improve the results in about half of the cases, which is not significant. In the following, we try to increase the domain adaptation power of the models, using transformer-based methods including domain adversarial training and the mixture of experts, which leads to increase in performance in 87% of multi-source out-of-domain tests. Of course, these methods are also effective in the performance of in-domain tests. An important issue that sometimes causes a significant drop in results is datasets. The similarity of the data and the similarity of the distribution of some domains increase the power of domain adaptation of the model and on the contrary. 125 142 2021/10/262023/07/12022/11/132022/08/32023/03/72021/04/102020/05/102022/11/22020/03/302022/09/24 1401/7/2 2023/02/222024/02/252024/02/252024/02/252024/02/242022/09/242024/02/172023/12/112024/02/172024/02/25 1402/12/6 سیده فاطمه نوراللهی Seyedeh Fatemeh Nourollahi دانشگاه قم ایران sfn1373@gmail.com راضیه برادران Razieh Baradaran دانشگاه قم ایران r.baradaran@stu.qom.ac.ir حسین امیرخانی Hossein Amirkhani دانشگاه قم ایران amirkhani@qom.ac.ir hate speech classification transformer domain adaptation generalization کلام نفرت‌انگیز تطبیق دامنه تعمیم طبقه‌بندی ترنسفورمر H. Almerekhi, H. Kwak, B. J. Jansen and J. Salminen, "Detecting Toxicity Triggers in Online Discussions," in Proceedings of the 30th ACM Conference on Hypertext and Social Media, Hof, Germany, pp. 291-292, September 2019.##A. Arango, J. Pérez and B. Poblete, "Hate Speech Detection Is Not as Easy as You May Think: A Closer Look at Model Validation," in, Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, pp. 45-54, July 2019.##P. Badjatiya, S. Gupta, M. Gupta, V. Varma, "Deep learning for hate speech detection in tweets," in Proceedings of the 26th International Conference on World Wide Web Companion, Perth Australia, pp. 759-760, April 2017.##S. Benesch, "Dangerous Speech: A Proposal to Prevent Group Violence," in Voices That Poison: Dangerous Speech Project Proposal Paper, 2013.##J. Blitzer, R. McDonald and F. Pereira, "Domain Adaptation with Structural Correspondence Learning," in Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, pp. 120-128, July 2006.##P. Burnap and M. Williams, "Us and Them: Identifying Cyber Hate on Twitter Across Multiple Protected Characteristics," EPJ Data Science, vol. 5, pp. 1-15, March 2016.##R. Cohen-Almagor, "Fighting Hate and Bigotry on the Internet," Policy & Internet, vol. 3, pp. 1-26, August 2011.##H. Daume III, "Frustratingly Easy Domain Adaptation," in Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic, pp. 256-263, June 2007.##T. Davidson, D. Warmsley, M. Macy and I. Weber, "Automated Hate Speech Detection and the Problem of Offensive Language," in International AAAI Conference on Web and Social Media, Montreal, Canada, vol. 11, pp. 512-515, May 2017.##J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, pp. 4171-4186, June 2019.##J. Donahue, J. Hoffman, E. Rodner, K. Saenko and T. Darrell, "Semi-supervised Domain Adaptation with Instance Constraints," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, pp. 668-675, June 2013.##R. Faris, A. Ashar, U. Gasser and D. Joo, "Understanding Harmful Speech Online," Berkman Klein Center Research Publication, vol. 21, December 2016.##J. R. Finkel and C. D. Manning, "Hierarchical Bayesian Domain Adaptation," in Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, USA, pp. 602-610, June 2009.##P. Fortuna, J. Soler-Company and L. Wanner, "How Well Do Hate Hpeech, Toxicity, Abusive and Offensive Language Classification Models Generalize Across Datasets?," Information Processing & Management, vol. 58, pp. 102524, May 2021.##Y. Ganin and V. Lempitsky, "Unsupervised Domain Adaptation by Backpropagation," in Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France, pp. 1180-1189, June 2015.##A. Gretton, K. Borgwardt, M. Rasch, B. Scholkopf and A. J. Smola, "A Kernel Method for the Two-Sample-Problem," Advances in neural information processing systems, vol. 8, pp. 513-520, December 2007.##T. Gröndahl, L. Pajola, M. Juuti, M. Conti and N. Asokan, "All You Need Is "Love": Evading Hate Speech Detection," in AISec '18, Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security, Toronto, Canada, pp. 2-12, January 2018.##T. Gui, Q. Zhang, H. Huang, M. Peng and X. Huang, "Part-of-Speech Tagging for Twitter with Adversarial Neural Networks," in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp. 2411-2420, September 2017.##J. Guo, D. Shah and R. Barzilay,"Multi-Source Domain Adaptation with Mixture of Experts," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 4694-4703, 2018.##J. Sorensen, J. Elliott, L. Dixon, M. McDonald and W. Cukierski, "Toxic Comment Classification Challenge," kaggle.com, Mar. 21, 2018. [Online]. Available: https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/overview/citation.##M. Karan and J. Šnajder, "Cross-Domain Detection of Abusive Language Online," in Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), Brussels, Belgium, pp. 132-137, October 2018.##Y. Kim, K. Stratos and D. Kim, "Domain Attention With an Ensemble of Experts," in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, pp. 643-653, July 2017.##B. Kulis, K. Saenko and T. Darrell, "What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms," in Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Los Alamitos, USA, pp. 1785-1792, June 2011.##Y. Li, T. Baldwin and T. Cohn, "What's in a Domain? Learning Domain-Robust Text Representations Using Adversarial Training," in Proceedings of the 2018 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, USA, pp. 474-479, June 2018.##Z. C. Lipton, Y. Wang and A. J. Smola, "Detecting and Correcting for Label Shift with Black Box Predictors," in Proceedings of the 35th International Conference on Machine Learning, pp. 3128-3136, July 2018.##X. Ma, P. Xu, Z. Wang, R. Nallapati. and B. Xiang, "Domain Adaptation with BERT-based Domain Classification and Data Selection," in Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP, Hong Kong, China, pp. 76-83, November 2019.##M. Mozafari, R. Farahbakhsh and N. Crespi, "A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media," in Proceedings of the Eighth International Conference on Complex Networks and Their Applications COMPLEX NETWORKS 2019, Cham, Germany, pp. 928-940, December 2019.##E. W. Pamungkas and V. Patti, "Cross-Domain and Cross-Lingual Abusive Language Detection: A Hybrid Approach with Deep Learning and a Multilingual Lexicon," in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, Florence, Italy, pp. 363-370, July 2019.##B. Parekh, "Is There a Case for Banning Hate Speech?," in The Content and Context of Hate Speech: Rethinking Regulation and Responses, M. Herz and P. Molnar, Cambridge: Cambridge University Press, pp. 37-56, 2012.##B. Plank, "Domain Adaptation for Parsing", Ph.D. Thesis, University of Groningen, 2011.##A. Ramponi and B. Plank, "Neural Unsupervised Domain Adaptation in NLP-A Survey," in The 28th International Conference on Computational Linguistics, Barcelona, Spain, pp. 6838-6855, December 2020.##J. Salminen, H. Almerekhi, M. Milenkovic, S. Jung, J. An, H. Kwak and B. Jansen, "Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for Identifying and Classifying Hate in Online News Media," in Proceedings of the twelfth International Conference on Web and Social Media, ICWSM 2018, Palo Alto, USA, vol. 12, pp. 330-339, June 2018.##J. Salminen, M. Hopf, S. A. Chowdhury, S. g. Jung, H. Almerekhi and B. J. Jansen, "Developing an Online Hate Classifier for Multiple Social Media Platforms," Human-Centric Computing and Information Sciences, vol. 10, pp. 1-34, January 2020.##V. Sanh, L. Debut, J. Chaumond and T. Wolf, "DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter," in CoRR, 2019.##M. S. Jahan and M. Oussalah, "A Systematic Review of Hate Speech Automatic Detection Using Natural Language Processing," in arXiv, May 2021.##A. Sellars, "Defining Hate Speech," Berkman Klein Center Research Publication, Boston Univ. School of Law, Public Law Research Paper, 2016.##A. A. Siegel, "Online Hate Speech," in Social Media and Democracy: The State of the Field, Prospects for Reform, N. Persily and J. A. Tucker, Cambridge, UK: Cambridge University Press, pp. 56-58, 2020.##B. Sun, J. Feng and K. Saenko, "Return of Frustratingly Easy Domain Adaptation," in Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, USA, vol. 30, pp. 2058-2065, March 2016.##S. D. Swamy, A. Jamatia and B. Gambäck, "Studying Generalisability Across Abusive Language Detection Datasets," in Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), Hong Kong, China, pp. 940-950, November 2019.##A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is All You Need," Advances in Neural Information Processing Systems, vol. 11, pp. 5998-6008, December 2017.##Z. Waseem, J. Thorne and J. Bingel, "Bridging the Gaps: Multi Task Learning for Domain Transfer of Hate Speech Detection," in Online Harassment, pp. 29-55, July 2018.##M. Wiegand, M. Siegel and J. Ruppenhofer, "Overview of the germeval 2018 shared task on the identification of offensive language," in Proceedings of the GermEval 2018 Workshop, September 2018.##D. Wright and I. Augenstein, "Transformer Based Multi-Source Domain Adaptation," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7963-7974, November 2020.##T. Yao, Y. Pan, C. Ngo, H. Li and T. Mei, "Semi-supervised Domain Adaptation with Subspace Learning for Visual Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 2142-2150, June 2015.##M. Zampieri, P. Nakov, S. Rosenthal, P. Atanasova, G. Karadzhov, H. Mubarak, L. Derczynski, Z. Pitenis and Ç. Çöltekin, "SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)," in Proceedings of the Fourteenth Workshop on Semantic Evaluation, Barcelona, Spain, pp. 1425-1447, December 2020.##M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra and R. Kumar, "SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)," in Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, USA, pp. 75-86, June 2019.##P. Rajpurkar, J. Zhang, K. Lopyrev and P. Liang, "SQuAD: 100,000+ Questions for Machine Comprehension of Text," in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, October 2016,##O. de Gibert, N. Pérez, A. García-Pablos and M. Cuadros, "Hate Speech Dataset from a White Supremacy Forum," in Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pp. 11-20, September 2018.‌##R. Kumar, A. K. Ojha, S. Malmasi and M. Zampieri, "Benchmarking Aggression Identification in Social Media," in Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 1-11, August 2018.##Z. Waseem and D. Hovy, "Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter," in Proceedings of the NAACL Student Research Workshop, pp. 88-93, June 2016.‌##T. Consigny, "Sesame Street Ensemble: A Mixture of DistiIBERT Experts," 2021.##M. Artetxe, S. Bhosale, N. Goyal, T. Mihaylov, M. Ott, S. Shleifer, X. V. Lin, J. Du, S. Lyer, R. Pasunuru, G. Anantharaman, X. Li, S. Chen, H. Akin, M. Baines, L. Martin, X. Zhou, P. Singh Koura, B. O'Horo, J. Wang, L. Zettlemoyer, M. Diab, Z. Kozareva and V. Stoyanov, "Efficient Large Scale Language Modeling with Mixtures of Experts," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11699-11732, December 2022.##A. Repetto, "Neural Networks: A Mixture of Experts with Attention," towardsdatascience.com, Jul. 23, 2017. [Online]. Available: https://towardsdatascience.com/neural-networks-a-mixture-of-experts-with-attention-30e196657065.##R. A. Jacobs, M. I. Jordan, S. J. Nowlan and G. E. Hinton, "Adaptive Mixtures of Local Experts," in Neural Computation, vol. 3, pp. 79-87, March 1991.##N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer," in International Conference on Learning Representations, November 2016.‌##W. Fedus, B. Zoph, and N. Shazeer, "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity," in Journal of Machine Learning Research, vol. 23, pp. 1-39, 2022.‌##Z. Du, J. Li, H. Su, L. Zhu, and K. Lu, "Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3937-3946, June 2021.##Q. Ye, J. Zha, and X. Ren, "Eliciting Transferability in Multi-Task Learning with Task-Level Mixture-of-Experts," in arXiv, 2022.‌##H. Almerekhi, H. Kwak, B. J. Jansen and J. Salminen, "Detecting Toxicity Triggers in Online Discussions," in Proceedings of the 30th ACM Conference on Hypertext and Social Media, Hof, Germany, pp. 291-292, September 2019.##A. Arango, J. Pérez and B. Poblete, "Hate Speech Detection Is Not as Easy as You May Think: A Closer Look at Model Validation," in, Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France, pp. 45-54, July 2019.##P. Badjatiya, S. Gupta, M. Gupta, V. Varma, "Deep learning for hate speech detection in tweets," in Proceedings of the 26th International Conference on World Wide Web Companion, Perth Australia, pp. 759-760, April 2017.##S. Benesch, "Dangerous Speech: A Proposal to Prevent Group Violence," in Voices That Poison: Dangerous Speech Project Proposal Paper, 2013.##J. Blitzer, R. McDonald and F. Pereira, "Domain Adaptation with Structural Correspondence Learning," in Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Australia, pp. 120-128, July 2006.##P. Burnap and M. Williams, "Us and Them: Identifying Cyber Hate on Twitter Across Multiple Protected Characteristics," EPJ Data Science, vol. 5, pp. 1-15, March 2016.##R. Cohen-Almagor, "Fighting Hate and Bigotry on the Internet," Policy & Internet, vol. 3, pp. 1-26, August 2011.##H. Daume III, "Frustratingly Easy Domain Adaptation," in Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic, pp. 256-263, June 2007.##T. Davidson, D. Warmsley, M. Macy and I. Weber, "Automated Hate Speech Detection and the Problem of Offensive Language," in International AAAI Conference on Web and Social Media, Montreal, Canada, vol. 11, pp. 512-515, May 2017.##J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, pp. 4171-4186, June 2019.##J. Donahue, J. Hoffman, E. Rodner, K. Saenko and T. Darrell, "Semi-supervised Domain Adaptation with Instance Constraints," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, USA, pp. 668-675, June 2013.##R. Faris, A. Ashar, U. Gasser and D. Joo, "Understanding Harmful Speech Online," Berkman Klein Center Research Publication, vol. 21, December 2016.##J. R. Finkel and C. D. Manning, "Hierarchical Bayesian Domain Adaptation," in Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Boulder, USA, pp. 602-610, June 2009.##P. Fortuna, J. Soler-Company and L. Wanner, "How Well Do Hate Hpeech, Toxicity, Abusive and Offensive Language Classification Models Generalize Across Datasets?," Information Processing & Management, vol. 58, pp. 102524, May 2021.##Y. Ganin and V. Lempitsky, "Unsupervised Domain Adaptation by Backpropagation," in Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France, pp. 1180-1189, June 2015.##A. Gretton, K. Borgwardt, M. Rasch, B. Scholkopf and A. J. Smola, "A Kernel Method for the Two-Sample-Problem," Advances in neural information processing systems, vol. 8, pp. 513-520, December 2007.##T. Gröndahl, L. Pajola, M. Juuti, M. Conti and N. Asokan, "All You Need Is "Love": Evading Hate Speech Detection," in AISec '18, Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security, Toronto, Canada, pp. 2-12, January 2018.##T. Gui, Q. Zhang, H. Huang, M. Peng and X. Huang, "Part-of-Speech Tagging for Twitter with Adversarial Neural Networks," in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp. 2411-2420, September 2017.##J. Guo, D. Shah and R. Barzilay,"Multi-Source Domain Adaptation with Mixture of Experts," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, pp. 4694-4703, 2018.##J. Sorensen, J. Elliott, L. Dixon, M. McDonald and W. Cukierski, "Toxic Comment Classification Challenge," kaggle.com, Mar. 21, 2018. [Online]. Available: https://www.kaggle.com/c/jigsaw-toxic-comment-classification-challenge/overview/citation.##M. Karan and J. Šnajder, "Cross-Domain Detection of Abusive Language Online," in Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), Brussels, Belgium, pp. 132-137, October 2018.##Y. Kim, K. Stratos and D. Kim, "Domain Attention With an Ensemble of Experts," in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, pp. 643-653, July 2017.##B. Kulis, K. Saenko and T. Darrell, "What You Saw is Not What You Get: Domain Adaptation Using Asymmetric Kernel Transforms," in Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, Los Alamitos, USA, pp. 1785-1792, June 2011.##Y. Li, T. Baldwin and T. Cohn, "What's in a Domain? Learning Domain-Robust Text Representations Using Adversarial Training," in Proceedings of the 2018 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, USA, pp. 474-479, June 2018.##Z. C. Lipton, Y. Wang and A. J. Smola, "Detecting and Correcting for Label Shift with Black Box Predictors," in Proceedings of the 35th International Conference on Machine Learning, pp. 3128-3136, July 2018.##X. Ma, P. Xu, Z. Wang, R. Nallapati. and B. Xiang, "Domain Adaptation with BERT-based Domain Classification and Data Selection," in Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP, Hong Kong, China, pp. 76-83, November 2019.##M. Mozafari, R. Farahbakhsh and N. Crespi, "A BERT-Based Transfer Learning Approach for Hate Speech Detection in Online Social Media," in Proceedings of the Eighth International Conference on Complex Networks and Their Applications COMPLEX NETWORKS 2019, Cham, Germany, pp. 928-940, December 2019.##E. W. Pamungkas and V. Patti, "Cross-Domain and Cross-Lingual Abusive Language Detection: A Hybrid Approach with Deep Learning and a Multilingual Lexicon," in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, Florence, Italy, pp. 363-370, July 2019.##B. Parekh, "Is There a Case for Banning Hate Speech?," in The Content and Context of Hate Speech: Rethinking Regulation and Responses, M. Herz and P. Molnar, Cambridge: Cambridge University Press, pp. 37-56, 2012.##B. Plank, "Domain Adaptation for Parsing", Ph.D. Thesis, University of Groningen, 2011.##A. Ramponi and B. Plank, "Neural Unsupervised Domain Adaptation in NLP-A Survey," in The 28th International Conference on Computational Linguistics, Barcelona, Spain, pp. 6838-6855, December 2020.##J. Salminen, H. Almerekhi, M. Milenkovic, S. Jung, J. An, H. Kwak and B. Jansen, "Anatomy of Online Hate: Developing a Taxonomy and Machine Learning Models for Identifying and Classifying Hate in Online News Media," in Proceedings of the twelfth International Conference on Web and Social Media, ICWSM 2018, Palo Alto, USA, vol. 12, pp. 330-339, June 2018.##J. Salminen, M. Hopf, S. A. Chowdhury, S. g. Jung, H. Almerekhi and B. J. Jansen, "Developing an Online Hate Classifier for Multiple Social Media Platforms," Human-Centric Computing and Information Sciences, vol. 10, pp. 1-34, January 2020.##V. Sanh, L. Debut, J. Chaumond and T. Wolf, "DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter," in CoRR, 2019.##M. S. Jahan and M. Oussalah, "A Systematic Review of Hate Speech Automatic Detection Using Natural Language Processing," in arXiv, May 2021.##A. Sellars, "Defining Hate Speech," Berkman Klein Center Research Publication, Boston Univ. School of Law, Public Law Research Paper, 2016.##A. A. Siegel, "Online Hate Speech," in Social Media and Democracy: The State of the Field, Prospects for Reform, N. Persily and J. A. Tucker, Cambridge, UK: Cambridge University Press, pp. 56-58, 2020.##B. Sun, J. Feng and K. Saenko, "Return of Frustratingly Easy Domain Adaptation," in Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, USA, vol. 30, pp. 2058-2065, March 2016.##S. D. Swamy, A. Jamatia and B. Gambäck, "Studying Generalisability Across Abusive Language Detection Datasets," in Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), Hong Kong, China, pp. 940-950, November 2019.##A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is All You Need," Advances in Neural Information Processing Systems, vol. 11, pp. 5998-6008, December 2017.##Z. Waseem, J. Thorne and J. Bingel, "Bridging the Gaps: Multi Task Learning for Domain Transfer of Hate Speech Detection," in Online Harassment, pp. 29-55, July 2018.##M. Wiegand, M. Siegel and J. Ruppenhofer, "Overview of the germeval 2018 shared task on the identification of offensive language," in Proceedings of the GermEval 2018 Workshop, September 2018.##D. Wright and I. Augenstein, "Transformer Based Multi-Source Domain Adaptation," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7963-7974, November 2020.##T. Yao, Y. Pan, C. Ngo, H. Li and T. Mei, "Semi-supervised Domain Adaptation with Subspace Learning for Visual Recognition," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 2142-2150, June 2015.##M. Zampieri, P. Nakov, S. Rosenthal, P. Atanasova, G. Karadzhov, H. Mubarak, L. Derczynski, Z. Pitenis and Ç. Çöltekin, "SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)," in Proceedings of the Fourteenth Workshop on Semantic Evaluation, Barcelona, Spain, pp. 1425-1447, December 2020.##M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra and R. Kumar, "SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)," in Proceedings of the 13th International Workshop on Semantic Evaluation, Minneapolis, USA, pp. 75-86, June 2019.##P. Rajpurkar, J. Zhang, K. Lopyrev and P. Liang, "SQuAD: 100,000+ Questions for Machine Comprehension of Text," in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392, October 2016,##O. de Gibert, N. Pérez, A. García-Pablos and M. Cuadros, "Hate Speech Dataset from a White Supremacy Forum," in Proceedings of the 2nd Workshop on Abusive Language Online (ALW2), pp. 11-20, September 2018.‌##R. Kumar, A. K. Ojha, S. Malmasi and M. Zampieri, "Benchmarking Aggression Identification in Social Media," in Proceedings of the First Workshop on Trolling, Aggression and Cyberbullying (TRAC-2018), pp. 1-11, August 2018.##Z. Waseem and D. Hovy, "Hateful Symbols or Hateful People? Predictive Features for Hate Speech Detection on Twitter," in Proceedings of the NAACL Student Research Workshop, pp. 88-93, June 2016.‌##T. Consigny, "Sesame Street Ensemble: A Mixture of DistiIBERT Experts," 2021.##M. Artetxe, S. Bhosale, N. Goyal, T. Mihaylov, M. Ott, S. Shleifer, X. V. Lin, J. Du, S. Lyer, R. Pasunuru, G. Anantharaman, X. Li, S. Chen, H. Akin, M. Baines, L. Martin, X. Zhou, P. Singh Koura, B. O'Horo, J. Wang, L. Zettlemoyer, M. Diab, Z. Kozareva and V. Stoyanov, "Efficient Large Scale Language Modeling with Mixtures of Experts," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 11699-11732, December 2022.##A. Repetto, "Neural Networks: A Mixture of Experts with Attention," towardsdatascience.com, Jul. 23, 2017. [Online]. Available: https://towardsdatascience.com/neural-networks-a-mixture-of-experts-with-attention-30e196657065.##R. A. Jacobs, M. I. Jordan, S. J. Nowlan and G. E. Hinton, "Adaptive Mixtures of Local Experts," in Neural Computation, vol. 3, pp. 79-87, March 1991.##N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer," in International Conference on Learning Representations, November 2016.‌##W. Fedus, B. Zoph, and N. Shazeer, "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity," in Journal of Machine Learning Research, vol. 23, pp. 1-39, 2022.‌##Z. Du, J. Li, H. Su, L. Zhu, and K. Lu, "Cross-Domain Gradient Discrepancy Minimization for Unsupervised Domain Adaptation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3937-3946, June 2021.##Q. Ye, J. Zha, and X. Ren, "Eliciting Transferability in Multi-Task Learning with Task-Level Mixture-of-Experts," in arXiv, 2022.‌## ##