Signal and Data Processing Journal A scientific journal officially licensed by the Commission for Scientific Publications of the (MSRT). Publisher: Research Ceter for Developmen of Technologies
Volume 23, Issue 1 (6-2026)
JSDP 2026, 23(1): 83-98 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Gholizadeh M, Soltanizadeh H, RahmaniManesh M. Multi-Source Transfer Learning Based on Pre-Trained Models: A Fuzzy Rule-Based Approach. JSDP 2026; 23 (1) : 6
URL: http://jsdp.rcisp.ac.ir/article-1-1471-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1471-en.html
Associate Professor, Faculty of Electrical and Computer Engineering, Semnan University, Semnan, Iran
Abstract: (747 Views)
In many real-world machine learning applications, access to data from multiple sources is often restricted due to privacy concerns, data ownership policies, legal regulations, or technical limitations. These restrictions significantly complicate the learning process, especially in scenarios where knowledge from several related domains could improve predictive performance. In particular, multi-source transfer learning has emerged as an important paradigm for leveraging information from multiple source domains to improve learning in a target domain with limited labeled data. However, most existing multi-source transfer learning methods assume that the data from all source domains are fully accessible during training. This assumption is often unrealistic in practical settings, where only partial access to data may be granted, or in some cases, only pre-trained models are available instead of the original datasets.
Motivated by this limitation, this paper addresses the problem of multi-source transfer learning under restricted data access conditions. Specifically, we consider scenarios in which full datasets from some source domains cannot be directly accessed, and instead only pre-trained models or partial information about these sources are available. Such conditions frequently arise in applications involving sensitive or distributed data, such as healthcare systems, industrial data analytics, financial systems, and collaborative environments where data sharing is limited. Under these circumstances, designing learning mechanisms that can effectively transfer knowledge without direct access to all source data becomes a critical challenge.
To tackle this problem, we propose a novel framework that combines neural networks, fuzzy clustering, and Takagi–Sugeno fuzzy rules to facilitate knowledge transfer while effectively managing uncertainty. The proposed approach leverages the representational power of neural networks to capture complex relationships within the available data, while fuzzy clustering is used to identify latent structures and group similar patterns across domains. The use of fuzzy logic enables the model to explicitly handle ambiguity and uncertainty, which are commonly encountered in real-world datasets. In particular, Takagi–Sugeno fuzzy rules provide a flexible mechanism for modeling nonlinear relationships and integrating information obtained from different sources in a transparent and interpretable manner.
The proposed method operates by first extracting informative representations from the available source models and the limited data accessible from the target domain. Fuzzy clustering is then applied to identify underlying data structures and generate clusters that represent distinct regions of the input space. Based on these clusters, a set of Takagi–Sugeno fuzzy rules is constructed to model the relationship between input variables and the target output. Each rule corresponds to a local model that captures specific patterns in the data, enabling the system to approximate complex nonlinear functions while maintaining interpretability. By combining neural learning with fuzzy reasoning, the proposed framework effectively transfers useful knowledge from multiple sources to the target domain even when direct access to source datasets is limited.
Another important advantage of the proposed approach lies in its ability to cope with uncertainty and incomplete knowledge. In multi-source transfer learning scenarios with restricted data access, the information obtained from different sources may be incomplete, noisy, or inconsistent. The integration of fuzzy logic allows the model to represent partial memberships and gradual transitions between data regions, which improves its robustness to such imperfections. As a result, the method can better generalize in situations where traditional crisp models may struggle to capture complex relationships.
To evaluate the effectiveness of the proposed framework, extensive experiments were conducted on four regression datasets representing different real-world scenarios. The experimental setup simulates conditions where the target domain has limited labeled data and some source domains provide only pre-trained models instead of raw datasets. The proposed method was compared with several state-of-the-art multi-source transfer learning approaches to assess its predictive performance under these challenging conditions.
The experimental results demonstrate that the proposed method consistently achieves superior or competitive performance compared to existing advanced techniques, particularly in situations characterized by limited knowledge and significant uncertainty. In many cases, the method improves prediction accuracy in the target domain by effectively leveraging information embedded in the available source models while maintaining robustness against incomplete or uncertain data. Furthermore, the fuzzy rule-based structure enhances the interpretability of the learned models, allowing practitioners to gain insights into the relationships between variables and the underlying decision mechanisms.
Overall, the findings of this study highlight the potential of integrating neural networks with fuzzy clustering and Takagi–Sugeno fuzzy systems for addressing the challenges of multi-source transfer learning with restricted data access. The proposed approach provides a practical solution for exploiting limited information from multiple sources while effectively managing uncertainty, making it particularly suitable for real-world applications where data sharing is constrained.
In conclusion, this work contributes a novel hybrid framework for multi-source transfer learning that operates effectively under incomplete data accessibility. By combining neural representation learning, fuzzy clustering, and interpretable fuzzy rule-based modeling, the proposed method enables efficient knowledge transfer across domains and improves predictive performance in the target domain. The promising experimental results suggest that the proposed approach can serve as a valuable tool for real-world machine learning tasks where access to comprehensive multi-source datasets is limited.
Motivated by this limitation, this paper addresses the problem of multi-source transfer learning under restricted data access conditions. Specifically, we consider scenarios in which full datasets from some source domains cannot be directly accessed, and instead only pre-trained models or partial information about these sources are available. Such conditions frequently arise in applications involving sensitive or distributed data, such as healthcare systems, industrial data analytics, financial systems, and collaborative environments where data sharing is limited. Under these circumstances, designing learning mechanisms that can effectively transfer knowledge without direct access to all source data becomes a critical challenge.
To tackle this problem, we propose a novel framework that combines neural networks, fuzzy clustering, and Takagi–Sugeno fuzzy rules to facilitate knowledge transfer while effectively managing uncertainty. The proposed approach leverages the representational power of neural networks to capture complex relationships within the available data, while fuzzy clustering is used to identify latent structures and group similar patterns across domains. The use of fuzzy logic enables the model to explicitly handle ambiguity and uncertainty, which are commonly encountered in real-world datasets. In particular, Takagi–Sugeno fuzzy rules provide a flexible mechanism for modeling nonlinear relationships and integrating information obtained from different sources in a transparent and interpretable manner.
The proposed method operates by first extracting informative representations from the available source models and the limited data accessible from the target domain. Fuzzy clustering is then applied to identify underlying data structures and generate clusters that represent distinct regions of the input space. Based on these clusters, a set of Takagi–Sugeno fuzzy rules is constructed to model the relationship between input variables and the target output. Each rule corresponds to a local model that captures specific patterns in the data, enabling the system to approximate complex nonlinear functions while maintaining interpretability. By combining neural learning with fuzzy reasoning, the proposed framework effectively transfers useful knowledge from multiple sources to the target domain even when direct access to source datasets is limited.
Another important advantage of the proposed approach lies in its ability to cope with uncertainty and incomplete knowledge. In multi-source transfer learning scenarios with restricted data access, the information obtained from different sources may be incomplete, noisy, or inconsistent. The integration of fuzzy logic allows the model to represent partial memberships and gradual transitions between data regions, which improves its robustness to such imperfections. As a result, the method can better generalize in situations where traditional crisp models may struggle to capture complex relationships.
To evaluate the effectiveness of the proposed framework, extensive experiments were conducted on four regression datasets representing different real-world scenarios. The experimental setup simulates conditions where the target domain has limited labeled data and some source domains provide only pre-trained models instead of raw datasets. The proposed method was compared with several state-of-the-art multi-source transfer learning approaches to assess its predictive performance under these challenging conditions.
The experimental results demonstrate that the proposed method consistently achieves superior or competitive performance compared to existing advanced techniques, particularly in situations characterized by limited knowledge and significant uncertainty. In many cases, the method improves prediction accuracy in the target domain by effectively leveraging information embedded in the available source models while maintaining robustness against incomplete or uncertain data. Furthermore, the fuzzy rule-based structure enhances the interpretability of the learned models, allowing practitioners to gain insights into the relationships between variables and the underlying decision mechanisms.
Overall, the findings of this study highlight the potential of integrating neural networks with fuzzy clustering and Takagi–Sugeno fuzzy systems for addressing the challenges of multi-source transfer learning with restricted data access. The proposed approach provides a practical solution for exploiting limited information from multiple sources while effectively managing uncertainty, making it particularly suitable for real-world applications where data sharing is constrained.
In conclusion, this work contributes a novel hybrid framework for multi-source transfer learning that operates effectively under incomplete data accessibility. By combining neural representation learning, fuzzy clustering, and interpretable fuzzy rule-based modeling, the proposed method enables efficient knowledge transfer across domains and improves predictive performance in the target domain. The promising experimental results suggest that the proposed approach can serve as a valuable tool for real-world machine learning tasks where access to comprehensive multi-source datasets is limited.
Article number: 6
Keywords: Multi-source transfer learning, fuzzy rules, domain adaptation, pre-trained models, Takagi-Sugeno fuzzy rules.
Type of Study: Applicable |
Subject:
Paper
Received: 2025/05/31 | Accepted: 2026/02/8 | Published: 2026/09/16 | ePublished: 2026/09/16
Received: 2025/05/31 | Accepted: 2026/02/8 | Published: 2026/09/16 | ePublished: 2026/09/16
References
1. Q. Gao, T. Huang, K. Zhao, H. Shao, and B. Jin, "Multi-source weighted source-free domain transfer method for rotating machinery fault diagnosis," Expert Syst. Appl., vol. 237, 2024, doi: 10.1016/j.eswa.2023.121585. [DOI:10.1016/j.eswa.2023.121585]
2. H. Guan and M. Liu, "Domain Adaptation for Medical Image Analysis: A Survey," IEEE Trans. Biomed. Eng., vol. 69, no. 3, pp. 1173-1185, 2022, doi: 10.1109/TBME.2021.3117407. [DOI:10.1109/TBME.2021.3117407]
3. G. Ma, J. Lu, and G. Zhang, "Multisource Domain Adaptation With Interval-Valued Target Data via Fuzzy Neural Networks," IEEE Trans. Fuzzy Syst., vol. 32, no. 5, pp. 3094-3106, 2024, doi: 10.1109/TFUZZ.2024.3367456. [DOI:10.1109/TFUZZ.2024.3367456]
4. F. Zhuang et al., "A Comprehensive Survey on Transfer Learning," Proc. IEEE, vol. 109, no. 1, pp. 43-76, Jan. 2021, doi: 10.1109/JPROC.2020.3004555. [DOI:10.1109/JPROC.2020.3004555]
5. M. Gholizade, H. Soltanizadeh, M. Rahmanimanesh, and S. S. Sana, "A review of recent advances and strategies in transfer learning," Int. J. Syst. Assur. Eng. Manag., Feb. 2025, doi: 10.1007/s13198-024-02684-2. [DOI:10.1007/s13198-024-02684-2]
6. M. Moradi, M. Rahmanimanesh, A. Shahzadi, and R. Monsefi, "Smooth unsupervised domain adaptation considering uncertainties," Inf. Sci. (Ny)., vol. 648, 2023, doi: 10.1016/j.ins.2023.119602. [DOI:10.1016/j.ins.2023.119602]
7. G. Tang, C. Chen, Y. Li, L. Liu, C. Yi, and J. Lin, "Integrating Demodulation Techniques With Multisource Transfer Learning for Intelligent Fault Diagnosis of Bearings," IEEE Trans. Instrum. Meas., vol. 74, pp. 1-10, 2025, doi: 10.1109/TIM.2025.3548074. [DOI:10.1109/TIM.2025.3548074]
8. قلی زاده معصومه، سلطانی زاده هادی، رحمانی منش محمد، «یادگیری انتقالی چندمنبعی مبتنی بر قواعد فازی برای بهبود دقت طبقهبندی تصاویر»، پردازش علائم و دادهها، ۱۴۰۳; ۲۱ (۴) :۴۹-۶۶.
8. M. Gholizade, H. Soltanizadeh, and M. Rahmanimanesh, "Multi-Source Transfer Learning Based on Fuzzy Rules for Improving Image Classification Accuracy," Signal Data Process., vol. 21, no. 4, pp. 49-66, 2025, doi: 10.61186/jsdp.21.4.49. [DOI:10.61186/jsdp.21.4.49]
9. S. Zhao et al., "Multi-source multi-modal domain adaptation," Inf. Fusion, vol. 117, 2025, doi: 10.1016/j.inffus.2024.102862. [DOI:10.1016/j.inffus.2024.102862]
10. M. Gholizade, H. Soltanizadeh, and M. Rahmanimanesh, "A Survey of Transfer Learning and Categories," Model. Simul. Electr. Electron. Eng., vol. 1, no. 3, pp. 17-25, 2021, doi: 10.22075/mseee.2021.23310.1062.
11. M. Moradi, M. Rahmanimanesh, and A. Shahzadi, "Unsupervised domain adaptation by incremental learning for concept drifting data streams," Int. J. Mach. Learn. Cybern., vol. 15, no. 9, pp. 4055-4078, Sep. 2024, doi: 10.1007/s13042-024-02135-1. [DOI:10.1007/s13042-024-02135-1]
12. M. Moradi, M. Rahmanimanesh, and A. Shahzadi, "Learning from streaming data with unsupervised heterogeneous domain adaptation," Int. J. Data Sci. Anal., vol. 19, no. 1, pp. 61-81, 2025, doi: 10.1007/s41060-023-00463-z. [DOI:10.1007/s41060-023-00463-z]
13. Y. Zhu, W. Yu, and X. Li, "A Multi-objective transfer learning framework for time series forecasting with Concept Echo State Networks," Neural Networks, vol. 186, p. 107272, Jun. 2025, doi: 10.1016/j.neunet.2025.107272. [DOI:10.1016/j.neunet.2025.107272]
14. S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, "Domain adaptation via transfer component analysis," IJCAI Int. Jt. Conf. Artif. Intell., pp. 1187-1192, 2009.
15. M. Moradi, M. Rahmanimanesh, and A. Shahzadi, "Transfer learning for concept drifting data streams in heterogeneous environments," Knowl. Inf. Syst., vol. 66, no. 5, pp. 2799-2857, 2024, doi: 10.1007/s10115-023-02043-w. [DOI:10.1007/s10115-023-02043-w]
16. K. Li, J. Lu, H. Zuo, and G. Zhang, "Multi-Source Contribution Learning for Domain Adaptation," IEEE Trans. Neural Networks Learn. Syst., vol. 33, no. 10, pp. 5293-5307, 2022, doi: 10.1109/TNNLS.2021.3069982. [DOI:10.1109/TNNLS.2021.3069982]
17. K. Yang, J. Lu, W. Wan, and G. Zhang, "Multi-source transfer regression via source-target pairwise segment," Inf. Sci. (Ny)., vol. 556, pp. 389-403, 2021, doi: 10.1016/j.ins.2020.09.074. [DOI:10.1016/j.ins.2020.09.074]
18. C. X. Ren, Y. H. Liu, X. W. Zhang, and K. K. Huang, "Multi-Source Unsupervised Domain Adaptation via Pseudo Target Domain," IEEE Trans. Image Process., vol. 31, pp. 2122-2135, 2022, doi: 10.1109/TIP.2022.3152052. [DOI:10.1109/TIP.2022.3152052]
19. X. Lu, T. Gong, and X. Zheng, "Multisource Compensation Network for Remote Sensing Cross-Domain Scene Classification," IEEE Trans. Geosci. Remote Sens., vol. 58, no. 4, pp. 2504-2515, 2020, doi: 10.1109/TGRS.2019.2951779. [DOI:10.1109/TGRS.2019.2951779]
20. A. Khattar and S. M. K. Quadri, "Multi-source domain adaptation of social media data for disaster management," Multimed. Tools Appl., vol. 82, no. 6, pp. 9083-9111, 2023, doi: 10.1007/s11042-022-13456-0. [DOI:10.1007/s11042-022-13456-0]
21. H. Zuo, J. Lu, G. Zhang, and F. Liu, "Fuzzy Transfer Learning Using an Infinite Gaussian Mixture Model and Active Learning," IEEE Trans. Fuzzy Syst., vol. 27, no. 2, pp. 291-303, 2019, doi: 10.1109/TFUZZ.2018.2857725. [DOI:10.1109/TFUZZ.2018.2857725]
22. H. Zuo, J. Lu, G. Zhang, and W. Pedrycz, "Fuzzy Rule-Based Domain Adaptation in Homogeneous and Heterogeneous Spaces," IEEE Trans. Fuzzy Syst., vol. 27, no. 2, pp. 348-361, 2019, doi: 10.1109/TFUZZ.2018.2853720. [DOI:10.1109/TFUZZ.2018.2853720]
23. J. Lu, H. Zuo, and G. Zhang, "Fuzzy Multiple-Source Transfer Learning," IEEE Trans. Fuzzy Syst., vol. 28, no. 12, pp. 3418-3431, 2020, doi: 10.1109/TFUZZ.2019.2952792. [DOI:10.1109/TFUZZ.2019.2952792]
24. H. Zuo, G. Zhang, W. Pedrycz, V. Behbood, and J. Lu, "Fuzzy Regression Transfer Learning in Takagi-Sugeno Fuzzy Models," IEEE Trans. Fuzzy Syst., vol. 25, no. 6, pp. 1795-1807, 2017, doi: 10.1109/TFUZZ.2016.2633376. [DOI:10.1109/TFUZZ.2016.2633376]
25. S. A. Solla, T. K. Leen, and C. E. Rasmussen, "in Advances in Neural Information Processing Systems 12 The Infinite Gaussian Mixture Model," pp. 554-560, 2000, [Online]. Available: http://bayes.imm.dtu.dk
26. H. Zuo, G. Zhang, W. Pedrycz, V. Behbood, and J. Lu, "Granular Fuzzy Regression Domain Adaptation in Takagi-Sugeno Fuzzy Models," IEEE Trans. Fuzzy Syst., vol. 26, no. 2, pp. 847-858, 2018, doi: 10.1109/TFUZZ.2017.2694801. [DOI:10.1109/TFUZZ.2017.2694801]
27. S. De Vito, E. Massera, M. Piga, L. Martinotto, and G. Di Francia, "On field calibration of an electronic nose for benzene estimation in an urban pollution monitoring scenario," Sensors Actuators, B Chem., vol. 129, no. 2, pp. 750-757, 2008, doi: 10.1016/j.snb.2007.09.060. [DOI:10.1016/j.snb.2007.09.060]
28. P. Tüfekci, "Prediction of full load electrical power output of a base load operated combined cycle power plant using machine learning methods," Int. J. Electr. Power Energy Syst., vol. 60, pp. 126-140, 2014, doi: 10.1016/j.ijepes.2014.02.027. [DOI:10.1016/j.ijepes.2014.02.027]
29. B. Gong, Y. Shi, F. Sha, and K. Grauman, "Geodesic flow kernel for unsupervised domain adaptation," Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 2066-2073, 2012, doi: 10.1109/CVPR.2012.6247911. [DOI:10.1109/CVPR.2012.6247911]
30. B. Fernando, A. Habrard, M. Sebban, and T. Tuytelaars, "Unsupervised visual domain adaptation using subspace alignment," Proc. IEEE Int. Conf. Comput. Vis., pp. 2960-2967, 2013, doi: 10.1109/ICCV.2013.368. [DOI:10.1109/ICCV.2013.368]
31. S. J. Pan, I. W. Tsang, J. T. Kwok, and Q. Yang, "Domain adaptation via transfer component analysis," IEEE Trans. Neural Networks, vol. 22, no. 2, pp. 199-210, 2011, doi: 10.1109/TNN.2010.2091281. [DOI:10.1109/TNN.2010.2091281]
32. C. Yang, Z. Deng, K. S. Choi, and S. Wang, "Takagi-Sugeno-Kang transfer learning fuzzy logic system for the adaptive recognition of epileptic electroencephalogram signals," IEEE Trans. Fuzzy Syst., vol. 24, no. 5, pp. 1079-1094, 2016, doi: 10.1109/TFUZZ.2015.2501438. [DOI:10.1109/TFUZZ.2015.2501438]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |