Signal and Data Processing Journal A scientific journal officially licensed by the Commission for Scientific Publications of the (MSRT). Publisher: Research Ceter for Developmen of Technologies
Volume 23, Issue 1 (6-2026)
JSDP 2026, 23(1): 1-18 |
Back to browse issues page
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Mizban Goush M, Soltanizadeh H, Zahraei M. A Hybrid Gaussian-Transformer Framework for Interpolation of Missing Data in Kinect Sensor. JSDP 2026; 23 (1) : 1
URL: http://jsdp.rcisp.ac.ir/article-1-1460-en.html
URL: http://jsdp.rcisp.ac.ir/article-1-1460-en.html
Associate Professor, Department of Electrical Engineering, Semnan University, Semnan, Iran
Abstract: (707 Views)
Accurate gait analysis plays a pivotal role in diagnosing neuromuscular disorders, such as Parkinson’s disease and stroke, and enhancing rehabilitation programs. The Kinect sensor, as a cost-effective and accessible tool, captures three-dimensional motion data but often faces challenges with missing data points due to environmental noise, physical obstructions, or rapid movements. Traditional interpolation methods, such as linear, spline, or random forest, exhibit poor performance in handling multidimensional and nonlinear data, failing to model complex spatiotemporal dependencies. This study introduces an innovative hybrid framework named HIM (Hybrid Interpolation Method), which integrates Gaussian Process (GP) for uncertainty modeling with a Transformer model for capturing long-term dependencies, optimizing outputs using the Particle Swarm Optimization (PSO) algorithm. Leveraging multi-head attention mechanisms and RBF kernels, this approach significantly enhances data reconstruction accuracy, offering an effective solution for medical and sports applications.
Motion data were collected from 53 healthy participants using the Kinect v2 sensor. Participants walked on a treadmill at a speed of 0.75 m/s, with three-dimensional knee coordinates recorded. To simulate missing data, 1 to 5 points were randomly removed. The HIM method involves data normalization using Z-score to mitigate scaling effects, initial estimation of missing values with GP and RBF kernels for uncertainty modeling, refinement of estimates using a Transformer model equipped with multi-head attention (8 heads) and feedforward layers to capture spatiotemporal dependencies, optimization of weights via PSO (50 particles, 100 iterations) for weighted output integration, and data smoothing with a moving average filter (window size 5) followed by performance evaluation using Root Mean Square Error (RMSE). HIM was compared against seven other algorithms (random forest, cubic spline, support vector machine, K-nearest neighbors, Gaussian fuzzy, inverse distance weighting, and autoencoder), implemented in Python on an Intel Core i7 system.
Results demonstrated that HIM achieved the best performance with an average RMSE of 2.6×10⁻³, showing significant improvement over baseline methods. For instance, random forest exhibited poor performance with an average RMSE of 2.2581, while faster methods like inverse distance weighting and Gaussian fuzzy offered higher computational efficiency. Statistical analysis (t-test) confirmed HIM’s significant superiority over other methods. The GP component dominated the final combination (weights close to 1), underscoring its critical role in uncertainty modeling. Although HIM’s execution time (16.2 to 16.5 seconds) was higher than simpler methods, its unprecedented accuracy stands out.
The HIM framework, by synergistically combining GP and Transformer, marks a significant advancement in Kinect motion data analysis, strengthening applications in medical rehabilitation monitoring and sports injury prevention. It aligns with prior studies like DK-Former and CGPT, highlighting the superiority of hybrid approaches. The primary limitation is its relatively high execution time, necessitating optimization for real-time applications. Future work includes compressing the Transformer model using knowledge distillation, evaluating performance on real patient data, and integrating with real-time intelligent systems. This research paves the way for developing deep learning-based systems for biometric data processing.
Motion data were collected from 53 healthy participants using the Kinect v2 sensor. Participants walked on a treadmill at a speed of 0.75 m/s, with three-dimensional knee coordinates recorded. To simulate missing data, 1 to 5 points were randomly removed. The HIM method involves data normalization using Z-score to mitigate scaling effects, initial estimation of missing values with GP and RBF kernels for uncertainty modeling, refinement of estimates using a Transformer model equipped with multi-head attention (8 heads) and feedforward layers to capture spatiotemporal dependencies, optimization of weights via PSO (50 particles, 100 iterations) for weighted output integration, and data smoothing with a moving average filter (window size 5) followed by performance evaluation using Root Mean Square Error (RMSE). HIM was compared against seven other algorithms (random forest, cubic spline, support vector machine, K-nearest neighbors, Gaussian fuzzy, inverse distance weighting, and autoencoder), implemented in Python on an Intel Core i7 system.
Results demonstrated that HIM achieved the best performance with an average RMSE of 2.6×10⁻³, showing significant improvement over baseline methods. For instance, random forest exhibited poor performance with an average RMSE of 2.2581, while faster methods like inverse distance weighting and Gaussian fuzzy offered higher computational efficiency. Statistical analysis (t-test) confirmed HIM’s significant superiority over other methods. The GP component dominated the final combination (weights close to 1), underscoring its critical role in uncertainty modeling. Although HIM’s execution time (16.2 to 16.5 seconds) was higher than simpler methods, its unprecedented accuracy stands out.
The HIM framework, by synergistically combining GP and Transformer, marks a significant advancement in Kinect motion data analysis, strengthening applications in medical rehabilitation monitoring and sports injury prevention. It aligns with prior studies like DK-Former and CGPT, highlighting the superiority of hybrid approaches. The primary limitation is its relatively high execution time, necessitating optimization for real-time applications. Future work includes compressing the Transformer model using knowledge distillation, evaluating performance on real patient data, and integrating with real-time intelligent systems. This research paves the way for developing deep learning-based systems for biometric data processing.
Article number: 1
Keywords: Missing Data Interpolation, Gaussian Process, Transformer, Kinect Sensor, Gait Analysis, Motion Data Reconstruction, Multi-Head Attention Mechanism.
Type of Study: Research |
Subject:
Paper
Received: 2025/02/23 | Accepted: 2026/02/8 | Published: 2026/06/21 | ePublished: 2026/06/21
Received: 2025/02/23 | Accepted: 2026/02/8 | Published: 2026/06/21 | ePublished: 2026/06/21
References
1. J. C. Arellano-González, H. I. Medellín-Castillo, J. J. Cervantes-Sánchez, and A. Vidal-Lesso, "A practical review of the biomechanical parameters commonly used in the assessment of human gait," Revista mexicana de ingeniería biomédica, vol. 42, no. 3, 2021.
2. S. S. S. Fathima, K. Jyotsna, T. Srinivasulu, K. Archana, and S. Ravichand, "Walking pattern analysis using GAIT cycles and silhouettes for clinical applications," Measurement: Sensors, vol. 30, p. 100893, 2023. [DOI:10.1016/j.measen.2023.100893]
3. S. Chen, J. Lach, B. Lo, and G.-Z. Yang, "Toward pervasive gait analysis with wearable sensors: A systematic review," IEEE journal of biomedical and health informatics, vol. 20, no. 6, pp. 1521-1537, 2016. [DOI:10.1109/JBHI.2016.2608720]
4. A. Dakkak and A. Husain, "Recovering missing depth information from Microsoft's Kinect," in Proc. Embedded Vis. Alliance, 2012, pp. 1-9.
5. دانشپور نگین، میرابوالقاسمی سیده فاطمه، «پرکردن دادههای گمشده در دادههای سری زمانی چندمتغیره»، پردازش علائم و دادهها، 19 (2)، 39-60، 1401.
5. N. Daneshpour and S. F. Mirabolghasemi, "Missing Data Imputation in Multivariate Time Series Data," Signal and Data Processing, 19 (2). 39-60, 2022. [DOI:10.52547/jsdp.19.2.39]
6. S. J. Howarth and J. P. Callaghan, "Quantitative assessment of the accuracy for three interpolation techniques in kinematic analysis of human movement," Computer methods in biomechanics and biomedical engineering, vol. 13, no. 6, pp. 847-855, 2010. [DOI:10.1080/10255841003664701]
7. C. K. Williams and C. E. Rasmussen, Gaussian processes for machine learning (no. 3). MIT press Cambridge, MA, 2006.
8. V. Ashish, "Attention is all you need," Advances in neural information processing systems, vol. 30, p. I, 2017.
9. P. W. Smith, "A practical guide to splines (Carl de Boor)," SIAM Review, vol. 22, no. 4, p. 520, 1980. [DOI:10.1137/1022106]
10. A. Yorozu, T. Moriguchi, and M. Takahashi, "Improved leg tracking considering gait phase and spline-based interpolation during turning motion in walk tests," Sensors, vol. 15, no. 9, pp. 22451-22472, 2015. [DOI:10.3390/s150922451]
11. T. Mukai and S. Kuriyama, "Geostatistical motion interpolation," in ACM SIGGRAPH 2005 Papers, 2005, pp. 1062-1070. [DOI:10.1145/1186822.1073313]
12. S. Roberts, M. Osborne, M. Ebden, S. Reece, N. Gibson, and S. Aigrain, "Gaussian processes for time-series modelling," Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, vol. 371, no. 1984, p. 20110550, 2013. [DOI:10.1098/rsta.2011.0550]
13. A. Vaswani, "Attention is all you need," Advances in Neural Information Processing Systems, 2017.
14. A. J. Slagell, K. Lakkaraju, and K. Luo, "FLAIM: A Multi-level Anonymization Framework for Computer and Network Logs," in LISA, 2006, vol. 6, pp. 3-8.
15. Y. Wang et al., "A unified hybrid transformer for joint MRI sequences super-resolution and missing data imputation," Physics in Medicine & Biology, vol. 68, no. 13, p. 135006, 2023. [DOI:10.1088/1361-6560/acdc80]
16. T. Wang, Y. Hu, Q. Fang, B. He, X. Gong, and P. Wang, "DK-Former: A Hybrid Structure of Deep Kernel Gaussian Process Transformer Network for Enhanced Traffic Sign Recognition," IEEE Transactions on Intelligent Transportation Systems, 2024. [DOI:10.1109/TITS.2024.3436911]
17. L. M. Bui, T. T. Huu, D. Dinh, T. M. Nguyen, and T. N. Hoang, "Revisiting kernel attention with correlated gaussian process representation," arXiv preprint arXiv:2502.20525, 2025.
18. C. Pheatt and J. Ballester, "Using the Xbox Kinect Sensor for positional data acquisition," 2011. [DOI:10.1119/1.4748853]
19. Z. Zhang, "Microsoft kinect sensor and its effect," IEEE multimedia, vol. 19, no. 2, pp. 4-10, 2012. [DOI:10.1109/MMUL.2012.24]
20. K. Ryselis, T. Petkus, T. Blažauskas, R. Maskeliūnas, and R. Damaševičius, "Multiple Kinect based system to monitor and analyze key performance indicators of physical training," Human-Centric Computing and Information Sciences, vol. 10, pp. 1-22, 2020. [DOI:10.1186/s13673-020-00256-4]
21. C. Zerpa, C. Lees, P. Patel, E. Pryzsucha, and P. Patel, "The use of microsoft Kinect for human movement analysis," International journal of sports science, vol. 5, no. 4, pp. 120-127, 2015.
22. L. Yeung, K. C. Cheng, C. Fong, W. C. Lee, and K.-Y. Tong, "Evaluation of the Microsoft Kinect as a clinical assessment tool of body sway," Gait & posture, vol. 40, no. 4, pp. 532-538, 2014. [DOI:10.1016/j.gaitpost.2014.06.012]
23. B. Bonnechere et al., "Validity and reliability of the Kinect within functional assessment activities: comparison with standard stereophotogrammetry," Gait & posture, vol. 39, no. 1, pp. 593-598, 2014. [DOI:10.1016/j.gaitpost.2013.09.018]
24. D. V. Griffiths and I. M. Smith, Numerical methods for engineers. Chapman and Hall/CRC, 2006. [DOI:10.1201/9781420010244]
25. B. Alhajahmad and M. Atas, "Boosting Predictive Power: Random Forest and Gradient Boosted Trees in Ensemble Learning," in Proceeding Book of 2nd International Conference on Contemporary Academic Research ICCAR, 2023.
26. نمیرانیان، فریبا، لطیف، علی محمد، «ارائة رویکردی نوین در بخشبندی تصاویر دیجیتال بهوسیلة الگوریتم ژنتیک و جنگل تصادفی»، پردازش علائم و دادهها، 20 (4)، ش 58، 35-44، 1402.
26. F. Namiranian and A. Latif, "A New Approach for Digital Image Segmentation with Genetic Algorithm and Random Forest," Signal and Data Processing, No. 58, 20 )4(, pp. 35-44, 2024. [DOI:10.61186/jsdp.20.4.35]
27. T. Zhu, "Analysis on the applicability of the random forest," in Journal of Physics: Conference Series, 2020, vol. 1607, no. 1: IOP Publishing, p. 012123. [DOI:10.1088/1742-6596/1607/1/012123]
28. B. Talekar and S. Agrawal, "A detailed review on decision tree and random forest," Biosci. Biotechnol. Res. Commun, vol. 13, no. 14, pp. 245-248, 2020. [DOI:10.21786/bbrc/13.14/57]
29. H. Zaynidinov, "ALGORITHM AND SOFTWARE TOOLS FOR IMPROVING INTERPOLATION ACCURACY BASED ON CUBIC SPLINE," 2022, No. 5 (135/1), vol. 135, no. 117, pp. 5-8, 2023. [DOI:10.59251/2181-1296.v5.1351.1130]
30. M. Zhu and H. Shi, "A novel support vector machine algorithm for missing data," in Proceedings of the 2nd International Conference on Innovation in Artificial Intelligence, 2018, pp. 48-53. [DOI:10.1145/3194206.3194214]
31. G. Tutz and S. Ramzan, "Improved methods for the imputation of missing data by nearest neighbor methods," Computational Statistics & Data Analysis, vol. 90, pp. 84-99, 2015. [DOI:10.1016/j.csda.2015.04.009]
32. E. R. Hruschka, E. R. Hruschka Jr, and N. F. Ebecken, "Evaluating a nearest-neighbor method to substitute continuous missing values," in Australasian Joint Conference on Artificial Intelligence, 2003: Springer, pp. 723-734. [DOI:10.1007/978-3-540-24581-0_62]
33. M. El-Bakry, F. Ali, A. El-Kilany, and S. Mazen, "Fuzzy based techniques for handling missing values," International Journal of Advanced Computer Science and Applications, vol. 12, no. 3, 2021. [DOI:10.14569/IJACSA.2021.0120306]
34. K. Muludi, R. Setianingsih, R. Sholehurrohman, and A. Junaidi, "Exploiting nearest neighbor data and fuzzy membership function to address missing values in classification," PeerJ Computer Science, vol. 10, p. e1968, 2024. [DOI:10.7717/peerj-cs.1968]
35. D. W. Wong, "Interpolation: Inverse‐distance weighting," International encyclopedia of geography: People, the earth, environment and technology: People, the earth, environment and technology, pp. 1-7, 2016. [DOI:10.1002/9781118786352.wbieg0066]
36. D. Kalota and M. F. Khan, "Comparing Performance of Inverse Distance Weighting and Modified Shepard Interpolation Models with Different Sampling Arrangements: A Case Study," in Re-envisioning Advances in Remote Sensing: CRC Press, 2022, pp. 239-249. [DOI:10.1201/9781003224624-18]
37. W. H. L. Pinaya, S. Vieira, R. Garcia-Dias, and A. Mechelli, "Autoencoders," in Machine learning: Elsevier, 2020, pp. 193-208. [DOI:10.1016/B978-0-12-815739-8.00011-0]
38. Y. Sun, J. Li, Y. Xu, T. Zhang, and X. Wang, "Deep learning versus conventional methods for missing data imputation: A review and comparative study," Expert Systems with Applications, vol. 227, p. 120201, 2023. [DOI:10.1016/j.eswa.2023.120201]
39. H. Zhou et al., "Informer: Beyond efficient transformer for long sequence time-series forecasting," in Proceedings of the AAAI conference on artificial intelligence, 2021, vol. 35, no. 12, pp. 11106-11115. [DOI:10.1609/aaai.v35i12.17325]
40. B. Lim, S. Ö. Arık, N. Loeff, and T. Pfister, "Temporal fusion transformers for interpretable multi-horizon time series forecasting," International journal of forecasting, vol. 37, no. 4, pp. 1748-1764, 2021. [DOI:10.1016/j.ijforecast.2021.03.012]
41. Y. Zhang and J. Yan, "Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting," in The eleventh international conference on learning representations, 2023.
42. R. Poli, J. Kennedy, and T. Blackwell, "Particle swarm optimization: An overview," Swarm intelligence, vol. 1, no. 1, pp. 33-57, 2007. [DOI:10.1007/s11721-007-0002-0]
43. Y. Zhang, S. Wang, and G. Ji, "A comprehensive survey on particle swarm optimization algorithm and its applications," Mathematical problems in engineering, vol. 2015, no. 1, p. 931256, 2015. [DOI:10.1155/2015/931256]
Send email to the article author
| Rights and permissions | |
 |
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |