Volume 17, Issue 3 (11-2020)                   JSDP 2020, 17(3): 87-100 | Back to browse issues page

XML Persian Abstract Print

Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:

Sekhavat Y, Namani M S. Believable Visual Feedback in Motor Learning Using Occlusion-based Clipping in Video Mapping. JSDP. 2020; 17 (3) :87-100
URL: http://jsdp.rcisp.ac.ir/article-1-915-en.html
Tabriz Islamic Art University
Abstract:   (207 Views)
Gait rehabilitation systems provide patients with guidance and feedback that assist them to better perform the rehabilitation tasks. Real-time feedback can guide users to correct their movements. Research has shown that the quality of feedback is crucial to enhance motor learning in physical rehabilitation. Common feedback systems based on virtual reality present interactive feedback in a monitor in front of a user. However, in this technique, there is a gap between where the feedback is presented and where the actual movement occurs. In particular, there is a discrepancy between where the actual movement occurs (e.g., on a treadmill) and the place of presenting feedback (e.g., a screen in front of the user). As a result, the feedback is not provided in the same location, which requires users perform additional cognitive processing to understand and apply the feedback. This discrepancy is misleading and can consequently result in difficulties to adapt the changes in rehabilitation tasks. In addition, the occlusion problem is not well handled in existing feedback systems that results in misleading the users to assume that the obstacle is on the foot. To address this problem, we need to make an illusion of putting a foot on the obstacle. In this paper, we propose a visual feedback system based on video mapping to provide a better understanding of the relationship between body perception and movement kinematics. This system is based on Augmented Reality (AR) in which visual cues in the form of light are projected on the treadmill using video projectors. In this system, occlusion-based clipping is used to enhance the believability of the feedback. We argue that this system contributes to the correct execution of rehabilitation exercises by increasing patients’ awareness of gait speed and step length. We designed and implemented two prototypes including the video projection with occlusion-based clipping (OC) and a prototype with no occlusion-based clipping (NOC). A set of experiments were performed to assess and compare the ability of unimpaired participants to detect real-time feedback and make modifications to gait using our feedback system. In particular, we asked 24 unimpaired participants to perform stepping and obstacle avoidance tasks. Since the focus of the paper is the quality of the feedback than the effect of feedback on training in long-term, unimpaired participants were recruited for this study. In the experiments, a motion capture device was used to measure the performance of participants. We demonstrated that our system is effective in terms of steps to adapt changes, obstacles to adapt changes, normalized accumulative deviation, quality of user experience, and intuitiveness of feedback. The results showed that projection-based AR feedback can successfully guide participants through a rehabilitation exercise. In particular, the results of this study showed statistically significant differences between the fault-rate of participants using OC and NOC prototypes in the stepping (p=0.0031) and obstacle avoidance (0.021) tasks. In addition, participates rated OC more intuitive than NOC in terms of the quality of feedback. Our feedback system showed a significant improvement in participants’ ability to adapt the changes while walking on the treadmill.
Full-Text [PDF 4328 kb]   (75 Downloads)    
Type of Study: Research | Subject: Paper
Received: 2018/10/18 | Accepted: 2020/08/18 | Published: 2020/12/5 | ePublished: 2020/12/5

1. ]1[ M. Hajizadeh, A. Hashemi Oskouei, F. Ghalichi, "The Reliability of Knee Kinematics and Ground Reaction Force During Stair Negotiation", Iranian journal of Biomedical Engineering, vol.11. No. 3, pp.17-31, 2017.
2. ]2[ H. Zamani, M. Dadgoo, I. Ebrahimi Takamjani, E. Hajouj, A. Jamshidi Khorneh , "The Effects of Two Months Body Weight Supported Treadmill Training on Balance and Quality of Life of Patients With Incomplete Spinal Cord Injury", jrehab, vol.18, No.4, 328-337, 2018. [DOI:10.21859/jrehab.18.4.7]
3. ]3[ A. Shamsi gooshki, H. Nezamabadi-pour, S. Saryazdi, E. Kabir, "a relevance feedback approach based on similarity refinement in content based image retrieval", JSDP, vol. 11, No.2, pp. 43-55, 2015.
4. ]4[ R. Sigrist, G. Rauter, R. Riener, & P. Wolf, "Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review" Psychonomic bulletin & review, vol. 20. No.1, pp.21-53, 2013. [DOI:10.3758/s13423-012-0333-8] [PMID]
5. ]5[ J. Lee, Y. Kim, & G. J.Kim, "Effects of Visual Feedback on Outof-Body Illusory Tactile Sensation When Interacting with AugmentedVirtual Objects", IEEE Transactions on Human-Machine Systems, vol.47, No.1, pp. 101-112, 2017. [DOI:10.1109/THMS.2016.2599492]
6. ]6[ E. Kearney, S. Shellikeri, R. Martino, & Y. Yunusova, "Augmented visual feedback-aided interventions for motor rehabilitation in Parkinson's disease: a systematic review", Disability and rehabilitation, pp.1-17, 2018. [DOI:10.1080/09638288.2017.1419292] [PMID]
7. ]7[ L. E. Sucar, F. Orihuela-Espina, R.L. Velazquez, D.J. Reinkensmeyer, R. Leder, & J. Hernandez-Franco, "Gesture therapy: An upper limb virtual reality-based motor rehabilitation platform", IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 22, No. 3, pp.634-643, 2014. [DOI:10.1109/TNSRE.2013.2293673] [PMID]
8. ]8[L. Y. Liu, S.Sangani, & A. Lamontagne, "A real-time visual feedback protocol to improve symmetry of spatiotemporal factors of gait in stroke survivors: In Virtual Rehabilitation (ICVR)", 2017 International Conference on, 2017, pp. 1-2.
9. ]9[ T. T. James, "Effect of Gaming Assisted Visual Feedback on Functional Standing Balance among Acute Hemiparetic Stroke Patients", "Indian Journal of Physiotherapy & Occupational Therapy", vol.11, No.4, 2017. [DOI:10.5958/0973-5674.2017.00137.X]
10. ]10[ L.M. Muratori, E.M. Lamberg, L. Quinn, & S. V. Duff, "Applying principles of motor learning and control to upper extremity rehabilitation", Journal of Hand Therapy, vol. 26, No.2, pp.94-103, 2013. [DOI:10.1016/j.jht.2012.12.007] [PMID] [PMCID]
11. ]11[ S. N. Omkar, & D. K. Ganesh, "Stability training and measurement system for sportsperson (P84)", In the Engineering of Sport 7 , pp. 435-442, 2008. [DOI:10.1007/978-2-287-99054-0_51]
12. ]12[ M. Y. Lee, C.F.Lin, & K.S. Soon, "Balance control enhancement using sub-sensory stimulation and visual-auditory biofeedback strategies for amputee subjects. Prosthetics and orthotics international", vol. 3, No.4, pp.342-352, 2007. [DOI:10.1080/03093640601058162] [PMID]
13. ]13[ E. Ivanova, M. Schrader, K.Lorenz, & M. Minge, "Developing motivational visual feedback for a new telerehabilitation system for motor relearning after stroke", In Proceedings of the 31st British Computer Society Human Computer Interaction Conference , BCS Learning & Development Ltd, 2017, pp. 75. [DOI:10.14236/ewic/HCI2017.75]
14. ]14[ HK.Wu, H. R. Chen, & C. H.Yu, "Development of posterior walker with adjustable visual cues to improve gait performance for patients with Parkinson's disease", In IECON 2010-36th Annual Conference on IEEE Industrial Electronics Society, 2010, pp. 1512-1516.
15. ]15[ J. Lieberman, & C. Breazeal, "TIKL: Development of a wearable vibrotactile feedback suit for improved human motor learning", IEEE Transactions on Robotics, vol.23 (5), pp.919-926, 2007. [DOI:10.1109/TRO.2007.907481]
16. ]16[ P. Celnik, K.Stefan, F. Hummel, J. Duque, J. Classen, L.G. Cohen, "Encoding a motor memory in the older adult by action observation", Neuroimage, vol. 29(2), pp. 677-684, 2006. [DOI:10.1016/j.neuroimage.2005.07.039] [PMID]
17. ]17[ R. Saegusa, K. Shigematsu, K. Terashima, "Audiovisual feedback for cognitive assistance toward walk training: In Robotics and Biomimetics (ROBIO)", 2014 IEEE International Conference, pp. 925-930, 2014. [DOI:10.1109/ROBIO.2014.7090451]
18. ]18[ D. Freides, "Human information processing and sensory modality: Cross-modal functions, information complexity, memory, and deficit", Psychological bulletin, vol.81, No. 5, pp.284, 2008. [DOI:10.1037/h0036331] [PMID]
19. ]19[ H. Zamani, M. Dadgoo, E. Ebrahimi, "Trans-Tibial Amputee Gait Correction through Real-Time Visual Feedback", vol. 1(3), pp. 25-32, 2015.
20. ]20[ S. Moshref-Razavi, M. Sohrabi, M. S. Sotoodeh, "Effect of Neurofeedback Interactions and Mental Imagery on the Elderly's Balance", sija, vol.12 (3), pp.288-299, 2017. [DOI:10.21859/sija.12.3.288]
21. ]21[ R. Tang, H. Alizadeh, A. Tang, S. Bateman, J.A. Jorge, " Physio@ Home: design explorations to support movement guidance", In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems, 2014, pp. 1651-1656. [DOI:10.1145/2559206.2581197] [PMID]
22. ]22[ M. W.van Ooijen, M. Roerdink, M. Trekop, T.W. Janssen, & P. J. Beek, " The efficacy of treadmill training with and without projected visual context for improving walking ability and reducing fall incidence and fear of falling in older adults with fall-related hip fracture: a randomized controlled trial", BMC geriatrics, vol.16(1), pp. 215, 2014. [DOI:10.1186/s12877-016-0388-x] [PMID] [PMCID]
23. ]23[ Y. Sekhavat, H. Zarei, "Dynamic Difficulty Adjustment of Rehabilitation Games using Reinforcement Learning", vol. 48(1), pp.62-70, 2018.
24. ]24[ F. Anderson, T. Grossman, J. Matejka, G. Fitzmaurice, "YouMove: enhancing movement training with an augmented reality mirror", In Proceedings of the 26th annual ACM symposium on User interface software and technology, pp. 311-320, 2013. [DOI:10.1145/2501988.2502045] [PMCID]
25. ]25[ A.Alamri, J. Cha, A. El Saddik, "AR-REHAB: An augmented reality framework for poststroke-patient rehabilitation", IEEE Transactions on Instrumentation and Measurement, vol.59 (10), pp.2554-2563, 2010. [DOI:10.1109/TIM.2010.2057750]
26. ]26[ E. Velloso, A. Bulling, H. Gellersen," MotionMA: motion modelling and analysis by demonstration", In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2013, pp. 1309- 1318. [DOI:10.1145/2470654.2466171]
27. ]27[ Y.Tian, Y. Long, D. Xia, H. Yao, J. Zhang, "Handling occlusions in augmented reality based on 3D reconstruction method", Neurocomputing, vol.156, pp.96-104, 2013. [DOI:10.1016/j.neucom.2014.12.081]
28. ]28[ Y.A.Sekhavat, M.S. Namani, " Projection-based AR: Effective visual feedback in gait rehabilitation", IEEE Transactions on Human-Machine Systems, vol.48(6), pp.626-636, 2018. [DOI:10.1109/THMS.2018.2860579]
29. [29] F. Clemente, S. Dosen, L. Lonini, M. Markovic, D. Farina, C. Cipriani, " Humans can integrate augmented reality feedback in their sensorimotor control of a robotic hand", IEEE Transactions on Human-Machine Systems, vol.47(4), pp.583-589, 2016. [DOI:10.1109/THMS.2016.2611998]

Add your comments about this article : Your username or Email:

Send email to the article author

© 2015 All Rights Reserved | Signal and Data Processing