Background: Few existing interactive rehabilitation systems can effectively communicate multiple aspects of movement performance simultaneously, in a manner that appropriately adapts across various training scenarios. In order to address the need for such systems within stroke rehabilitation training, a unified approach for designing interactive systems for upper limb rehabilitation of stroke survivors has been developed and applied for the implementation of an Adaptive Mixed Reality Rehabilitation (AMRR) System.

Results: The AMRR system provides computational evaluation and multimedia feedback for the upper limb rehabilitation of stroke survivors. A participant's movements are tracked by motion capture technology and evaluated by computational means. The resulting data are used to generate interactive media-based feedback that communicates to the participant detailed, intuitive evaluations of his performance. This article describes how the AMRR system's interactive feedback is designed to address specific movement challenges faced by stroke survivors. Multimedia examples are provided to illustrate each feedback component. Supportive data are provided for three participants of varying impairment levels to demonstrate the system's ability to train both targeted and integrated aspects of movement.

Conclusions: The AMRR system supports training of multiple movement aspects together or in isolation, within adaptable sequences, through cohesive feedback that is based on formalized compositional design principles. From preliminary analysis of the data, we infer that the system's ability to train multiple foci together or in isolation in adaptable sequences, utilizing appropriately designed feedback, can lead to functional improvement. The evaluation and feedback frameworks established within the AMRR system will be applied to the development of a novel home-based system to provide an engaging yet low-cost extension of training for longer periods of time.

Background: Although principles based in motor learning, rehabilitation, and human-computer interfaces can guide the design of effective interactive systems for rehabilitation, a unified approach that connects these key principles into an integrated design, and can form a methodology that can be generalized to interactive stroke rehabilitation, is presently unavailable.

Results: This paper integrates phenomenological approaches to interaction and embodied knowledge with rehabilitation practices and theories to achieve the basis for a methodology that can support effective adaptive, interactive rehabilitation. Our resulting methodology provides guidelines for the development of an action representation, quantification of action, and the design of interactive feedback. As Part I of a two-part series, this paper presents key principles of the unified approach. Part II then describes the application of this approach within the implementation of the Adaptive Mixed Reality Rehabilitation (AMRR) system for stroke rehabilitation.

Conclusions: The accompanying principles for composing novel mixed reality environments for stroke rehabilitation can advance the design and implementation of effective mixed reality systems for the clinical setting, and ultimately be adapted for home-based application. They furthermore can be applied to other rehabilitation needs beyond stroke.

Parkinson's disease (PD) is a neurodegenerative disorder that produces a characteristic set of neuromotor deficits that sometimes includes reduced amplitude and velocity of movement. Several studies have shown that people with PD improved their motor performance when presented with external cues. Other work has demonstrated that high velocity and large amplitude exercises can increase the amplitude and velocity of movement in simple carryover tasks in the upper and lower extremities. Although the cause for these effects is not known, improvements due to cueing suggest that part of the neuromotor deficit in PD is in the integration of sensory feedback to produce motor commands. Previous studies have documented some somatosensory deficits, but only limited information is available regarding the nature and magnitude of sensorimotor deficits in the shoulder of people with PD. The goals of this research were to characterize the sensorimotor impairment in the shoulder joint of people with PD and to investigate the use of visual feedback and large amplitude/high velocity exercises to target PD-related motor deficits. Two systems were designed and developed to use visual feedback to assess the ability of participants to accurately adjust limb placement or limb movement velocity and to encourage improvements in performance of these tasks. Each system was tested on participants with PD, age-matched control subjects and young control subjects to characterize and compare limb placement and velocity control capabilities. Results demonstrated that participants with PD were less accurate at placing their limbs than age-matched or young control subjects, but that their performance improved over the course of the test session such that by the end, the participants with PD performed as well as controls. For the limb velocity feedback task, participants with PD and age-matched control subjects were less accurate than young control subjects, but at the end of the session, participants with PD and age-matched control subjects were as accurate as the young control subjects. This study demonstrates that people with PD were able to improve their movement patterns based on visual feedback of performance and suggests that this feedback paradigm may be useful in exercise programs for people with PD.

Stroke is a leading cause of disability with varying effects across stroke survivors necessitating comprehensive approaches to rehabilitation. Interactive neurorehabilitation (INR) systems represent promising technological solutions that can provide an array of sensing, feedback and analysis tools which hold the potential to maximize clinical therapy as well as extend therapy to the home. Currently, there are a variety of approaches to INR design, which coupled with minimal large-scale clinical data, has led to a lack of cohesion in INR design. INR design presents an inherently complex space as these systems have multiple users including stroke survivors, therapists and designers, each with their own user experience needs. This dissertation proposes that comprehensive INR design, which can address this complex user space, requires and benefits from the application of interdisciplinary research that spans motor learning and interactive learning. A methodology for integrated and iterative design approaches to INR task experience, assessment, hardware, software and interactive training protocol design is proposed within the comprehensive example of design and implementation of a mixed reality rehabilitation system for minimally supervised environments. This system was tested with eight stroke survivors who showed promising results in both functional and movement quality improvement. The results of testing the system with stroke survivors as well as observing user experiences will be presented along with suggested improvements to the proposed design methodology. This integrative design methodology is proposed to have benefit for not only comprehensive INR design but also complex interactive system design in general.

As the application of interactive media systems expands to address broader problems in health, education and creative practice, they fall within a higher dimensional space for which it is inherently more complex to design. In response to this need an emerging area of interactive system design, referred to as experiential media systems, applies hybrid knowledge synthesized across multiple disciplines to address challenges relevant to daily experience. Interactive neurorehabilitation (INR) aims to enhance functional movement therapy by integrating detailed motion capture with interactive feedback in a manner that facilitates engagement and sensorimotor learning for those who have suffered neurologic injury. While INR shows great promise to advance the current state of therapies, a cohesive media design methodology for INR is missing due to the present lack of substantial evidence within the field. Using an experiential media based approach to draw knowledge from external disciplines, this dissertation proposes a compositional framework for authoring visual media for INR systems across contexts and applications within upper extremity stroke rehabilitation. The compositional framework is applied across systems for supervised training, unsupervised training, and assisted reflection, which reflect the collective work of the Adaptive Mixed Reality Rehabilitation (AMRR) Team at Arizona State University, of which the author is a member. Formal structures and a methodology for applying them are described in detail for the visual media environments designed by the author. Data collected from studies conducted by the AMRR team to evaluate these systems in both supervised and unsupervised training contexts is also discussed in terms of the extent to which the application of the compositional framework is supported and which aspects require further investigation. The potential broader implications of the proposed compositional framework and methodology are the dissemination of interdisciplinary information to accelerate the informed development of INR applications and to demonstrate the potential benefit of generalizing integrative approaches, merging arts and science based knowledge, for other complex problems related to embodied learning.

Millions of Americans live with motor impairments resulting from a stroke and the best way to administer rehabilitative therapy to achieve recovery is not well understood. Adaptive mixed reality rehabilitation (AMRR) is a novel integration of motion capture technology and high-level media computing that provides precise kinematic measurements and engaging multimodal feedback for self-assessment during a therapeutic task. The AMRR system was evaluated in a small (N=3) cohort of stroke survivors to determine best practices for administering adaptive, media-based therapy. A proof of concept study followed, examining changes in clinical scale and kinematic performances among a group of stroke survivors who received either a month of AMRR therapy (N = 11) or matched dosing of traditional repetitive task therapy (N = 10). Both groups demonstrated statistically significant improvements in Wolf Motor Function Test and upper-extremity Fugl-Meyer Assessment scores, indicating increased function after the therapy. However, only participants who received AMRR therapy showed a consistent improvement in their kinematic measurements, including those measured in the trained reaching task (reaching to grasp a cone) and in an untrained reaching task (reaching to push a lighted button). These results suggest that that the AMRR system can be used as a therapy tool to enhance both functionality and reaching kinematics that quantify movement quality. Additionally, the AMRR concepts are currently being transitioned to a home-based training application. An inexpensive, easy-to-use, toolkit of tangible objects has been developed to sense, assess and provide feedback on hand function during different functional activities. These objects have been shown to accurately and consistently track hand function in people with unimpaired movements and will be tested with stroke survivors in the future.

Motion capture using cost-effective sensing technology is challenging and the huge success of Microsoft Kinect has been attracting researchers to uncover the potential of using this technology into computer vision applications. In this thesis, an upper-body motion analysis in a home-based system for stroke rehabilitation using novel RGB-D camera - Kinect is presented. We address this problem by first conducting a systematic analysis of the usability of Kinect for motion analysis in stroke rehabilitation. Then a hybrid upper body tracking approach is proposed which combines off-the-shelf skeleton tracking with a novel depth-fused mean shift tracking method. We proposed several kinematic features reliably extracted from the proposed inexpensive and portable motion capture system and classifiers that correlate torso movement to clinical measures of unimpaired and impaired. Experiment results show that the proposed sensing and analysis works reliably on measuring torso movement quality and is promising for end-point tracking. The system is currently being deployed for large-scale evaluations.