A multimodal gait and ocular geometric representation to generate a Parkinson progression report

Parkinson's disease (PD) is a progressive neurological condition, primarily associated with a deficiency in dopamine neurotransmitters, generating premotor, motor control, emotional, and executive dysfunctions. The characterization and PD diagnosis are principally based on the analysis of observed motion alterations, such as slowed movements (bradykinesia), wrong posture, and freezing of gait. Computational methods to support some of these observations remain limited to distinguishing between PD patients and control patients on the basis of protocols at advanced stages, according to current clinical PD guidelines. This work introduces a multi-item PD progression support statistically consistent with both the modified Hoehn and Yahr (H&Y) scale and the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) part III, which is based on a multimodal geometric representation that combines gait and oculomotor markerless video sequences. The proposed representation evaluates gait autonomy, posture impairment, freezing, bradykinesia, bilateral gait issues, and ocular bradykinesia. To do so, a three dimensional (3D) convolutional neural network (CNN) first captures spatiotemporal PD patterns. Then, a Riemannian network learns second order relationships among observed patterns, which are further fused at early or intermediate stages to output multi-item PD predictions. In a retrospective study with 13 control subjects and 19 diagnosed Parkinson's patients, the proposed approach (early fusion) achieved F1-scores of 95% for bilateral impairment, 94% for gait autonomy, 81% for freezing of gait, 92% for wrong posture, 91% for gait bradykinesia, and 94% for ocular bradykinesia in Parkinson's patients. The proposed approach is a promising tool to support routine and standard clinical PD analysis.