MobiEye: An Efficient Shopping-Assistance System for the Visually Impaired with Mobile Phone Sensing

The lack of rich visual information affects the shopping experience of the visually impaired (VI), including identifying and selecting commodities. Recent studies on VI assistance have focused on commodity identification but neglected to provide fine-grained and intuitive pick-up guidance, which is not user-friendly enough. Therefore, we propose a user-driven shopping assistance system to improve the shopping experience for VI users. We first conduct an in-depth interview with VI, then implement a prototype shopping assistance system-MobiEye, with real-time video analysis. Further, we evaluate the prototype system and identify two directions to optimizing the existing system: (1) Improving the pick-up accuracy in dense placement; and (2) reducing the latency and communication overhead. To address these two problems, we design a new guidance strategy for dense placements and propose a mobile-edge cocomputing strategy with a motion predictor and a communication gate to filter the transmitted images. Finally, we invited VI participants to evaluate the effectiveness and efficiency of MobiEye. The experimental results show that MobiEye achieved a 13% improvement in pick-up success rate and a 12 s reduction in average pick-up time compared with other shopping assistance systems.