An Intelligent Mobile Application for Assisting Visually Impaired in Daily Consumption Based on Machine Learning with Assistive Technology

Visually impaired people are facing many difficulties in daily life because of the disability in visual sensation. In daily consumption, they are hard to perform the transactions solely. They need to be assisted by other people in daily life, especially in purchasing, since they cannot recognize the different products with similar shapes and sizes. Also, in processing payment, visually impaired people are hard to differentiate the denominations (values) of banknotes. And many derived problems, such as cheating and providing the improper products, are being occurred. In this paper, an intelligent recognition system has been developed for visually impaired people in Hong Kong. The application system can help the users recognize canned foods and differentiate the nominal values of the banknotes so as to complete the monetary transactions efficiently and effectively. The application can be installed in smartphones with voice and vibration. By applying neural networks with assistive technology, the accuracies of recognizing banknotes and canned food are over 95% and 75%, respectively.