Unified Deep Learning Framework for ISL and ASL Sign Language Recognition

People with hearing and speech impairments can express their thoughts and feelings manually by doing hand gestures known as sign language. Sign language is not universal but is specific to a particular region or country. In order to facilitate communication between normal people and physically challenged people, an interpreter is needed. The unavailability of such interpreters motivates us to do research into making a virtual interpreter. The development and advancement of computing technology help us study, utilize, and evaluate different deep models like Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short Term Memory (LSTM) on Indian Sign Language (ISL) and American Sign Language (ASL) datasets for English alphabets. In our work, we have used both publicly available datasets and custom built datasets for analyzing the efficiency and robustness of these deep learning models. In addition to these models, our modified LSTM model gives a better accuracy of 77.52 % on Indian sign language English alphabets and 88.66 % on American sign language English alphabets.