EMBEDDED SIGN LANGUAGE TRANSLATOR: DEEP LEARNING-BASED RECOGNITION AND SPEECH CONVERSION OF NIGERIAN SIGN LANGUAGE GESTURES

Communication is a crucial aspect of human interaction; however, speech impaired individuals often face difficulties in expressing themselves to others unfamiliar with sign language. This study aims to address this challenge by proposing the design and implementation of a real-time hand gesture recognition system aimed at bridging this communication gap. The system utilizes a USB webcam connected to a Raspberry Pi microprocessor to capture hand gestures, which are processed using a trained YOLOv8 deep learning model. The model was specifically trained on 36 hand gestures representing the Nigerian Sign Language (NSL). Recognized gestures are displayed on an LCD screen and converted to audio using a Text-to-Speech (TTS) engine. The development process included data acquisition, annotation & pre-processing using Roboflow, model training and inference, as well as the integration of hardware components. Performance was evaluated using metrics such as precision, recall, and confusion matrix analysis where the model achieved an accuracy of 98.3% after being trained with 9604 hand gestures at 50 epochs. Experimental results show that the system is capable of accurately recognizing multiple static hand gestures under controlled lighting and positioning conditions. The solution offers a low-cost, portable means of facilitating communication for speech-impaired individuals, and provides a foundation for future enhancements such as dynamic gesture recognition and multi-language support.