Development of an American Sign Language Recognition System using Canny Edge and Histogram of Oriented Gradient
Keywords:
American Sign Language, Recognition system, Otsu algorithm, Feature extraction, Histogram of oriented gradient, K-Nearest Neighbour (K-NN)Abstract
Sign language is used by people who have hearing and speaking difficulties, but not understood by many without these difficulties. Therefore, sign language recognition systems are developed to aid communication between hearing impaired people and others. This paper developed a static American Sign Language Recognition (ASLR) system using canny-edge and histogram of oriented gradient (HOG) for feature extraction with K-Nearest Neighbour (K-NN) as classifier. The sign language image datasets used consist of English alphabets from both Massey University and Kaggle, and numbers (0-9) from Massey University. Median filter was used to remove noise after images were converted to grayscale. Otsu algorithm was used for segmentation while edges in the images were preserved using canny edge detection technique with HOG parameters tuning to obtain feature vectors. The extracted features were used by K-NN for classification. An average recognition accuracy and computational testing time of 97.6% and 0.39s respectively were obtained based on experiments with the Massey University dataset. Similarly, an average recognition accuracy and computational testing time of 99.0% and 0.43s respectively were obtained based on experiments with the Kaggle dataset. The developed system successfully recognized static English alphabets and numbers and outperformed some existing systems.
References
Adeyanju, I.A.; O. O. Bello and M. A. Adegboye. (2021). Machine learning methods for sign language recognition: A critical review and analysis. Intelligent Systems with Applications, 12, (2021): 1-36. https://doi.org/10.1016/j.iswa.2021.200056.
Alabi, S. (2019) Design and Implementation of a Robotic Hand for Sign Language Communication. Unpublished Undergraduate Project, Department of Computer Engineering, Federal University Oye-Ekiti (FUOYE), Nigeria.
Barczak, A. L. C.; N. H. Reyes; M. Abastillas; A. Piccio and T. Susnjak. (2011). A New 2D Static Hand Gesture Colour Image Dataset for ASL Gestures. Res. Lett. Inf. Math. Sci.15: 12–20
Brahmankar, V.; N. Sharma; S. Agrawal; A. Saleem and P. Borse. (2021). Indian Sign Language Recognition Using Canny Edge Detection. International Journal of
Advanced Trends in Computer Science and Engineering, 10 (3): 1576–1583. https://doi.org/10.30534/ijatcse/2021/131032021
Cheok, M. J.; Z. Omar and M. H. Jaward. (2019). A review of hand gesture and sign language recognition techniques. International Journal of Machine Learning and Cybernetics 10 (1): 131–153 https://doi.org/10.1007/s13042-017-0705-5
Dalal, N. and Triggs, B. (2005). Histograms of Oriented Gradients for Human Detection. Presented at the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, 886–893, IEEE. https://doi.org/10.1109/CVPR.2005.177
Das, P.; T. Ahmed and Md. F Ali. (2020). Static Hand Gesture Recognition for American Sign Language using Deep Convolutional Neural Network, in: 2020 IEEE Region 10 Symposium (TENSYMP). Presented at the 2020 IEEE Region 10 Symposium (TENSYMP), Dhaka, Bangladesh, 1762–1765, IEEE. https://doi.org/10.1109/TENSYMP50017.2020.9230772
Dudhal, A.; H. Mathkar; A. Jain; O. Kadam and M. Shirole. (2019). Hybrid SIFT feature extraction approach for Indian sign language recognition system based on CNN. Presented at the 2018 International Conference on ISMAC in Computational Vision and Bio-Engineering, 727-738, Springer, Cham. https://doi.org/10.1007/978-3-030-00665-5_72
Hartanto, R.; A. Susanto and P. I. Santosa. (2014). Real time static hand gesture recognition system prototype for Indonesian sign language. Presented at the 6th International Conference on Information Technology and Electrical Engineering (ICITEE), Yogyakarta, Indonesia, 1-6, IEEE. https://doi.org/10.1109/ICITEED.2014.7007911.
Hasan, M. M.; M. Khaliluzzaman; S. A. Himel and R. T. Chowdhury. (2017). Hand sign language recognition for Bangla alphabet based on Freeman Chain Code and ANN, in: 4th International Conference on Advances in Electrical Engineering, ICAEE 2017. Institute of Electrical and Electronics Engineers Inc., 749–753. https://doi.org/10.1109/ICAEE.2017.8255454
Hassan, S. T.; J. A. Abolarinwa; C. O. Alenoghena; S. A Bala; M. David and P. Enenche. (2018). Intelligent Sign Language Recognition Using Image Processing Techniques: A Case of Hausa Sign Language. ATBU Journal of Science, Technology and Education 6 (2):
Huong, T. N. T.; T. V.; Huu; T. L. Xuan and S. V. Van. (2016). Static hand gesture recognition for Vietnamese sign language (VSL) using principal components analysis. Presented at International Conference on Computing, Management and Telecommunications (ComManTel), DaNang, Vietnam, 138–141, IEEE https://doi.org/10.1109/ComManTel.2015.7394275.
Jain, V.; A. Jain; A. Chauhan; S. S. Kotla and A. Gautam. (2021). American Sign Language recognition using Support Vector Machine and Convolutional Neural Network. International Journal of Information Technology 13 (3): https://doi.org/10.1007/s41870-021-00617-x
Jimoh, K.; T. M. Adepoju; A. A. Sobowale and O. A. Ayilara. (2018). Offline Gesture Recognition System for Yorùbá Numeral Counting. Asian Journal of Research in Computer Science 1(4): 1–11. 1–11. https://doi.org/10.9734/ajrcos/2018/v1i424753
Jimoh, K.O.; A. O. Ajayi and I. K. Ogundoyin. (2020). Template Matching Based Sign Language Recognition System for Android Devices. FUOYE Journal of Engineering and Technology, 5(1): 42-48. https://doi.org/10.46792/fuoyejet.v5i1.465
Jin, C. M.; Z. Omar and M. H. Jaward. (2016). A mobile application of American sign language translation via image processing algorithms, in: Proceedings Presented at the 2016 IEEE Region 10 Symposium (TENSYMP), Bali, Indonesia, 104–109. https://doi.org/10.1109/TENCONSpring.2016.7519386
Kulkarni, V. S. and S. D. Lokhande. (2010). Appearance Based Recognition of American Sign Language Using Gesture Segmentation. International Journal on Computer Science and Engineering (IJCSE), 2 (3): 560–565.
Lawend, H. O.; A. M. Muad and A. Hussain, A. (2005). Robust Edge Detection Based on Canny Algorithm for Noisy Images. Journal of Theoretical & Applied Information Technology, 95(19): 5104- 5114.
Mahmud, I.; T. Tabassum; M. P. Uddin; E. Ali.; A. M. Nitu and M. I Afjal. (2019). Efficient Noise Reduction and HOG Feature Extraction for Sign Language Recognition, in: 2018 International Conference on Advancement in Electrical and Electronic Engineering, ICAEEE 2018. Institute of Electrical and Electronics Engineers Inc., 1–4. https://doi.org/10.1109/ICAEEE.2018.8642983
Maliki, R.; D. Alhaidar; K. Attallah; S. Alsalem; M. Morris and S. Tosunoglu (2017). Robotic Hands to Teach Sign Language 7. Presented at 30th Florida Conference on Recent Advances in Robotics, Boca Raton, Florida, 1-7.
Masood, S.; H. C Thuwal and A. Srivastava. (2018). American Sign Language Character Recognition Using Convolution Neural Network, in: Satapathy, S.C., Bhateja, V., Das, S. (Eds.), Smart Computing and Informatics, Smart Innovation, Systems and Technologies. Springer Singapore, Singapore, 403–412. https://doi.org/10.1007/978-981-10-5547-8_42
Mohammed, M. G. and Melhum A. I. (2020). Implementation of HOG Feature Extraction with Tuned Parameters for Human Face Detection. International Journal of Machine Learning and Computing, 10 (5): 654-661. https://doi.org/10.18178/ijmlc.2020.10.5.987
Shah, B. K.; V. Kedia; R. Raut; S. Ansari and A. Shroff. (2020). Evaluation and Comparative Study of Edge Detection Techniques. IOSR Journal of Computer Engineering (IOSR-JCE) 22 (5): 6–15.
Sharma, A.; A. Mittal; S. Singh and V. Awatramani. (2020). Hand Gesture Recognition using Image Processing and Feature Extraction Techniques. Procedia Computer Science 173:181–190 181–190. https://doi.org/10.1016/j.procs.2020.06.022
Tolentino, L. K. S.; R. O. Serfa Juan; A. C. Thio-ac; M.A. B. Pamahoy; J. R. R. Forteza and X. J. O. Garcia. (2019). Static sign language recognition using deep learning. International Journal of Machine Learning and Computing 9 (6): 821–827. https://doi.org/10.18178/ijmlc.2019.9.6.879.
Treat, S. (2016). Deaf Education. Available online at: https://prezi.com/ckdvqq0rv5cx/deaf-education/ (accessed 9.1.21).
Wadhawan, A. and Kumar, P. (2020). Deep learning-based sign language recognition system for static signs. Neural Computing and Applications 32 (12): 7957–7968. https://doi.org/10.1007/s00521-019-04691-y
World Health Organizations. (2021). Deafness and hearing loss - Key facts. Available online at: https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss Accessed on May 12.2022.
Yasir, F.; P. W. C. C Prasad; A. Alsadoon and A. Elchouemi. (2016). SIFT based approach on Bangla sign language recognition. Presented at 2015 IEEE 8th international workshop on computational intelligence and applications (IWCIA), 35-39, IEEE. https://doi.org/10.1109/IWCIA.2015.7449458
Additional Files
Published
Issue
Section
License
Copyright (c) 2022 Nigerian Journal of Technological Development
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
In accordance with the Copyright Act of 1976, which became effective January 1, 1978, the following statement signed by each author must accompany the manuscript submitted: "I, the undersigned author, transfer all copyright ownership of the manuscript referenced above to the Nigerian Journal of Technological Development, in the event the work is published. I warrant that the article is original, does not infringe upon any copyright or other proprietary right of any third party, is not under consideration by another journal, and has not been published previously. I have reviewed and approve the submitted version of the manuscript and agree to its publication in the Nigerian Journal of Technological Development." A copyright transfer form should be downloaded from the NJTD Website ( http://njtd.com.ng/index.php/njtd). Author(s) will be consulted, whenever possible, regarding republication of material. All authors must have access to the data presented and the authors and sponsor (if applicable) must agree to share original data with the editor if requested.
