Sign language learning using the hangman videogame

Abstract

This paper describes the preliminary study and development of a videogame aimed at learning the alphabet in Portuguese sign language, through gestures. Leap Motion Controller is used, allowing the detection of fingers and hands with high accuracy and resolution. The project is based on the well-known hangman game. The player inputs the gestures that represent the letters of the alphabet in the sign language. The Leap Trainer framework for gesture and pose learning and recognition was used. This framework is helpful to the development of the proposed game since it allows the recording of a gesture and a subsequent comparison with the recorded gesture, giving a match percentage result. There are many sign language learning games, but they are not interactive. The proposed game is primarily for children but also adequate for adults.