Learning sign language with IA
Facts about deafness and sign language
- 466 million of deaf people in the World
- 34 million of deaf children
- 200h to learn sign language (3h per week online)
- Many impacts on child development
- 1.1 billion people between 12-35 years old are at risk
- 4-5 million deaf people in France but 80-120k speakers
Nowadays, sign language is becoming more and more democratic. As proof, some international brands have adapted their logos.
credit: Amazon brand
credit: Burger King brand
The project
This project is based on a simple observation: although it is essential for the integration and socialization of people with deafness, sign language is not widely spoken throughout the world.The richness of this language today requires more than 200 hours of online work to achieve a partial mastery. The goal is therefore to simplify and make more playful a tedious learning process. In our case this requires the use of an AI capable of recognizing the different signs and assessing the degree of accuracy of execution.
"In public, I sometimes feel rejected or isolated. This must and can change" Nyle Di Marco (a famous deaf model)
How does it works ?
1.The recording of as many signs as possible by an experienced speaker. This will be done step by step thanks to a Teachable Machine as presented on BS : teachablemachine.com Once the library has been established, the aim is to use it to train the user to reproduce the signs.
credit: youtube.com
2.Classification: According to their meaning, their degree of use, their theme, their difficulty. This will be done by a sign language expert.This will allow, as the user progresses, to propose adapted content. Learning begins with simple basic signs and ends with learning idiomatic expressions.
credit: pinterest.com
3.Setting up the platform: the user logs in to his account. Depending on his level of progress, a session of a duration yet to be determined (30min - 1h) is proposed to him. The screen is divided into three parts: on the left is the example previously recorded by the expert, in the middle is the word to be translated and on the right is the user's webcam. The aim is to reproduce the sign shown. The AI will determine how successful the execution is.
4.The final use: The more the user uses the platform, the more he progresses and the more the algorithm proposes signs, then complex expressions.In the future, we can imagine launching the possibility of learning with friends/family. If a relative is a sign language speaker, he or she could record his or her own examples, so that the user can copy them. This would make learning even more fun.
Which users ?
- Deaf people: strangely the majority of deaf people do not master this language.
- Deaf children: for them, a good mastery is essential for social integration. The sooner they learn, the better it is.
- The parents of a deaf child: for the majority, their child is the first deaf person they meet, so it requires a quick adaptation.
- You ! : even if deafness does not affect you personally, mastering this language is an important plus and a nice gesture of social acceptance.
Some competitors
-
Mobile applications
- SignEveil
- Bébé signe
- Elix
Some sources
Articles and books : Sign Language Recognition (Helen Cooper), Sign Language Recognition and Translation with Kinect (Xiujuan Chai, Guang Li), Language, Cognition, and the Brain: Insights From Sign Language Research (Karen Emmorey)
A project by Justin Dupont