Sign language can be a powerful tool for people with hearing disabilities. However, there are still barriers between this language and the spoken ones. Intending to facilitate this interaction, a young group from Bolivia developed a computer program that translates Spanish words into signs in real-time. The creation was the winner of Solve for Tomorrow, in its first edition in the country (2023).
The students were between 16 and 17 years old and in the last year of secondary and compulsory schooling. In Bolivia, this stage of teaching has a humanistic and technical focus. At the end, students obtain a bachillerato degree, that is a diploma from a Technical School. “In the morning, basic subjects such as mathematics and social sciences are taken. In the afternoon, specialty areas are taken. These project students are from Computer Systems,” says the group´s mediator teacher, Alfredo Huanca.
When Solve for Tomorrow put out the call for registrations, several groups from the Educational Unit began to form ideas, as it was the case with “Visionarios” (“Visionaries”, in English), the real-time sign language detection program team. For them, the idea began in a class by teacher Huanca about artificial vision, the technology that uses computer intelligence combined with information obtained from images. In other words, based on what the machine sees, it can generate actions or make decisions.
The young people thought they might be able to turn signs into words. “They came with ideas, they already had had difficulties communicating with people with hearing disabilities and they wanted to help with their technical knowledge,” highlights the educator.
First, they thought about creating a physical prototype, with a screen, sensors, and microphone. But then they realized that everything could be done from the computer, with an internal microphone and camera. “We use a computer vision model that already has a database. So when you move your hand to make a signal that is in the system, you can already interpret what that represents in words or numbers,” he explains.
But artificial intelligence needs to be “taught” to do its job. The team then had to video record the words they would like the system to translate at that stage. The coordinator of the Humanistic Technical Baccalaureate of the Educational Unit was an important ally because she already knew sign language in depth and suggested the vocabulary that would be essential to include. “She also taught us about the precision of movements, pointing out that sometimes people do not show their hands well, so we needed to record the movements more slowly to improve the system’s ability to detect them,” the teacher explained.
Other support was from the Autonomous Municipal Government of El Alto, which has a Care Unit for People with Disabilities. “My coordinator and I decided to visit the Unit and they welcomed us with open arms. They told us that this technology is what they needed and they helped us do the word choice and the tests,” says teacher Alfredo Huanca. The Unit’s specialists were the recipients of the tests and gave their opinions. Since it was a prototype, they jointly decided to focus only on basic words and expressions, which the students learned and reproduced in the software, such as “Hello” and “What time is it?”
He states that the Solve for Tomorrow mentorships were very important to implement the STEM methodology and understand how class projects can contribute to sustainable development goals. “The program coordinators were always with me and, as an educator, I learned a lot”, evaluates.
We learn that we need not only to have patience and a lot of attention but also to always try to see things from another perspective to find a solution in programming, the teacher says.
Now, the software is ready to be implemented on any computer with an updated operating system and can be useful for educational institutions, hospitals, public management, and banks, among others. In 2024, the educator even intends to create a website for free downloading of the system, perhaps involving other students, becoming a school-wide project. In addition, they already received an invitation to ally with a local private university, after the results of Solve for Tomorrow came out. The objective is to still test the program with the final audience: people with hearing disabilities.
“The impact has been quite large not only in the group but also in the Educational Unit. The students of the lower grades now have this vision that it is possible to achieve such an achievement and colleagues from different areas evaluated the project positively and said that we have to continue doing this type of work,” reinforces the teacher.
Now, the young people from “Visionarios” are in university and one of them, José Pablo, is studying for a degree in Systems Engineering. “He has already seen that a lot can be done with technology and he wants to continue further,” believes Huanca.
