Teen in Kazakhstan Develops AI Sign Language Translator

Fifteen-year-old Sanzhar Yerezhepov is developing an artificial intelligence (AI) translator to make online information more accessible to deaf people in Kazakhstan. Called Tusin (“Understand”), the platform converts speech and text into lifelike sign language videos that reproduce facial expressions as well as hand movements. Yerezhepov attends Daryn, a specialized school for gifted children in Kokshetau, northern Kazakhstan. A boarding school for children with hearing and speech impairments is located nearby. Watching its students communicate in sign language made him wonder how easily they could access information online, giving him the idea for Tusin. Yerezhepov is from Shiyeli, a small village in Kazakhstan’s Burabay district. Neither of his parents works in science or IT. He is now working with the Kazakhstan Society of the Deaf to test an early version of Tusin and says five organizations have expressed willingness to work with him. The project reached the semifinals of an IT competition at Astana Hub, Kazakhstan’s international technology park for startups. In an interview with The Times of Central Asia, Yerezhepov explains why translating into sign language is more complicated than adding subtitles. He discusses the medical startup he put on hold and what he hopes to change in Kazakhstan’s education system. TCA: Tusin translates online content into sign language, but many videos already have subtitles. Why are those sometimes not enough for deaf viewers? Yerezhepov: For some deaf people, especially those whose primary language is sign language, understanding written text can be difficult. First, subtitles are not always available in a language they are comfortable with. They are often automatically generated and do not convey information accurately. Second, sign language has its own grammar and structure. So even someone living in Kazakhstan may have difficulty understanding subtitles in Kazakh or Russian. Many deaf people can read subtitles fluently, but others need sign language translation. TCA: Isn’t sign language the same around the world? Yerezhepov: No. Different countries use different sign languages. There is no single universal sign language that everyone on Earth understands. TCA: So which sign language is used in Kazakhstan? Yerezhepov: Kazakhstan, like some other former Soviet countries, uses Russian Sign Language. We do not have a separate Kazakh Sign Language, although some signs have been adapted. TCA: How do you teach artificial intelligence to understand and reproduce signs? Who checks its work? Yerezhepov: We work with representatives of the Akmola branch of the Kazakhstan Society of the Deaf and with sign language interpreters. Together, we examine each sign, check how accurately it is reproduced, and analyze the results. We have already demonstrated our technology to deaf and hard-of-hearing people, and they understand what the platform communicates. TCA: A sign can change meaning depending on the situation. How does Tusin handle that? Yerezhepov: Quite well, although we are still working on it. The same sign can have different meanings depending on the context. That is exactly the problem we are addressing with artificial intelligence, which can take context into account and determine what is being communicated. Our platform...