Bioengineers develop glove that translates sign language into speech

The glove developed by bioengineers from UCLA, can translate American Sign Language into English speech.

The system, which includes a pair of gloves with thin, stretchable sensors that run the length of each of the five fingers, is made from electrically conducting yarns.

The yarns locate hand motions and finger placements that stand for individual letters, numbers, words and phrases.

The device then turns the finger movements into electrical signals, which are sent to a coin–sized circuit board worn on the wrist.

This board transmits those signals wirelessly to a smartphone that translates them into spoken words at the rate of about one word per second.

Jun Chen, assistant professor of bioengineering at UCLA Samueli School of Engineering and the principal investigator on the research, said that the device offers “hope” for people who use sign language to communicate and for non-signers without needing someone else to translate for them.

“In addition, we hope it can help more people learn sign language themselves,” she said.

The research, which is published in the journal Nature Electronics, was helped by four participants who are deaf and use American Sign Language.

Along with the glove, researchers also added adhesive sensors to the testers’ faces — in between their eyebrows and on one side of their mouths — to capture facial expressions that are a part of American Sign Language.

The wearers then repeated each hand gesture 15 times.

A custom machine-learning algorithm turned these gestures into the letters, numbers and words they represented.

The system remarkably recognised 660 signs, including each letter of the alphabet and numbers 0 through 9.

Though previous devices have been developed, those systems that offered translation from American Sign Language were limited by bulky and heavy device designs or were uncomfortable to wear, Chen said.

Another positive from this device is how lightweight and inexpensive but long-lasting its stretchable polymers and electric sensors are.

In addition to Chen, the study’s UCLA authors included co-lead author Zhihao Zhao, Kyle Chen, Songlin Zhang, Yihao Zhou and Weili Deng.

All co-authors are a part of Chen’s Wearable Bioelectronics Research Group at UCLA, except Jin Yang, of China’s Chongqing University.

While UCLA has filed for a patent on the technology, Chen said a commercial model based on this technology would require added vocabulary and a faster translation time.

---