'Singing gloves' turn motion into music
(Credit: Video screenshot by Bonnie Cha/CNET)
Hand gestures can add a lot to a conversation. They can convey excitement and help you describe a scene or object. And, of course, the simple act of lifting a certain finger can quickly let someone know you're not too happy with them. In all, gestures are an effective form of communication, and now, researchers in Vancouver have found a way to take them to the next level.
A team of engineers from the University of British Columbia has developed a pair of gloves that read hand gestures and convert them to speech and song, potentially giving those with speech and/or hearing disabilities another way to communicate.
The project, called Digital Ventriloquized Actor (DIVA) and led by UBC professor of electrical and computer engineering Sidney Fels, tries to replicate the movements of real vocal cords through the use of hand gestures read by a system of sensors.
The right glove features 3D motion sensors that can detect whether your hand is open or closed. When open, DIVA produces vowel sounds (much like real life, where vowels require the throat and lips to be open), while closing your hand creates consonant sounds. Meanwhile, the left-hand glove has finger contacts that produce stop consonants, such as "b" and "p," when touched together.
What's unique about DIVA compared with other text-to-speech systems is its ability to sing. The 3D sensors allow the system to detect where your right hand is in a three-dimensional space, so you can change the pitch by moving your hand up, down, and around. In addition, a foot pedal allows you to control the volume.
As you'll see in the end of the video below, the results are, well, frankly a bit creepy. Mastering DIVA isn't particularly easy either, as it requires about 100 hours of practice to pick up all the sounds. More than a musical instrument, though, I see DIVA as a great tool for those who are hearing impaired, as it allows them to communicate with more people without the need for an interpreter or sign language.
Fels said DIVA could also be used in the future to control heavy machinery and admitted that while the current system is quite cumbersome, he and his team are working on a model that can be operated using a tablet.
Via CNET Crave
Be the first to comment
Sign In with Facebook or create an account to post a review.
