'Mind-Reading' Device Can Translate Your Brain Activity Into Audible Sentences

Have you ever had such an engrossing inner monologue that you felt inclined to ask: "Did I just say that out loud?"

For most of us, the thought of accidentally blabbing our inner voice in public is a mortifying one; the conversations we have with ourselves are often fraught with secret feelings or social faux pas.

But now a breakthrough new technology developed by researchers at the University of California, San Francisco (UCSF), threatens to make us all paranoid about the content of our private musings. It's a veritable mind-reading implant capable of translating your brain activity into synthetic speech, and it's shockingly accurate, reports MedicalXpress.com.

Not only does the technology translate sentences that you're thinking in your brain into audible speech, but the synthetic voice that gets generated operates with a virtual vocal chord that can mimic your manner of speaking, too. So any meaning that's contained in your inflections or emphases — like when conveying sarcasm, for instance — will also come across.

It's pretty spooky how accurate it is. You can hear some examples in the video provided by UCSF at the top of this story.

"For the first time, this study demonstrates that we can generate entire spoken sentences based on an individual's brain activity," said Edward Chang, MD, a professor of neurological surgery and member of the UCSF Weill Institute for Neuroscience.

Of course, the purpose of the technology is not to snoop on everyone's secret thoughts, though it could certainly be used in this way. Rather, it has real medical benefits for individuals who have lost the ability to speak, such as individuals suffering from conditions like locked-in syndrome, ALS, or paralysis.

"This is an exhilarating proof of principle that with technology that is already within reach, we should be able to build a device that is clinically viable in patients with speech loss," said Chang.

Virtual vocal tract is the key

The alarming accuracy of the device really comes down to the development of the virtual vocal tract, which is an anatomically detailed computer simulation that includes the lips, jaw, tongue, and larynx.

What researchers realized was that much of what our brain encodes when we talk out loud are instructions that coordinate the movements of the mouth and throat during speech. Previous research had tried to directly represent the acoustic properties of speech sounds from brain activity, which proved futile. Brains don't operate on that level. Rather, they direct muscular movements, and it's the muscular movements (in our throat and mouth, mostly), that produce the acoustic speech.

"The relationship between the movements of the vocal tract and the speech sounds that are produced is a complicated one," said Gopala Anumanchipalli, who led the research team. "We reasoned that if these speech centers in the brain are encoding movements rather than sounds, we should try to do the same in decoding those signals."

The virtual vocal tract was perfected by modeling it on subjects that could still speak. But researchers found that the neural code for vocal movements partially overlapped across individuals, such that one subject's vocal tract simulation could be adapted to respond to the neural instructions recorded from another subject's brain. In other words, there's enough universality to create a generalized algorithm here, so that speech can be translated from inner dialogue even from subjects that the device was never designed for.

Of course, it's not a perfect mind-reading machine. It can only translate inner thoughts that are encoded for the purpose of vocal speech. So it only works for inner dialogue, not for all thoughts, some of which might merely be mental images, representations, or non-linguistic feelings. Still, imagine all of the private conversations that go on in your head that it could translate.

"I'm proud that we've been able to bring together expertise from neuroscience, linguistics, and machine learning as part of this major milestone towards helping neurologically disabled patients," said Anumanchipalli.