Neuroengineers have created a new system that can translate simple thoughts into recognizable speech, using artificial intelligence and a speech synthesizer, according to a study published recently.
A team of New York-based researchers was able to reconstruct words using only brain activity, an innovation that could pave the way for brain-controlled technologies like, say, a smartphone that can translate your thoughts into text messages.
Dr. Nima Mesgarani, an associate professor at Columbia University, led the study and tells Inverse that he sees great potential to help restore speech to people recovering from a stroke or living with amyotrophic lateral sclerosis (ALS). Further down the line, this type of tech could also open up doors to brain-connected smartphones that could let users text using their minds, though that’s still a ways away. His work was published in the journal Scientific Reports.
“One of the motivations of this work…is for alternative human-computer interaction methods, such as a possible interface between a user and a smartphone,” he says. “However, that is still far from reality, and at the moment, the information that can be extracted using non-invasive methods is not good enough for a speech brain-computer interface application.”
Listen to the speech generated by the brain-computer interface.
To develop the new technique, Mesgarani and his colleague, Dr. Ashesh Dinesh Mehta from the Northwell Health Physician Partners Neuroscience Institute, began by examining the brain activity of epilepsy patients for their study. These patients already had electrode implants in their brains to monitor seizures, which Mesgarani and Mehta were able to use to gather data for their research.
The duo asked willing participants to listen to speakers recite the numbers between zero and nine, and then recorded the brain signals from that interaction. Next, they trained a neural network — a program that imitates neuron structure in the human brain — to recognize patterns in the signals and translate them into robotic-sounding words using a speech synthesizer, known as a vocoder.
The result was a short voice clip of what sounds like Microsoft Sam counting from zero to nine. The impressive part is just how clear the speech is compared to other methods the researchers tested. There’s still a lot of work to be done, though.
“It may take a decade before this technology becomes available,” says Mesgarani. “We need more progress both in long-term, bio-compatible implantable electrodes and/or breakthrough technologies in non-invasive neural recording methods. We also need a better understanding of how the brain represents speech, so that we can refine our decoding methods.”
The patients who were a part of this study, for example, all had brain surgery to implant electrocorticography monitors. This is an extremely invasive process that requires open brain surgery, something that most people might not be willing to undergo, even if there was a possibility of restoring some of their speech capabilities.
For now, this study introduced a method for decoding brain signals into speech. If we figure out how to accurately detect brain activity without surgery, we’ll be one step closer to not only revolutionizing speech therapy, but potentially toward bringing about brain-connected smartphones.
Brain-computer interface research has been receiving newfound interest in the past few years. In April 2017, Facebook announced it was working on a BCI during its annual F8 conference. And Elon Musk announced in November 2018 that Neuralink, his own BCI startup, was hiring.
(For the source of this, and many other important articles, please visit: https://www.inverse.com/article/52812-brain-computer-interface-translates-thoughts-to-speech/)