As shown in the diagram above there are a variety of methods available which are capable of reading neuronal LFP’s (local field potentials) . LFP’s are basically electrical signals produced by individual neurons or neuronal clusters when they they are activated .
Specific functions in the human body are governed by specific areas of the brain .
As we can see from the above diagram the area that governs human speech is termed Broca’s area , named after French surgeon Paul Broca .
One particular neural recording method is termed electrocorticography (ECoG) .
ECoG requires a MEMS implant which involves invasive brain surgery .
This method differs from EEG recording which is non-invasive as the electrodes are placed over the cranium and scalp .
ECoG has a much higher resolution and accuracy than EEG recording and is less susceptible to signal noise .
As can be seen in the above diagram an ECoG implant can take two specific forms :
- A flexible electrode grid which sits under the cranium and lies on the surface of the brain .
- An electrode array , which again sits under the cranium but actually penetrates the brain tissue .
Scientists have used the ECoG method to read , record and analyze a wide variety of neural signals .
For example scientists have used cranial electrode implants to record the neural signals which govern motor functions in a living monkey , the neural signals have then been decoded in real time and used to control a robotic arm .
- Brain Input Aids Devices That Move Injured Or Artificial Limbs
- DARPA funded Monkeys Consciously Control a Robot Arm Using Only Brain Signals
To illustrate how varied the research in this particular field is , another scientist claims that he has decoded the pattern of electrical impulses and neural signaling which form memories .
- Memory Implants : A maverick neuroscientist believes he has deciphered the code by which the brain forms long-term memories.
In fact , software and computerized tools are available that can measure and analyze almost any electrical signal from any area of the human brain .
- NeuralAct : A Tool to Visualize Electrocortical (ECoG) Activity
- Recording Human Electrocorticographic (ECoG) Signals for Neuroscientific Research and Real-time Functional Cortical Mapping
As can be seen in the acknowledgments in the above links , a large portion of the research is funded by the military .
Speech from ECoG
Continuous speech has been decoded from ECoG implants using specialized software .
A few research papers are listed below with key excerpts from the papers .
“Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings.Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation.”
“The high temporal and spatial resolution of ECoG recordings allowed us to trace the temporal dynamics of speech production through the areas in the brain relevant for continuous natural speech production.”
- Automatic Speech Recognition from Neural Signals: A Focused Review
- Automatic Speech Recognition from Neural Signals: A Focused Review (PDF)
“Automatic Speech Recognition is a technology that enables the recognition of spoken language into a textual representation by computers.”
“KEY CONCEPT 1. Brain-computer Interfaces (BCIs)
A Brain-Computer Interface is a system which sends messages or commands to a computer without using the brain’s normal output pathways of peripheral nerves and muscles.”
“KEY CONCEPT 2. Automatic Speech Recognition (ASR)
Automatic Speech Recognition is a technology that enables the recognition of spoken language into a textual representation by computers. These technologies often rely on statistical models like Hidden-Markov-Models and can now be found in a large variety of consumer electronics from cars to mobile phones.”
“KEY CONCEPT 3. Phone
A phone is a distinct speech sound that can be perceptually differentiated from other speech sounds.”
“KEY CONCEPT 4. ECoG Phone Models
ECoG phone models can be used to estimate the likelihood that an internal of ECoG activity is a certain phone. This generative models might for example return that newly recorded data have a probability of 0.6 of being a /l/, but only a probability of 0.1 of being a /b/.”
“Conclusion and Discussion
We show that ECoG is the most promising technique and demonstrate how audibly spoken speech can be recognized from ECoG data using ASR technology in our Brain-to-text system.”
“Continuous speech production is a highly complex process involving many parts of the human brain. To date, no fundamental representation that allows for decoding of continuous speech from neural signals has been presented. Here we show that techniques from automatic speech recognition can be applied to decode a textual representation of spoken words from neural signals. We model phones as the fundamental unit of the speech process in invasively measured brain activity (intracranial electrocorticographic (ECoG)) recordings. These phone models give insights into timings and locations of neural processes associated with the continuous production of speech and can be used in a speech recognizer to decode the neural data into their textual representations.”
“Electrical activity measured from the cortex, or electrocorticography (ECoG), offers several advantages over other neuroimaging modalities for characterization and real-time decoding of brain activity. Specifically, ECoG is well-suited for the study of speech and language owing to its unique spatial and temporal resolution capabilities that allow it to accurately capture the fast-changing dynamics of the large cortical networks underlying speech processing .”
So there we have it , ‘silent speech’ can be recorded , measured , analyzed and decoded in real time . In other words , no pun intended , these particular systems can actually decode the thought patterns of individual words from the human brain .
Part II coming soon (ish)