Skin electrodes turn silent speech into words after a patient’s laryngectomy

A man who had undergone removal of his voice box silently formed dates and times while electrodes on his face and neck recorded the movements. A machine-learning system converted those muscle signals into audible speech. In a separate rendering step, researchers used recordings made before surgery to give the output the sound of his former voice.[1]

The September pilot in Laryngoscope Investigative Otolaryngology involved one laryngectomy patient and one healthy reference speaker. It investigates a way to decode intended speech without an implanted interface. Its two achievements—recovering words and personalising their sound—were evaluated differently, a distinction that matters when interpreting the result.[1]

From muscle activity to audio

The researchers recorded surface electromyography, known as sEMG, using a montage of 20 electrodes on the face, jaw and neck. They also enrolled a healthy reference speaker. Each participant silently articulated a set of 499 prompts consisting of dates and times; a model learned the relationship between the electrical patterns and the target speech. The researchers reserved 19 phrases for validation and 19 for a fixed test, using the others to train the system.[1]

Surface electromyography records the electrical activity associated with muscle contraction. Here, the decoder turned those signals into an intermediate representation from which a speech synthesiser generated audio. The person deliberately articulated each prompt; the system was not decoding unspoken thoughts. Dates and times provided a constrained vocabulary with which to test the connection between movement and output.[1]

Five listeners transcribed the generated speech. The patient’s mean word error rate was 20.2%, with a reported 95% confidence interval of roughly 11.0–31.5%; the healthy speaker’s mean was 14.9%. Word error rate counts substitutions, omissions and insertions against a reference transcript. These measurements were made before the output was rendered in the participant’s own voice.[1]

For personalisation, the researchers used approximately eight minutes of pre-surgical recordings. That demonstration concerned the identity of the voice, while the listening test concerned the accuracy of the decoded words. The paper does not establish that the final personalised output has the same intelligibility score. Keeping those stages separate preserves what is distinctive about each.[1]

Toward a wearable interface

An analysis of electrode channels suggested that fewer sensors might capture much of the recorded signal: nine of the patient’s 20 channels reconstructed approximately 95% of the multichannel information. In the healthy speaker, 11 of 18 usable channels did so. That figure concerns signal reconstruction, not a 95% speech-recognition score or the demonstrated performance of a smaller wearable device.[1]

The experiment involved one laryngectomy patient, a single recording session and a restricted vocabulary. Dates and times can be easier to model than unconstrained conversation because the range of possible words and phrases is narrow. Electrode placement and signal quality can also vary day to day, especially when skin conditions and muscle fatigue change. The paper did not show robustness over weeks, automatic setup at home or conversations with strangers in uncontrolled noise.[1]

Existing rehabilitation offers several routes, including an electrolarynx, tracheoesophageal speech, esophageal speech and augmentative communication. ASHA’s clinical guidance stresses that anatomy, recovery, dexterity, preferences and access to follow-up influence the choice. A future muscle-sensing interface would join a set of options whose suitability differs from person to person.[2]

The next engineering work is concrete: preserve the signal when electrodes are repositioned, support a wider vocabulary and make recording comfortable enough for repeated use. A larger patient study could then test whether the approach helps people communicate across days and situations. The pilot establishes that intentionally produced muscle activity can carry recoverable speech information even after the larynx has been removed.[1]

Report an error or suggest a correction