Stretchable Silent Speech Glove using Myopotentials and Machine Learning

The field of Silent Speech Interface (SSI) exists as an interface for sending information without speaking. Personal and confidential information can be heard by others when spoken, but if speech can be recognized without voice, confidential information can be transmitted safely. In addition, silent speech can send information to machines and people without disturbing the environment. There are many methods for predicting what is uttered from silent speech, one of which is the use of muscle potentials around the mouth. Recently, it has been shown that muscle potentials can be measured by placing sensors around the mouth, but the integration of a mechanism that uses soft materials to recognize speech using electrodes, muscle potential amplification circuits, and machine learning models has not yet been realized. Although it is possible to place circuits and elements as well as sensors around the mouth, the area around the mouth is limited, and attaching sensors would interfere with mouth movements. The myopotential amplification circuit requires many elements to be placed on the circuit, which would result in a loss of aesthetics in terms of appearance.<br/><br/>To solve these problems, we developed a myopotential measurement device that can be attached to the human hand using soft materials, liquid metal, transparent FPCs (Flexible Printed Circuits), and a microcomputer. The device can be placed near the mouth and electrodes placed around the mouth to collect myopotential information, which can then be used in a machine learning model to classify speech only when speech recognition is performed without speech. By enabling sensing by placing electrodes on the mouth only when the device is being used, there is no need to worry about information being read at times other than when the device is being used, and since there is no need to always wear sensors or circuitry around the mouth, speech will not be read at unintended times. It does not interfere with the movement of the face or mouth, eliminating aesthetic concerns.<br/><br/>In this study, electrodes based on soft materials were created to realize a device that is worn on the hand and repeatedly measures myoelectric potentials around the mouth. The electrodes were wired to circuits on transparent polyimide film using a liquid metal mixture of Galinstan and nickel powder and bonded with a hard-soft pattern of silicone rubber and epoxy. The electrodes and substrate, which fit the human hand, can capture information around the mouth without breaking, even when the hand is moved, or the fingers are bent. In addition, a speech classification model was created using machine learning based on the acquired data of myoelectric potentials in silent speech around the mouth. We propose a new form of silent speech interface that allows many people to send information to machines and people in more situations and with the same privacy as in their conventional lives.