Can AI Help Voice Recognition?

Written: May 2023

Word Count: 1,042. Reading time: 3.9 minutes

Artificial intelligence and facial recognition have the potential to offer many benefits. Travelers will be very happy to have a faster process for boarding a flight. Citizens may feel more secure knowing criminals can be apprehended. Business transactions may be streamlined with facial recognition, and mobile devices will be more secure if no one can access data from them but the owner. The tradeoff is potential loss of privacy and civil liberty. How can we be sure an airline or a bank will not suffer a data breach where our faceprints are stolen by bad actors? Will we be able to trust our local and national governments are not spying on us? It is clear some form of regulation is needed. In addition to technology to use AI with our faces, AI and our voices also has great potential. Voice is the subject of this article.

If a friend or relative calls you on the phone, and something is wrong in his or her life, you can tell immediately. Could an AI tell also? Yes, and a whole lot more. Using the same AI machine learning technology used to tell a cat from a dog or recognize a person’s face, an AI can be trained to recognize your voice. An AI with access to a database containing many voice samples along with a description of whose voice each sample is associated with, the AI can recognize you. Based on characteristics of the voice and what state of mind those characteristics are associated with, it can also know if you are not feeling well, are upset about something, or in a big hurry. Some of the characteristics of voice data which can be detected include tone, tempo, volume, language, dialect used, and other voice characteristics.

John McCormick, Deputy Editor of WSJ Pro Artificial Intelligence, wrote an excellent article about voice recognition called, “What AI Can Tell From Listening to You”.  McCormick said that by analyzing voice data, an AI can determine a person’s emotions, mental and physical health, and detect if you are depressed, in danger of a heart attack, or dozing at the wheel of your car. AI voice technology is already in use in a number of application areas. For example, Google released an algorithm in June 2019 which can translate what you say and say it back in another language BUT in your voice.

A major area of opportunity using voice recognition is in mental health. According to the National Institute of Mental Health, an estimated 57.8 million adults in the United States, or 1 in 5 adults, had a mental illness in 2021. This includes both mild and severe mental illnesses. The most common mental illnesses in the United States are:

• Anxiety disorders (25.4%)
• Depressive disorders (17.3%)
• Substance use disorders (15.1%)
• Bipolar disorder (2.6%)
• Schizophrenia (1.1%)

Mental illnesses can have a significant impact on a person’s life, affecting their ability to work, go to school, maintain relationships, and take care of themselves. The Institute of Mental Health estimates only half of those needing treatment receive it. Emerging AI and voice technology may be able to make problems easier to spot.

CompanionMX, founded in 2015 by Dr. David Moher, a professor of psychiatry at Harvard Medical School, is a digital health company that develops and markets a mobile app called Companion which uses voice analysis to track and predict mental health conditions. Patients are encouraged to talk to the app describing how they are feeling. The app is designed to be used by people with mood disorders, such as depression and bipolar disorder. Companion uses AI-powered algorithms to analyze a user’s voice recordings and identify changes in their emotional state. These changes can then be used to predict the onset of a mood episode. Companion can also be used to monitor the effectiveness of treatment and to track progress over time. Clinicians can potentially make better decisions to improve the mental health of the patient. Researchers who have studied the Companion system have found it very encouraging.

CompanionMX is one of a number of companies which are developing digital health solutions for mental health conditions. The development of digital health solutions for mental health is a rapidly growing field. These solutions have the potential to not only improve access to care, but reduce the stigma associated with mental illness.

Another interesting area of opportunity using voice recognition and AI is keeping drivers awake. According to the National Safety Council, each year, drowsy driving accounts for about 100,000 crashes, 71,000 injuries, and 1,550 fatalities. Today’s cars have a lot of computing power which can help. Many cars already have voice recognition for making phone calls or telling the car GPS system where you want to go. Some cars have external cameras to help avoid collisions. Cars could also have a camera on the dashboard watching you. Many companies are designing AI which uses voice analysis combined with facial recognition to assess the alertness and emotional state of a driver.

Toyota is using AI facial recognition to find drivers who are not alert. The technology, developed by ITOCHU Techno-Solutions, uses cameras to track the driver’s eyes and face to determine if they are paying attention. If the driver is not alert, the system will sound an alarm and alert the driver. It could even blast some music.

Some of the benefits of using AI facial recognition to find drivers who are not alert include preventing accidents, improving driver safety, and reducing driver fatigue. There are challenges including high cost to develop and implement the technology. The system can be inaccurate, especially if the driver is wearing sunglasses or a hat. Some people, perhaps many, would find the system to be intrusive, as it is constantly monitoring the driver’s face. There may also be privacy concerns.

Overall, AI voice and facial recognition have the potential to be valuable tools. However, like all things AI, it is important to weigh the benefits and challenges of the technology before implementing it. Europe and China are moving quickly to put guardrails in regulations. The United States Congress continues to be laggards. There is work underway but moving at a snail’s pace.