From fever-identifying drones to 3D-printed ventilator components to the tracking of mobile location data, there’s no shortage of ways that technologists are harnessing modern tech to track (and hopefully help slow) the spread of the coronavirus pandemic.
Researchers from Carnegie Mellon University have come up with one of the most sophisticated and unlikely-sounding methods yet — but if it works it could be a game-changer for a world in which there’s a drastic shortage of proper COVID-19 testing kits. The idea? A free app that can diagnose COVID-19 simply by listening to and analyzing a user’s voiceprint.
“First and foremost, what we have here is a data-collection effort,” Bhiksha Raj, a professor at Carnegie Mellon working on the project, told Digital Trends. “After having analyzed and run preliminary tests on COVID and non-COVID voices and coughs, we believe that it may be possible to obtain a reliable flag for potential COVID by analyzing voice. We have set up a system to encourage people to contribute voice recordings. We need tens of thousands of recordings from COVID-infected subjects, and hundreds of thousands of uninfected people, both healthy and those with other problems. Our hope is that people will contribute.”
The app, titled the COVID Voice Detector, isn’t just collecting data for some as-yet-unspecified endpoint, however. The app is available right now for users to test out for themselves.
By visiting Carnegie Mellon’s dedicated website, users can submit to a battery of tests intended to determine their estimated “COVID-19 Score.” To gain this, they must first fill in a brief profile with information about their age, weight, sex, height, and the racial or ethnic group with which they most identify. They must also answer questions about any current cough or fever they’re currently exhibiting, along with any previous diagnosis of COVID-19. After that, it’s onto a series of audio tests such as coughing three times into your microphone, making prolonged “ooo” and “eee” sounds, and repeating the alphabet. The data is then submitted to researchers, who will use it to further hone the system. Within a minute, the user gets to find out their own results, which are presented as a sliding scale to suggest how concerned we should be.
“The COVID Score, as you currently get it, is more of a guess of how much the signatures in your voice match those of the other COVID patients we have thus far analyzed,” Raj acknowledged. “Coughs of people with infected lungs sound different. The elongated vowels like ‘aaaa’ not only sound different, but the duration to which they can be kept up is reduced. When you speak a long string of numbers, followed immediately by the alphabet, you’re being asked to speak for an extended period of time. COVID patients tend to get breathless, have runny noses, somewhat sore throats, and tire easily. All of these will affect the way they speak — particularly when they must speak for longer durations.”
But does it work?
Of course, the multi-trillion dollar question (at least, measured by the amount of money the stock market has lost since the coronavirus pandemic began) is just how well this works. As with so much about coronavirus — from questions about whether or not those who catch it develop immunity to how long our current coronavirus lockdown will last — the simple answer is that we don’t yet know.
The COVID Voice Detector team stresses that this is still an in-development tool that should not be considered as a valid alternative for an official diagnosis. “I understand that this is an experimental system which is still under development,” reads a checkbox on the website when the user creates their profile. “It is not a diagnostic system. It has not been verified by medical professionals. It is not FDA or CDC-approved, and must not be used as a substitute for a medical test or examination.”
Nonetheless, there is some impressive talent involved. This is no fly-by-night operation. Rita Singh, a professor of computer science at Carnegie Mellon who took a lead role in developing the A.I., has long worked toward identifying micro-signatures in human voices. She believes that the voice can be used to mine a plethora of psychological, physiological, and medical data about individuals. (Other researchers have previously used the voice as a possible source of diagnostic data for diseases such as Parkinson’s.)
Meanwhile, the COVID Voice Detector’s use of a lung capacity score was created by an A.I. company called telling.ai, drawing on multiple years of research, in collaboration with hospitals and doctors, to develop an accurate measure of lung capacity. (What is not clear at this stage, due to lack of data, is how this relates to COVID-19, which is known to attack the lungs.)
The availability of tools such as the COVID Voice Detector highlights a current challenge faced by the tech world. For some time, it has been evident that there is a conflict between Silicon Valley’s famous mantra of “move fast and break things” and the real world’s need for empirical proofs, proper testing, and verification. Put simply, technologists’ belief in hacking together tools and pushing them out into the world, safe in the knowledge that agile methodology will let them tweak and hone until the finished product works as well as hoped, doesn’t always mesh well with the slow, but steady, approach needed for peer review and clinical testing. This challenge is particularly pronounced in the field of medicine.
This is further tested by the coronavirus pandemic where a third factor is added in: Lack of time and resources. This scenario encourages speed-based experimentation, which must be adopted wisely to ensure that the best ideas can be rapidly deployed, without giving snake oil solutions oxygen that they don’t deserve.
Move fast and… treat things?
Which of the two solutions will the COVID Voice Detector fall into? Its own developers freely admit that there is much that still needs to be proven about efficacy. Bhiksha Raj notes that, for instance, just because someone gets a possible match score for coronavirus it does not necessarily mean that they do. There has simply not been enough analysis of COVID-19 patients to make this link clear. “I do not, by any means, want anyone to mistake it for professional or semi-professional opinion and take healthcare decisions based on [this],” Raj said. “If they did, they could endanger themselves and others.”
This is, at best, currently a triage tool that could prompt people to seek out further medical opinions. But if it functions as hoped, the potential upside could be huge.
“If this works, we will have a very simple and easy way of monitoring millions of people,” Raj said. “Not only can we get instantaneous evaluations, but also look at longitudinal trends among subjects who use it repeatedly. This could provide a way tracking health outbreaks in general in future — particularly ones that affect voice.”
For this reason, Carnegie Mellon plans to share the data it gathers with other researchers around the world, encouraging them to work on it either collaboratively or independently. The one catch? “We will need the assurance from anyone who uses our data that they will not be protecting or commercializing any IP that comes out of it either.”
