“Our feeling is that cough tracking is for everyone with lungs,” said Joe Brew, co-founder and CEO of Hyfe AI — the company behind the AI-powered CoughTracker app
But this actually undersells the life-changing potential of this little-known —but fascinating — health-monitoring technology.
To illustrate how tracking coughs could change — and one day even save — your life, Brew took us through the interesting beginnings of the CoughTracker app, how the company’s AI has grown to understand the difference between a cough and a dog barking, and how mobile technology has made it possible to track coughs in such detail.
Why you should track your coughs
If you’re like me, you probably don’t think much about your coughs, but Brew explained why this should change.
“Even if you don’t have a chronic cough, you don’t have tuberculosis, you don’t have one of those classic coughing diseases, but multiple times per year you get a cough just like other healthy people, and that’s valuable information. Just like having a thermometer in your bathroom cupboard is a useful thing, you are now able to measure this very prevalent, very frequent symptom,” Brew said.
“In the future, when it comes to treating cough — cough, by the way, is the most prevalent symptom that people present with at doctor’s offices — doctors will look back and wonder, ‘How did we used to do this before automated cough counting?’ [Now] they just ask the patient and expect the patient to know, but does anybody actually know how much they cough? Absolutely not.”
Doctors have been dealing with coughing patients for centuries, so surely they know all there is to know about coughs. Given all that, does tracking matter? This is where using AI and mobile tech, in the same way we count our steps, can change the way doctors deal with them.
“One-hundred percent of doctors will say, I use cough all the time. I look at cough, I treat cough, I diagnose based on cough,” Brew said. “They even use the sound of coughs, so absolutely, doctors understand that it matters. and they know that more cough is bad, less cough is good. But once you get into the super-granular stuff, doctors don’t know what to do because they have never sat by your bedside all night. There’s just no body of evidence because when you go to visit the doctor, they’re with you for 10 minutes and they can’t build up a data set that’s useful. Now, we’re offering a greater level of precision.”
Mobile tech makes it possible
Hyfe’s AI builds and analyzes that data set, but it couldn’t do so without having access to the sound of us coughing in the first place. Brew laughed as he said, “If you tried to build a universal cough tracker in the year 2000, good luck, right?” But we now carry devices that have all the sensors required to make cough tracking a reality. Brew considers wearables the best device for the job.
“I think the ultimate form factor is going to be a smartwatch, and you can make the app light enough to run on one. The best way [to track coughs] will be with the watch we’re building, and the main reason is just its unobstructed proximity to mouths. A lot of things happen to phones that shouldn’t happen if you want to do audio tracking. They go in purses, they go in back pockets, and they get left on the kitchen counter. There’s a consistency to having a watch on your wrist.”
Hyfe’s current app runs on iOS and Android phones, where the impact on the processor and battery life is fairly low. However, it’s audio quality where things get complicated.
“Really cheap phones do a lot of strange things with audio, so our algorithms have learned this is what a cough sounds like on an iPhone, and this is what a cough sounds like on an $80 phone.”
Hyfe has people annotate the audio it collects, which incidentally is only collected and analyzed when you agree to let it do so — otherwise, everything remains on your device, removing concerns about privacy. Brew said the human annotation process is essential to help its algorithm learn, as not everyone coughs in the same way. It also needs to differentiate between similar sounds, so it’s tracking coughs and not sneezes or dog barks.
“It looks for abrupt, explosive sounds. Because, of course, a cough doesn’t fade in or out. It starts with a loud pop. When it finds an explosive sound, it takes a half sample and sends it to a classifier trained on millions of human-confirmed sounds, which assesses if that was a cough or not.”
To get to this point, Hyfe uses a method called a “convolutional neural network,” where the recorded sound is converted into a spectrogram, and in a similar way to AI recognizing faces in photos, it looks for explosive sounds and then labels them a cough, a dog bark, a sneeze, or something else, because each looks different when examined at this level.
“That’s our method,” Brew said, adding, “and our classifier is right 99% of the time, or something absurdly high like that.”
How this all came about
“We’ve got centuries of sound data to process, and we are always behind,” Brew said. “We will never catch up with our database in terms of how much audio we’ve captured versus how much we’re human annotating.”
Brew and his team have been working on this since early 2020, when the idea for a cough-tracking system came up at the beginning of the COVID-19 pandemic. Early versions of the database were built around data collected from friends and family and annotated personally by the team. Brew admits the first version of the Hyfe cough tracking app wasn’t very good, having been hastily put together and launched during lockdown, but people continued to use it.
“The reason they stuck around,” Brew explained, “is because they have this huge problem, which is a chronic cough. It’s not that they have an underlying disease that’s causing the cough — the cough is the problem. They have a hypersensitivity and can cough all day and all night, and it interferes with their sleep and with their social lives, and causes severe irritation. Reality slapped us in the face, as this was a real group of people that are completely underserved.”
Brew said some estimates put the number of adults with a chronic cough at 30% of the population, and said there aren’t many available treatments because there hasn’t been a way to measure effectiveness before. An always-listening cough tracker has the potential to change that, giving people with chronic coughs access to detailed information that doesn’t rely on memory or guesses. Through this, they can better understand how eating certain types of food affects them, how humidity and air quality change the cough, or if allergic reactions are involved.
“Just like when you’re struggling with blood pressure, you get a blood pressure monitor. If you’re struggling with cough, having a cough counter really makes sense in your arsenal of tools,” Brew said.
Hyfe’s AI and the importance of tracking and understanding coughs go beyond helping those with chronic coughs. Brew spoke about a study that took place in Canada, where cough trackers placed near sick patients hospitalized with COVID-19 revealed a reduction in coughing and was an indicator of worsening health rather than an improvement.
What makes it really exciting is when you go beyond using the audio to track coughs, as the same AI and methods could also be used to analyze other types of audio. Brew talked about using it to assist with recognizing developmental disabilities in children, to care of the elderly and ensure there’s regular communication, and to understand other health problems ranging from GERD and sleep apnea to depression.
The future of cough tracking is bright
You can download and use Hyfe’s CoughTracker app now, but what’s next, and what about the distant future for cough tracking?
“We think that in early 2024, our cough monitor wearable will be approved by the [Food and Drug Administration] in the United States,” Brew confirmed. “That’s what we’re hoping for, and we’re doing trials right now. But because the regulatory space is so complex and slow, and the tech moves so fast, we’re also really interested in what we can do in the wellness space. which is less about you have this specific disease or infection, and more about these are your trends, and these things can help make your life better.”
As more tech companies focus on health and wellness, cough tracking may become as commonplace as blood oxygen monitoring and respiratory rate tracking on our wearables. The Apple Watch already monitors loud noises to protect our hearing and uses microphones to detect distinct sounds to enable its Crash Detection feature. Tracking coughs using this same hardware doesn’t seem like much of a stretch.
Mobile devices and AI will continue to have a positive impact on our health further in the future too, according to Brew, who made this exciting prediction:
“I do think by 2030 our devices will diagnose disease, track continuously, and even give you warnings like, there’s a have high probability you have lung cancer. I think that’s absolutely going to come, and it’s going to come relatively fast. We’re definitely under a decade away from that.”
This is an app that initially took two weeks to create before it was launched in app stores. It’s now offering centuries’ worth of data and potentially life-changing (and even lifesaving) advice that comes from listening to and analyzing your coughs. It’s not hard to see why Brew is so excited about the technology’s potential.
