They’re no longer just used for calls. Voice-controlled “intelligent assistants” like Apple’s Siri, Google Assistant, and Microsoft’s Cortana are commonplace on modern smartphones. We use them to search for answers, set reminders, and fix appointments. Apps and games can also request microphone access, and they frequently do. But do we really know when they’re listening and what happens to the audio they record?
“Smartphones are small tracking devices,” Michelle De Mooy, Acting Director of the Center for Democracy and Technology’s Privacy & Data Project, told Digital Trends. “We may not think of them like that because they’re very personal devices — they travel with us, they sleep next to us. But they are in fact collectors of a vast amount of information including audio information.”
If you doubt that, and you’re a user of Google services, take a minute and visit this link. If you’re logged into your Google account, it will show all your activity across Google’s services, from Chrome and Search to Android and YouTube. Tap Filter by date & product at the top, choose Voice & Audio, and then hit Search. If you’ve ever used voice search on Google, you’ll see a list of audio recordings that you can play back and listen to right now.
Few people realize that the data is there. Google is relatively transparent, in that it allows you to review what has been collected, but it doesn’t go out of its way to publicize that it’s there in the first place. Most other companies that record you don’t let you see what they have at all, and it’s far from clear what they’re doing with the audio.
If you worry that your smartphone might be listening to your conversations, you’re not alone.
Your recordings probably aren’t being listened to by an actual human being (though if there’s any truth to this Reddit thread, they might be). But companies are certainly applying algorithms to look for patterns and determine potentially useful things about your behaviors and interests.
Audio data could reveal all sorts of things. The ambient noise could determine whether you’re in the living room or the bathtub. Background voices could reveal who you’re with. Using the microphone to measure noise levels can even reveal when you’re asleep.
“Even if you think you’re not saying anything very interesting or worthwhile, the data gets married and mingled with lots of other kinds of data that can create a very detailed picture of you,” explains De Mooy. “Most of these technologies aren’t in a vacuum, they’re not siloed, they really are interacting with every other type of technology that we have.”
Last year the CDT alerted the Federal Trade Commission (FTC) to a technology called SilverPush. It uses audio beacons to track your activities across devices: Your TV emits a tone during a commercial break, a tone that’s inaudible to you, but your phone is listening for it. Now they can link the TV and phone as belonging to the same person.
Advertisers have developed lots of techniques for device-matching, because the more accurately they can track your activities, the easier it is for them to advertise to you. But it’s not difficult to imagine other applications for this technology. Any government interested in who you are meeting with could play a tone through the TV and effectively ping all the phones in the room, identifying the whole group.
If you worry that your smartphone might be listening in to your conversations, you’re not alone. The internet is rife with anecdotal stories about digital eavesdropping. Many people feel that conversations they’ve had within earshot of their phones have been used to tailor advertising.
After Facebook was accused of this practice back in the summer, it released a short denial saying “Facebook does not use your phone’s microphone to inform ads or to change what you see in News Feed…. We only access your microphone if you have given our app permission and if you are actively using a specific feature that requires audio.”
So why were so many people convinced that Facebook was listening in and changing advertising based on what they said?
“There are two possible explanations,” Professor Jason Hong of the School of Computer Science at Carnegie Mellon University explained to Digital Trends. “It’s either coincidence, or they were browsing on a website and happened to see that topic and then talked about it with friends later on. Facebook is actually linked into lots of websites and news sites around the world, so they do collect data about what content you’re looking at and then base ads off of that.
“But as far as we know, they’re not listening on the microphone.”
Facebook’s statement notably doesn’t mention what it does use audio recordings for, but its entire business model is based on data collection and laser-focused targeting for advertising. You may feel like there are worse things than relevant adverts, but the implications run a lot deeper than that.
“Smartphones are small tracking devices.”
“When you’re using a free service, you’re paying for it with your information, but the trade-off we’re making is really unclear to most people,” says De Mooy. “The internet that I see is vastly different than the internet you see. The content we see reflects the data that has been collected on us.”
That means you might see different job adverts based on your gender, or you may be targeted with high-interest loans because you belong to a specific ethnic group.
What do you do if you’re concerned about listening?
With services like Siri or Google Assistant, your phone is always listening for a keyword, but that’s processed locally. It doesn’t start recording your audio until it hears “Ok Google” or “Hey Siri.” At that point, it records and uploads an audio file. You can turn these features off quite easily; for example, on Android, go to Settings > Google > Search & Now > Voice and turn “Ok Google” detection off.
“The big companies have a strong incentive to be mostly upfront about what they’re doing because the Federal Trade Commission and other government organizations would fine them if they were using deceptive practices,” suggests Hong. “There are also a lot of researchers analyzing these kinds of apps.”
It’s highly unlikely that they’re going to listen in uninvited, but it’s still not clear what’s happening to your recordings. Even if you take the time to read privacy policies, which few people do, it’s often far from easy to decipher them.
“Nobody really reads these privacy policies and it would surprise a lot of people just how much data is being collected,” says Hong.
More worrying, perhaps, is the potential for a dubious or outright malicious app to turn on the microphone without you knowing.
“With GPS, at least there’s the little GPS icon, so you know that an app is using it, but with the microphone and other sensor data we don’t really know what’s being captured and what’s not,” explains Hong.
Sticking to popular apps and not granting microphone access is a smart move.
“Don’t be the first penguin in the water,” says Hong. “Don’t be the first one to download an app after it gets released, wait for one or two weeks. Google and Apple both have ways of finding malicious apps and removing them.”
You should also review the permissions you’ve already granted to apps. On Android go to Settings > Privacy and safety > App permissions. On iOS go to Settings > Privacy. Both have an entry for microphone, which will list all the apps that have access. If you see something suspicious, investigate.
Should you be worried?
The audio space is still relatively immature, compared to the techniques and tricks companies are using to track our wider online activity, but as our devices start to listen in more and more, it’s an increasingly legitimate worry.
“As more smartphones, smart TVs, and smart toys start to listen in on us all the time, it’s going to be very hard for anyone to understand where all the data is flowing, because every company wants to connect to its own cloud service,” says Hong. “This makes it a real hassle, even for experts, to try and understand what’s going on.”
The CDT believes there should be baseline privacy legislation to protect us, but it has also been trying to convince companies that collecting less data is actually in their interests. They don’t necessarily need all this information, and it’s increasing their potential legal liability. Many companies seem to be collecting everything they can get their hands on with the idea that they’ll figure out a use for it later.
“There’s a term for that in real life,” says De Mooy. “It’s called hoarding.”
A lot of this is driven by a genuine belief that technology is making lives better, but good intentions can be subverted, and we don’t know how our data might be used down the line. At the moment, it’s not easy to make an informed decision, because we don’t know much about what’s happening behind the scenes.
“If you’re going to use these services on your smartphone, understand that it’s not private,” suggests De Mooy. “It’s not a personal assistant or friend, it’s a small tracking device. Social apps are meant to collect data and make it public. Pay attention, watch out for default settings, and take action to protect your privacy.”
