Blog by Satu Rantakokko

It’s been years since the public first reacted to revelations that smartphones may be listening to users—allegedly to enable targeted advertising. This often manifested as ads appearing on social media shortly after discussing a particular topic. Today, most people have experienced some form of technological eavesdropping. Targeted ads typically appear within hours or a day of the conversation. (For more on the experience and emotional impact of technological eavesdropping, see Segijn et al., 2024)

For me, one pivotal moment occurred years ago in a shopping mall. I walked past a store and casually told my companion, “Let’s turn after store X.” I had no interest in that store, which made it easy to link the sudden appearance of related ads on social media to that offhand comment. My phone was in my backpack, behind me.

At the time, I wondered how my voice could have been picked up so clearly in a noisy mall—especially since I wasn’t even facing the phone. Years later, that moment resurfaced in my mind after another experience.

I was catching up with a friend who belongs to a small, privacy-conscious minority. He doesn’t use smartphones and shares no personal information on his limited social media accounts. As a result, social media platforms struggle to target ads to him.

Except once. He described a situation where he began receiving targeted ads related to topics he had discussed during a car ride—with a friend who owns a smartphone.

So, the familiar eavesdropping-to-advertising scenario played out again—but this time, the ads were directed at a bystander, not the phone’s owner. Can someone else’s smart device really recognize you or me by voice? Is that even possible?

Technically, yes—it’s entirely possible.

Smartphones are designed to recognize their owners regardless of context. This is essential, as voice-activated devices must be able to distinguish between authorized users and others. Given the vast amount of sensitive personal and professional data stored on smartphones, the risk of data breaches is high if the device falls into the wrong hands. That’s why user authentication methods are actively developed—including voice profiling, which can be created from even a short speech sample.

Voice is considered a behavioral biometric identifier. It can reveal a speaker’s ethnicity, age, gender, and emotional state. Each person has a unique vocal signature shaped by rhythm, pauses, pitch, volume, word choice, sighs, and more. Voice-based identification is a growing field of research, increasingly powered by AI and deep learning technologies. (See Hanifa, Isa & Mohamad, 2021; Ohi et al., 2021.)

Back in that shopping mall years ago, I don’t recall receiving a flood of unrelated ads afterward. But I might not have noticed, as I was less vigilant about phone privacy back then. I can’t say for sure whether someone else’s comment nearby might have triggered advertising.

What’s clear is that for a phone to distinguish its owner’s voice profile, it must listen broadly when others are present. It must analyze surrounding voices to determine whether the speaker is the owner or not. (See Ohi et al., 2021.)

And what about my friend’s case, where it seems an app on someone else’s phone recognized him and began serving targeted ads based on spoken topics? After he told his friend, the friend revoked microphone permissions for all apps and began disabling the mic at the software level when not in use. The data leak stopped, and no further targeted ads appeared. It was never determined which app—or apps—were responsible for identifying the bystander and triggering the ads.

Even though it wasn’t me, the idea of being technologically eavesdropped on as a bystander felt disturbing—almost repulsive. If automated listening of bystanders is this advanced, it raises unsettling questions: What else is being heard and profiled about me, even when I protect my own phone, block listening, and disable location tracking? Is my voice and location still casually logged through other people’s devices? Where does this data go, and what is known about me? Who has access to it, and how is it used?

Such data leakage is hard to detect, especially since most of us carry smart devices constantly. That’s why we rarely consider how vulnerable we are as bystanders—and when targeted ads appear, we don’t always realize that the triggering conversation may have been captured by someone else’s device, not our own.

The old idea that “I have nothing to hide” is due for an update. Does that same carefree mindset apply to all your friends, family, and even your workplace?

Dear reader, have you ever experienced technological eavesdropping as a bystander—or suspected it? Have your data ever been linked in surprising ways—like a conversation near your work phone leading to ads on your personal profile, or vice versa?

Or have you noticed—or suspected—something even more advanced? If so, don’t hesitate to reach out to us @ satu.rantakokko[at]uwasa.fi 

 

References

Hanifa, R. M., Isa, K. & Mohamad, S. (2021). A review on speaker recognition: Technology and challenges. Computers & Electrical Engineerin, vol. 90. https://doi.org/10.1016/j.compeleceng.2021.107005

Ohi, A. Q., Mridha, M. F., Hamid, M. A. & Monowar, M. M. (2021). Deep speaker recognition: Process, progress, and challenges. IEEE Access, 9, 89619-89643. https://doi.org/10.1109/ACCESS.2021.3090109

Segijn, C. M., Strycharz, J., Turner, A. & Opree, S.  J. (2024). Conversation-Related Advertising and Electronic Eavesdropping: Mapping Perceptions of Phones Listening for Advertising in the United States, the Netherlands, and Poland. https://journals.sagepub.com/doi/pdf/10.1177/20563051241288448