48 votes

In leak, Facebook partner brags about listening to your phone’s microphone to serve ads for stuff you mention

37 comments

  1. [15]
    skybrian
    Link
    Which seems more likely? A salesperson who doesn't even work at Facebook lied about their technology to potential customers, or that Facebook secretly implemented a feature to listen to customers'...

    Which seems more likely? A salesperson who doesn't even work at Facebook lied about their technology to potential customers, or that Facebook secretly implemented a feature to listen to customers' microphones, and also told an outside salesperson about it so they can brag about their secret technology to potential customers?

    I don't know anything about Facebook's advertising business, but I would guess that the salesperson was lying. It's a leak that reveals some sleazy sales tactics.

    57 votes
    1. [10]
      stu2b50
      Link Parent
      If you read the actual page they never say that they listen to microphones through facebook or anything. Facebook is just merely one of the companies listed under their "partners" page. I think it...

      If you read the actual page they never say that they listen to microphones through facebook or anything. Facebook is just merely one of the companies listed under their "partners" page. I think it was once again some clickbait craftsmanship to link the two in the title like that.

      In general, the archived page just sounds like bullshit. It never actually describes how they're getting this supposed microphone data, or how they bypass things like the hardware level microphone indicator on iOS?

      36 votes
      1. [9]
        blivet
        Link Parent
        Yeah, I’m sure that some companies would love to be able to do it, but from what I’ve read the user’s device would have to constantly be sending tremendous amounts of data to Meta’s (or whoever’s)...

        Yeah, I’m sure that some companies would love to be able to do it, but from what I’ve read the user’s device would have to constantly be sending tremendous amounts of data to Meta’s (or whoever’s) servers, which could easily be detected and would also drain the device’s battery so quickly that no one could fail to notice it.

        17 votes
        1. stu2b50
          Link Parent
          I'd also HIGHLY doubt Facebook, if they did manage to covertly take all this data, would just casually send to this random company. Data is the business of Facebook and Google, it's their moat -...

          I'd also HIGHLY doubt Facebook, if they did manage to covertly take all this data, would just casually send to this random company. Data is the business of Facebook and Google, it's their moat - they will sell you advertising services that use the data to be more effective, but they're not going to give you their secret sauce.

          21 votes
        2. [5]
          Amarok
          Link Parent
          The smart way to do it would be to process the audio into text locally and send that, minimally as possible. An entire conversation about food at the grocer's could be sent back simply as a couple...

          The smart way to do it would be to process the audio into text locally and send that, minimally as possible. An entire conversation about food at the grocer's could be sent back simply as a couple of encrypted text words such as the names or types of products. Basically, you want the nouns in the sentences and that's all. It would be single digit kilobytes for the whole conversation. This is such a small amount of data that it would never overload anything and it would easily get lost in the noise of regular internet traffic unless you were good enough at forensic network analysis to catch it happening. Even if you capture the packets you can't read through the encryption, it'll just look like random binary bits. It would be extremely difficult to definitively identify.

          That'll still crimp your battery life, but not nearly as much since speech to text is relatively low on the power use. Microphones don't take much power either. I can't really see a technical limitation to doing it this way. It's certainly possible, not that I think they do it (yet).

          11 votes
          1. [4]
            sparksbet
            Link Parent
            We can look at how the keywords for software like Siri, Alexa, and Google Assistant work to get an idea of what's doable locally. Afaik they don't process audio to text constantly due to how...

            The smart way to do it would be to process the audio into text locally and send that, minimally as possible.

            We can look at how the keywords for software like Siri, Alexa, and Google Assistant work to get an idea of what's doable locally. Afaik they don't process audio to text constantly due to how resource-intensive it is, but rather operate directly on the audio. This is possible because they're searching for such a limited range of words (and also because when you initialize these features, they'll usually ask you to say the keyword a couple times so they can recognize an individual's specific quirks and incorporate them). Detecting a wider variety of keywords would require actually doing speech-to-text full transcription, as you surmise.

            Based on the increased battery usage of the Live Captions feature built into my pixel when I turn it on, I wager it would definitely be a much more noticeable battery hit to do full speech-to-text on all surrounding audio than you think. Relatively low compared to sending all the audio to the cloud doesn't necessarily mean it's so low that you wouldn't notice it. I think you would need a lot of support from the cell phone OS itself to conceal that this was happening even if the processing was occurring locally in the way you describe, since the increased battery usage would have to be treated as part of your phone's baseline somehow and whatever app or service was processing this audio would still need recording permissions. So if someone is doing this, it's either Google and Apple themselves or someone working very closely with them.

            15 votes
            1. [2]
              gary
              Link Parent
              Since you already have to trust Google or Apple, this is why I view Alexa as an unacceptable potential Trojan. An always-plugged in home assistant with far field mic sidesteps the battery issue....

              Since you already have to trust Google or Apple, this is why I view Alexa as an unacceptable potential Trojan. An always-plugged in home assistant with far field mic sidesteps the battery issue. Same thing for TVs, which is why mine will never connect to the internet.

              5 votes
              1. Amarok
                Link Parent
                Do you know what happens when you tell Alexa to turn on the living room lights when the internet is down? Nothing, that's what. She vapor locks. That tells you all you need to know right there.

                Do you know what happens when you tell Alexa to turn on the living room lights when the internet is down? Nothing, that's what. She vapor locks. That tells you all you need to know right there.

                5 votes
            2. Amarok
              Link Parent
              Everything in the phone except the radios is not really an issue. The wifi and especially cell radios take up more than the CPU on does for most phones (except the exotic flagship phones, which is...

              Everything in the phone except the radios is not really an issue. The wifi and especially cell radios take up more than the CPU on does for most phones (except the exotic flagship phones, which is a small slice of the total pie). If you want to do this sneakily, minimizing the radio/wifi is not negotiable. You have to cache it and then quietly send it when the radios turn on for other things. Just piggyback on that burst. The spying doesn't require real time reporting.

              Since the source on the operating system is closed, we will simply never know, until someone takes the time to reverse engineer the whole stack. One might be able to tease it out in other ways - for example, disable the internet on the phone and let it have an opportunity to record for a couple weeks, see if you can catch it bursting a suspiciously large amount of data out when it regains internet access.

              Frankly this task seems pretty basic to me, not even a real programming challenge. I'd class it not just as possible but as relatively easy, especially for state actors who develop much more malicious and clever software like stuxnet or clarity. We already know the security on android phones is so bad they can be hacked and enslaved with a text message (it does get better, just too slowly). I try but I don't see a lot of reasons to be optimistic about it. At the end of the day, all that data is worth a fortune to them, and they don't want to pay us for it, so they will find a way to take it. The only question is, has that happened yet, or do we have wait five more years until speech to text is computationally as cheap as mp3 playback?

              4 votes
        3. [2]
          Crespyl
          Link Parent
          Although I agree that this "active listening" is highly unlikely, it is (or at least was last I heard) the case that the Facebook app is a notorious battery hog.

          drain the device's battery

          Although I agree that this "active listening" is highly unlikely, it is (or at least was last I heard) the case that the Facebook app is a notorious battery hog.

          3 votes
          1. Englerdy
            Link Parent
            Anecdotal, but my battery life improved tremendously after deleting the Facebook app and locking down the pre-installed version. Messenger doesn't seem to have the same battery drain issue.

            Anecdotal, but my battery life improved tremendously after deleting the Facebook app and locking down the pre-installed version. Messenger doesn't seem to have the same battery drain issue.

            4 votes
    2. [4]
      lou
      (edited )
      Link Parent
      I feel this opinion fails to take into consideration that presuming a negative about Facebook is often and consistently the option most consistent with reality. Do I believe this? Probably not....

      I feel this opinion fails to take into consideration that presuming a negative about Facebook is often and consistently the option most consistent with reality. Do I believe this? Probably not. Would I dismiss this? Probably not either. I don't see why the notorious sleaziness of sales people would be more persuasive than the notorious sleaziness of Facebook.

      5 votes
      1. [3]
        skybrian
        Link Parent
        To me it’s more about the difficulty of keeping secrets and what sort of secret this would be. It seems like it would be difficult to keep a secret like that in a high-profile mobile app. Security...

        To me it’s more about the difficulty of keeping secrets and what sort of secret this would be. It seems like it would be difficult to keep a secret like that in a high-profile mobile app. Security researchers will notice things. Employees will notice things, and it would be hard to make sure they’re all loyal to a cause that the company publicly denies exists. It would be a very high-risk secret project, and I think managers at Facebook would understand how high-risk it is? Why take such a risk?

        But this is speculation based on what I think Silicon Valley employees and managers are like, not any particular knowledge of Facebook.

        8 votes
        1. [2]
          vczf
          (edited )
          Link Parent
          My read of the article is that Facebook et al. are recipients of Active Listening data, not sources. That is much more believable to me, because juicy external data would make the data they’ve...

          My read of the article is that Facebook et al. are recipients of Active Listening data, not sources. That is much more believable to me, because juicy external data would make the data they’ve siloed more valuable for ad targeting, without incurring the existential reputational risk of literally eavesdropping on people.

          Where does CMG get the data? My guess is they don’t ask too many questions.

          Could be smart TVs (Samsung, Roku, TCL, etc.), scummy apps, low-end sleazy Android brands, and possibly even malware networks targeting vulnerable versions of Android or IOT devices.

          8 votes
          1. skybrian
            (edited )
            Link Parent
            Yes, that makes more sense. Why a high-risk project to collect data yourself when there are outside suppliers? I don’t know the industry well enough to know how much it’s true, but it’s more...

            Yes, that makes more sense. Why a high-risk project to collect data yourself when there are outside suppliers? I don’t know the industry well enough to know how much it’s true, but it’s more plausible.

            (Big companies are wary about legal risks created by outside suppliers too, though. I was only once involved in that, and the lawyers wanted a lot of favorable contract terms from a software company.)

            3 votes
  2. [3]
    CannibalisticApple
    Link
    This has always been speculated, to the point I can't remember if there's been any previous confirmations of such technology. This is the part that's most frustrating though: Really tired of all...

    This has always been speculated, to the point I can't remember if there's been any previous confirmations of such technology. This is the part that's most frustrating though:

    "We know what you're thinking. Is this even legal?" a since-deleted Cox blog post from November 2023 noted. "It is legal for phones and devices to listen to you. When a new app download or update prompts consumers with a multi-page term of use agreement somewhere in the fine print, Active Listening is often included."

    Really tired of all the sites and apps with stuff like this in fine print...

    24 votes
    1. [2]
      raze2012
      Link Parent
      How is that something that gets past android permissions? That whole fine grained list of permissions was precisely to let users disagree to stuff like this? Not sure about IOS, but I imagine it...

      How is that something that gets past android permissions? That whole fine grained list of permissions was precisely to let users disagree to stuff like this?

      Not sure about IOS, but I imagine it should catch and reject stuff like this before it even pushes out.

      7 votes
      1. 0xSim
        Link Parent
        It doesn't, and if Facebook was somehow able to circumvent it, it would be known by now. This is something that is way too big to stay hidden, and there are probably many researchers and...

        How is that something that gets past android permissions?

        It doesn't, and if Facebook was somehow able to circumvent it, it would be known by now. This is something that is way too big to stay hidden, and there are probably many researchers and developers who carefully look at the the app and its behavior, specifically looking for something like that.

        Not saying it's impossible, but there is 0 concrete proof. Users who claim it just have anecdotes, and this is the most likely explanation for the OP.

        17 votes
  3. winther
    Link
    While it can certainly feel very uncanny when you encounter situations where it seems like someone has been listening, I think it is more a testament to how much these companies already know about...

    While it can certainly feel very uncanny when you encounter situations where it seems like someone has been listening, I think it is more a testament to how much these companies already know about us using other datascraping techniques. Which is just as problematic. It is not just what you yourself do on your devices, but also triangulated with people you may interact with by simply visiting their house for example. The amount of data collected makes very correct profiling possible, sometimes showing us ads for stuff before we ourselves know we need it. Having the infrastructure to listen in on millions of peoples conversations for hours a day is likely not even worth it compared to what they can already gather.

    11 votes
  4. [3]
    vczf
    Link
    I’ve had many (n>5) incidents in the last two years where I’m having a conversation at home, and an ad targeting the exact topic under discussion appears on my mom’s Google app or on my YouTube...

    I’ve had many (n>5) incidents in the last two years where I’m having a conversation at home, and an ad targeting the exact topic under discussion appears on my mom’s Google app or on my YouTube account on the TV.

    I suspect the Samsung TV is the likely source of the spying, though I can’t prove it. I opted out of behavioral marketing and set up NextDNS with extra filters for smart TVs, but unfortunately can’t disconnect it from the internet entirely since we need it for Samsung TV+.

    I used to be an hard-liner about this kind of thing. Now, I’ve grown to accept that it’s a societal ill that requires political solutions. I’m skeptical if sacrificing convenience for a dubious and hard-to-quantify privacy benefit is worth it, though I still do what I can.

    9 votes
    1. caliper
      Link Parent
      I still try to be a hardliner about it, and it is PITA. As you say, I have no idea if all that effort really gets me anywhere and I fail to convince people around me to be more careful about which...

      I still try to be a hardliner about it, and it is PITA. As you say, I have no idea if all that effort really gets me anywhere and I fail to convince people around me to be more careful about which apps/platforms they use. People just don’t see or care about the issue. And I think you’re entirely correct about it needing a political push to make things better, because the market is entirely focused on making things as bad as possible.

      8 votes
    2. imperator
      Link Parent
      Same here and we also have a Samsung. I also think Samsung phones do it too some degree. My ads are shit on my phone but are some times so specific on my partners.

      Same here and we also have a Samsung. I also think Samsung phones do it too some degree.

      My ads are shit on my phone but are some times so specific on my partners.

      1 vote
  5. [15]
    Oslypsis
    Link
    Absolutely checks out. I just talked with my granny yesterday where she mentioned her needing eye drops that costed around $900 per bottle (after insurance), then she told them she cant afford...

    Absolutely checks out. I just talked with my granny yesterday where she mentioned her needing eye drops that costed around $900 per bottle (after insurance), then she told them she cant afford that, so the doctor "got a hold of someone" and had the price dropped to $45 per bottle. And what ads am I seeing over and over now? Yep, eye drops.

    6 votes
    1. [14]
      0xSim
      Link Parent
      That's 100% confirmation bias. You're talking about hundreds of things with many people everyday, but of course you'll remember that thing you're getting ads for. It could be totally random, it...

      That's 100% confirmation bias. You're talking about hundreds of things with many people everyday, but of course you'll remember that thing you're getting ads for. It could be totally random, it could be because an algorithm with enough data about you statistically determined that you might be interested in eye drops.

      15 votes
      1. [13]
        Oslypsis
        Link Parent
        No, I'm not. I mostly watch youtube and play Minecraft. Also, I wasn't getting eye drop ads before this. Confirmation bias also doesn't explain how, back from last Xmas, I got ads specifically for...

        You're talking about hundreds of things with many people everyday,

        No, I'm not. I mostly watch youtube and play Minecraft. Also, I wasn't getting eye drop ads before this.

        Confirmation bias also doesn't explain how, back from last Xmas, I got ads specifically for some cheap floating ball toy that was a gift. I'd never search for something like that because tbh, it wasn't all that up my alley. I read the name of it out loud, and the next time (that day) I was on Amazon (iirc), there it was, in an ad.

        9 votes
        1. [6]
          0xSim
          Link Parent
          Confirmation bias doesn't explain why you're getting ads, it explains why you think you're getting ads because you talked about something. Imagine this Venn diagram with 2 circles: Things you talk...

          Confirmation bias also doesn't explain how, back from last Xmas, I got ads specifically for some cheap floating ball toy that was a gift

          Confirmation bias doesn't explain why you're getting ads, it explains why you think you're getting ads because you talked about something. Imagine this Venn diagram with 2 circles:

          • Things you talk about
          • Things you're getting ads for

          You talked about a ball toy, it ended up in the overlap area, and you noticed it. It happened twice in a year. Five times in two years for vczf.

          Those are both very small overlap areas, compared to all the things you may talk about in one or two years, and the thousands of ads you saw in that timeframe. But yes, those occurrences stand out, obviously. The first time it was suspicious, and now you're maybe half-looking for them, so they stand out even more.

          What do you think is more likely? That Facebook managed to evade, for years, an increasing surveillance from specialists that specifically want to prove the app is actively listening, or that you just got 2 random occurrences happen in a year? And like I said, it's also totally possible that those ads were targeted through other means. Maybe because you searched for adjacent things, maybe because that ball toy was popular with people with a similar profile. And even then it could be a dumb coincidence.

          I'm not saying it does not happen, I'm saying it's statistically irrelevant. That's what confirmation bias is.

          9 votes
          1. [3]
            Weldawadyathink
            Link Parent
            There is another factor that plays into this. The human brain is really bad at understanding large numbers, especially when mixed with low probabilities. The best example is the lottery. The...

            There is another factor that plays into this. The human brain is really bad at understanding large numbers, especially when mixed with low probabilities. The best example is the lottery. The chance that you win the lottery is extremely low. It is so extremely low that the probability is effectively zero. But there are millions or billions of people who play the lottery. Multiply that almost zero chance by a billion and it’s almost guaranteed that someone wins the lottery.

            To relate this back to the example, Facebook serves probably many millions of ads each day. For the sake of argument, let’s say they were served completely randomly. It would be surprising if they served you an ad about something you just talked about. But it should not be surprising that they served someone an ad about something they talked about. They served so many ads that it’s basically guaranteed. Now we add in the fact that ads are not completely random, and they have very good ways to target their ads, it becomes even more likely for this random occurrence to happen.

            13 votes
            1. Lapbunny
              (edited )
              Link Parent
              Not to mention they have five billion heuristics to nudge these numbers towards the right people the right way with metrics. Are you home, or at the supermarket, and you have an app tracking your...

              Not to mention they have five billion heuristics to nudge these numbers towards the right people the right way with metrics. Are you home, or at the supermarket, and you have an app tracking your location? Did you just look up a recipe at 2 in the afternoon? Is there a holiday coming up that you may shop for? Remind someone they need chili powder.

              Expecting mothers always are surprised they get telepathic crib advertisements, but it's really just one "I missed my period" search plus like 3-5 months.

              5 votes
            2. Protected
              Link Parent
              Yes, the targeting bias is key here. Facebook does have a lot of data about a lot of people's habits, and there are statistical correlations that are difficult for us to imagine informally.

              Yes, the targeting bias is key here. Facebook does have a lot of data about a lot of people's habits, and there are statistical correlations that are difficult for us to imagine informally.

              2 votes
          2. [2]
            Oslypsis
            Link Parent
            Seems like we're going to have to agree to disagree because I'm still not convinced, and I don't feel like continuing this. I appreciate the effort to clarify, though.

            Seems like we're going to have to agree to disagree because I'm still not convinced, and I don't feel like continuing this. I appreciate the effort to clarify, though.

            3 votes
            1. 0xSim
              Link Parent
              That's alright, the whole thread was interesting. Have a nice day o/

              That's alright, the whole thread was interesting. Have a nice day o/

              3 votes
        2. [6]
          vczf
          Link Parent
          I once made a remark about how I liked the size and quality of the Badia Cayenne Pepper we had in our pantry, and minutes later there was an ad on YouTube for McCormick spices. I had never seen a...

          I once made a remark about how I liked the size and quality of the Badia Cayenne Pepper we had in our pantry, and minutes later there was an ad on YouTube for McCormick spices. I had never seen a McCormick ad before that day, and never saw it ever again.

          5 votes
          1. [5]
            Oslypsis
            Link Parent
            Yeah, idk if it's Facebook or Amazon or what, but something/someone is listening for advertising purposes. It's now just a matter of finding out who.

            Yeah, idk if it's Facebook or Amazon or what, but something/someone is listening for advertising purposes.

            It's now just a matter of finding out who.

            9 votes
            1. [4]
              RoyalHenOil
              Link Parent
              The culprit need not be recording conversations to collect this data. I'm guessing that it's an app on your phone, and it is collecting a huge array of data about you and your contacts (including...

              The culprit need not be recording conversations to collect this data. I'm guessing that it's an app on your phone, and it is collecting a huge array of data about you and your contacts (including your grandmother and the person who gifted you that ball).

              If you are not very careful about granting permissions, phone apps can see your contacts, log your texts and phone calls, see what you type even if you don't send it, see who you are physically located near, take screenshots of your phone, etc. There are thousands of apps out there doing these things.

              This has implications not only about the ads you see, but also potentially ID theft (e.g., apps getting screenshots of passwords and credit card numbers). If I were you, I would go on a major app purge and only re-download apps that don't require permissions or that you absolutely trust.

              6 votes
              1. [2]
                redwall_hp
                Link Parent
                Is Android really that much Swiss cheese? That is flat out not permitted for any reason on iOS. No app can request permission to view SMS/iMessage or call history, no API exists to allow it, and...

                log your texts and phone calls

                Is Android really that much Swiss cheese? That is flat out not permitted for any reason on iOS. No app can request permission to view SMS/iMessage or call history, no API exists to allow it, and apps' file system access is throughly sandboxed.

                2 votes
                1. asuka
                  Link Parent
                  Android provides an API available to third party apps to read and send SMS, but it has to explicitly request the permission to do so.

                  Android provides an API available to third party apps to read and send SMS, but it has to explicitly request the permission to do so.

                  4 votes
              2. Oslypsis
                Link Parent
                Thank you, I will go through my app settings and fix what I don't like. This shit is so creepy and we NEED laws to protect us and our privacy on all fronts.

                Thank you, I will go through my app settings and fix what I don't like. This shit is so creepy and we NEED laws to protect us and our privacy on all fronts.

                2 votes