35 votes

How did deepfake images of me end up on a porn site?

29 comments

  1. [2]
    Pioneer
    Link
    Was reading this article this morning. Absolute nightmare fuel. This had my eyebrow raised in so many ways. Poor lass, must be a constant niggling sensation whenever you meet people for a coffee...

    Was reading this article this morning. Absolute nightmare fuel.

    It wasn’t difficult to get the pictures taken down. In a weird twist, the creator of the website profile deleted their page and all the galleries shortly after I was first made aware of the images. That lends an extra layer of strangeness to the whole experience. I was left with the screenshots as evidence and there were a few other places on the internet where they had appeared. But the original material had been removed. Was it a coincidence? Did they somehow know that I’d discovered the images? Again, I don’t think it would be helpful to dwell on it.

    This had my eyebrow raised in so many ways.

    Poor lass, must be a constant niggling sensation whenever you meet people for a coffee of "was it you?" no matter how much you try and put it off.

    32 votes
    1. dirthawker
      Link Parent
      I would guess that, having been discovered by the victim, that the perpetrator is just going to move the images to a different URL.

      I would guess that, having been discovered by the victim, that the perpetrator is just going to move the images to a different URL.

      9 votes
  2. [3]
    winther
    Link
    This is just one story out of probably many. I am genuinely afraid of what mass access to this kind of tech will do of damages, especially for my daughters and their upcoming school years. I know...

    This is just one story out of probably many. I am genuinely afraid of what mass access to this kind of tech will do of damages, especially for my daughters and their upcoming school years. I know what it is like being a horny teenage boy with fantasies about classmates, but we didn't have stuff like this available. I have already seen some mentions of things like this going around in schools, and I can't honestly say I would have been mature enough to handle such possibilities myself any better. I do see however a larger awareness about consent and personal boundaries in younger generations, that is way beyond how our culture on those kind of topics was back in the late 90s.

    24 votes
    1. [2]
      Thrabalen
      Link Parent
      There are a lot of areas where youth have evolved massively since the 80s, when I grew up. Young people handle race, sexuality, foreign nationality, on the whole better (in many cases, far better)...

      There are a lot of areas where youth have evolved massively since the 80s, when I grew up. Young people handle race, sexuality, foreign nationality, on the whole better (in many cases, far better) than I remember my peers when I was in my 20s.

      16 votes
      1. [2]
        Comment deleted by author
        Link Parent
        1. Thrabalen
          Link Parent
          Oh, I absolutely agree that in this case, no group is mature enough to handle it because the technology is too new to have a consensus solidly built about it.

          Oh, I absolutely agree that in this case, no group is mature enough to handle it because the technology is too new to have a consensus solidly built about it.

          6 votes
  3. [18]
    DefiantEmbassy
    Link
    The scary part is how this is only going to get easier, with the new wave of image generation models. A recent post about an event in Spanish schools:...

    The scary part is how this is only going to get easier, with the new wave of image generation models.

    A recent post about an event in Spanish schools: https://tildes.net/~life.women/1al0/in_spain_dozens_of_girls_are_reporting_ai_generated_nude_photos_of_them_being_circulated_at_school

    15 votes
    1. [17]
      Dr_Amazing
      Link Parent
      Honestly I don't think it can get much easier. Right now with a few free programs and a midrange computer, you can pump out AI pictures using a single photo of the victim. It takes a little tech...

      Honestly I don't think it can get much easier. Right now with a few free programs and a midrange computer, you can pump out AI pictures using a single photo of the victim.

      It takes a little tech knowledge to get it all running, but not a lot. Once it's set up it's simple.

      The next big advancement will be video. Right now deep fake video is a slow complicated process. You need a ton of pictures and video of the person and you can only swap their face onto an existing video. But we're really close to being able to just ask for a video of anything.

      14 votes
      1. [6]
        Casocial
        (edited )
        Link Parent
        Pandora's box has been opened, and I have a hard time seeing it going away the further technology advances. IMO the best case scenario would have to be when making a deepfake is so trivial that...

        Pandora's box has been opened, and I have a hard time seeing it going away the further technology advances. IMO the best case scenario would have to be when making a deepfake is so trivial that nobody cares anymore when someone's "nudes" are leaked.

        14 votes
        1. [3]
          Comment deleted by author
          Link Parent
          1. [2]
            Dr_Amazing
            Link Parent
            Interestingly I don't even think the author is using "deep fake" correctly. This story takes place in 2020 when the technology barely existed. It sounds like she's using the new buzzword but she...

            Interestingly I don't even think the author is using "deep fake" correctly. This story takes place in 2020 when the technology barely existed.

            It sounds like she's using the new buzzword but she was dealing with someone photoshopping pictures. A technology that's excited for decades.

            3 votes
            1. winther
              Link Parent
              Deep fakes was definitely a thing in 2020. The famous Obama fake speech video is from 2017 and the FakeApp thing is also from around that time, and various deep fakes of celebrities has existed...

              Deep fakes was definitely a thing in 2020. The famous Obama fake speech video is from 2017 and the FakeApp thing is also from around that time, and various deep fakes of celebrities has existed for several years now. Though definitely gotten easier and better in the last couple of years.

              9 votes
        2. redbearsam
          Link Parent
          I couldn't agree more. There's no going back, but once deep fakes are literally everywhere it could become liberating. Actual nudes that are leaked will be able to hide in plain sight and nobody...

          I couldn't agree more. There's no going back, but once deep fakes are literally everywhere it could become liberating. Actual nudes that are leaked will be able to hide in plain sight and nobody will know what's what. At this point I think the only solution is to lean in.

          We'll probably have a deepfake button integrated into android/iphone keyboards next to emoji and stickers to razz your friends with (not all deep faking is pornographic).

          1 vote
        3. [2]
          winther
          Link Parent
          Maybe at some point but will likely take many years for such a culture change. And what will that do our intimate relationships? Of course seeing someone naked is not what defines an intimate...

          Maybe at some point but will likely take many years for such a culture change. And what will that do our intimate relationships? Of course seeing someone naked is not what defines an intimate relationship, but would we really be comfortable with say our coworkers creating basically their own porn stash of us?

          I am not sure Helen here or other victims of this will feel comfortable even if this tech become more normalized, and I don't think that I as a probably more privileged male can say that it is going to be non issue for everybody at some point.

          1 vote
          1. Casocial
            Link Parent
            In my view, one reason why deepfakes can feel so violating is because of how it affects others' perception of you. I hope that with normalization, it will stop the victims from being looked down...

            In my view, one reason why deepfakes can feel so violating is because of how it affects others' perception of you. I hope that with normalization, it will stop the victims from being looked down on or ostracized by their peers because of such leaks. Not that such a reaction to someone whose nudes, real or not, got leaked is warranted at all, but if it could happen to literally anyone with the click of a button, perhaps it will make the public in general more sympathetic.

            The dismay and outrage felt by current victims of leaked deepfakes are of course completely valid.

            4 votes
      2. [10]
        Grumble4681
        Link Parent
        I think it definitely can get much easier. From what I have seen on some image generation things, for legal and non-nefarious purposes, you often need high end Nvidia GPUs because some of these...

        Honestly I don't think it can get much easier. Right now with a few free programs and a midrange computer, you can pump out AI pictures using a single photo of the victim.

        I think it definitely can get much easier. From what I have seen on some image generation things, for legal and non-nefarious purposes, you often need high end Nvidia GPUs because some of these tools seem to require CUDA which is an Nvidia thing.

        Those GPUs aren't exactly that cheap. I think you're greatly underestimating how much easier it could actually get.

        7 votes
        1. [6]
          ShrubOfRegret
          Link Parent
          I think you're overestimating. For Training AI you need something beefy. For Running AI an old laptop might be good enough; I've been using a 6-7 year old Ideapad for Stable Diffusion without too...

          I think you're overestimating. For Training AI you need something beefy. For Running AI an old laptop might be good enough; I've been using a 6-7 year old Ideapad for Stable Diffusion without too much trouble. Looking at cheap Nvidia cards on Newegg, there are a number of options in the $120-$150 range that should be enough to get by. That's not insignificant, but it's an order of magnitude less than the top end.

          Not only that, but most AI can be run in CPU only mode. It's definitely slower. But in my experience, it's less slow than you might think.

          8 votes
          1. [5]
            Grumble4681
            (edited )
            Link Parent
            OK lets look at it this way. What does it take to browse the web, or send a picture to someone, send messages to people, or any numerous other things that people can do on a phone/computer easily?...

            OK lets look at it this way. What does it take to browse the web, or send a picture to someone, send messages to people, or any numerous other things that people can do on a phone/computer easily? Well everything you just described (and the other replies I've gotten) is more difficult than all of those things I just mentioned.

            My point is precisely that. I just named very common things most people do online, and in comparison, what you described is nowhere near as common or easy. So it can get substantially easier. It's not an overestimate to say that what you're describing is not as convenient as using something everyone already has (phone, laptop, whatever), and doing things that are free. That's all the examples I gave and many more I didn't. That's completely unlike what you just described.

            If you search "send a picture to someone online" you can get a million different results that would accomplish the job easily.

            But let's just say someone happens to have the correct hardware. Now let's say you search "stable diffusion download"

            https://duckduckgo.com/?hps=1&q=stable+diffusion+download&ia=web

            Straight off the bat, just figuring out how to use the program on your own doesn't seem all that accessible to a casual user. Even if you go with a guide like the howtogeek at the top, it says you need a gpu with 6gb of vram and a myriad of other things, with a more complicated setup process than just clicking an install button.

            Do you know how many other things you can do on a computer that don't have such requirements and are drastically easier?

            So again I'll go back to my original statement and say that it can get much easier. This is also easily provable by finding many other things you can do that are much easier, and then comparing them to what you're doing.

            1 vote
            1. [4]
              ShrubOfRegret
              Link Parent
              You were talking about hardware, so that's what I addressed. Despite the requirements people often repeat, you don't even need a GPU if you don't care about it being slow. You're right that setup...

              You were talking about hardware, so that's what I addressed. Despite the requirements people often repeat, you don't even need a GPU if you don't care about it being slow.

              You're right that setup for AI could be easier. There's no single click setup, and, if there is, it isn't the first search result or even on the first page of results. The tutorials for what we do have aren't good; if you're in a terminal on Windows you're doing it wrong. All the pieces are there for someone to make an easy automatic deepfake program, but for right now it still requires a bit of effort.

              However, I'd say this is similar to the saying that locks exist to keep an honest man honest. We still have a lock on the door, but anyone inclined to actually look at it is going to realize it's unlocked.

              1 vote
              1. [3]
                Grumble4681
                Link Parent
                I specifically mentioned an example about the hardware because that's what I also encountered looking into any of these models for image/audio/text generation/manipulation etc. to back the broad...

                I specifically mentioned an example about the hardware because that's what I also encountered looking into any of these models for image/audio/text generation/manipulation etc. to back the broad point I was making, it was not an exclusive case. It was just one way that I knew it could get easier.

                Just to give one example that I remember enough to easily find the page

                https://softology.pro/tutorials/tensorflow/tensorflow.htm

                An NVIDIA 3090 or 4090 GPU with 24GB VRAM is highly recommended and will get the best performance and highest resolution outputs.
                An NVIDIA 20xx GPU with 8GB VRAM would be the bare minimum hardware spec and is not recommended. Most of the machine learning modes in Visions of Chaos will not work with an 8GB GPU.
                If you have an older GPU or one with less than 8GB VRAM, do not continue. You will only be disappointed and complain "it doesn't work".
                You MUST have an NVIDIA GPU for the Machine Learning related modes to work in Visions of Chaos. If you do not have an NVIDIA GPU stop now and do not continue with the rest of this page.

                Now that's one particular program, which I only came across because I found a thread about easy ways to get into the tech without knowing a lot, but in pretty much every thread I came across for other software, it was the same repeated message. Requires Nvidia GPU because of CUDA, and a mid-high end GPU in general. I don't have an Nvidia GPU and I was merely curious what possibilities there were much like everyone was when ChatGPT first came out, but it was just one roadblock after another. I didn't care that much to go buy an Nvidia GPU and just realized that without one it's seemingly pointless at the moment to explore anything at all because it was consistently mentioned as a requirement. Again, I linked to one specific software but I saw it on multiple different ones.

                1. ShrubOfRegret
                  Link Parent
                  Yeah, I'm not entirely sure why people pick the "requirements" they do. They act like if you can't generate a 4k image in less than 30 seconds that it doesn't work at all. I've played around with...

                  Yeah, I'm not entirely sure why people pick the "requirements" they do. They act like if you can't generate a 4k image in less than 30 seconds that it doesn't work at all. I've played around with image, voice, and text AI. The only one that I truly needed better hardware for was the text AI because those models are huge.

                  As far as image AI/Stable Diffusion, like I said, an old 4GB Nvidia laptop GPU is working well enough for me. The setup is different, but you can get AMD cards that support ROCm working. Even then if your card at least supports DirectX 12, Windows has DirectML (DML) as another option. Then failing everything else if you have a decent CPU that can work too.

                  There are tons of ways to get Stable Diffusion working. It's just that anything other than the easiest route isn't well represented.

                  2 votes
                2. Dr_Amazing
                  Link Parent
                  These requirements are way too high. I'm getting just by just fine with 6gb of vram and I know you can get by with 4 if you really have to.

                  These requirements are way too high. I'm getting just by just fine with 6gb of vram and I know you can get by with 4 if you really have to.

                  2 votes
        2. Dr_Amazing
          Link Parent
          I guess it depends on your definition of "high end". My computer can handle it. When I bought it 3 years ago it was pretty powerful but not top if the line.

          I guess it depends on your definition of "high end". My computer can handle it. When I bought it 3 years ago it was pretty powerful but not top if the line.

          1 vote
        3. sparksbet
          Link Parent
          What you need to train a model like this and what you need to use one are very different things. A good DeepFake model may well be very difficult to train initially but use significantly fewer...

          What you need to train a model like this and what you need to use one are very different things. A good DeepFake model may well be very difficult to train initially but use significantly fewer resources when you're using an already-trained model (that is, to make a deepfake with a couple images).

        4. bonedriven
          Link Parent
          Pricing isn't really an issue as GPUs can be rented from cloud services. I can rent an A6000 for less than $1 an hour and train a stable diffusion model within about an hour with a set of 20...

          Pricing isn't really an issue as GPUs can be rented from cloud services. I can rent an A6000 for less than $1 an hour and train a stable diffusion model within about an hour with a set of 20 images, and I don't claim to be any kind of expert in the area just curious about the technology.

  4. [6]
    ignorabimus
    (edited )
    Link
    What a horrible act to commit against another human. This is slightly tangential, so I apologise in advance. I think that eventually that the internet deregulation advocates are going to have to...

    What a horrible act to commit against another human.

    This is slightly tangential, so I apologise in advance.

    I think that eventually that the internet deregulation advocates are going to have to face the music on the nasty parts of the internet. There's a subset of online privacy advocates who feel that government should never intervene when it comes to the internet. I think a lot of these people are reasonably privileged white men who haven't had to face any of the evils the modern internet has to offer. They often talk of the need to protect people's "privacy" in a very simplistic way (i.e. talking about privacy without asking about privacy for whom).

    There are lots of valid fears, for example that the NSA/GCHQ want to advance their surveillance of the internet. I really feel that the correct response, however, is not to deny that the internet needs more regulation, but to work out how we can write legislation that protects people's privacy while they are using the internet and also in real life.

    One thing I do think will happen is age restrictions for online pornography – we're already seeing moves towards browser-based identification such as the Web Integrity API and Privacy Pass. It's not great for Google to design browser-based identity systems, especially if they can convince states to use them later. Clearly we want to have a system which preserves the anonymity of adults who are watching pornography, without making it possible for children to do so and I think it would be more powerful for privacy campaigners to advocate for better and safer such systems.

    11 votes
    1. [3]
      tesseractcat
      Link Parent
      I think it's questionable how much you can maintain anonymity and still regulate individual internet usage... This is why you see so much backlash to even small regulation (it's slippery slopey)....

      I think it's questionable how much you can maintain anonymity and still regulate individual internet usage... This is why you see so much backlash to even small regulation (it's slippery slopey). All part of the larger war on general purpose computing, which is great if you're a victim of deepfakes but not so great if you're posting anything subversive.

      12 votes
      1. [2]
        ignorabimus
        Link Parent
        I'm not a cryptographer (I'm in an adjacent field) so please take what I say with a grain of salt, but I believe that it is technically possible to do so.

        I'm not a cryptographer (I'm in an adjacent field) so please take what I say with a grain of salt, but I believe that it is technically possible to do so.

        1. Moonchild
          (edited )
          Link Parent
          Your previous comment was (intentionally) very vague and general, and so was the parent; the slide deck is about something very specific. I guess that is about keeping kids from watching porn....

          Your previous comment was (intentionally) very vague and general, and so was the parent; the slide deck is about something very specific. I guess that is about keeping kids from watching porn. But, aside my opinions on that, it's hardly the most important issue here, and it's not obvious what significance the content of the slide deck has to any other attendant issues.

          6 votes
    2. [2]
      gowestyoungman
      Link Parent
      We need to get the point of acknowledging that porn is a net negative for ALL of society, not just women who are deepfaked and not just naive minors. Like many things, its permissible to many, but...

      We need to get the point of acknowledging that porn is a net negative for ALL of society, not just women who are deepfaked and not just naive minors. Like many things, its permissible to many, but the world would not be worse off it if it was highly controlled. There is clear and growing evidence that it has hugely damaging effects on young people who are exposed to it in their formative years - which these days is EVERY kid with a phone. It's NOT good. When your 12 yr old son is watching girls being choked, slapped and degraded or viewing kink and fetishes before he's even held a girl's hand, it's not a wonder that many men come into a dating relationship with really disturbing views on what healthy, respectful sex looks like.

      Instead of being something easily accessed by anyone with the ability to click 'Yes Im 18' on a screen, it would be much better if the viewer had to prove you were an adult by at least entering a credit card number for that click, even if there was no charge. They dont give out credit cards to minors and it would be a far more reliable system than just a screen click. Is there a chance the card data would be stolen? Absolutely. But that would also encourage people to use 'regulated' sites and not just any skeezy site that pops up on their browser.

      3 votes
      1. ignorabimus
        Link Parent
        I'm not sure about this, but I do think that it's not a good thing for children to be watching porn. I actually disagree quite strongly here. I think cryptographic tokens which allow you to prove...

        We need to get the point of acknowledging that porn is a net negative for ALL of society, not just women who are deepfaked and not just naive minors.

        I'm not sure about this, but I do think that it's not a good thing for children to be watching porn.

        Instead of being something easily accessed by anyone with the ability to click 'Yes Im 18' on a screen, it would be much better if the viewer had to prove you were an adult by at least entering a credit card number for that click, even if there was no charge. They dont give out credit cards to minors and it would be a far more reliable system than just a screen click. Is there a chance the card data would be stolen? Absolutely. But that would also encourage people to use 'regulated' sites and not just any skeezy site that pops up on their browser.

        I actually disagree quite strongly here. I think cryptographic tokens which allow you to prove that you are of a certain age without actually having to disclose any other data (e.g. who you are) are a better way to go. I also don't think that credit cards are an appropriate tool for identity verification either.

        9 votes