34 votes

Creator of "DeepNude" releases it as GPL licensed Open Source Software after previously taking it down

34 comments

  1. [4]
    spit-evil-olive-tips
    (edited )
    Link
    It's really telling to see how incredibly narrowly they're thinking of "misuse" here - to them, it's someone taking their code, attaching a Bitcoin miner or some other malware, and distributing...

    If you followed our story you will know that we have decided not to continue selling DeepNude because we could no longer guarantee enough safety.
    ...
    With multiple illecit DeepNude version in the web, anonymous and unknown users, virus and malware, the assumption of security dissolved soon. There are no valid security systems, when hackers from all over the world attack you.
    ...
    We could have made a lot of money, but we preferred to limit the misuse as much as possible.

    It's really telling to see how incredibly narrowly they're thinking of "misuse" here - to them, it's someone taking their code, attaching a Bitcoin miner or some other malware, and distributing that as DeepNude. That's the exploitative and immoral behavior they want to prevent.

    29 votes
    1. [3]
      cadadr
      Link Parent
      Well playing the devils advocate, isn't it possible that the unofficial stuff upload resulting pictures to internet involuntarily, whereas the official version wouldn't? (Tho I'm pretty unfamiliar...

      Well playing the devils advocate, isn't it possible that the unofficial stuff upload resulting pictures to internet involuntarily, whereas the official version wouldn't? (Tho I'm pretty unfamiliar with the story here so maybe IDK what I am talking about)

      1 vote
      1. moocow1452
        Link Parent
        It's only more ethical towards the person using the software, which as stated above, is where the priority of the devs lie. And if you make an argument that a corrupted version could be placed on...

        It's only more ethical towards the person using the software, which as stated above, is where the priority of the devs lie. And if you make an argument that a corrupted version could be placed on the web somewhere, it is much easier to do that with the source.

        17 votes
      2. stu2b50
        Link Parent
        Neither of these upload to the internet. This is very bare-bones, it doesn't even include the model checkpoints. It just has the models and a basic CLI for running it on an image.

        Neither of these upload to the internet. This is very bare-bones, it doesn't even include the model checkpoints. It just has the models and a basic CLI for running it on an image.

        3 votes
  2. [4]
    stu2b50
    Link
    It is amusing to go on the issues page and see people who clearly have no idea what they're doing, like "It runs but does nothing" (it literally spit out an error and you copy pasted it into the...

    It is amusing to go on the issues page and see people who clearly have no idea what they're doing, like "It runs but does nothing" (it literally spit out an error and you copy pasted it into the issue box) and "no module named cv2" (maybe try installing opencv2?).

    HM I wonder why these people are so interested in running this.

    22 votes
    1. json
      Link Parent
      Script kiddies want their free fake nudes.

      Script kiddies want their free fake nudes.

      4 votes
  3. [12]
    cptcobalt
    Link
    Releasing the source is a terrible idea. Do I really need to say anything more? We can all figure out the nuance of why this is terrible on our own, right?

    Releasing the source is a terrible idea.

    Do I really need to say anything more? We can all figure out the nuance of why this is terrible on our own, right?

    19 votes
    1. [10]
      Octofox
      Link Parent
      Is it really though? The cat is out of the bag now. AI research has progressed far enough that any programmer with the desire to create this could have and similar things have been popping up all...

      Is it really though? The cat is out of the bag now. AI research has progressed far enough that any programmer with the desire to create this could have and similar things have been popping up all over the place for the last few years.

      20 votes
      1. [9]
        umbrae
        (edited )
        Link Parent
        Yes, it is, here are two reasons in my view: First is a purely moral one. Put yourself in the shoes of the author. Regardless of where you feel the world at large is headed, do you feel releasing...

        Yes, it is, here are two reasons in my view:

        First is a purely moral one. Put yourself in the shoes of the author. Regardless of where you feel the world at large is headed, do you feel releasing this software to the world will cause on balance more harm than good?

        The answer here is obviously more harm. Being responsible for that regardless of where the technical zeitgeist is headed makes this reprehensible in my view.

        Second is more logistical: it’s likely there will be regulatory or legal corrections made for creating these over time. There may even be laws on the books now, I’m not sure. Putting this out in public before the regulatory engine can catch up increases the time delta in between which it can be used for harm without legal repercussion.

        16 votes
        1. [2]
          babypuncher
          Link Parent
          Laws like this are often reactionary. I would argue that the laws you suggest here won't come about until there is a proven need for them, so the timing of the release of this software does not...

          Putting this out in public before the regulatory engine can catch up increases the time delta in between which it can be used for harm.

          Laws like this are often reactionary. I would argue that the laws you suggest here won't come about until there is a proven need for them, so the timing of the release of this software does not change the delta between then and the passing of relevant legislation.

          11 votes
          1. umbrae
            Link Parent
            I think it’s true that laws won’t come about until there’s a proven need for them, but I think it’s a big logical leap to presume that the release of this software will hasten those laws being...

            I think it’s true that laws won’t come about until there’s a proven need for them, but I think it’s a big logical leap to presume that the release of this software will hasten those laws being written by a commensurate amount. As @octofox said, the cat is already out of the bag.

            It’s not impossible that this hastens legislation by being made public - it’s for sure hard to estimate what impact this being public could have. Maybe some impact! But I don’t think it overrides the individual and very real harm that this could cause for individuals now, and I think there’s greater odds that the legal engine will take about the same amount of time, and all this has done is cause more harm in between the time it was released and the time the laws caught up to it.

            All that aside though, there’s still the moral point.

            5 votes
        2. [6]
          stu2b50
          Link Parent
          To play devil's advocate, there were already replications on Github before they released, which was probably a motivating factor. If someone wanted it for nefarious or non-nenfarious reasons, they...

          To play devil's advocate, there were already replications on Github before they released, which was probably a motivating factor. If someone wanted it for nefarious or non-nenfarious reasons, they already had it. Releasing their version shouldn't cause more harm now, since it already exists in the wild

          And can be beneficial. I was curious on how they managed such good results without paired data (which is obv impossible). They ostensible released to help other people doing work in the field.

          4 votes
          1. [5]
            doas
            Link Parent
            That's extremely variable. I was also curious, and found that the "nude" of this image is this monstrosity.

            I was curious on how they managed such good results

            That's extremely variable. I was also curious, and found that the "nude" of this image is this monstrosity.

            4 votes
            1. [2]
              ali
              Link Parent
              it's always possible to find examples in which a neural network fails. The important and interesting cases are the ones where it works 'too' well

              it's always possible to find examples in which a neural network fails. The important and interesting cases are the ones where it works 'too' well

              3 votes
              1. doas
                Link Parent
                That's true. But this is hardly some edge case/example, it's about as ideal an input as the algorithm should want. High contrast between person and background, clear boundaries of underwear, only...

                That's true. But this is hardly some edge case/example, it's about as ideal an input as the algorithm should want. High contrast between person and background, clear boundaries of underwear, only one forward facing person etc.

                I've seen very few examples, actually, where the deepnude actually looked anything near realistic. Most are malformed in some way.

                4 votes
            2. [2]
              Diff
              Link Parent
              Is that an explosion in the middle there?

              Is that an explosion in the middle there?

              1. doas
                Link Parent
                She god damn imploded.

                She god damn imploded.

    2. hook
      Link Parent
      I would be interested in having this debate. To add some nuance, I'm throwing in the Low Orbit Ion Cannon as another, yet different, example of FOSS that is presumably primarily used for nefarious...

      I would be interested in having this debate.

      To add some nuance, I'm throwing in the Low Orbit Ion Cannon as another, yet different, example of FOSS that is presumably primarily used for nefarious purposes.

      9 votes
  4. jclishman
    Link
    This sort of stuff is just what we're going to have to deal with now. As humanity progresses up the tech tree, the problems and challenges it will have to face get harder.

    This sort of stuff is just what we're going to have to deal with now. As humanity progresses up the tech tree, the problems and challenges it will have to face get harder.

    12 votes
  5. moocow1452
    Link
    I would have appreciated the honesty if they had said, "We want this to exist in the world, and we can't find a payment processor that will let us charge for it." But that would probably be a big...

    I would have appreciated the honesty if they had said, "We want this to exist in the world, and we can't find a payment processor that will let us charge for it." But that would probably be a big leap for the spineless sort who would make this kind of program in the first place.

    9 votes
  6. onyxleopard
    Link
    This was inevitable, but I have to think that people had the exact same reactions around the introduction of manual photo editing technology. This is just automated photoshop, and the world hasn’t...

    This was inevitable, but I have to think that people had the exact same reactions around the introduction of manual photo editing technology. This is just automated photoshop, and the world hasn’t ended just because pro photo editors can make realistic face swaps with porn actresses. If this guy didn’t release his source, one of the umpteen others who were trying to replicate his results eventually would have. I just hope someone is collecting a training set for images of men so that we can have equal gender representation in the fake nude-filled future.

    5 votes
  7. [3]
    Elronnd
    Link
    Seems to have been taken down again...

    Seems to have been taken down again...

    5 votes
    1. [2]
      Keegan
      Link Parent
      I'm assuming its because it violated GitHub's terms of service. First of all, the repository had several nude images of a 3D model, which could be said was "educational" due to it being there to...

      I'm assuming its because it violated GitHub's terms of service.

      Sexually obscene content - Don’t post content that is pornographic. This does not mean that all nudity, or all code and content related to sexuality, is prohibited. We recognize that sexuality is a part of life and non-pornographic sexual content may be a part of your project, or may be presented for educational or artistic purposes. We do not allow obscene sexual content or content that may involve the exploitation or sexualization of minors.

      First of all, the repository had several nude images of a 3D model, which could be said was "educational" due to it being there to explain the process of how the program works.

      The big issue is that this program does enable possible sexualization of minors, since I guarantee it has been used for that at least once.

      Maybe I'm wrong, but I really suspect it was taken down by GitHub rather than deleted by the owner.

      9 votes
      1. Deimos
        Link Parent
        Motherboard posted an article today about it that includes a statement from GitHub and seems to confirm that it was taken down: GitHub Removed Open Source Versions of DeepNude (@Elronnd)

        Motherboard posted an article today about it that includes a statement from GitHub and seems to confirm that it was taken down: GitHub Removed Open Source Versions of DeepNude

        (@Elronnd)

        7 votes
  8. [2]
    mrbig
    Link
    What exactly does this app do?

    What exactly does this app do?

    3 votes
    1. alyaza
      Link Parent
      undresses any picture of a woman you feed into it. VICE had an article on it and a followup when it seemed like they were going to take it offline a few days ago.

      undresses any picture of a woman you feed into it. VICE had an article on it and a followup when it seemed like they were going to take it offline a few days ago.

      9 votes
  9. [5]
    alexandria
    Link
    Hold on, what exactly is the code in this instance? Is it the software to create and train the neural net, sans the gigabyte/terabyte collection of images used to train it? or is it a pre-trained...

    Hold on, what exactly is the code in this instance? Is it the software to create and train the neural net, sans the gigabyte/terabyte collection of images used to train it? or is it a pre-trained neural net?

    3 votes
    1. [4]
      stu2b50
      Link Parent
      That. It defines the architecture of the networks, and there's a relatively basic CLI that takes in an image and runs the network on it. The model checkpoints aren't included (though other people...

      Is it the software to create and train the neural net

      That. It defines the architecture of the networks, and there's a relatively basic CLI that takes in an image and runs the network on it. The model checkpoints aren't included (though other people reverse engineered them).

      It of course doesn't have the training data, and I doubt it's anywhere near a terabyte. From the readme, the pix2pix instance that masks bodies was trained on paired data where they manually drew circles on boobs in photoshop, and I doubt they had an army of slave laborers to do that.

      6 votes
      1. [3]
        Wes
        (edited )
        Link Parent
        Seems like something you could accomplish for a small fee via Mechanical Turk or Fiverr. I'd actually be surprised if these services weren't being used already to generate training data for...

        Seems like something you could accomplish for a small fee via Mechanical Turk or Fiverr.

        I'd actually be surprised if these services weren't being used already to generate training data for various projects.

        4 votes
        1. burkaman
          Link Parent
          There are whole companies that are essentially just "Mechanical Turk for your machine learning project": https://scale.ai/

          There are whole companies that are essentially just "Mechanical Turk for your machine learning project": https://scale.ai/

          4 votes