45 votes

Stack Overflow moderators are striking to stop garbage AI content from flooding the site

28 comments

  1. [15]
    sparksbet
    Link
    I 100% think the moderators are in the right here, and I'm honestly kinda floored that Stack Exchange would override mod decisions like this. The moderators have generally been very forthright...

    I 100% think the moderators are in the right here, and I'm honestly kinda floored that Stack Exchange would override mod decisions like this. The moderators have generally been very forthright about why they chose to have a policy against AI-generated content and as someone who works in NLP, they're entirely justified and well-reasoned.

    It's kinda bizarre to me why they would think an order from on high forcing AI-generated content to be allowed would be good for the site. There are times when overriding the consensus of users is important but there has to be a good reason, and I cannot see one here.

    30 votes
    1. [14]
      stu2b50
      Link Parent
      It’s because it’s not feasible to filter out LLM text. Stackoverflow said as much in their press release. There’s multiple studies now showing that a) none of the “ai detectors” can actually...

      It’s because it’s not feasible to filter out LLM text. Stackoverflow said as much in their press release. There’s multiple studies now showing that

      a) none of the “ai detectors” can actually detect LLM written text with any kind of accuracy

      b) humans also cannot detect LLM written text with any kind of accuracy

      Trying to ban chatgpt text just causes witch hunts, unreasonable moderator attention and resourcing, and lots of false positives which are very negative experiences for users.

      24 votes
      1. [5]
        teaearlgraycold
        Link Parent
        I agree it's not a fight you can win. And it's a shame. I think ChatGPT and Copilot somewhat obsolete Stack Overflow. But there should still be a place to go where you know you're getting a...

        I agree it's not a fight you can win. And it's a shame. I think ChatGPT and Copilot somewhat obsolete Stack Overflow. But there should still be a place to go where you know you're getting a response from a human. Why would you want an AI response on SO? If you're there it's because you specifically don't want an LLM response.

        12 votes
        1. [4]
          0x29A
          Link Parent
          Exactly, and yeah it does feel currently like a nearly unsolvable problem. Currently, I will trust a StackOverflow answer with 100x the level of trust I would put in a ChatGPT answer, and with...

          If you're there it's because you specifically don't want an LLM response.

          Exactly, and yeah it does feel currently like a nearly unsolvable problem. Currently, I will trust a StackOverflow answer with 100x the level of trust I would put in a ChatGPT answer, and with influx of AI-generated data, I feel like crowdsourced, higher-quality information in places like SO is at a huge risk of being lost and replaced by something absolutely inferior. That seems like a net negative, but how do we go about preventing it?

          Maybe, non-AI or at least, accurate AI answers would potentially still surface to the top? I'm not sure. But it feels like generative AI is about to accelerate the internet's already severe "noise vs signal" problem

          6 votes
          1. teaearlgraycold
            Link Parent
            They could ramp up their reputation system. Lazy people spamming Codex responses should be too lazy to spend time getting past it.

            They could ramp up their reputation system. Lazy people spamming Codex responses should be too lazy to spend time getting past it.

            6 votes
          2. [2]
            AssPennies
            Link Parent
            I'm curious/worried that this aspect is going to create a negative feedback loop, one like we've never seen before, and it's going to devolve content into something abysmal at an ever increasing...

            AI is about to accelerate the internet's already severe "noise vs signal" problem

            I'm curious/worried that this aspect is going to create a negative feedback loop, one like we've never seen before, and it's going to devolve content into something abysmal at an ever increasing rate.

            One can already see this starting to happen on some poorly moderated social media sites, where bots are responding to bots, and threads eventually turn nonsensical.

            2 votes
            1. Lonan
              Link Parent
              In some ways I wonder if AI is the best it can ever be right now, because it has been trained on "pure" human information. Future generations could give diminishing returns as they are trained on...

              In some ways I wonder if AI is the best it can ever be right now, because it has been trained on "pure" human information. Future generations could give diminishing returns as they are trained on worse and worse input from mixed AI / human information. Similar to how photocopies get worse the more copies you make. You can make a better photocopier (make a better AI model), but if the input is diluted you have to work harder than ever to get a meaningful output.

              5 votes
      2. [3]
        balooga
        Link Parent
        I think you've really keyed in on the core issue here. Pandora's box is open, AI-generated content is here and not going anywhere, and there is currently no reliable way to detect it at scale....

        I think you've really keyed in on the core issue here. Pandora's box is open, AI-generated content is here and not going anywhere, and there is currently no reliable way to detect it at scale. Enacting a policy against it will not make it vanish.

        I assume most SO users are drive-bys like me: Google directs us to a probable solution and we consume it then close the tab. I've spent an enormous amount of time on the site but I'm not a contributing part of the community. It's like Wikipedia, I've gleaned a ton of knowledge from it but I'm not writing or editing articles. So far be it from me to suggest strategies for dealing with the AI inaccuracy problem; as an outsider my ideas are surely out of step with their cultural ethos.

        With that disclaimer out of the way, I'd think there could be some way for users to flag answers as "incorrect." Not necessarily AI-generated, but simply wrong/misinformation. Which IMHO should be the real criteria for acceptability anyway — I don't care about an answer's provenance if it's factually correct and well-written. I'm pretty sure SO has only an upvote/downvote system. Usually this weeds out the bad responses (low effort, didn't understand the question, wrong, etc.) but I guess GPT answers are still getting upvoted because they are so good at sounding correct even when they're not. So a more powerful tool for users that functions as an uber-downvote might counteract that, and take some of the responsibility off mods for sorting this out themselves?

        10 votes
        1. Fiachra
          Link Parent
          That was my line of thinking as well - the problem with AI-generated answers is that they hide factual errors in well-written paragraphs. So come down hard on the problem: be stricter about...

          That was my line of thinking as well - the problem with AI-generated answers is that they hide factual errors in well-written paragraphs. So come down hard on the problem: be stricter about factual errors. Have a far lower removal threshold for this, and any other problems that are increasing due to AI spam.

          3 votes
        2. riQQ
          Link Parent
          There's also a system to flag posts for being off-topic, spam, not in English, plagiated and some more reasons. But for incorrect answers the current way to handle these is downvoting.

          There's also a system to flag posts for being off-topic, spam, not in English, plagiated and some more reasons. But for incorrect answers the current way to handle these is downvoting.

          1 vote
      3. [5]
        sparksbet
        Link Parent
        I agree that using automated tools or just human intuition would not be useful (except in certain obvious cases where there would probably be other grounds for removal anyway). But this is not the...

        I agree that using automated tools or just human intuition would not be useful (except in certain obvious cases where there would probably be other grounds for removal anyway). But this is not the only way to moderate AI-generated content aside from just wholesale allowing it. A lot of the stuff I've seen on banning AI-generated content, at least from subreddit mods, has been focused on getting rid of content that is openly AI-generated ("I asked Chat GPT and it said...") or posted in such quick succession as to be undoubtedly not written by a human. The latter might still be moderatable on StackOverflow on the grounds of being spam (I don't know what there ruleset is there, but they do have some limits on new users in that respect at least), but the former is a very clear and easy way to get rid of low-effort content without the risk of false positives you describe. My reading of Stack Exchanges's new policy is that it wouldn't allow them to moderate such posts.

        2 votes
        1. [4]
          stu2b50
          Link Parent
          What is the actual threat model there? Those are presumably people acting in good faith (since they're announcing that they're using AI). Not only can you just ignore them, downvote them, say mean...

          What is the actual threat model there? Those are presumably people acting in good faith (since they're announcing that they're using AI). Not only can you just ignore them, downvote them, say mean things to them in replies, but they're also the kind of people that will stop once there's a negative signal.

          Because the people that aren't acting in good faith, will just remove the part where they say they used chatGPT and we're back to "there is no actual way to reliably distinguish LLM and human text".

          If nothing else, it's a small issue in the grand scheme of things.

          1 vote
          1. [2]
            skybrian
            Link Parent
            Ideally there would be some kind of educational process: I think StackOverflow is dealing with an Eternal September problem, though? Maybe they need to get better at onboarding clueless...

            Ideally there would be some kind of educational process:

            Great that you're trying to help, but this is low-effort. You can use ChatGPT in your own investigations, but you need to verify that the result works before posting it.

            I think StackOverflow is dealing with an Eternal September problem, though? Maybe they need to get better at onboarding clueless volunteers, somehow.

            1 vote
            1. sparksbet
              Link Parent
              What worries me is that my understanding of StackExchange's new policy would prohibit mods from even taking this kind of action against self-proclaimed ChatGPT posts.

              What worries me is that my understanding of StackExchange's new policy would prohibit mods from even taking this kind of action against self-proclaimed ChatGPT posts.

              1 vote
          2. sparksbet
            Link Parent
            Do you have much experience moderating communities like this? An influx of low-effort posts that can be produced much more quickly than they could by a human writing even low-effort content can...

            Do you have much experience moderating communities like this? An influx of low-effort posts that can be produced much more quickly than they could by a human writing even low-effort content can really inundate moderators who are trying to keep content on-topic and informative -- and people seeing other ChatGPT posts not removed are only incentivized to post such things, especially on a site that removes posts pretty readily like Stack Overflow and the various Stack Exchange sites.

            As for "just downvote them", obviously people can already do that. But it's not like the mods and communities haven't had discussion about why they think that's inaccurate. The fact that the admins are overruling the decisions of their users and mods at all here is weird. Stack Exchange sites typically give their communities and especially moderators a lot of leeway to decide their rules and norms, and it's extremely abnormal to completely overrule them like this on that site. There was already plenty of discussion about what types of posts could be removed and how effective such moderation could be in the stack exchange meta sections, but then the company came out of the blue and overruled the conclusions the community and moderators came to. That's a big move to make for a site with such a history of mod and community autonomy and it seems like even in the best-case scenario the site has extremely little benefit from forcing their will over the users and mods like this.

            1 vote
  2. simplify
    Link
    I don’t blame them, a lot of AI generated code is indeed wrong and people spamming SO with it are just trying to get upvotes for internet clout. But at the same time, since beginning to use...

    I don’t blame them, a lot of AI generated code is indeed wrong and people spamming SO with it are just trying to get upvotes for internet clout. But at the same time, since beginning to use Copilot, I’ve been looking at SO a lot less. I’d be interested to know if the number of daily visitors has dwindled some since LLM plugins were released. As the models and plugins get better, I can see SO becoming obsolete or just used in very hyper-specific cases.

    5 votes
  3. [2]
    SirDeviant
    Link
    What does StackOverflow gain by allowing AI-generated content? Bots don't watch ads, and users obviously aren't a fan. Are they trying to bump up their activity numbers for an IPO?

    What does StackOverflow gain by allowing AI-generated content? Bots don't watch ads, and users obviously aren't a fan. Are they trying to bump up their activity numbers for an IPO?

    1 vote
    1. [2]
      Comment deleted by author
      Link Parent
      1. supergauntlet
        Link Parent
        ultimately I think the solution is that stackoverflow mods will have to make their own, probably invite only or similar, site that doesn't allow bot accounts. Of course that also means you have to...

        ultimately I think the solution is that stackoverflow mods will have to make their own, probably invite only or similar, site that doesn't allow bot accounts.

        Of course that also means you have to have a lot of community members that do interviews and let lots of new users in, or else there's no point to such a site.

  4. krlombs
    Link
    We are now reaching a time where AI created content can begin to pass enough as “real” content. From this point forward most things involving any kind of story/content/user moderation will be...

    We are now reaching a time where AI created content can begin to pass enough as “real” content. From this point forward most things involving any kind of story/content/user moderation will be spammed with content that is sub-par at best.

    1 vote
  5. AppleVision
    Link
    If the AI is correct, why be biased against AI generated content? How to even detect AI generated content? Should we be against spellcheck as well?

    If the AI is correct, why be biased against AI generated content? How to even detect AI generated content? Should we be against spellcheck as well?

    1 vote
  6. [2]
    DawnPaladin
    Link
    This is far from the first time Stack Exchange has pissed everyone off by forcing an unpopular policy change. I wonder if it would be feasible to make an open-source version of StackOverflow with...

    This is far from the first time Stack Exchange has pissed everyone off by forcing an unpopular policy change. I wonder if it would be feasible to make an open-source version of StackOverflow with a better leadership structure.

    Writing the software wouldn't be too difficult, I don't think. The Q&A format isn't super complicated. All StackOverflow content is licensed Creative Commons Attribution-ShareAlike, so you could legally import all of StackOverflow's content to help get your fork started.

    I feel like the hard part would be devising a leadership structure that keeps the site running and doesn't repeat the mistakes of Stack Exchange. As @arp242 described, the promise of StackOverflow is that if you put in the work of moderation, you'll be rewarded with a say in how the site is run. But that's not really true; at the end of the day, Stack Exchange has the final say, and they have no real obligation to listen to anyone. How would you design an organization that functioned democratically, where the vote of the moderators has real sway?

    1. [2]
      Comment deleted by author
      Link Parent
      1. DawnPaladin
        Link Parent
        Codidact is exactly the sort of thing I was looking for! Glad to see that exists. Adding that to my bookmarks - thank you!

        Codidact is exactly the sort of thing I was looking for! Glad to see that exists. Adding that to my bookmarks - thank you!

  7. [6]
    Comment deleted by author
    Link
    1. [3]
      Adys
      Link Parent
      I really don't, and tbh I think it's one of those "LLM-newbie traps": It's easy to end up thinking that the AI has opinions. It does not. As one of Tildes' staunchest supporters of the "this is...

      I think it's interesting to see what AI "thinks" of itself.

      I really don't, and tbh I think it's one of those "LLM-newbie traps": It's easy to end up thinking that the AI has opinions. It does not.

      As one of Tildes' staunchest supporters of the "this is the beginning of AGI" narrative, I make a very very strong distinction between reasoning and opinion. Opinions are a fairly human thing. They're a shortcut. Now, it doesn't mean an AI (especially an AGI) couldn't be trained to have opinions, but the current frameworks don't make this particularly easy or obvious.

      GPT-4 is better at this as OpenAI has taught it to explicitly avoid expressing opinions, and going by the whole "I don't have opinions" / "I don't form opinions" explicit warnings in your text, this looks like GPT-4 to me, am I correct? There's multiple ways to tweak your prompt to get it to react differently to the article; I'm sure if I were bored enough I could get strikingly different results and "opinions" from two fairly similar and innocuous prompts, even bypassing the whole "as an AI" thing.

      11 votes
      1. [3]
        Comment deleted by author
        Link Parent
        1. [2]
          skybrian
          Link Parent
          I'm wary of treating this as an "average" opinion. First of all, chatbots use random number generators when generating responses. You would need to regenerate the response multiple times and see...
          • Exemplary

          I'm wary of treating this as an "average" opinion.

          First of all, chatbots use random number generators when generating responses. You would need to regenerate the response multiple times and see how its responses vary to get a better idea of the average chatbot response.

          But let's suppose they're all much like this one. I still don't think it makes sense as a human average opinion. Whenever there's an average, you have to think about what's in the denominator. Which people do chatbot responses average over? Does it give the people surveyed equal weight? Surveys are cursed instruments, but a properly-run poll is a much better way of trying to get an average opinion of a human population.

          So instead, I think it's better to use chatbots as tools for brainstorming. If it generates an argument that you hadn't considered, you can look into it. If it just gives you arguments that you would have thought of on your own, it didn't help.

          In this case, it just generated bland boilerplate, like a student trying to complete a homework assignment who doesn't really care. So, it's not that good from a brainstorming perspective.

          But maybe we could do better at getting it to generate some interesting ideas with a more creative prompt? For example, you could ask what a Marxist would think of this situation. (Or substitute someone more interesting.)

          1 vote
          1. skybrian
            Link Parent
            So here's what I tried: Yeah, these jokes are mostly pretty awful, but I think a few have potential. (It would look better if I threw out the bad ones.)

            So here's what I tried:

            Could you generate some funny quotes from famous people commenting on the following article? https://www.vice.com/en/article/4a33dj/stack-overflow-moderators-are-striking-to-stop-garbage-ai-content-from-flooding-the-site

            Sure, here are some humorous quotes from famous individuals commenting on the situation:

            Albert Einstein: "Two things are infinite: the universe and AI-generated content on Stack Overflow, and I'm not sure about the universe."

            Mark Twain: "The reports of the usefulness of AI-generated content on Stack Overflow are greatly exaggerated."

            Oscar Wilde: "Experience is simply the name moderators give to their AI content removals."

            Groucho Marx: "Behind every successful AI, there are a lot of unsuccessful Stack Overflow moderators."

            Stephen Hawking: "I have noticed even people who claim everything is predestined, and that we can do nothing to change it, look before they post AI-generated content."

            Winston Churchill: "The AI content on Stack Overflow is a riddle, wrapped in a mystery, inside an enigma; but perhaps there is a key. That key is human moderation."

            Dorothy Parker: "The first thing I do in the morning is brush my teeth and sharpen my tongue... to argue with AI content on Stack Overflow."

            Mae West: "Between two evils, I always pick the one I never tried before... so I guess it's time to moderate some AI content."

            Yogi Berra: "The future ain't what it used to be. It's now full of AI-generated content."

            Benjamin Franklin: "In this world, nothing can be said to be certain, except death, taxes, and AI content on Stack Overflow."

            Yeah, these jokes are mostly pretty awful, but I think a few have potential. (It would look better if I threw out the bad ones.)

    2. [2]
      balooga
      Link Parent
      As an aside, I'm curious if the 1 at the end of the second-to-last paragraph you pasted was put there by GPT or if it's some kind of formatting error. It looks like a footnote, maybe? I don't have...

      As an aside, I'm curious if the 1 at the end of the second-to-last paragraph you pasted was put there by GPT or if it's some kind of formatting error. It looks like a footnote, maybe? I don't have access to ChatGPT-4, not sure if it does those.

      1 vote
      1. [2]
        Comment deleted by author
        Link Parent
        1. balooga
          Link Parent
          Ah, cool cool. I'm weirdly interested in catching GPT making grammatical or formatting mistakes. It's rare but it happens. So I tend to get hung up on anything that looks like it could be an...

          Ah, cool cool. I'm weirdly interested in catching GPT making grammatical or formatting mistakes. It's rare but it happens. So I tend to get hung up on anything that looks like it could be an example of that!

          1 vote
  8. riQQ
    Link

    Volunteer moderators of the forum are striking over a policy that says AI-generated content can practically not be moderated.

    4 votes