22 votes

EFF's Red Flag Machine: Guess why GoGuardian flagged a site

11 comments

  1. [6]
    vord
    Link
    A quick glance into one of the terrible tools to further prevent student learning via automation. I have many thoughts WRT censorship and the tools appropriate for the job, but as this highlights,...

    A quick glance into one of the terrible tools to further prevent student learning via automation.

    I have many thoughts WRT censorship and the tools appropriate for the job, but as this highlights, keyword-based banning is rarely a useful endeavor. I'd love to hear some thoughts from our resident teachers.

    11 votes
    1. [5]
      OBLIVIATER
      Link Parent
      It's worth noting that these filters are set up by the school administration and configured by them to block things. It's entirely within their power to either block or allow certain keywords and...

      It's worth noting that these filters are set up by the school administration and configured by them to block things. It's entirely within their power to either block or allow certain keywords and things like that.

      Every teacher I know pretty much entirely relies on software provided by companies like GoGuardian to even begin to keep up with the kids on their computers at school. Without it, it'd be the wild west for school issued devices, with pretty much no good way to have any oversight. There are also liability concerns with students viewing inappropriate content on school premises or on school issued devices so a scorched earth approach is often taken to protect the schools from litigation.

      9 votes
      1. [4]
        vord
        (edited )
        Link Parent
        I'd point to the Research page, which shows that by and large, students mostly aren't trying to do that. Admittedly some of that is just the chilling effect, so in that vein the software is...

        I'd point to the Research page, which shows that by and large, students mostly aren't trying to do that. Admittedly some of that is just the chilling effect, so in that vein the software is working as intended. That page is probably the one I should have linked to in retrospect, as it gives much more nuance to the discussion.

        Maybe it's just my age showing, but IMO kids need that wild west paired with some accountability for messing it up. Course that ship probably sailed long ago.

        The litigation and liability issue is definitely something seriously wrong with this country.

        My privileged responses I'm sure most teachers would love to say but probably can't: Oh your kid was looking at porn? Well maybe they shouldn't have been doing that. Now they'll have to use paper or provide their own device.

        Oh your kid failed because they weren't paying attention or doing the work? Sucks to be them, maybe they'll try harder next time.

        And to be clear, I'm not opposed to blocking on school networks as a whole. But I remember keyword filtering from my high school 20+ years ago and it looks like the status quo has not improved since then. Nowadays I feel a porn and piracy DNS block would provide 99.9% of the protection without the overbearing noise and frustration caused by keyword filtering.

        8 votes
        1. [3]
          OBLIVIATER
          (edited )
          Link Parent
          I work for an education software company; though not in the filtering depart, I'm in suicide prevention. I can confidently say that its a lot more complicated than it was 20 years ago...
          • Exemplary

          I work for an education software company; though not in the filtering depart, I'm in suicide prevention. I can confidently say that its a lot more complicated than it was 20 years ago unfortunately. Kids can and will do everything they can to bypass filtering software and access inappropriate content, and I've heard of multiple instances of schools getting into serious trouble when this happens. I've even heard of kids getting groomed on their school issued devices and the school was held liable for it.

          This is a battle that was fought and lost years ago and we're still picking up the pieces. I'm not thrilled about what has been happening and I support the EFF, but they're coming at it from an angle like schools aren't pretty much forced to use software like ours by the government and parents.

          My privileged responses I'm sure most teachers would love to say but probably can't: Oh your kid was looking at porn? Well maybe they shouldn't have been doing that. Now they'll have to use paper or provide their own device.

          Maybe, but I have several teachers in my family, and I've talked to hundreds of teachers for work and I've yet to interact with one who wasn't reliant on some form of classroom management software to even begin to do their job. Class sizes are larger than ever in most public schools and a single teacher who is underpaid, overworked, emotionally and physically exhausted simply can't do their job with 20-30 kids on their chromebooks working completely unsupervised. They are already on tiktok and youtube shorts literally all day (and this isn't some old man yells at cloud moment, I have the first hand experience to back it up) and things aren't getting any better on that front.

          Nowadays I feel a porn and piracy DNS block would provide 99.9% of the protection without the overbearing noise and frustration caused by keyword filtering.

          Keyword filtering solutions exist specifically because DNS blocks weren't proving to be sufficient. They can be configured poorly like the ones shown on the EFF quizz, but companies provide out of the box filtering configurations that do a much better job. They usually break down if they're tweaked by well-meaning, but usually tech-ignorant school administration staff because most school IT departments are underfunded skeleton crews with one 63 year old man who's been working there forever and hasn't bothered to learn anything new about computers since Windows 95.

          I obviously can't speak about this unbiasedly because I literally work in the industry, but I can provide my educated (if biased) opinion on the subject. I come from a place of someone who used to hate this kind of thing as well, but now I view it as a necessary evil. As always the real solution to this problem would simply be to actually give schools the funding they need to properly educate children without having to resort to mass control and surveillance, but as we all know due to recent events that's not going to happen any time soon; and I fear it will never happen due to the trajectory of how much public services have gone neglected. It's simply another (fairly low) priority on the mountain of problems that society needed to sort out 15 years ago.

          14 votes
          1. [2]
            vord
            Link Parent
            That sounds about right. Laptops in the classrooms was a horrible mistake. Bring back computer labs.

            That sounds about right.

            Laptops in the classrooms was a horrible mistake. Bring back computer labs.

            8 votes
            1. OBLIVIATER
              Link Parent
              Honestly you're probably right, but they do a great job in helping lighten the load for things like researching, playing classroom games, etc but they probably do more harm than good

              Honestly you're probably right, but they do a great job in helping lighten the load for things like researching, playing classroom games, etc but they probably do more harm than good

              6 votes
  2. [5]
    balooga
    Link
    This dumb keyword filtering approach has never worked, for a million obvious reasons. I suspect LLMs are a lot more capable of determining whether a page, in context, is NSFW. Seems like it’s only...

    This dumb keyword filtering approach has never worked, for a million obvious reasons. I suspect LLMs are a lot more capable of determining whether a page, in context, is NSFW. Seems like it’s only a matter of time before this type of software starts running page content through a lightweight model to make its decisions. Which will have its own flaws of course, but I imagine the reduced rate of false positives would be worth it.

    3 votes
    1. [4]
      OBLIVIATER
      Link Parent
      It already does, at least from GoGuardian. This data from the EFF is fairly outdated or sourced from schools who manually configured their software to filter these keywords.

      It already does, at least from GoGuardian. This data from the EFF is fairly outdated or sourced from schools who manually configured their software to filter these keywords.

      5 votes
      1. [3]
        vord
        (edited )
        Link Parent
        If there is anything I know about system upgrades, the answer is always "do the absolute bare minimum to get it done ASAP without breaking anything.". Feature enhancement is rarely considered at...

        If there is anything I know about system upgrades, the answer is always "do the absolute bare minimum to get it done ASAP without breaking anything.". Feature enhancement is rarely considered at these junctures and (as you mentioned) no budget for giving a reasonable workload for IT (including hiring sufficiently competent staff).

        I would wager a good majority of schools are running more or less the same configs that they did 15+ years ago.

        3 votes
        1. [2]
          OBLIVIATER
          Link Parent
          I doubt this because its all cloud based services now, which didn't exist that long ago. Those configurations for GoGuardian would be at most a few years old because that's when they redid their...

          I would wager a good majority of schools are running more or less the same configs that they did 15+ years ago.

          I doubt this because its all cloud based services now, which didn't exist that long ago. Those configurations for GoGuardian would be at most a few years old because that's when they redid their Admin/Teacher filtering suite.

          3 votes
          1. vord
            Link Parent
            While I'll happily hand over my metaphorical gambled money on that, I'll share an ancedote you've reminded me of. When I moved to my current employer, they had just completed a migration off of a...

            While I'll happily hand over my metaphorical gambled money on that, I'll share an ancedote you've reminded me of.

            When I moved to my current employer, they had just completed a migration off of a mainframe into an ERP system backed by an Oracle database. They retrained their entire staff to use SQL and PL/SQL. And the end result was porting the mainframe processes as directly as possible, damn the consequences (and there were a lot). Talking stuff like duplicating data in the ERP data model into 200-wide column tables to perform data manipulation then copy it back into the data model. It ignored the vast majority of the benefits that an ERP migration would bring (using it as the time to reevaluate business processes and rework the data model to fit the ERP system).

            It really was an interesting technical achievement, if accomplished for completely the wrong reasons that they suffer the consequences from even today, over 13 years later.

            To circle back to the reason it reminded me: I agree that you're correct that its not the literal same config. But while I wouldn't double-down on it, I still have a hunch it was literally a case of holding up the current config next to the new config and implementing it as close as possible. Maybe that's just the trauma (sweet job-security trauma) from dealing with this aftermath that makes me cynical in that vein.

            2 votes