53 votes

I used to teach students. Now I catch ChatGPT cheats.

56 comments

  1. [18]
    scarecrw
    Link
    I work with students, but not in a scenario where I'm faced with these challenges directly (I don't focus on written work nor am I grading assignments). I do, however, get to see and hear how they...

    I work with students, but not in a scenario where I'm faced with these challenges directly (I don't focus on written work nor am I grading assignments). I do, however, get to see and hear how they interact with ChatGPT and similar tools and have come away with a couple of insights:

    • Most of them use these tools very poorly. Students are mostly unaware of the strengths and weaknesses of LLMs, and, if the response they get isn't immediately useful, they give up quickly. This paper lays out this difficulty more clearly and formally than I ever could. Ultimately, this makes me very wary of any suggestion that students are going to have success in using these as learning aids, at least not without substantial guidance.

    • Students are very aware of others using these tools to cheat, and it's worsening the already enormous academic pressure many students feel. While I've only had a few students confess to their own usage of these tools, many are eager to complain about their classmates' cheating and how they feel their own work is being devalued as a result.

    I think the only thing in this piece that I see differently than the author, is how large or small of a shift this really is.

    I once believed my students and I were in this together, engaged in a shared intellectual pursuit.

    Really? I don't want to sound overly negative, but come on... Students doing the bare minimum to get a passing grade is not some new phenomenon. I suppose the accessibility of these new tools has laid bare just how many students are willing to take the easier path, but the connection between educational achievement and career success has long undermined the idea that schools could be so pure in their ideals.

    My optimistic vision for the future of student writing is focused on deep investigation of topics and, to put it bluntly, student ego. One sad commonality I've found in student work is an utter lack of pride. It's so beaten into students that the purpose of work is to be graded, that the concept of putting in effort for their own self-value is completely foreign. When I think back to my own struggles with writing as a student, I can't help but think that it's no wonder I couldn't write: there wasn't anything I knew well enough to write about with confidence. I'd love to see students given the opportunity to become experts at something, to the point where writing about it to share their knowledge and opinions would just be the natural step.

    61 votes
    1. cycling_mammoth
      Link Parent
      I know this might be veering off-topic somewhat, but I found one of the best ways to combat this was the promotion of essay competitions and the possibility of having your work be published in a...

      One sad commonality I've found in student work is an utter lack of pride. It's so beaten into students that the purpose of work is to be graded, that the concept of putting in effort for their own self-value is completely foreign.

      I know this might be veering off-topic somewhat, but I found one of the best ways to combat this was the promotion of essay competitions and the possibility of having your work be published in a student journal. I recognize that this perhaps is not useful for secondary school pedagogy, but at a university level this really can encourage students to at least think of their work as more than something to be graded. Likewise, I guess it can still be seen as reductive in that it is now equating a monetary (prize) value to the paper. But I still feel like one takes more pride in writing a course paper if it is also going to be submitted for competitions / publication opportunities.

      16 votes
    2. [15]
      DawnPaladin
      Link Parent
      Agreed with a lot of this. The paper you linked is based on GPT-3, which is very old; newer models need much less prompt engineering. You can just talk to them. I think students who want to figure...

      Agreed with a lot of this. The paper you linked is based on GPT-3, which is very old; newer models need much less prompt engineering. You can just talk to them. I think students who want to figure out how to use these tools to learn instead of using them to cheat will find it straightforward to do so.

      13 votes
      1. [14]
        merry-cherry
        Link Parent
        The new models are easier to use than before but they're no better at producing functional results. The limits of GPTs are starting to show and it's much lower than many thought. So while a...

        The new models are easier to use than before but they're no better at producing functional results. The limits of GPTs are starting to show and it's much lower than many thought. So while a student can more easily produce AI slop than before, the slop is hardly any better.

        13 votes
        1. [12]
          DawnPaladin
          Link Parent
          I disagree. I was studying math yesterday, watching a Khan Academy video about complex numbers. The whole concept of what they were doing seemed completely arbitrary and disconnected from reality....

          I disagree. I was studying math yesterday, watching a Khan Academy video about complex numbers. The whole concept of what they were doing seemed completely arbitrary and disconnected from reality. I asked ChatGPT for background, and it explained the historical context of complex numbers and gave me a bunch of useful information that filled in big gaps in my understanding. I've checked the stuff it gave me; it's real. This is far from the only time this sort of thing has happened.

          You are acting as if LLMs produce nothing but lies and garbage, and that just isn't so. They are not perfect, but they are useful research tools, and ignoring them is folly.

          12 votes
          1. [10]
            merry-cherry
            Link Parent
            Complex numbers the concept is not basic arithmetic. It's perfectly valid to learn of the concept via a teaching source. What would be an issue is if you never handled any math problems involving...

            Complex numbers the concept is not basic arithmetic. It's perfectly valid to learn of the concept via a teaching source. What would be an issue is if you never handled any math problems involving complex numbers yourself and relied on tools to do it for you. You'd never learn how to handle complex numbers as you'd never experience the struggle necessary to learn them.

            10 votes
            1. [6]
              DawnPaladin
              Link Parent
              100% agree! This is the difference between using LLMs as a tool/tutor/assistant vs trying to have it do the entire job for you. Academic settings should be encouraging the former and punishing the...

              100% agree! This is the difference between using LLMs as a tool/tutor/assistant vs trying to have it do the entire job for you. Academic settings should be encouraging the former and punishing the latter, IMO.

              2 votes
              1. [4]
                mordae
                Link Parent
                You can detect gaps in understanding in at most five minutes by just conversing. It's this stupid idea people are "graded" by exactly "how much did they not understand" that's causing most of the...

                You can detect gaps in understanding in at most five minutes by just conversing. It's this stupid idea people are "graded" by exactly "how much did they not understand" that's causing most of the issues in education.

                Also, I fail to see why should anyone be punished for being lazy. Maybe for being dishonest, but lazy? We literally have in the declarations of human rights that compelled work shall be outlawed.

                3 votes
                1. [3]
                  papasquat
                  Link Parent
                  No one in college is compelled to be there. They're adults who are choosing to earn a degree. Failing them isn't punishing them, it's grading their work.

                  No one in college is compelled to be there. They're adults who are choosing to earn a degree. Failing them isn't punishing them, it's grading their work.

                  6 votes
                  1. [2]
                    mordae
                    Link Parent
                    Well, academic setting includes primary and secondary education. It is also debatable whether participation in tertiary education truly is voluntary, since some parents are pretty forceful about...

                    Well, academic setting includes primary and secondary education. It is also debatable whether participation in tertiary education truly is voluntary, since some parents are pretty forceful about it and the labor market also kinda insists.

                    Failing them means that despite having successfully completed e.g. 95% of required courses you still have to repeat the whole thing from scratch. That's punishment, not "grading". Look up Sisyphus.

                    Also, coursework is not actual useful work for oneself or to help out others. It's mostly synthetic busywork to prove stuff. Kinda reminds me of the wasteful proof-of-work blockchain crap.

                    Now I understand that playing with stuff is actually critical for learning, but I have yet to see a student who's having fun engaging with their coursework, because it's stimulating. The whole setting where they are punished for not completing on time doesn't really help.

                    1. papasquat
                      Link Parent
                      Well we're talking about college, so yes, it is voluntary. Even if some parents are forceful about their kids going to college, they're adults who are able to make their own decisions, and they've...

                      Well we're talking about college, so yes, it is voluntary. Even if some parents are forceful about their kids going to college, they're adults who are able to make their own decisions, and they've decided to go to college.

                      A degree has set requirements that are outlined at the beginning. Not meeting those requirements means you don't get the degree. It's not a punishment to not get the degree if you don't meet the requirements, it's just a consequence of not doing the work.

                      Part of the issue with higher education is that it's treated as a requirement for most jobs, and as such, the whole thing is viewed as transactional. You pay your tuition, and you automatically get a degree for showing up. That's not how it's supposed to work.

                      You get a degree as proof to the world that you've learned and absorbed the material. How do you prove to the university that you've learned the material without actually showing it through coursework and exams though?

                      And yeah, it's not always fun doing coursework. It wasn't assigned to be fun. It was designed to help you learn.

                      4 votes
              2. Akir
                Link Parent
                As someone who has just began to work in teaching, I am starting to realize that one of the biggest problems with education is that it’s really hard to get the time to sit down and talk to...

                As someone who has just began to work in teaching, I am starting to realize that one of the biggest problems with education is that it’s really hard to get the time to sit down and talk to individual students long enough to get them fully engaged and fill in their understanding. My classes have been mostly 1-on-1 but even with that, being limited to one hour chunks means that I’ll often need to just give up on teaching a concept before they have good understanding and then next week we waste a lot more time doing “review” which is actually going back and finishing what we were supposed to have already learned, which can take longer than if we had just finished the first time.

                While I’m glad that students have access to something that can help them to fill in those gaps, let’s be honest - most of them probably won’t. They will take the path of least resistance and use it to simply give them the correct answers rather than teaching them how to come to it themselves.

            2. [3]
              mordae
              (edited )
              Link Parent
              I am sorry but this sounds a lot like "I have suffered in my studies so now you have to as well." Complex numbers are not that hard. They're just 2D vectors. At least from the point of practicing...

              I am sorry but this sounds a lot like "I have suffered in my studies so now you have to as well."

              Complex numbers are not that hard. They're just 2D vectors. At least from the point of practicing programmer. The whole i^2 = -1 boils down to sin/cos behaviour.

              Also, I have routinely used tools to simplify algebraic expressions via symbolic manipulation such as SymPy way before LLMs.

              I learned complex math by arguing with Sonnet and writing the code that needed it, after reading PySDR. This is a valid strategy even if you let the LLM do the bulk of the work. The point is to engage with the material and it's kinda hard to start from a blank slate.

              LLMs are currently very close to people who got all As in my high school. Good memorizers with crappy understanding and little creativity outside language.

              1 vote
              1. [2]
                merry-cherry
                Link Parent
                All learning requires some level of struggle to really learn it. No one is ever perfect right out of the gate. They make mistakes or get confused or whatever. I'm not talking about suffering, I'm...

                All learning requires some level of struggle to really learn it. No one is ever perfect right out of the gate. They make mistakes or get confused or whatever. I'm not talking about suffering, I'm talking about the learning process. Someone that uses AI as a crutch when learning something new can severely limit their ability to learn. Using it as a learning tool can be fine, but using it as an answer generator that you then copy means you don't actually learn anything, your just a integration tool between the AI and whatever you're pasting into.

                8 votes
                1. mordae
                  (edited )
                  Link Parent
                  I hear you. Aider can now feed linter and compiler outputs back to the model, so you don't have to copy-paste manually. The actual problem here is that our socioeconomic system is pushing people...

                  I hear you. Aider can now feed linter and compiler outputs back to the model, so you don't have to copy-paste manually.

                  The actual problem here is that our socioeconomic system is pushing people to be more "productive" and to be jacks of all trades. Taking the shortcuts works in the short term. And who knows what's in the future.

                  If you want people to learn, to get deep into stuff, I guess neither academic sector nor their workplace encourage that anyway.

          2. lou
            (edited )
            Link Parent
            I don't think anyone is saying everything AI produces is of poor quality but rather that it often does not produce student essays of sufficient quality. I would expect AI to be remarkably good at...

            I don't think anyone is saying everything AI produces is of poor quality but rather that it often does not produce student essays of sufficient quality. I would expect AI to be remarkably good at explaining fundamental mathematical concepts, but it is perhaps not as good for other subjects. It is certainly not as good when you must combine different concepts to achieve a synthesis between them that is not cliche or entirely trivial. That is what many graded essays require.

            4 votes
        2. sparksbet
          Link Parent
          There are definitely some improvements in quality between, say, GPT-3 and GPT-4, and the specific additions to the training that led to ChatGPT did additionally improve outputs. These improvements...

          There are definitely some improvements in quality between, say, GPT-3 and GPT-4, and the specific additions to the training that led to ChatGPT did additionally improve outputs. These improvements are, however, pretty small in the grand scheme of things and very much incremental improvements in nature -- they aren't game-changers in terms of the fundamental weaknesses of these types of models, which are the things average people using them are most likely to run into and need to learn to avoid.

          I do also think we're probably close to the limit in terms of diminishing returns for this type of model, but I suppose we'll see what happens.

          7 votes
    3. streblo
      Link Parent
      My wife teaches at a middle school level and she's trying to teach this. She has them utilize it to both edit their own writing or synthesize and edit entire reports with a lot of focus on...

      Most of them use these tools very poorly. Students are mostly unaware of the strengths and weaknesses of LLMs, and, if the response they get isn't immediately useful, they give up quickly. This paper lays out this difficulty more clearly and formally than I ever could. Ultimately, this makes me very wary of any suggestion that students are going to have success in using these as learning aids, at least not without substantial guidance.

      My wife teaches at a middle school level and she's trying to teach this. She has them utilize it to both edit their own writing or synthesize and edit entire reports with a lot of focus on iteration and fact checking. But she's just one teacher and this is her own initiative, not something in the curriculum yet or even widely embraced by her peers.

      I think there's an (understandable) distaste associated with AI; but I hope that doesn't prevent governments/schools from incorporating it into their curriculum in the near future. It's a fact of life that these things exist and they can be useful given the right environment.

      7 votes
  2. [10]
    hobblyhoy
    Link
    I'm disappointed by these takes and hoped we'd be seeing better adaptation by now. This idea that we're locked into the options of let 'em cheat or pen and paper essays during class time is just...

    I'm disappointed by these takes and hoped we'd be seeing better adaptation by now. This idea that we're locked into the options of let 'em cheat or pen and paper essays during class time is just so unimaginative. How about teaching them how it works so they can understand the pitfalls and power of it themselves? How about gearing some work around proper outlining and the structural planning rather than the finished product? They will sit there and proclaim "oh okay sure but when you get the real world with such and such situation chatGPT won't help then" ...then design your assignments around that instead! It's like the whole teaching institution has been so stuck with 5 paragraph papers for so long they don't know how to do anything else.

    It doesn't have to be this way. If we didn't go in with such a defeatist attitude ChatGPT could be the impetus for revitalizing and reimagining education as we know it. In fact I think it's inevitable, I'm just uncertain how long it will take us to get there.

    22 votes
    1. [6]
      Shevanel
      Link Parent
      I don’t disagree with what you’re saying, but as a former teacher, I feel comfortable saying that, at least in the US, everyone in education is stretched so incredibly thinly as it is, that to get...

      I don’t disagree with what you’re saying, but as a former teacher, I feel comfortable saying that, at least in the US, everyone in education is stretched so incredibly thinly as it is, that to get buy-in on a revamped curriculum that involves embracing an incredibly quickly-shifting technological advancement (especially compared to the glacial pace of academia) is a lost cause. Most teachers don’t have the time or the funding they need to effectively teach the subjects they know like the back of their hand—it’s not feasible to skill up the entire workforce on what is essentially (at least in the eyes of academia) entirely greenfield tech.

      19 votes
      1. [5]
        kfwyre
        Link Parent
        Couldn’t agree more as a current teacher. Add to this the fact that it’s happening on the heels of COVID, which already massively upended education and accelerated teacher burnout. Also, schools...

        Couldn’t agree more as a current teacher.

        Add to this the fact that it’s happening on the heels of COVID, which already massively upended education and accelerated teacher burnout. Also, schools and teachers also became targets in political culture wars.

        15 votes
        1. [4]
          Shevanel
          Link Parent
          You are a saint among saints, @kfwyre! Thank you for doing what you do. I happened to leave education in 2020 (just very coincidental timing, my departure had been planned since pre-COVID), and I...

          You are a saint among saints, @kfwyre! Thank you for doing what you do.

          I happened to leave education in 2020 (just very coincidental timing, my departure had been planned since pre-COVID), and I can't imagine what it must be like now. I hate that conversations around tech in teaching, especially pushback against AI, always ends up painting the classroom teacher like a fuddy-duddy who hates tech and refuses to adjust. In my experience, teachers want what's best for students. Full stop. If that means that, in the long run, curriculum is slowly adapted to embrace this new path of learning, and it's demonstrable that this is what's best for kids, I guarantee that the majority of teachers will be behind it (even if the older generation of them drags their heels a bit!). Y'know, if it wasn't for the insurmountable budgetary/political/cultural issues holding us back from doing much of anything. Too many teachers need to literally justify their very jobs year after year to be able to teach their current curriculum effectively, let alone make advancements in it.

          To bring to mind the average "tsk tsk"-stuck-in-their-ways old teacher as some sort of obstinate obstacle to the evolution of learning in these conversations is counterproductive. If everyone needs a scapegoat so badly, let's look at who's funding (or more appropriately, not funding) education departments. And if we really need to drive it home to one individual, it would be tough to find a better one than George W. Bush. No Child Left Behind was a well-intentioned effort that has done more damage to the educational environment than any other "advancement" in education since, well, probably the advent of factory-model schools that we've used for the last 100 years, but that ship has long sailed unless we burn the whole thing to the ground.

          7 votes
          1. [3]
            kfwyre
            Link Parent
            I appreciate the kind words. I'm not a saint, I'm just someone who likes getting two months off each year! 😂 Out of curiosity, what did you end up moving to, career-wise? I'm always curious about...

            I appreciate the kind words. I'm not a saint, I'm just someone who likes getting two months off each year! 😂

            Out of curiosity, what did you end up moving to, career-wise? I'm always curious about where people who leave teaching end up. (Also if you're not wanting to share this information, don't feel obligated. I want to respect your privacy as well!)

            4 votes
            1. [2]
              Shevanel
              Link Parent
              lol, just trying to feed the flames with the “two months off” comment, I see! No sweat; I’ve discussed it before on Tildes, and given the musical side of my prior life, I already have a decent...

              lol, just trying to feed the flames with the “two months off” comment, I see!

              No sweat; I’ve discussed it before on Tildes, and given the musical side of my prior life, I already have a decent presence on the Internet, whether I like it or not. I’m now a software engineer. A pretty far cry from teaching, but I feel like both jobs engage the problem-solving part of my brain (in very different ways), so I enjoy(ed) them both in their own ways. I also lead a small team now, so that coaching / human side of the role is still intact, which is very important to me.

              You may have mentioned it previously on Tildes, but mind sharing what subject you teach? I’ll break the ice; I was a high school band director.

              1 vote
              1. kfwyre
                Link Parent
                I know this is unsatisfying, but I actually don't mention what I teach. I'm in a somewhat unique position, which makes sharing the specific details about my position somewhat identifying. I'm fine...

                I know this is unsatisfying, but I actually don't mention what I teach. I'm in a somewhat unique position, which makes sharing the specific details about my position somewhat identifying. I'm fine with sharing that I teach in a secondary setting, but nothing more than that. I leave the specific subject(s) as an exercise to my readers!

                Also it's awesome that you were a band director! I was a band kid in high school myself, so I have bigtime affinity for anyone in that role.

                I'm also glad that it sounds like you've settled well into a new position that still works with your strengths.

                3 votes
    2. teaearlgraycold
      Link Parent
      Exactly. Same for programming. But going back to the first principles of computer science pedagogy and redesigning the curriculum is a lot of work. And in the process you'll need to figure out why...

      then design your assignments around that instead!

      Exactly. Same for programming. But going back to the first principles of computer science pedagogy and redesigning the curriculum is a lot of work. And in the process you'll need to figure out why what we've been doing worked in the first place. So it could take years for a successful approach to emerge.

      18 votes
    3. [2]
      ThrowdoBaggins
      Link Parent
      It seriously has echoes of my school teachers telling me “you won’t just carry a calculator around in your pockets” — showing my age a bit here but the iPhone came out within a few years of...

      It seriously has echoes of my school teachers telling me “you won’t just carry a calculator around in your pockets” — showing my age a bit here but the iPhone came out within a few years of finishing school and suddenly, actually, everyone was carrying a calculator around in their pockets!

      I’m glad that even before the advent of the smartphone, some of my final years at school had math teachers who would give us calculators and open book tests from day one, and structure the course and tests around knowing we could just have the answers in front of us. Marks suddenly became a lot less about the final result, and a lot more about showing the steps to get there, and I think I learned the core concepts of maths deeper than I otherwise would have if our tests were designed to be unassisted

      15 votes
      1. merry-cherry
        Link Parent
        That works for things that have a process but not for work that is produced in few steps. Basic arithmetic learning is not aided by calculators and that's just as true now as it was then. Unless...

        That works for things that have a process but not for work that is produced in few steps. Basic arithmetic learning is not aided by calculators and that's just as true now as it was then. Unless you're going to force every student to draw 100 circles to prove how they got 10 x 10, you're going to need to accept raw answers.

        It's similar with writing. These students need to learn the process of taking ideas and shaping them into thoughts and words. AI is strictly a hindrance to that process at this stage. It bypasses the learning process meaning these students fail to learn the fundamentals of thought. They can have an AI that baby steps them through but they'll be forever shackled to that AI for the rest of their lives for even the most basic tasks. It's not much different than calculator kids that struggle to add basic numbers long into adulthood because they never internalized arithmetic.

        There's loads of room for AI and calculators once students are past the fundamentals but you still have to get the fundamentals first without those tools. I can't think of a worse future than one beholden to mega tech stocks for basic functions.

        18 votes
  3. [2]
    TonesTones
    Link
    I'm surprised by the number of responses arguing that this is a failure of education. Many commenters here seem to think that the professors need to adapt their curriculum to better fit the age of...

    I'm surprised by the number of responses arguing that this is a failure of education. Many commenters here seem to think that the professors need to adapt their curriculum to better fit the age of AI and to start testing things that AI cannot do.

    The comparison to "you won't always carry around a calculator in your pocket" from @ThrowdoBaggins is poignant, but also important in where it falls short. We still need to teach basic arithmetic to children so they have some intellectual leverage with which to grasp the stronger concepts. We still need to teach basic reading and writing skills to students so they can learn higher-order skills. You cannot expect someone to become an architect of software systems without first learning how to program.

    AI is fundamentally different than previous in that in can do most introductory-level work at a level that most students can't; in other words, LLMs are smarter than most students. If students learn the LLM-first workflow, they literally will not have the ability to adapt when LLMs fall short.
    Part of my educational research was in designing programing assignments that LLMs struggled with that were still accessible to introductory programmers, and we came up with lots of introductory coding assignments that all 2023 LLMs consistently struggled with, even with good prompt engineering. Then GPT 4o came out and one-shot all of them.

    In my mind, there are two reasonable conclusions from the existence of systems with increasing intelligence that can complete student assignments. Either (A) humans cannot use LLMs to do higher-order tasks at work, in which case LLMs are actively harmful towards education, or (B) humans can use LLMs to do those higher-order tasks, in which case we are rapidly moving towards a time when most humans trained in white-collar language-based work are unemployable (since what's the human middleman needed for?).
    My perspective is that (B) is what will hold given to what extraordinary extent AI researchers are using AI to make novel developments (and that we've only just begun using RL to actually improve LLMs). If that's the case, then we really will need to rethink education.

    9 votes
    1. mordae
      (edited )
      Link Parent
      We knew from the outset that technology replaces human labour. That was the idea. That was why we invested in it. To avoid labour. Once sufficient portion of labour is automated, capitalism will...

      We knew from the outset that technology replaces human labour. That was the idea. That was why we invested in it. To avoid labour.

      Once sufficient portion of labour is automated, capitalism will collapse. How depends on what we do:

      1. People will agree to redistribute the products roughly equitably and then we'll see.
      2. People will let the society regress to feudalism where they are not fundamentally necessary and thus have zero leverage for their continued existence.

      Currently we are on track for (2).

      I hope we choose (1), have two decade breather and then slowly get back to the fact that some people still wish to impose their ideas on others with force and work on that for a while.

      2 votes
  4. [11]
    cycling_mammoth
    Link
    As someone who is pursuing a career in education—what options do educators even have that preserve writing as an important skill? Depriving students of the experience of researching, creating an...

    As someone who is pursuing a career in education—what options do educators even have that preserve writing as an important skill? Depriving students of the experience of researching, creating an argument, and developing said argument into a concrete and well thought paper feels wrong. Is this more so a question of institutions providing the necessary tools to students? For example, larger libraries with computers that have access to academic databases, a word processor, and nothing else. A sort of "semi-closed-book" environment to prevent AI usage. But even then, policing AI usage like that just feels frankly wrong and unfair to students who would not partake in such behaviour.

    7 votes
    1. [3]
      mieum
      Link Parent
      In my courses, the assignments a designed in a way that would make it hard to outright delegate all or most of the work to AI. In my philosophy of education course, for example, students write a...

      In my courses, the assignments a designed in a way that would make it hard to outright delegate all or most of the work to AI. In my philosophy of education course, for example, students write a “manifesto” about their views on education. It begins with a simple, brief statement or outline of their thoughts at the beginning of the course. Then I engage each of them in a regular, asynchronous dialog throughout the semester in which I try to get them to reflect on their views in the context of things discussed in class. They then revise this twice, during midterms and finals.

      It isn’t bulletproof, and not applicable to all subjects, but I have had a lot of success with it. And it is easy to catch cheaters with this assignment.

      The other thing I recommend is having students evaluate themselves. I find that giving them this power reduces the incentive to delegate their thinking to AI. Having to quantitatively and qualitatively evaluate themselves also makes evaluation a part of their experience in the class, rather than just a fateful judgement the professor makes.

      Edit: Furthermore, adding an oral evaluation to a written assignment is another barrier to AI-based cheating. That way they will have to discuss their paper with you even if AI wrote it or provided them with all of the ideas. Finally, making them “show their work” when researching and writing may also help.

      17 votes
      1. [2]
        cycling_mammoth
        Link Parent
        I have found the same in my literature courses, at this point it is frankly difficult to delegate most of the work to AI, as it cannot feasibly integrate enough textual evidence or valid secondary...

        In my courses, the assignments a designed in a way that would make it hard to outright delegate all or most of the work to AI.

        I have found the same in my literature courses, at this point it is frankly difficult to delegate most of the work to AI, as it cannot feasibly integrate enough textual evidence or valid secondary criticism to support an argument. I guess part of my “fear” is how we go forward as large language models become more capable in this regard.

        Then I engage each of them in a regular, asynchronous dialog throughout the semester in which I try to get them to reflect on their views in the context of things discussed in class. They then revise this twice, during midterms and finals.

        Furthermore, I do think this is a good method, however, as it ensures there is some form of continuity based on prior work in the course. I have seen plenty of educators create new assessments to combat the use of AI, but lack the continuity that you have achieved with this.

        Finally, making them “show their work” when researching and writing may also help.

        To what extent should they show their work? I have seen some variations of this throughout my university life, and I have found it quite helpful, but as a future educator I do not want to have “show your work” steps that just seem like a means to 'prevent procrastination'. Ideally, I would want to have steps that actually contribute to the development of their argumentative skills and writing capability.

        P.S. Sorry if some of my thoughts are not perfectly clear, I am writing this a little late, but I really wanted to respond to some of the lovely feedback here before I forget

        3 votes
        1. Minori
          Link Parent
          It doesn't definitively prove students are doing their own thinking, but it's increasingly common for professors to ask for the edit history for a file to prove the student wrote and edited it. Of...

          Ideally, I would want to have steps that actually contribute to the development of their argumentative skills and writing capability.

          It doesn't definitively prove students are doing their own thinking, but it's increasingly common for professors to ask for the edit history for a file to prove the student wrote and edited it. Of course students could still copy by typing out a response from an LLM, but that forces at least a little more thinking than copying and pasting.

          5 votes
    2. [3]
      DawnPaladin
      Link Parent
      There are two questions here: How do we use essays as a tool to help students learn? How do we evaluate how much students have learned? I think it will help to separate these two. In other words,...

      There are two questions here:

      1. How do we use essays as a tool to help students learn?
      2. How do we evaluate how much students have learned?

      I think it will help to separate these two. In other words, the percentage of a student's grade that comes from evaluating their essays needs to go way down. Getting AI to write your essay is easy to do and hard to catch. It doesn't make sense to write rules you can't enforce and which you know won't be followed.

      The students in the article have a point. Why is the professor constraining which tools students are allowed to use for research? Once they graduate, odds are good they will be using AI as part of their workflow. Shouldn't they be learning effective use of these research tools during university?

      The professor still has to assign grades, though. Here are some ideas for doing that effectively:

      • Quizzes, tests, and final exams take place in class. These can take any form you like. If you want essays on the tests, hand out blue books as in days of old. If student handwriting isn't up to the task, use non-networked computers. Or electric typewriters. Whatever allows students to produce an essay without the possibility of outside help.
      • More emphasis on discussions in class. Ask the students questions about what they've written. Class participation becomes a bigger part of the grade.
      9 votes
      1. [2]
        cycling_mammoth
        Link Parent
        I do think this is important. Some of the most influential classes on my intellectual development within my university career were seminar classes — predominately focussing on individual seminar...

        More emphasis on discussions in class. Ask the students questions about what they've written. Class participation becomes a bigger part of the grade.

        I do think this is important. Some of the most influential classes on my intellectual development within my university career were seminar classes — predominately focussing on individual seminar presentations that led to group class discussion of the material afterwards.

        Regarding your other points, I think I mostly agree with you. Evaluating what students have learned can easily be shifted to more in-class assessments, and is becoming pedagogically necessary. I guess my concern more so stems from the critical thinking and reading skills that library / online database research provides, as well as the argumentative processes involved in creating an argumentative essay. If we really have to shift away from traditional essay writing, how do educators preserve the skills developed from writing and research?

        Shouldn't they be learning effective use of these research tools during university?

        I actually agree with you here as well, but sadly academia is a slow moving institution. I actually could see these tools becoming used during university — but they would need access to the corpus of scholarly articles that universities subscribe to, with proper citations for where it is getting information from. As it stands, the main use of AI that seems undeniably effective in a university context is grammar editors that integrate AI for reformulating sentences (removing passive voice, change of register, or improving your second language grammar).

        P.S. As I stated in another reply, it is a little late, so I apologize if my thoughts are not 100% cohesive. There are just so many great responses to this post, and I really wanted to get to some of them before I forget. Also, I am quite aware that I am a Luddite when it comes to AI, I do want to emphasize that I am genuinely grateful for comments like yours that are challenging my perspective.

        6 votes
        1. DawnPaladin
          Link Parent
          It is lovely to talk with someone who's engaging this topic with curiosity! I am not an educator, so I apologize in advance for any misunderstandings of academia I commit here. I would not want...

          It is lovely to talk with someone who's engaging this topic with curiosity! I am not an educator, so I apologize in advance for any misunderstandings of academia I commit here.

          I guess my concern more so stems from the critical thinking and reading skills that library / online database research provides, as well as the argumentative processes involved in creating an argumentative essay. If we really have to shift away from traditional essay writing, how do educators preserve the skills developed from writing and research?

          I would not want people to stop using libraries and online research to write essays. I don't want essays to go away. We just have to figure out some new techniques for grading. Handing in the essay for the professor to grade in isolation isn't going to work any more.

          I think your idea to do seminars is a really good one—those are exactly the same lines I'm thinking along. So how do we incorporate essay writing into that plan? Maybe something like this:

          1. Teacher assigns a seminar. Students decide on a research topic.
          2. As prep for the seminar, over the course of several weeks, each student does a bunch of research. Every week or two, they summarize their current research in an essay. These essays are easy to auto-generate, so they're not worth much of your grade. Mainly they serve as a series of checkpoints—they give the student a chance to discuss their research progress with the teacher. They provide those prone to procrastination with intermediate deadlines so they don't wind up doing the entire research project over 48 hours.
          3. Students present their seminar. The teacher and class ask them questions to test the depth of their knowledge. This is the bulk of their grade.

          Another idea would be to have students write essays arguing one side or another of various topics, worth a small amount of grade. Then after X rounds of back-and-forth, host a live debate in class. Grade students based on how well they know their own arguments and the arguments of their opponents.

          Do you think either of these would work? The general pattern I'm going for here is that essays are easy to cheat on, we know this and accept it, and the rubric is designed with this in mind. The essays are not assigned for their own sake—they are stairsteps to get you over a wall that's looming at the end of the semester. Students who want to learn and who engage with the material will finish the semester with a big, cool project they can show off. Students who are checked out and coasting will hit the wall and fail; hopefully they'll learn from the experience and do better next time.

          2 votes
    3. [4]
      Turtle42
      Link Parent
      We will need to restructure our entire existing notion of what education is. LLMs need to be a part of the learning process and we need to reconsider the kind of homework we assign to kids, and...

      We will need to restructure our entire existing notion of what education is. LLMs need to be a part of the learning process and we need to reconsider the kind of homework we assign to kids, and accept that they're going to use them. Study after study proves homework doesn't help kids learn, why do we keep assigning it?

      Most public education (in my jurisdiction) teaches kids to simply pass standardized tests, hence the busy work like math problems and book reports. Most kids don't view the forming of a thesis, creating an argument, research, etc. fun the way you or I might about a particular topic we're interested in. I think that's the key. It's busy work to most of them. The passion for the subject needs to be instilled natively, somehow.

      I've heard of something called the 4 C's of education: critical thinking, communication, collaboration, and creativity. And from the way I feel like as a full grown adult, free without the confines of school and subjects I hated and homework I hated even more, I would have benefited immensely from an education with that kind of focus over the things they did focus on like algebra. I'm now free to do whatever I want, and it turns out I love computers and Linux. I've been pouring over the most boring documentation literature just to soak up everything as much as possible. I regret not learning this stuff sooner.

      I found immense passion in having the freedom to read or write what I wanted whenever we had the opportunity to do so. Of course kids need to be taught a general foundation of knowledge, but alongside that knowledge needs to be a focus on the 4 Cs so then from a younger age than we probably think, need to let them roam free at school and tinker with literally everything we can provide them so interest can be sparked in something, where they can then use that knowledge to teach themselves where they will inevitably fail and have to try again, which is the most beneficial way I have found to learn as an adult. By messing with something myself, not from being lectured to. Which was definitely something that was missing from my own remarkably average public school experience.

      We need to start treating an LLMs as a tool, kind of like a language calculator. And yes, I'm aware that models can hallucinate whereas a calculator by it's very nature does not. But they can also accomplish the kind of tasks incredibly well and efficiently when prompted intelligently and provided adequate context. Resistance is futile at this point.

      I use a calculator to do basic arithmetic. We all do. I could do it in my head, but sometimes I just want to be sure. I kind of use Claude the same way, and I'm tired of being ashamed of it. I'm a photographer and enter art shows, and have to write artist statements. I like writing in general, but for some reason I hate writing artist statements. They're often dripping with ick and super egotistical and often meaningless, just using art buzzwords about identity and juxtaposition and I just hate it. I can show Claude a couple of pictures, give it my thoughts and feelings about the work in casual language while telling him a little of my own meaning. Then it does some AI magic with it's training content of literally everything ever created ever, and then it spits out something better than I ever could have wrote myself. Why is this a bad thing? We can pawn off this kind of busy rote work to a computer, let's focus on real stuff now.

      Claude AI has taught me more in two months about literally whatever I'm working on than I could have ever taught myself alone, or even had access to previously. I cannot stress how average I am as a human. Someone much smarter than me with enough gumption could do great things if they were taught how to use this incredible tool correctly. We can't ignore this technology.

      5 votes
      1. gary
        Link Parent
        This is frankly a dangerous idea. The most charitable I can be here is that some studies show a lack of positive effect and some studies show a positive effect. There are meta analyses that show a...

        Study after study proves homework doesn't help kids learn, why do we keep assigning it?

        This is frankly a dangerous idea. The most charitable I can be here is that some studies show a lack of positive effect and some studies show a positive effect. There are meta analyses that show a generally positive correlation. And what about at grade level? I've seen doubt that homework is effective at the primary school level (I still push back on this), but that it's more effective for high school aged kids. It is too widely scoped of a statement to say so definitively.

        4 votes
      2. [2]
        ThrowdoBaggins
        Link Parent
        I’ll very slightly disagree on a technicality, which is that calculators have limits when it comes to how many decimal places it can store at a time. I know this from first hand experience because...

        And yes, I'm aware that models can hallucinate whereas a calculator by it's very nature does not.

        I’ll very slightly disagree on a technicality, which is that calculators have limits when it comes to how many decimal places it can store at a time.

        I know this from first hand experience because when I was bored in math class, I would occupy myself by trying to memorise the digits of pi, by playing around with the numbers stored in the calculator. But eventually I hit a wall because while the calculator screen could show something like 10 digits, it stored the value of pi accurate to maybe 14 digits, and when I tinkered with it to force it to display more and more trailing digits, I ran out of new digits to learn!

        In my final year of high school, I did a more advanced math class and got to use a more fancy graphing calculator that I think stored 20 or so digits, but I only memorised like 17 or 18 of them.

        Anyway, tangent aside, while a calculator won’t hallucinate the way an LLM might, it certainly can lose precision as you delve deeper, especially with fractions and irrational numbers that don’t play nice with decimal (or sometimes binary) counting

        2 votes
        1. wowbagger
          Link Parent
          If anything this just highlights the difference, IMO. A calculator's limitations are knowable and you can compensate for them. An LLM is the quintessential black box, there is no way to predict...

          If anything this just highlights the difference, IMO. A calculator's limitations are knowable and you can compensate for them. An LLM is the quintessential black box, there is no way to predict the mistakes it will make.

          1 vote
  5. [5]
    skybrian
    Link
    One possibility might be to use oral exams more. Maybe students could be asked questions about their paper after they turn it in, which would at least ensure that they've read the paper and...

    One possibility might be to use oral exams more. Maybe students could be asked questions about their paper after they turn it in, which would at least ensure that they've read the paper and understand the ideas in it. The trouble is that this sort of one-on-one teaching scales poorly.

    It turns out that if there is anything more implausible than the idea that they might need to write as part of their jobs, it is the idea that they might have to write, or want to write, in some part of their lives other than their jobs. Or, more generally, the idea that education might be valuable not because it gets you a bigger paycheque but because, in a fundamental way, it gives you access to a more rewarding life.

    A problem with the argument against careerism is that students are sometimes going hundreds of thousands of dollars into debt. It's irresponsible to do this without having a plan to pay it back, so those students, at least, can and should be concerned that this enormous bet will pay off for them.

    So I think it's important to acknowledge that it's a valid concern. But that doesn't mean that students should be solely concerned about how their study will prepare them for their careers. Study can serve multiple purposes.

    7 votes
    1. [4]
      C-Cab
      (edited )
      Link Parent
      As a college instructor, I love the idea of using oral exams to evaluate understanding. It's obviously stressful for the student, but I'd say it's gold standard of assessment. But as you...

      As a college instructor, I love the idea of using oral exams to evaluate understanding. It's obviously stressful for the student, but I'd say it's gold standard of assessment.

      But as you rightfully point out, there is no way I could do that for my class of 50 let alone my two classes of 120 each. One of my graduate courses had oral exams, and we had to schedule time with the professor. I think we had 30 minutes to discuss randomly selected topics, which means he spent at least 5 hours, which he did twice during the semester. We also were scrambling for the times that worked for ours and his schedule - imagine trying to make that work for 50!

      I think I might start moving to group presentations. I'm generally not a fan of them, but public speaking is a good skill to develop. And even if students use LLMs in organizing their thoughts, writing a script, or whatever they still would have to engage with the material in some way in order to communicate it.

      8 votes
      1. [3]
        updawg
        Link Parent
        I agree (it sounds especially hard for 3.04x10^64 students) but, to add a counterpoint of unknown value, over 70% of all French high school students have to take an oral exam to graduate. So they...

        We also were scrambling for the times that worked for ours and his schedule - imagine trying to make that work for 50!

        I agree (it sounds especially hard for 3.04x10^64 students) but, to add a counterpoint of unknown value, over 70% of all French high school students have to take an oral exam to graduate. So they obviously somehow make it work.

        1 vote
        1. mordae
          Link Parent
          Czechia as well. 60 minutes per student in 4 sessions for different subjects usually. They could do it in half the time for pass/fail, but someone wants them to add an extra bit of precision and...

          Czechia as well. 60 minutes per student in 4 sessions for different subjects usually. They could do it in half the time for pass/fail, but someone wants them to add an extra bit of precision and make the grade 1/2/3/F instead of just 1/F. Especially silly given students answer single randomly selected question, which dominates the measurement outcome.

          2 votes
        2. C-Cab
          Link Parent
          I'm not saying it's an impossible task, but I think it would take change at the institutional level. As an individual instructor, I could cut out time from my curriculum and dedicate it to oral...

          I'm not saying it's an impossible task, but I think it would take change at the institutional level. As an individual instructor, I could cut out time from my curriculum and dedicate it to oral exams, but that means less information that is taught and I've already cut things that I think are relevant to the subject.

          1 vote
  6. [2]
    gt24
    Link
    One paragraph from the article seems like something AI may have trouble with. I don't think AI would do well creating a first draft, refining it based off of feedback in a realistic manner, and...

    One paragraph from the article seems like something AI may have trouble with.

    I have said this sort of thing to students from time to time, in explaining why most of their grade—all of it, in some cases—is based on such long-term writing assignments. Such assignments have been the most powerful teaching tool in my arsenal, particularly when they are developed in stages: a first draft followed by feedback, followed by a revision, and so on. Assuming that the student puts in the effort, the paper inevitably gets better through such a process.

    I don't think AI would do well creating a first draft, refining it based off of feedback in a realistic manner, and then iterating each time. I would imagine that the "wheels will fall off the bus" at some point as the AI is typically expected to give the answer as opposed to an emulation of a iterative writing process.

    While iteration can be a difficult thing to embrace for students trying to cram completing an assignment in mere hours before it is due, it may be beneficial to emphasize that iteration more since it is where AI could have a decent amount of difficulty (so I hypothesize, I haven't tested this myself).

    One last thing that may help is an introduction of a physically printed source that AI would know nothing about such as an essay provided by the instructor (or a larger body of work). Since it is unique to the professor, AI will hopefully have not seen it to refer to. The AI would need to know of it first requiring effort from a lazy student to enter it (either by typing or some sort of OCR). Still, I would think that the AI would tend to not focus on the newly entered work with a ton of weight, will not cite it correctly, and will just be given a more difficult time figuring things out. When the student's work is submitted, they can then be asked to write a quick opinion on that source since they should have reviewed it when writing their work.

    Basically, AI doesn't know everything but that is a claim without proof... so perhaps it helps to introduce something that the AI doesn't know about to prove that point.

    6 votes
    1. Minori
      Link Parent
      They're very very good at this actually! If someone knows how to chain prompts, it's easy to get some iterative results until the model inevitably breaks down from having too many characters input...

      I don't think AI would do well creating a first draft, refining it based off of feedback in a realistic manner, and then iterating each time. I would imagine that the "wheels will fall off the bus" at some point as the AI is typically expected to give the answer as opposed to an emulation of a iterative writing process

      They're very very good at this actually! If someone knows how to chain prompts, it's easy to get some iterative results until the model inevitably breaks down from having too many characters input at once. Basically you give the LLM the text you want improved as part of the prompt. They're very capable of quoting text from the prompt and touching it up.

      One last thing that may help is an introduction of a physically printed source that AI would know nothing about such as an essay provided by the instructor (or a larger body of work). Since it is unique to the professor, AI will hopefully have not seen it to refer to.

      I like this unique approach. I had a history professor in college do something similar way before LLMs got big. We had to write several essays over topics in world history, but he strictly required students to make arguments rooted in the textbook!

      If a student tried writing a generic essay on "what motivated the Spanish Inquisitions" with arguments not supported by the textbook he'd assume the student cheated and give them a zero. It was also a great way to force students to make arguments based on a source (the textbook) rather than coming up with arguments then going to find evidence for them.

      5 votes
  7. [8]
    mordae
    (edited )
    Link
    OP's moral grandstanding is as hollow as the credentials his students are chasing. He’s part of a system that prioritizes credentials over genuine learning, and he’s now grappling with the...

    OP's moral grandstanding is as hollow as the credentials his students are chasing. He’s part of a system that prioritizes credentials over genuine learning, and he’s now grappling with the consequences of that system. While he laments the rise of AI-generated essays and the erosion of intellectual curiosity, he fails to acknowledge that the educational system he’s a part of has long been complicit in fostering this environment. Grades, degrees, and job prospects have become the primary focus of higher education, often at the expense of fostering critical thinking, creativity, and a true love of learning.

    The author’s frustration is understandable, but it’s also somewhat hypocritical. He’s part of a system that has, for decades, treated education as a transactional process—students pay for degrees, and institutions provide them. The rise of AI is merely exposing the cracks in this system. Students are using AI not because they’re inherently lazy or disinterested, but because they’ve been conditioned to see education as a means to an end—a degree that will get them a job, not as an opportunity to grow intellectually or morally.

    The author’s moralizing about the value of education rings hollow because it doesn’t address the systemic issues that have led to this point. Standardized testing, grade inflation, and the commodification of education have all contributed to a culture where the end goal is the credential, not the learning. Students are simply responding to the incentives that the system has created. If the system rewards shortcuts, why wouldn’t students take them?

    Moreover, the author’s insistence on handwritten, in-class essays as a solution is both impractical and outdated. It’s a Band-Aid fix that doesn’t address the root problem: the disconnect between what education is supposed to be and what it has become. Instead of clinging to traditional methods, educators need to rethink how they assess learning in an age where AI is ubiquitous. This might mean embracing new forms of assessment, such as oral exams, project-based learning, or collaborative assignments, that are harder to automate and more reflective of real-world skills.

    In the end, the author’s lament is less about the students and more about the failure of the educational system to adapt to a changing world. AI isn’t the problem—it’s a symptom of a deeper issue. Until educators and institutions confront the systemic flaws that have led to this point, the cycle of cheating, policing, and moral grandstanding will continue. The real question isn’t how to stop students from using AI, but how to create an educational system that values learning over credentials and prepares students for a future where critical thinking and creativity are more important than ever.

    And while the author emphasizes the importance of ethics, particularly in fields like healthcare, he overlooks the reality that law and liability often take precedence in professional decision-making. In practice, many professionals—especially in high-stakes fields like medicine—are conditioned to prioritize legal compliance over ethical idealism. The author’s focus on ethical education is commendable, but it risks being undermined by a system that ultimately rewards legal caution over moral courage.

    -- Sonnet 3.5

    7 votes
    1. teaearlgraycold
      Link Parent
      Regardless of content quality I don’t want to engage with LLM content posted under someone’s username.

      Regardless of content quality I don’t want to engage with LLM content posted under someone’s username.

      11 votes
    2. [2]
      chris-evelyn
      Link Parent
      How much of that reply did you write, how much was Sonnet?

      How much of that reply did you write, how much was Sonnet?

      8 votes
      1. mordae
        Link Parent
        I have originally wrote up the counterpoints myself and had Sonnet tidy it up, but then I've just tried feeding it it's own opening "OP's moral grandstanding is as hollow as the credentials his...

        I have originally wrote up the counterpoints myself and had Sonnet tidy it up, but then I've just tried feeding it it's own opening "OP's moral grandstanding is as hollow as the credentials his students are chasing. He’s part of a system that..." and asked it to finish writing the commentary alone. It gave me most of the text. Finally I've asked it to mention that it's mostly liability and law that dictates what healthcare professionals will do and it revised and wrote the last paragraph. I think I've asked it not to write that confidently.

        :shrug:

        2 votes
    3. sparksbet
      Link Parent
      Honestly, I can see handwritten, in-class essays being the solution under some circumstances for, say, high schoolers. But such essays really only work when they're on topics that anyone can have...

      Moreover, the author’s insistence on handwritten, in-class essays as a solution is both impractical and outdated. It’s a Band-Aid fix that doesn’t address the root problem: the disconnect between what education is supposed to be and what it has become.

      Honestly, I can see handwritten, in-class essays being the solution under some circumstances for, say, high schoolers. But such essays really only work when they're on topics that anyone can have an opinion on regardless of research, which is rarely the case for anything worth writing a high school/college-style essay about even in high school. Even when the right topic exists, it doesn't assess most of the actual writing skills that a student needs to learn, and moreover it can reinforce bad habits (like not editing one's work). There's a reason most universities when I was applying in the mid-2010s did not even ask for your scores on the essay-writing portions of the SAT and ACT, which were administered this way. Even compared to standardized testing, in-class handwritten essays aren't a great assessment tool even of a student's ability to write an essay.

      For college students, I think the idea goes from kinda bad to straight-up ridiculous. None of the courses in which I wrote essays in my pre-ChatGPT university career were the types of courses in which handwritten, in-class essays would remotely be an effective way to teach or measure the educational goals of the class. Even a class that is solely dedicated to teaching how to write an essay is going to need to teach students how to research, outline, and edit an essay -- none of which can be effectively done if you limit your material to in-class essays.

      7 votes
    4. oliak
      Link Parent
      I just wanted to say this and so much more. Yes. I almost gagged reading all the moralizing and the lack of insight into the actual problems our society has foisted upon these students. It's...

      I just wanted to say this and so much more. Yes. I almost gagged reading all the moralizing and the lack of insight into the actual problems our society has foisted upon these students. It's myopic and disgusting.

      5 votes
    5. [2]
      hobblyhoy
      Link Parent
      That opening line is pretty slick for an LLM. That threw me off the scent long enough I didn't suss out it was generated until paragraph three. Do you have any particular prompting you use to give...

      That opening line is pretty slick for an LLM. That threw me off the scent long enough I didn't suss out it was generated until paragraph three. Do you have any particular prompting you use to give it this voice?

      3 votes
      1. mordae
        Link Parent
        I am not sure. These models are always hit or miss.

        I am not sure. These models are always hit or miss.

        1 vote