Pretty interesting. The first part is a collection of interviewed viewpoints from students as well as educators; pretty standard, you and I already know the pros and cons. The last part, the...
Pretty interesting. The first part is a collection of interviewed viewpoints from students as well as educators; pretty standard, you and I already know the pros and cons. The last part, the Science behind learning, is more insightful: what happens to how we understand the world after certain tools or methods are introduced. How does learning actually happen?
Chapters in the video:
Intro
Path 1: banning AI
Path 2: allowing AI
The problem with the calculator analogy
The science of learning
Conclusion
Interesting bit I didn't know: the IB program allows AI use because
The IB believes that artificial intelligence (AI) technology will become part of our everyday lives—like spell checkers, translation software and calculators. (Full statement here)
Also: desirable difficulty vs fluency is news to me.
The IB link is interesting. My kids are in IB, and they have been adamant that AI is unacceptable and evil. I've been trying to get them interested in learning how to use it as a tool. I'll have...
The IB link is interesting. My kids are in IB, and they have been adamant that AI is unacceptable and evil. I've been trying to get them interested in learning how to use it as a tool. I'll have to show them this.
Your kids might have heard it from adults who didn't actually read the statement. Do warn them that adults who are administering their exam or course may not share the official view points, and...
Your kids might have heard it from adults who didn't actually read the statement. Do warn them that adults who are administering their exam or course may not share the official view points, and getting good grades quietly while knowing something different to be true is an exam skill set too.....
So I assume they're nearly college age? I recently discovered ModernStates.org CLEP challenge exams entirely for free, which basically means first year of college free, online, and at one's own pace. my AP / IB highschool career would have been very different if I had that option, back in the day.
Seems to me like school needs to evolve past boring, pointless assignments and more towards active learning and in-classroom projects. I know its not that simple because schools (at least in...
Seems to me like school needs to evolve past boring, pointless assignments and more towards active learning and in-classroom projects. I know its not that simple because schools (at least in America) are basically underwater, but that's something we need to address first. School can't be the same forever, we need to innovate.
The biggest focus for me is not that they are getting the items correct, but that they understand the steps along the way to get there. AI will get the answer in the same way that a calculator...
The biggest focus for me is not that they are getting the items correct, but that they understand the steps along the way to get there. AI will get the answer in the same way that a calculator solves equations, or a dictionary defines vocabulary. The instruction methods need to change to account for this. Perhaps this means the end of dumping endless homework on students?
They touch on your points in the video. A) It's not like a calculator, since calculators don't do confabulations. B) It doesn't teach students how to get there on their own if needed, so it...
They touch on your points in the video.
A) It's not like a calculator, since calculators don't do confabulations.
B) It doesn't teach students how to get there on their own if needed, so it doesn't help them learn.
literally B) is what everyone said about calculators when I was in school in the 90s. Worse still, writing (your own) calculator programs to use on tests was banned. Imagine if instead kids were...
literally B) is what everyone said about calculators when I was in school in the 90s. Worse still, writing (your own) calculator programs to use on tests was banned.
Imagine if instead kids were encouraged to learn to use calculators quickly, sanity-check the result in case of a typo, and program.
The LLM argument is no different from calculators - educators don't have a deep intuition of how to use them or how transformational they'll be, and so they teach from a place of fear, not understanding.
If you, in high school, were chafing against the administration's restrictions on writing your own calculator program to use on tests, you're a terrible baseline to use when talking about the use...
If you, in high school, were chafing against the administration's restrictions on writing your own calculator program to use on tests, you're a terrible baseline to use when talking about the use of tools in education. You're clearly way above baseline, and make for a terrible anecdote.
Regarding calculators, it's all well and good to say "Just sanity check the results." But if someone never learned to do the math, depending on those simple heuristics only gets you far enough for day to day math making sure your bill at a restaurant is close enough. It doesn't get people far enough to ensure precision, and a lot of professions do.
It's even worse with LLMs. If you're ignorant of the subject matter (as the students inherently are,) they don't have the capability to sanity check the output of the LLM. How would they? And an answer about geopolitics doesn't fall into the binary of correct or incorrect you can depend on with math and a calculator.
You're missing my point. Sanity-checking a calculator's output is an essential skill to have in today's world. "Wait I tried to multiply this large number by a small negative one, but I have a...
You're missing my point.
Sanity-checking a calculator's output is an essential skill to have in today's world. "Wait I tried to multiply this large number by a small negative one, but I have a large positive number. Oh, I guess I didn't press the * key and instead subtracted them." "Wait, I multiplied by 2 and the result is odd. Hmm maybe I typed 3 instead? Let me try again." Someone who can't do this is going to suck a lot at being productive in today's world.
It doesn't matter what the school teaches, if they don't also teach you how to use ubiquitous technology well, your education was pretty useless.
How can you practice working with LLMs giving you qualitative results?
Well:
You can search the individual claims it makes and see if they're corroborated in human-written, trusted text.
You can even look for primary sources to verify.
You can repeat some of its claims back to it in isolation and ask if they're true.
You can rephrase your question and see how it answers.
You can ask a similar question you know the answer to and see if it answers correctly (e.g. recently I asked it if a book is magical realism cos I wasn't sure; it said yes. So then I asked, is {a high fantasy novel} magical realism? It said no and explained why not with what I knew to be a correct answer. So I trusted its first answer.)
You can repeat that last step multiple times with both positive and negative controls.
A lot of the above are the same as doing pre-LLM research. A lot of them aren't. But if you don't know how to do all of these things, then I think you are bad at using modern technology, and you're gonna be worse at doing things than someone who uses LLM and does these things.
So yes, schools need to teach how to use LLMs, and that's more important than teaching how not to use LLMs. But when learning to use an LLM you'll also learn how not to use one. (And it's the same deal with calculators btw, a lot of sanity checking is, for example, checking if the ones column is correct, which requires you've memorized the basic operations on all single-digit numbers.)
I think we're talking past one another, because you seem to have missed my point as well. Of course schools need to teach people how to use LLMs. Using them or their descendants is going to be an...
I think we're talking past one another, because you seem to have missed my point as well. Of course schools need to teach people how to use LLMs. Using them or their descendants is going to be an entirely necessary skill very quickly across the board.
But the LLM itself will not help the students develop the skills. With a calculator (especially a decent one with a history function,) you can see the steps you took to get to the result. You're giving an input, and you already have a sense of what the output should be. But the reason you have a sense of what that output should be is because someone else taught you maths. In the same way, teachers show their students how to engage in independent research. If students replace that active independent research with asking an LLM for an answer and trusting it, that's a failure, because (at least for now) they aren't worthy of that trust.
By comparison, the only way a calculator can get a wrong answer is if you give it the wrong inputs. Trying to conflate the use of calculators and LLMs obscures the deep and fundamental differences between the two.
I just wanted to respond to, "A) It's not like a calculator, since calculators don't do confabulations." It's exactly like a calculator in one specific way (and the only way that matters imo):...
I think we're talking past one another
I just wanted to respond to, "A) It's not like a calculator, since calculators don't do confabulations."
It's exactly like a calculator in one specific way (and the only way that matters imo): It's a brand-new technology that will be ubiquitous when today's kids are adults.
So any time you teach something, it has to be done with the understanding that outside of the simulated classroom mode, this technology will ALWAYS be available. Then ask, "does this lesson make sense to bother teaching?"
For example, learning how to summarize is not that important a skill anymore. In fact, in many cases neither is learning to write. What are important skills are critiquing an LLM's summary based on something else you read, or editing LLM prompts to get closer to what you want to say & then further editing their output to be precisely what you want. Attention to detail in editing LLM output is a LOT more important than original composition now and forever.
The skills needed to do (applied) math pre- and post-calculator are incredibly different from each other; now using numpy matters and who cares if you are dyslexic and can't do column addition.
The only thing that is guaranteed is that LLMs are applying this exact same revolution to...pretty much everything else you learn in school. So which was more important when we were in school, learning to do things by hand because our teachers didn't believe in calculators, or learning how to interact with calculators as skillfully as possible, which includes an underlying intuition of what's going on so you can check the output?
And today, which is more important, doing things by hand or incorporating LLMs and learning to work with them?
I suspect there’s a fundamental issue with education in that it’s “answer” oriented. School should be about questions. The real world isn’t a test. You need to be able to take an arbitrary...
I suspect there’s a fundamental issue with education in that it’s “answer” oriented. School should be about questions. The real world isn’t a test. You need to be able to take an arbitrary situation and know what to do with it. And the problem is most of the time you’re familiar with the situations you’re running into, so you get comfortable. But if you can realize when something is subtly off, ask questions, and then work on getting answers to them, you’ve made it.
The work to do the answer-finding is slowly being given to the machines. But us humans still need to know what questions to ask and when.
Yeah, what we're seeing is the unraveling of Goodhart's Law vis a vis school testing. Many people are trying to defend essays as if they were the point, but they're not the point. Education is the...
Yeah, what we're seeing is the unraveling of Goodhart's Law vis a vis school testing. Many people are trying to defend essays as if they were the point, but they're not the point. Education is the point, and essays and other tests are how we've in the past proved that education has occurred. What educators are going to do next to prove education is an open question, but essays really can't be it.
Anecdotal experience incoming; I have learned how to use Linux almost exclusively (as a hobby) by using chatGPT. Anytime I wanted to learn to do something that I couldn’t directly find in my Linux...
Anecdotal experience incoming; I have learned how to use Linux almost exclusively (as a hobby) by using chatGPT.
Anytime I wanted to learn to do something that I couldn’t directly find in my Linux sysadmin reference book, or I needed a little more hand-holding with commands spelled out for me, I asked chatGPT and over time I think it actually taught me how to administrate my home servers pretty effectively. I rarely need it for anything sysadmin related anymore.
Now when it comes to learning something a bit less abstract and more objective like coding, I found it impossible to actually learn. It simply spits out code that more often than not does work, and as a result has enabled me to do some cool stuff and I have learned how code works enough to tinker with it, but it has not taught me how to actually write code from scratch in any effective way.
That’s not to say it doesn’t have the potential to. As a society it's going to be impossible to avoid LLMs in the education field, we need to teach students and teachers how to use it effectively now on the ground floor before it gets out of hand. Especially as a brainstorming tool. Homework has proven time and time again to be ineffective, so what's the point? Good teachers likely know when students are learning in the class.
Unfortunately I work in a school district and see how behind the times and bureaucratic public school districts can be and I'm not optimistic about anything AI/LLM related getting integrated into the curriculum within any reasonable time frame.
I think these concerns miss the forest for the trees. We're worried about students using it to cheat or as a shortcut to learning. In the working world, more and more people are using AI to aid...
I think these concerns miss the forest for the trees.
We're worried about students using it to cheat or as a shortcut to learning.
In the working world, more and more people are using AI to aid their profession, so discouraging them will only reduce people's productivity.
Students should learn how AI works, when it's appropriate to use it and why, and show how it can make mistakes.
Even this is missing the point, in my opinion.
Learning how to learn is arguably the most important skill. The working world is increasingly dependent on problem solving, so I think that skill - not proving you've learned something or how to pass a test - is the real challenge.
I don't know the how to teach this or encourage policies that do, but I've been fortunate to have good teachers in my life. The best teachers are ones who can explain why your thinking was wrong, because that knowledge isn't subject specific or limited to certain tools.
I think this is how we can tackle the topic of AI in schools, and whatever the next innovation might be. Learning how to find the limitations of our tools, and when/how to use them responsibly to solve new problems.
Good. Good riddance to homework entirely. I had teachers that tried to bury us in homework and made my stress level go through the roof. School should end at school, kids deserve the "right to...
Good. Good riddance to homework entirely. I had teachers that tried to bury us in homework and made my stress level go through the roof. School should end at school, kids deserve the "right to disconnect" just like working adults do. How come no one ever talks about that?
Even on salary, there should be some work life barriers. I know there can be on-call expectations and the like, but salary shouldn't mean "available 24/7".
Even on salary, there should be some work life barriers. I know there can be on-call expectations and the like, but salary shouldn't mean "available 24/7".
I mostly agree that homework is bullshit to make the system look like it's working. But teachers already have it hard enough as it is and don't need the extra work of rethinking the system.
I mostly agree that homework is bullshit to make the system look like it's working. But teachers already have it hard enough as it is and don't need the extra work of rethinking the system.
Pretty interesting. The first part is a collection of interviewed viewpoints from students as well as educators; pretty standard, you and I already know the pros and cons. The last part, the Science behind learning, is more insightful: what happens to how we understand the world after certain tools or methods are introduced. How does learning actually happen?
Chapters in the video:
Interesting bit I didn't know: the IB program allows AI use because
Also: desirable difficulty vs fluency is news to me.
The IB link is interesting. My kids are in IB, and they have been adamant that AI is unacceptable and evil. I've been trying to get them interested in learning how to use it as a tool. I'll have to show them this.
Your kids might have heard it from adults who didn't actually read the statement. Do warn them that adults who are administering their exam or course may not share the official view points, and getting good grades quietly while knowing something different to be true is an exam skill set too.....
So I assume they're nearly college age? I recently discovered ModernStates.org CLEP challenge exams entirely for free, which basically means first year of college free, online, and at one's own pace. my AP / IB highschool career would have been very different if I had that option, back in the day.
Seems to me like school needs to evolve past boring, pointless assignments and more towards active learning and in-classroom projects. I know its not that simple because schools (at least in America) are basically underwater, but that's something we need to address first. School can't be the same forever, we need to innovate.
The biggest focus for me is not that they are getting the items correct, but that they understand the steps along the way to get there. AI will get the answer in the same way that a calculator solves equations, or a dictionary defines vocabulary. The instruction methods need to change to account for this. Perhaps this means the end of dumping endless homework on students?
They touch on your points in the video.
A) It's not like a calculator, since calculators don't do confabulations.
B) It doesn't teach students how to get there on their own if needed, so it doesn't help them learn.
literally B) is what everyone said about calculators when I was in school in the 90s. Worse still, writing (your own) calculator programs to use on tests was banned.
Imagine if instead kids were encouraged to learn to use calculators quickly, sanity-check the result in case of a typo, and program.
The LLM argument is no different from calculators - educators don't have a deep intuition of how to use them or how transformational they'll be, and so they teach from a place of fear, not understanding.
If you, in high school, were chafing against the administration's restrictions on writing your own calculator program to use on tests, you're a terrible baseline to use when talking about the use of tools in education. You're clearly way above baseline, and make for a terrible anecdote.
Regarding calculators, it's all well and good to say "Just sanity check the results." But if someone never learned to do the math, depending on those simple heuristics only gets you far enough for day to day math making sure your bill at a restaurant is close enough. It doesn't get people far enough to ensure precision, and a lot of professions do.
It's even worse with LLMs. If you're ignorant of the subject matter (as the students inherently are,) they don't have the capability to sanity check the output of the LLM. How would they? And an answer about geopolitics doesn't fall into the binary of correct or incorrect you can depend on with math and a calculator.
You're missing my point.
Sanity-checking a calculator's output is an essential skill to have in today's world. "Wait I tried to multiply this large number by a small negative one, but I have a large positive number. Oh, I guess I didn't press the
*key and instead subtracted them." "Wait, I multiplied by 2 and the result is odd. Hmm maybe I typed 3 instead? Let me try again." Someone who can't do this is going to suck a lot at being productive in today's world.It doesn't matter what the school teaches, if they don't also teach you how to use ubiquitous technology well, your education was pretty useless.
How can you practice working with LLMs giving you qualitative results?
Well:
A lot of the above are the same as doing pre-LLM research. A lot of them aren't. But if you don't know how to do all of these things, then I think you are bad at using modern technology, and you're gonna be worse at doing things than someone who uses LLM and does these things.
So yes, schools need to teach how to use LLMs, and that's more important than teaching how not to use LLMs. But when learning to use an LLM you'll also learn how not to use one. (And it's the same deal with calculators btw, a lot of sanity checking is, for example, checking if the ones column is correct, which requires you've memorized the basic operations on all single-digit numbers.)
I think we're talking past one another, because you seem to have missed my point as well. Of course schools need to teach people how to use LLMs. Using them or their descendants is going to be an entirely necessary skill very quickly across the board.
But the LLM itself will not help the students develop the skills. With a calculator (especially a decent one with a history function,) you can see the steps you took to get to the result. You're giving an input, and you already have a sense of what the output should be. But the reason you have a sense of what that output should be is because someone else taught you maths. In the same way, teachers show their students how to engage in independent research. If students replace that active independent research with asking an LLM for an answer and trusting it, that's a failure, because (at least for now) they aren't worthy of that trust.
By comparison, the only way a calculator can get a wrong answer is if you give it the wrong inputs. Trying to conflate the use of calculators and LLMs obscures the deep and fundamental differences between the two.
I just wanted to respond to, "A) It's not like a calculator, since calculators don't do confabulations."
It's exactly like a calculator in one specific way (and the only way that matters imo): It's a brand-new technology that will be ubiquitous when today's kids are adults.
So any time you teach something, it has to be done with the understanding that outside of the simulated classroom mode, this technology will ALWAYS be available. Then ask, "does this lesson make sense to bother teaching?"
For example, learning how to summarize is not that important a skill anymore. In fact, in many cases neither is learning to write. What are important skills are critiquing an LLM's summary based on something else you read, or editing LLM prompts to get closer to what you want to say & then further editing their output to be precisely what you want. Attention to detail in editing LLM output is a LOT more important than original composition now and forever.
The skills needed to do (applied) math pre- and post-calculator are incredibly different from each other; now using numpy matters and who cares if you are dyslexic and can't do column addition.
The only thing that is guaranteed is that LLMs are applying this exact same revolution to...pretty much everything else you learn in school. So which was more important when we were in school, learning to do things by hand because our teachers didn't believe in calculators, or learning how to interact with calculators as skillfully as possible, which includes an underlying intuition of what's going on so you can check the output?
And today, which is more important, doing things by hand or incorporating LLMs and learning to work with them?
I suspect there’s a fundamental issue with education in that it’s “answer” oriented. School should be about questions. The real world isn’t a test. You need to be able to take an arbitrary situation and know what to do with it. And the problem is most of the time you’re familiar with the situations you’re running into, so you get comfortable. But if you can realize when something is subtly off, ask questions, and then work on getting answers to them, you’ve made it.
The work to do the answer-finding is slowly being given to the machines. But us humans still need to know what questions to ask and when.
Yeah, what we're seeing is the unraveling of Goodhart's Law vis a vis school testing. Many people are trying to defend essays as if they were the point, but they're not the point. Education is the point, and essays and other tests are how we've in the past proved that education has occurred. What educators are going to do next to prove education is an open question, but essays really can't be it.
Anecdotal experience incoming; I have learned how to use Linux almost exclusively (as a hobby) by using chatGPT.
Anytime I wanted to learn to do something that I couldn’t directly find in my Linux sysadmin reference book, or I needed a little more hand-holding with commands spelled out for me, I asked chatGPT and over time I think it actually taught me how to administrate my home servers pretty effectively. I rarely need it for anything sysadmin related anymore.
Now when it comes to learning something a bit less abstract and more objective like coding, I found it impossible to actually learn. It simply spits out code that more often than not does work, and as a result has enabled me to do some cool stuff and I have learned how code works enough to tinker with it, but it has not taught me how to actually write code from scratch in any effective way.
That’s not to say it doesn’t have the potential to. As a society it's going to be impossible to avoid LLMs in the education field, we need to teach students and teachers how to use it effectively now on the ground floor before it gets out of hand. Especially as a brainstorming tool. Homework has proven time and time again to be ineffective, so what's the point? Good teachers likely know when students are learning in the class.
Unfortunately I work in a school district and see how behind the times and bureaucratic public school districts can be and I'm not optimistic about anything AI/LLM related getting integrated into the curriculum within any reasonable time frame.
Parents could always do your homework. In some cases, they did.
I think these concerns miss the forest for the trees.
We're worried about students using it to cheat or as a shortcut to learning.
In the working world, more and more people are using AI to aid their profession, so discouraging them will only reduce people's productivity.
Students should learn how AI works, when it's appropriate to use it and why, and show how it can make mistakes.
Even this is missing the point, in my opinion.
Learning how to learn is arguably the most important skill. The working world is increasingly dependent on problem solving, so I think that skill - not proving you've learned something or how to pass a test - is the real challenge.
I don't know the how to teach this or encourage policies that do, but I've been fortunate to have good teachers in my life. The best teachers are ones who can explain why your thinking was wrong, because that knowledge isn't subject specific or limited to certain tools.
I think this is how we can tackle the topic of AI in schools, and whatever the next innovation might be. Learning how to find the limitations of our tools, and when/how to use them responsibly to solve new problems.
Good. Good riddance to homework entirely. I had teachers that tried to bury us in homework and made my stress level go through the roof. School should end at school, kids deserve the "right to disconnect" just like working adults do. How come no one ever talks about that?
Not necessarily if you’re salary.
Even on salary, there should be some work life barriers. I know there can be on-call expectations and the like, but salary shouldn't mean "available 24/7".
I mostly agree that homework is bullshit to make the system look like it's working. But teachers already have it hard enough as it is and don't need the extra work of rethinking the system.