What about having an LLM teach you to code?
My daughter (11) is doing a week long Python class, which is not using LLMs.
It got me thinking about how I learned to program in the pre-internet days (laboriously, from books), and then what a marvel it was when you could just search for information, especially for troubleshooting. But for her, the first answer in the Google search is going to be the AI summary, and most of her search tools are going to be AI tools.
I wonder if it would be possible to make an LLM that has a didactic/socratic mode. So if you said, "help me write a program to do madlibs" maybe it would give you a skeleton of a function, then prompt you to come to with a plan, then critique that plan. Or if you said, "I'm getting this error", it wouldn't just fix it, it would explain what the error means and nudge you towards the answer.
Thinking in a larger sense, it could have a rubric of important concepts, even tiers of understanding. It could be using the interactions to track the user's understanding, which could let it then tune how it answers future questions, or even be used to customize assignments.
I recognize that this is potentially replacing a teacher with a machine, which wouldn't be my goal. Good teachers are more holistic in their teaching than a machine is ever likely to be. But for people who don't have access to good teachers, or need more directed support than is available from a teacher, or just want to self study, it seems like it could be a valuable addition.
Until they solve the obsequiousness problem, it would be vulnerable to prompt hacking, so really more of a tool for someone who recognizes the value of learning over just being given the answer.
What do folks think about using such a tool? What would you want it to do, or not do?
Aside: I forgot until I reached the end of this post, but this is also (somewhat) the plot of The Diamond Age, or A Young Lady's Illustrates Primer by Neal Stephenson.
Gemini, at least, has an explicit teaching mode that works pretty well. I used it for months to study something in detail, in conjunction with real-life study. Pretty similar to what your daughter is looking at, actually.
It's not perfect, of course. You pointed out the social issue - the obsequiousness. Mostly, for me, that manifested as it being like "you've made so much progress this session!" when I knew I hadn't - not an issue so much with the actual lessons, which were generally excellent. (After a lot of prompting work getting it to stop being like "what an insightful question!", that is.) And of course human teachers are also often overgenerous with praise and unwilling to correct you when you're wrong. I do worry a bit about the effect of large amounts of a AI-style flattery on a developing mind, though. If both bad ideas and insightful ideas get called insightful, you'll never learn to distinguish between them. I would want to edit the system context pretty heavily before I let a preteen spend significant time with it.
The other critical issue is the fact that models don't remember anything. They can't really remember where you are, what you've done, etc., even if you use a model with "memory." This is a much bigger challenge that doesn't present itself with humans. You can sort of get around this by having it make you a lesson plan, then giving it back its own plan later and being like "I'm on day 12," but that's pretty clunky in practice. For that reason, models work fantastically to explain specific concepts and answer specific questions, but it's better to have something already prepared that you want to talk over. Again, the concept of man-machine teaming works quite well here.
I'm in favor of the idea, all considered. I'd just proceed carefully and give as much supervision as possible.
When writing code, I will have the coding agent write a design doc with a checklist and check off things when they're done. So, when it restarts it can see what's been done already. It seems like something like ought to work for lessons if it can store a persistent file somewhere.
I imagine someday, that will be built in. We're just at the beginning of figuring out how to use these things.
I do the plan-based approach too, and I have taken to leaving the plans in the code she so that the AI can go back and reference them later.
Now that the agents have coding skills built in, one of the areas that I think will be interesting will be to give an agent the ability to create and manage its own knowledge graph. If they get better at context-sensitive recall, then they can be more useful in the long term.
An interesting aside: the stateless nature of the LLM is one of the strongest arguments (to me) that they are not really conscious. However, once you start to give the AI continuity of experience so that or can grow and learn, I think that answer becomes less clear.
I think that's a good practice. The way they use context works almost infinitely better with code, where it can absorb everything you're working with at a glance and keep going, whereas for something more conversational or didactic it can be quite a bit more frustrating.
I actually used Claude to teach myself for an online university class that had no lecture and a practically non-existent professor. It turned out ok, but only because I’m a grown-ass adult and I knew what I was doing. I had a curriculum which I fed to it as context and had it teach me, but it had problems because it wasn’t really 100% sure about what was in the readings and videos, but it was capable of synthesizing because it knew the topic well.
Claude did a good job of teaching me to a point. This was a class on low level data structures and while I knew some of them I didn’t know all of them and Claude generally preferred a Socratic approach where it would feed me bits of information and have me to synthesize it into greater understanding by asking me questions. Context would always be a problem because it would be asking me questions about things I had already learned about in other days set by other chat sessions. Sometimes it wouldn’t understand that I already knew about the difference between an array and a linked list for instance.
I passed that class and I do credit Claude for being the primary reason why I did, however, I think this is a special case. The reason why I used Claude to begin with is that the course was absolutely terrible. The readings were from books that were written in an Indian dialect of English which made them extremely difficult to parse, and they were from a number of different sources which made them even more disjointed. I didn’t have the support of a professor, and the amount of time I would spend just reading and watching the videos would have taken me something like 8-12 hours depending on how bad they were that week. In previous classes the readings were from technical reference books which required knowledge of mathematical symbols which I had never encountered up to that point, and of course nothing in the readings explained it.
But what would have happened if I didn’t have a curriculum to guide me? To put things simply, I wouldn’t have learned everything I needed to. AI is sycophantic regardless of which model you choose, it seems. It was always congratulating me and telling me I had it, and I was constantly asking me if there were things I hadn’t covered that I should have. There usually was. If I hadn’t been feeding it my curriculum piecemeal, it would have had me going all over the place and probably covering things I didn’t need to know.
While I did pass the final exam, I don’t believe that I really learned the material very well. I couldn’t explain how red-black trees self-balance or organize, or much detail into how the KNP search algorithm works.
With all this in mind I would absolutely caution against making AI your child’s teacher. AI can be a good supplement to a teacher and could be a great tutor, but I don’t think it is good as a teacher who has a far more complex job than just explaining concepts.
I think there are enough online curricula that you could probably have had Opus crosswalk them and build you a credible curriculum for most topics. Probably not useful for a class since it might add topics that weren't tested, but if you were learning for general knowledge, I think this would help side tiny what the core concepts really are.
Yes, I agree with this. In this case, the MVP is to give her a tool to search for answers on her own that engages her in a less passive way than the default AI. But I will definitely be supervising the journey.
Just want to add that if you're in the Chat GPT camp there is the equivalent feature, "Study mode".
Certainly. You can accomplish most of this with a good system prompt. This is obviously more advanced than would be ideal for an eleven year old, but see Stanford's Agent Guidelines for an idea of where to start.
I don't think a new tool is strictly needed here. You can plug this into any available model through an interface like LM Studio. Or if that UI is too complicated, it can be run as a server and a simpler frontend could sit in front of it.
That said, with much more effort, one could likely write a specific coding teacher tool that examines a student's work and gently corrects or nudges them in the right direction, fully integrated into the IDE. I'm not aware of one that exists yet, but it seems easier than ever to prompt one into existence with a few magic words...
You can go the other way, where you just elimate Googles AI summary. There's a few ways of doing this, like so: https://tenbluelinks.org/
Or using an addon to tack udm=14 to Google searches, which handles the case of people manually visiting Google before search (the former just deals with when you use the omni bar to search).
I have some experience teaching coding (c2004). Back then the problem we had the most difficulty solving was finding something people wanted to code. That's is: the best students brought their own problems they wanted code to solve. It was a problem of motivation rather than tools or educational materials.
I do think LLM can be a good teacher.. if this is a hobby-project you're on a good track. If you want to build a business you'll need some primary research on customer-problem fit. LLMs cannot do that.
I was thinking of it as a hobby, or maybe even as a thought experiment. However, I have a friend whose husband runs a tutoring company, so it might be worth a chat with them.
I have basically been doing this for myself off and on to learn Python. I have a simple project I’m working on where I’ve broken an application down into features and functions and have been using a combination of Cursor and Claude to write the code and explain it back to me as a study aid.
It helps that I already have some basic understanding of python and understand sql and JavaScript pretty well, so I have something to build off of already though. It basically ends up working similarly to searching Stack Exchange, except the responses come instantly and nobody tells me to “get a real computer” if I mention I’m using a Mac.
I've been using it to learn a new programming language. Ban it from writing code for you and even generating easy copy/paste solutions. All of it can be bypassed with enough effort but definitely can be a tool worth using.
I've been using this tutor prompt: https://www.patreon.com/posts/guide-to-turn-ai-153388398
The creator used to work for Veritasium before he split off to make videos about AIs. This is the video where he explains the system: https://youtu.be/Wn-17_6m_w4
This is really helpful, thanks! I think adding this to a skill that lets the agent save and recall concepts that the student has learned would make it useful in the long term, not just one session.
You might be interested in Matt Pocock’s
teachskill as well. I haven’t personally tried it yet, but it looks quite promising.There is also an accompanying video which introduces the skill a bit more.
Thanks, I will check it out!
This could work. I actually started building a coding agent with this in mind. The idea was an agent that does all the things you'd expect an agent to do, except for a simple thing: the agent can't edit your files or run bash commands. It could read, look up documentation, etc. but not modify anything by itself. It would have to teach you how to make the changes.
However, as it goes with these projects, I got distracted by other things in my life and never got very far with it. But during test runs, I thought the idea really worked well. LLMs are already trained to be helpful to the user. I found that I really did not have to do that much prompting to make it behave as a teacher.
Same for me, so many project ideas! If only there were infinite time.
The linkage between present events and Neal Stephenson's book has been top of mind for me for the last few years, too.
That said, I'm truly hoping for those of us who aren't young girls reading primers, that we're spared John Percival Hackworth's marathon, decade-long drummer orgy.
AI fluid exchange -- eeew
I think WALL-E was more prophetic tbh, it was made just before the iPhone.
Lol, I was thinking the Exact Same Thing after I posted that.
I love that at some point, Stephenson always gets super gross and high concept.