Maybe antirez is some god-tier prompter, but I find myself constantly dissatisfied by the decisions that anthropic's frontier models make. I think they are 100% great at automated reviews...
Maybe antirez is some god-tier prompter, but I find myself constantly dissatisfied by the decisions that anthropic's frontier models make.
I think they are 100% great at automated reviews (relatively low cost to verify, I can overrule without negotiation, endless attention any time of the day, can be fed house review checklists). I think they are great sounding boards for design. Exploiting their encyclopedic knowledge of general techniques to direct research yields better designs faster than I could unassisted. I am just struggling with the idea that you can be so sure that your ideas are being faithfully reflected if you look never. That's not at all my experience. I am constantly finding inconsistencies and stupidity any time I take my eye off the underlying evolution of what's being spit out.
I'm a software engineer with 15+ years in the industry. Over the years, I have occasionally tried AI through whatever the Intellij IDEA Ultimate IDE offered when I was really stuck on something...
Exemplary
I'm a software engineer with 15+ years in the industry. Over the years, I have occasionally tried AI through whatever the Intellij IDEA Ultimate IDE offered when I was really stuck on something fairly self contained (e.g. sql query optimization). Every time I was disappointed and wondered how anyone got anything worthwhile out of AI.
Earlier this year, my boss got us all a 1-month trial to Claude Code (Opus 4.6 I think), and told us genuinely no-pressure "give it a shot, see if it helps". I went in very skeptically, with a baseline zero trust for the AI output. The first few tasks were self-contained, dead simple, and easily verifiable - stuff like "translate this legacy Twirl template to React". I reviewed every line it produced and tested every result. It did pretty well, so I started using it a bit more.
It's been a few months now and I find that I use Claude Code (now Opus 4.8, thinking high) for the majority of my work. I've created some tight agent files for the projects and a short global one based on actual usage and results. I've heard my process called "architecture driven development", though I don't really participate in the AI industry? culture? fandom? Detailed design docs / plans vastly improve the AI implementation results and subsequent review, like @stu2b50 mentioned. Anyway, here's what's gotten me good results:
Personally draft as detailed of a requirement set as I can, with high level design/architecture. Reference local code precedent where possible.
Give that to Claude and prompt it to draft a plan. Anything beyond a known simple fix, always draft a plan. It'll ask me various questions during the planning, and state any assumptions or decisions it made at the end.
In a clean session (so it's a cold-read), prompt Claude to critically review the plan it just wrote. I created a skill for this after a while based on how I would usually prompt it, refined with Claude itself. This digs in to everything from verifying all claims against live source / data, reviewing high level architecture and cost, preemptively looking for contradictions and ambiguities and edge cases, etc. It'll come back to me with a list of findings, organized by estimated severity. Giving it a read-only user for the database and several other key integrations has helped a lot here, and sometimes even surfaced bad assumptions on my part.
Manually walk through each point in the plan review findings with Claude, prompting a resolution or further investigation, until all are resolved.
Depending on the nature of what was found and how much the resolutions altered the plan, repeat step #3 as many times as reasonable.
Manually skim the final plan. At this point, I'm pretty familiar with it from the drafting and reviews, so it's mostly a sanity check.
In a new session, have Claude implement the plan and check against whatever automated verification the specific thing might have (compile, linting, the rare tests, etc.)
8 - 10. Same as step #3-5, except for the implementation. In a clean session audit the implementation. I've got another skill for this based on what I'd usually ask it to do. Faithfulness to plan, trace impacted code flows, etc. Walk through findings, rinse and repeat as necessary.
A skim of the actual code diff. Working with a large, mature codebase gives Claude a lot of precedent to follow, so it usually organizes and writes things consistent with our existing code. Sometimes I don't like exactly how it organized things, or the specific implementation of something, and I'll either directly change it or prompt Claude to redo that bit.
Manual end to end testing. Usually some adjustments fall out of this.
Note that the various clean sessions are important, or having claude run the review skill with subagents in clean sessions. Aside from growing large contexts eventually degrading results, a cold reading of the plan and the implementation are key. They don't let old assumptions skip over investigating potential faults, dropped branches from investigation get brought back in, or anything like that.
If nothing else, it's a good way to force yourself to articulate and record all of your requirements, all of the desired flows, and account for "all" edge cases. Like extreme rubber-ducking.
EDIT:
Oh, and one other thing that I don't think people often consider: Maybe once every few months, ask the AI to review all of it's memories and agent files for consistency, redundancy, verbosity, and organization (different project memories, memory vs agent file, project agent file vs global agent file). Over time, especially if you've got multiple related projects, it'll accumulate a lot. Cleaning up can make a difference in result quality and context token usage.
This is an excellent workflow! Mine isn't a lot different. In general I'd argue for more review, both human and agent, but it really depends on the code/application. At least for production/user...
This is an excellent workflow! Mine isn't a lot different. In general I'd argue for more review, both human and agent, but it really depends on the code/application. At least for production/user facing code.
One thing that's different... I don't use auto memory and I don't have agents write skills or subagent definitions. I've never been satisfied with their results in either making memories or building scaffolding. Personally I think the "self improving" angle is a trap, at least for the time being. Agents suck at it. Instead I manually formalize things that come up repeatedly into context files that are read/injected in various ways.
Working with a large, mature codebase gives Claude a lot of precedent to follow
The value of this cannot be overstated, and this is more true with each generation of models. If you give them access to a comprehensive codebase with patterns, utilities and established architecture, they will do a reasonably good job of following conventions. Which removes a lot of opportunities to go off the rails.
For greenfield projects, even though you don't have to write code, it doesn't mean you can't! It's a great way to give agents a pile of conventions, patterns and design philosophies very quickly.
One thing I'd add to the list:
Create design docs about larger areas of the codebase. This is one area where I let the agents do most of it because it's essentially summarization, which they're great at, and there aren't too many opportunities for mistakes that might matter. It's very useful to be able to say "read X file" and have the agent immediately understand a particular feature, functionality or system and where it exists in the codebase. Cuts down on both time and tokens.
Always comforting to hear people arrived at a similar usages and conclusions. Either we're doing something right, or we're at least in the same wrong boat together! Yeah, memories are a bit goofy....
Always comforting to hear people arrived at a similar usages and conclusions. Either we're doing something right, or we're at least in the same wrong boat together!
Yeah, memories are a bit goofy. They seem to be useful short term, helping to bridge the gap between workdays and task switches, but can come back to bite weeks down the line when they're no longer in sync with the current state of the world. I try to get it to write as much as possible and useful to the plan files, and much more rarely to the agent files after hitting the same problem or deriving the same information over and over again.
For skills and the agent files, I had Claude write the prose, but built them with nearly the same flow as the feature flow. Detailed objective scope and requirements, known bits of information or where to gather them from, desired behavior, etc. up front, iterative back and forth, cold reading reviews by both me and Claude, and tweaks over time based on their performance. I think it's important for people to use the AI tools and harness as vanilla as possible for a while to find what fits you personally rather than cargo-culting a bunch of skills and prompts and agent file "hacks".
Definitely agree about the design docs for large areas of the code base, as well as the boundaries and interactions between code bases if your project has multiple repos/applications. Before I did that, anything non-trivial would either get confused if I didn't tell it explicitly where things lived, or would spend time and tokens poking around every time. For the same reason, I keep a database schema dump on hand so it can do some quick grepping to find most things.
Oh, and I just remembered another tidbit, agents are great with CLI tools. I rarely need to reach for some feature-subset MCP when the thing has a mature cli tool I can install. Plus I don't have to keep authentication info in some plaintext MCP config.
I'm pretty sure it's a good boat! In addition to using agents in my own workflows, I've setup solutions for other organizations, outside of software, and many of the same principles work well...
I'm pretty sure it's a good boat! In addition to using agents in my own workflows, I've setup solutions for other organizations, outside of software, and many of the same principles work well there too.
Also with each model generation, more and more of these concepts are being formalized onto fine tuning rather just the harnesses. Presumably there's a pretty high bar to make it into training.
I think it's important for people to use the AI tools and harness as vanilla as possible for a while to find what fits you personally rather than cargo-culting a bunch of skills and prompts and agent file "hacks".
Agreed, especially when you consider that many skills and prompts become outdated with newer models/harnesses. The more you understand how the tool works, and what its current limitations are, the better. If you only ever see how the agent behaves in scaffolding created by someone else, you're not building an accurate mental model.
For the same reason, I keep a database schema dump on hand so it can do some quick grepping to find most things.
Yes. I have a schema that lives in each project root that agents are instructed to read or grep for everything DB related. They also keep it updated with changes. That's another area with low error rates that I'm comfortable letting agents handle mostly unsupervised.
Oh, and I just remembered another tidbit, agents are great with CLI tools.
It sounds like we agree on pretty much everything. In my experience CLI tools (or sometimes APIs) beat MCP servers nearly all of the time.
Ok. So point 8, is effectively the antithesis of what the proposed article is saying right? It says don't look at the code, you say "I look at the code". Planning, adversarial review, session...
Ok. So point 8, is effectively the antithesis of what the proposed article is saying right? It says don't look at the code, you say "I look at the code". Planning, adversarial review, session clearing, it's all a big dance to try to get the AI to do what you'd like, but in the end you still aren't actually delegating in the way that the article is suggesting. I'll also give a concrete example of the type of thing that I see all of the time. I wanted claude to produce a network manager dispatch script to update a particular route on an interface. This is a 5 line prompt at best and definitely not worthy of the complete process you have described. I am using opus 4.8 max effort. It produced something like the following
interface=$1
gateway=$IP4_GATEWAY
if [[ -n $gateway ]]; then gateway=$(<<bunch of shell to try to calculate a gateway); fi
[[ -n $gateway ]] || exit 0
ip route replace 10.0.0.0/8 via $gateway dev $interace metric 100
Now the problem with this (ignore quoting, action checking, it's a snippet) is basically that on a network manager managed interface if $IP4_GATEWAY isn't populated it's because there isn't a default route yet. So trying to calculate one is pointless. When one shows up the script will be called again, and so it will be populated. This type of shit happens all of the time. AIs cannot keep enough in their brains afaict to stop just jamming in extra unnecessary backstops. They do it for scripts, they do it for functions, then never lean up or unify error handling, or trace architectural invariants. They just double up piles and piles of redundant checking, which they themselves then fail to keep in sync.
All this to say, nice process, but no amount of planning gets frontier models to produce code that I am happy with, because the gap isn't in understanding what I want in terms of functional requirements. The above script would have worked. It's just needlessly complicated.
Not GP, but in my opinion that's because the OP article is wrong. Not about everything, but definitely about the implication that you can skip review. The classes of mistakes and errors you're...
So point 8, is effectively the antithesis of what the proposed article is saying right?
Not GP, but in my opinion that's because the OP article is wrong. Not about everything, but definitely about the implication that you can skip review.
The classes of mistakes and errors you're talking about are definitely real, and far from the only ones.
Note that you can mitigate a lot of said issues with custom scaffolding and tooling. Just not effectively enough that you can forego review entirely. So far :|
I agree that the premise of the article was wrong. That was my conclusion (at the root of this thread). My summary of initial chain: Me: I don't agree with article, AI often generates code I don't...
I agree that the premise of the article was wrong. That was my conclusion (at the root of this thread). My summary of initial chain:
Me: I don't agree with article, AI often generates code I don't like.
DrStone: I too was disappointed, until I started doing X.
Me: I don't think X helps with my disappointment.
To be clear: I think X is a useful collection of techniques. If it isn't clear already, I use AI a lot, generate a lot of code with it, inspect a lot of code. I am not at the toe-dipping staging :)
Yeah, you're right, I do still look at the code a bit, which is what the article advocates completely against. That said, I've been looking at it a lot less than when I started, based on what I've...
Yeah, you're right, I do still look at the code a bit, which is what the article advocates completely against. That said, I've been looking at it a lot less than when I started, based on what I've actually seen it produce in my situation. In the beginning I was meticulously reviewing every line of code it produced, far more carefully than I would any human PR, even after our extensive interactive planning practically wrote the thing already. Over time, I've still looked, but substantially less, now more like what I would for a colleague's PR who has done well in the past. Mostly a sanity skim, a high level architecture and organization kind of eyeballing, for complex or critical changes. Just a minute or few. Small tweaks and bug fixes I might let pass straight through at this point.
I think the softening has come down to a few things (not just laziness or complacency). I've figured out a workflow that works for me and spends a lot of time planning the details up front with me driving; I've definitely caught things in this stage where it's planning to overbuild or hedge against impossible conditions. My general trust in the tool has improved after using it successfully on both small and large things for months. I've seen how the large, mature code base gives it a lot of prior art to work with that it usually follows. Auto-memories and, much more importantly, my agents files and design docs have explicitly codified a lot of architecture, convention, and more subtle rules, growing and sharpening over time, which help the agent do the "right" thing more often.
I don't think I'll ever go full vibe (I have trouble completely delegating even to humans), but it's trending in that direction.
Yeah, seconding. I have a /prune-docs skill which actively prunes both memories and developer documentation for redundancies, outdated bits and pieces, things that shouldn't be in there, etc....
Oh, and one other thing that I don't think people often consider: Maybe once every few months, ask the AI to review all of it's memories and agent files for consistency, redundancy, verbosity, and organization (different project memories, memory vs agent file, project agent file vs global agent file). Over time, especially if you've got multiple related projects, it'll accumulate a lot. Cleaning up can make a difference in result quality and context token usage.
Yeah, seconding. I have a /prune-docs skill which actively prunes both memories and developer documentation for redundancies, outdated bits and pieces, things that shouldn't be in there, etc.
Maintenance and code quality are key to getting great results.
AI is an accelerator and a multiplier. If your code's shit and your docs are shit, it'll multiply the shit. If it's good, it'll multiply the good.
The key is design docs. Lots of design docs. Always have the agent make a design doc first. Only have it start generating code once the details in the docs are agreeable. As a bonus it’s really...
The key is design docs. Lots of design docs. Always have the agent make a design doc first. Only have it start generating code once the details in the docs are agreeable. As a bonus it’s really nice having that kind of doc easily available.
Do you not find that docs produced by ai tend to miss the forest from the trees over indexing on minutiae to the point of rendering them nearly useless for humans (probably very good for other...
Do you not find that docs produced by ai tend to miss the forest from the trees over indexing on minutiae to the point of rendering them nearly useless for humans (probably very good for other agents)?
Also, I don't know about you, but personally I find I arrive at leaner designs by stress-testing them against existing code. Sure, if it's something that fits an obvious archetype or design pattern you can probably anticipate a fair amount.
I find that AI basically never simplifies ever. Neither in design or code and without active anti entropy you end up in a bad spot, and no amount of design gets you lower entropy.
What kind of design docs are you advocating for? Technical, functional, architecture, or something else? Do you mean for each piece of functionality you build or high level system things? How big...
What kind of design docs are you advocating for? Technical, functional, architecture, or something else? Do you mean for each piece of functionality you build or high level system things? How big are the docs you prefer?
Im curious how it compares to our setup and my preferred method
My tip is to add this to AGENTS.md: And in the first prompt I always ask a question. If it's about a bug it's "why did this happen?" If it's a feature request, I describe the change and say...
My tip is to add this to AGENTS.md:
If a prompt contains questions, stop after answering the questions, so the user has a chance to reply before you edit any more code.
And in the first prompt I always ask a question. If it's about a bug it's "why did this happen?" If it's a feature request, I describe the change and say "what's involved in making this change?" If there's an API change then I ask "How should we change the API?" I don't tell it to go ahead until I'm pretty sure I understand what it will do.
Also, if it's a new project, I write a plan.md first and we go over it, making multiple passes. Then I ask "are there any important questions to answer before starting work?" And I edit the plan myself until it's good enough to start Phase 1, because coding agents tend to put in excessive detail.
I guess Claude Code has a /plan mode, but my coding agent doesn't and I find I don't need it. I just make sure to ask a question if I don't want it to start right away.
Then again I'm just writing web apps, not anything complicated.
Interesting idea. I'll give it a whirl. I tend to just roll with auto-mode because I've found empirically (literally drag racing the same problem) that by the time my colleagues have finished...
Interesting idea. I'll give it a whirl. I tend to just roll with auto-mode because I've found empirically (literally drag racing the same problem) that by the time my colleagues have finished their glorious round of dialectics with some planning mode i have iterated my way to a better design and solution just by hammering on the problem and getting deeper into the problem than can be discovered by looking at from afar.
The coding agent landscape has changed a lot in the past year, but in my experience it's still true that agents make dumb decisions. Less than before, but it's a long way from going away. Because...
The coding agent landscape has changed a lot in the past year, but in my experience it's still true that agents make dumb decisions. Less than before, but it's a long way from going away.
Because of that, if you're not keeping track of what it's doing, dumb decisions are going to end up coded into your project. You can mitigate some of the problem by creating detailed enough plans that there aren't a lot of decisions to make, but as soon as an agent runs into a blocker, it will start making decisions. Which you'll have no idea is happening if you aren't in the loop. This is actually more true of the current frontier models because they're trained to keep working through issues more aggressively than previous generations.
I agree with the author that's it's no longer necessary to review every single line of code, at least in some circumstances (boilerplate mostly). The latest models have much better taste than models from even 6 months ago. But I completely disagree with the framing that reviewing code slows down your velocity and therefore makes you worse. Velocity is not the only metric! I feel like this comes from a corporate "engineers are replaceable cogs" mindset. There are a lot of situations where going slow enough to maintain a mental model, reason sufficiently about choices and tradeoffs and continually update and maintain decent architecture are really really important. In those situations you're not "falling behind" by not cramming the maximum amount of coding into an 8 window, you're being a responsible engineer.
So mine is a trick. People feel more and more programming is completely modified by AI and don’t know what they should do, if they can really start coding in a completely different way, without looking much at the code as their main output. They feel like they are betraying their own field. So my intention is to arrive and say “look at me, In can write code, you know, I’m not hiding behind AI: yet, things changed, it’s not your weakness, it’s not that you are AI-pilled. It is just that our field is evolving in an incredible *and* painful (but also joyful) direction”.
This is why yesterday, on X, I said that I believe many programmers at this point have less impact they could have because they look at the code. I truly believe into that. And note that this does not mean to vibe code something just asking for the final product. The point is: if you control the ideas of your software, looking at the code itself is suboptimal and often pointless. For the following reasons:
I don't mean for this to be rude or offensive, and I wouldn't normally point this out, except for the subject matter. This blog post is horribly written and very difficult to read from a grammar...
I don't mean for this to be rude or offensive, and I wouldn't normally point this out, except for the subject matter.
This blog post is horribly written and very difficult to read from a grammar perspective. Its really disconcerting to see someone advocate so full-throatedly for a complete hands off vibe coding and code review setup if they don't even trust the technology enough to use it to proofread a blog post.
I get that the author isn't a native english speaker, but LLMs are explicitly good at translation. Its one of their main use cases, and understandability is one of the core metrics that they're trained on. I make a lot of allowances for people who don't natively speak the language, but not if you're an AI evangelist.
If you're not even using the thing to fix mistakes in your blog posts; you know, one of the few things that people almost universally agree that they're good at, it doesn't exactly lend a lot of weight to the central thesis of the post.
Given how people react to AI generated prose, me included, I find myself feeling much more forgiving towards an ESL writer who didn't do an agent editing pass.
Given how people react to AI generated prose, me included, I find myself feeling much more forgiving towards an ESL writer who didn't do an agent editing pass.
Speaking as an amateur, my approach has been to not touch LLMs for code help unless I'm absolutely desperate, and part of his closing vindicates that choice, I feel: I take that to mean, roughly,...
Speaking as an amateur, my approach has been to not touch LLMs for code help unless I'm absolutely desperate, and part of his closing vindicates that choice, I feel:
I have a doubt only regarding young programmers that don't have enough experience, and can't build a mental model. We don't know, yet, if they will require or not to understand very well how a given piece of code works, but I believe they should learn how to write programs.
I take that to mean, roughly, "If you couldn't write the code/solve the problem yourself, you shouldn't be asking the LLM to do it for you." So while I find a lot of what he says in there disturbing (e.g. "I believe many programmers at this point have less impact they could have because they look at the code. [...] The working day is 8 hours. If you read the code, it is a tradeoff."), the ending bit kind of qualifies the rest of it in a way that I'm slightly less uncomfortable with, I guess?
Idk. Looking forward to hearing what the professionals on here think.
There is one type of cancerous coding pattern all models make, no matter how good the model is, and no matter how detailed the AGENTS.md. I found it's best described here: The models would rather...
There is one type of cancerous coding pattern all models make, no matter how good the model is, and no matter how detailed the AGENTS.md. I found it's best described here:
Present-day models tend to produce code that is too defensive, too complex, too local in its reasoning. They avoid strong invariants. They add fallbacks instead of making bad states impossible.
[...]
Furthermore it’s well understood that models tend to observe some local failure and add a local defense. Karpathy mentioned how they are “mortally terrified of exceptions”. In systems with important invariants, especially persisted data formats or core infrastructure, the right fix is not “handle every malformed case.” The right fix is to make the malformed case unrepresentable or impossible to write in the first place. Yet even with a lot of manual steering, that type of code does not come out of LLMs naturally, and even if the code comes out naturally like that, they will still attempt to handle now impossible errors.
The models would rather add a if (property !== null) check all over the code than just add that to the type definition and handle the null case in the single place it could possibly arise.
Until this is somehow improved, there is no way I'd ever prompt AI without fixing the generated code. Sometimes the model does a bad job and I have to refactor most of it myself, sometimes it does a good job and I dont. In any case, I don't think I've ever had a situation where I didn't have to first remove a bunch of useless defensive checks from the generated code.
It seems like the "best" models are even worse in this regard. I've tried GPT-5.5, but I keep going back to DeepSeek v4, because GPT keeps insisting on adding that stuff all over even when I remove it.
I've run into this a few times in a personal project that I've let the AI have relatively free reign over since the consequences are very low, and I get to learn about the kinds of mistakes it...
I've run into this a few times in a personal project that I've let the AI have relatively free reign over since the consequences are very low, and I get to learn about the kinds of mistakes it makes. The most egregious example of what you're describing was when built a redundant switch statement in 6 places, where in each case the local function only cared about 1-3 of the 12 cases and the rest after pass through. IMO the reason it happened was that it was an emergent pattern that occurred over a dozen or so sessions. Since each session was focused on a task, the agent started using the first occurrence as precedence and it then compounded. Once I pointed out the reasons why that pattern was fragile and unsustainable it auto recorded a memory and I guided it on how to refactor it, but I'm not convinced it won't happen again.
It has definitely happened to me in TypeScript when using a type from a third-party library which has a nullable property, but most of my code only had to accept the object if that property is not...
It has definitely happened to me in TypeScript when using a type from a third-party library which has a nullable property, but most of my code only had to accept the object if that property is not null
When I think about code becoming a hidden layer. I wonder, doesn't it make sense to have it write things as close to metal as possible? why have all this high level stuff that was designed to make...
When I think about code becoming a hidden layer. I wonder, doesn't it make sense to have it write things as close to metal as possible? why have all this high level stuff that was designed to make it easier to develop if were aren't doing that anymore.
One day maybe can we just bake the website into the server, in assembly.
Coding agents benefit from high-level languages too. For example, finicky high-level type systems will catch more bugs, and coding agents can usually fix them. So you want the sort of language...
Coding agents benefit from high-level languages too. For example, finicky high-level type systems will catch more bugs, and coding agents can usually fix them. So you want the sort of language where, if it compiles, it probably works. That's not going to be true of assembly.
Maybe antirez is some god-tier prompter, but I find myself constantly dissatisfied by the decisions that anthropic's frontier models make.
I think they are 100% great at automated reviews (relatively low cost to verify, I can overrule without negotiation, endless attention any time of the day, can be fed house review checklists). I think they are great sounding boards for design. Exploiting their encyclopedic knowledge of general techniques to direct research yields better designs faster than I could unassisted. I am just struggling with the idea that you can be so sure that your ideas are being faithfully reflected if you look never. That's not at all my experience. I am constantly finding inconsistencies and stupidity any time I take my eye off the underlying evolution of what's being spit out.
I'm a software engineer with 15+ years in the industry. Over the years, I have occasionally tried AI through whatever the Intellij IDEA Ultimate IDE offered when I was really stuck on something fairly self contained (e.g. sql query optimization). Every time I was disappointed and wondered how anyone got anything worthwhile out of AI.
Earlier this year, my boss got us all a 1-month trial to Claude Code (Opus 4.6 I think), and told us genuinely no-pressure "give it a shot, see if it helps". I went in very skeptically, with a baseline zero trust for the AI output. The first few tasks were self-contained, dead simple, and easily verifiable - stuff like "translate this legacy Twirl template to React". I reviewed every line it produced and tested every result. It did pretty well, so I started using it a bit more.
It's been a few months now and I find that I use Claude Code (now Opus 4.8, thinking high) for the majority of my work. I've created some tight agent files for the projects and a short global one based on actual usage and results. I've heard my process called "architecture driven development", though I don't really participate in the AI industry? culture? fandom? Detailed design docs / plans vastly improve the AI implementation results and subsequent review, like @stu2b50 mentioned. Anyway, here's what's gotten me good results:
8 - 10. Same as step #3-5, except for the implementation. In a clean session audit the implementation. I've got another skill for this based on what I'd usually ask it to do. Faithfulness to plan, trace impacted code flows, etc. Walk through findings, rinse and repeat as necessary.
Note that the various clean sessions are important, or having claude run the review skill with subagents in clean sessions. Aside from growing large contexts eventually degrading results, a cold reading of the plan and the implementation are key. They don't let old assumptions skip over investigating potential faults, dropped branches from investigation get brought back in, or anything like that.
If nothing else, it's a good way to force yourself to articulate and record all of your requirements, all of the desired flows, and account for "all" edge cases. Like extreme rubber-ducking.
EDIT:
Oh, and one other thing that I don't think people often consider: Maybe once every few months, ask the AI to review all of it's memories and agent files for consistency, redundancy, verbosity, and organization (different project memories, memory vs agent file, project agent file vs global agent file). Over time, especially if you've got multiple related projects, it'll accumulate a lot. Cleaning up can make a difference in result quality and context token usage.
This is an excellent workflow! Mine isn't a lot different. In general I'd argue for more review, both human and agent, but it really depends on the code/application. At least for production/user facing code.
One thing that's different... I don't use auto memory and I don't have agents write skills or subagent definitions. I've never been satisfied with their results in either making memories or building scaffolding. Personally I think the "self improving" angle is a trap, at least for the time being. Agents suck at it. Instead I manually formalize things that come up repeatedly into context files that are read/injected in various ways.
The value of this cannot be overstated, and this is more true with each generation of models. If you give them access to a comprehensive codebase with patterns, utilities and established architecture, they will do a reasonably good job of following conventions. Which removes a lot of opportunities to go off the rails.
For greenfield projects, even though you don't have to write code, it doesn't mean you can't! It's a great way to give agents a pile of conventions, patterns and design philosophies very quickly.
One thing I'd add to the list:
Always comforting to hear people arrived at a similar usages and conclusions. Either we're doing something right, or we're at least in the same wrong boat together!
Yeah, memories are a bit goofy. They seem to be useful short term, helping to bridge the gap between workdays and task switches, but can come back to bite weeks down the line when they're no longer in sync with the current state of the world. I try to get it to write as much as possible and useful to the plan files, and much more rarely to the agent files after hitting the same problem or deriving the same information over and over again.
For skills and the agent files, I had Claude write the prose, but built them with nearly the same flow as the feature flow. Detailed objective scope and requirements, known bits of information or where to gather them from, desired behavior, etc. up front, iterative back and forth, cold reading reviews by both me and Claude, and tweaks over time based on their performance. I think it's important for people to use the AI tools and harness as vanilla as possible for a while to find what fits you personally rather than cargo-culting a bunch of skills and prompts and agent file "hacks".
Definitely agree about the design docs for large areas of the code base, as well as the boundaries and interactions between code bases if your project has multiple repos/applications. Before I did that, anything non-trivial would either get confused if I didn't tell it explicitly where things lived, or would spend time and tokens poking around every time. For the same reason, I keep a database schema dump on hand so it can do some quick grepping to find most things.
Oh, and I just remembered another tidbit, agents are great with CLI tools. I rarely need to reach for some feature-subset MCP when the thing has a mature cli tool I can install. Plus I don't have to keep authentication info in some plaintext MCP config.
I'm pretty sure it's a good boat! In addition to using agents in my own workflows, I've setup solutions for other organizations, outside of software, and many of the same principles work well there too.
Also with each model generation, more and more of these concepts are being formalized onto fine tuning rather just the harnesses. Presumably there's a pretty high bar to make it into training.
Agreed, especially when you consider that many skills and prompts become outdated with newer models/harnesses. The more you understand how the tool works, and what its current limitations are, the better. If you only ever see how the agent behaves in scaffolding created by someone else, you're not building an accurate mental model.
Yes. I have a schema that lives in each project root that agents are instructed to read or grep for everything DB related. They also keep it updated with changes. That's another area with low error rates that I'm comfortable letting agents handle mostly unsupervised.
It sounds like we agree on pretty much everything. In my experience CLI tools (or sometimes APIs) beat MCP servers nearly all of the time.
It's a great boat, can I hope in with you guys. I want to share so much...
Please do
Ok. So point 8, is effectively the antithesis of what the proposed article is saying right? It says don't look at the code, you say "I look at the code". Planning, adversarial review, session clearing, it's all a big dance to try to get the AI to do what you'd like, but in the end you still aren't actually delegating in the way that the article is suggesting. I'll also give a concrete example of the type of thing that I see all of the time. I wanted claude to produce a network manager dispatch script to update a particular route on an interface. This is a 5 line prompt at best and definitely not worthy of the complete process you have described. I am using opus 4.8 max effort. It produced something like the following
Now the problem with this (ignore quoting, action checking, it's a snippet) is basically that on a network manager managed interface if $IP4_GATEWAY isn't populated it's because there isn't a default route yet. So trying to calculate one is pointless. When one shows up the script will be called again, and so it will be populated. This type of shit happens all of the time. AIs cannot keep enough in their brains afaict to stop just jamming in extra unnecessary backstops. They do it for scripts, they do it for functions, then never lean up or unify error handling, or trace architectural invariants. They just double up piles and piles of redundant checking, which they themselves then fail to keep in sync.
All this to say, nice process, but no amount of planning gets frontier models to produce code that I am happy with, because the gap isn't in understanding what I want in terms of functional requirements. The above script would have worked. It's just needlessly complicated.
Not GP, but in my opinion that's because the OP article is wrong. Not about everything, but definitely about the implication that you can skip review.
The classes of mistakes and errors you're talking about are definitely real, and far from the only ones.
Note that you can mitigate a lot of said issues with custom scaffolding and tooling. Just not effectively enough that you can forego review entirely. So far :|
I agree that the premise of the article was wrong. That was my conclusion (at the root of this thread). My summary of initial chain:
Me: I don't agree with article, AI often generates code I don't like.
DrStone: I too was disappointed, until I started doing X.
Me: I don't think X helps with my disappointment.
To be clear: I think X is a useful collection of techniques. If it isn't clear already, I use AI a lot, generate a lot of code with it, inspect a lot of code. I am not at the toe-dipping staging :)
Yeah, you're right, I do still look at the code a bit, which is what the article advocates completely against. That said, I've been looking at it a lot less than when I started, based on what I've actually seen it produce in my situation. In the beginning I was meticulously reviewing every line of code it produced, far more carefully than I would any human PR, even after our extensive interactive planning practically wrote the thing already. Over time, I've still looked, but substantially less, now more like what I would for a colleague's PR who has done well in the past. Mostly a sanity skim, a high level architecture and organization kind of eyeballing, for complex or critical changes. Just a minute or few. Small tweaks and bug fixes I might let pass straight through at this point.
I think the softening has come down to a few things (not just laziness or complacency). I've figured out a workflow that works for me and spends a lot of time planning the details up front with me driving; I've definitely caught things in this stage where it's planning to overbuild or hedge against impossible conditions. My general trust in the tool has improved after using it successfully on both small and large things for months. I've seen how the large, mature code base gives it a lot of prior art to work with that it usually follows. Auto-memories and, much more importantly, my agents files and design docs have explicitly codified a lot of architecture, convention, and more subtle rules, growing and sharpening over time, which help the agent do the "right" thing more often.
I don't think I'll ever go full vibe (I have trouble completely delegating even to humans), but it's trending in that direction.
Yeah, seconding. I have a /prune-docs skill which actively prunes both memories and developer documentation for redundancies, outdated bits and pieces, things that shouldn't be in there, etc.
Maintenance and code quality are key to getting great results.
AI is an accelerator and a multiplier. If your code's shit and your docs are shit, it'll multiply the shit. If it's good, it'll multiply the good.
The key is design docs. Lots of design docs. Always have the agent make a design doc first. Only have it start generating code once the details in the docs are agreeable. As a bonus it’s really nice having that kind of doc easily available.
Do you not find that docs produced by ai tend to miss the forest from the trees over indexing on minutiae to the point of rendering them nearly useless for humans (probably very good for other agents)?
Also, I don't know about you, but personally I find I arrive at leaner designs by stress-testing them against existing code. Sure, if it's something that fits an obvious archetype or design pattern you can probably anticipate a fair amount.
I find that AI basically never simplifies ever. Neither in design or code and without active anti entropy you end up in a bad spot, and no amount of design gets you lower entropy.
What kind of design docs are you advocating for? Technical, functional, architecture, or something else? Do you mean for each piece of functionality you build or high level system things? How big are the docs you prefer?
Im curious how it compares to our setup and my preferred method
My tip is to add this to AGENTS.md:
And in the first prompt I always ask a question. If it's about a bug it's "why did this happen?" If it's a feature request, I describe the change and say "what's involved in making this change?" If there's an API change then I ask "How should we change the API?" I don't tell it to go ahead until I'm pretty sure I understand what it will do.
Also, if it's a new project, I write a plan.md first and we go over it, making multiple passes. Then I ask "are there any important questions to answer before starting work?" And I edit the plan myself until it's good enough to start Phase 1, because coding agents tend to put in excessive detail.
I guess Claude Code has a /plan mode, but my coding agent doesn't and I find I don't need it. I just make sure to ask a question if I don't want it to start right away.
Then again I'm just writing web apps, not anything complicated.
Interesting idea. I'll give it a whirl. I tend to just roll with auto-mode because I've found empirically (literally drag racing the same problem) that by the time my colleagues have finished their glorious round of dialectics with some planning mode i have iterated my way to a better design and solution just by hammering on the problem and getting deeper into the problem than can be discovered by looking at from afar.
Well, I'm a hobbyist programmer so I have no deadlines and I'm not competing with anyone. So that's a different situation.
Also make sure to look into gitnexus.
Looks interesting but it doesn’t mention support for Deno. (Generic TypeScript support won’t do because import resolution is custom.)
The coding agent landscape has changed a lot in the past year, but in my experience it's still true that agents make dumb decisions. Less than before, but it's a long way from going away.
Because of that, if you're not keeping track of what it's doing, dumb decisions are going to end up coded into your project. You can mitigate some of the problem by creating detailed enough plans that there aren't a lot of decisions to make, but as soon as an agent runs into a blocker, it will start making decisions. Which you'll have no idea is happening if you aren't in the loop. This is actually more true of the current frontier models because they're trained to keep working through issues more aggressively than previous generations.
I agree with the author that's it's no longer necessary to review every single line of code, at least in some circumstances (boilerplate mostly). The latest models have much better taste than models from even 6 months ago. But I completely disagree with the framing that reviewing code slows down your velocity and therefore makes you worse. Velocity is not the only metric! I feel like this comes from a corporate "engineers are replaceable cogs" mindset. There are a lot of situations where going slow enough to maintain a mental model, reason sufficiently about choices and tradeoffs and continually update and maintain decent architecture are really really important. In those situations you're not "falling behind" by not cramming the maximum amount of coding into an 8 window, you're being a responsible engineer.
From the article:
I don't mean for this to be rude or offensive, and I wouldn't normally point this out, except for the subject matter.
This blog post is horribly written and very difficult to read from a grammar perspective. Its really disconcerting to see someone advocate so full-throatedly for a complete hands off vibe coding and code review setup if they don't even trust the technology enough to use it to proofread a blog post.
I get that the author isn't a native english speaker, but LLMs are explicitly good at translation. Its one of their main use cases, and understandability is one of the core metrics that they're trained on. I make a lot of allowances for people who don't natively speak the language, but not if you're an AI evangelist.
If you're not even using the thing to fix mistakes in your blog posts; you know, one of the few things that people almost universally agree that they're good at, it doesn't exactly lend a lot of weight to the central thesis of the post.
I noticed that too. I assume it was hastily written.
These days, though, the bad grammar is a good sign that he wrote it himself :)
Given how people react to AI generated prose, me included, I find myself feeling much more forgiving towards an ESL writer who didn't do an agent editing pass.
Speaking as an amateur, my approach has been to not touch LLMs for code help unless I'm absolutely desperate, and part of his closing vindicates that choice, I feel:
I take that to mean, roughly, "If you couldn't write the code/solve the problem yourself, you shouldn't be asking the LLM to do it for you." So while I find a lot of what he says in there disturbing (e.g. "I believe many programmers at this point have less impact they could have because they look at the code. [...] The working day is 8 hours. If you read the code, it is a tradeoff."), the ending bit kind of qualifies the rest of it in a way that I'm slightly less uncomfortable with, I guess?
Idk. Looking forward to hearing what the professionals on here think.
There is one type of cancerous coding pattern all models make, no matter how good the model is, and no matter how detailed the AGENTS.md. I found it's best described here:
The models would rather add a
if (property !== null)check all over the code than just add that to the type definition and handle the null case in the single place it could possibly arise.Until this is somehow improved, there is no way I'd ever prompt AI without fixing the generated code. Sometimes the model does a bad job and I have to refactor most of it myself, sometimes it does a good job and I dont. In any case, I don't think I've ever had a situation where I didn't have to first remove a bunch of useless defensive checks from the generated code.
It seems like the "best" models are even worse in this regard. I've tried GPT-5.5, but I keep going back to DeepSeek v4, because GPT keeps insisting on adding that stuff all over even when I remove it.
I've run into this a few times in a personal project that I've let the AI have relatively free reign over since the consequences are very low, and I get to learn about the kinds of mistakes it makes. The most egregious example of what you're describing was when built a redundant switch statement in 6 places, where in each case the local function only cared about 1-3 of the 12 cases and the rest after pass through. IMO the reason it happened was that it was an emergent pattern that occurred over a dozen or so sessions. Since each session was focused on a task, the agent started using the first occurrence as precedence and it then compounded. Once I pointed out the reasons why that pattern was fragile and unsustainable it auto recorded a memory and I guided it on how to refactor it, but I'm not convinced it won't happen again.
YES! Example from this evening.
I haven’t really seen this in TypeScript, perhaps because types are non-null by default and you have to write “| null” to make them nullable.
It has definitely happened to me in TypeScript when using a type from a third-party library which has a nullable property, but most of my code only had to accept the object if that property is not null
When I think about code becoming a hidden layer. I wonder, doesn't it make sense to have it write things as close to metal as possible? why have all this high level stuff that was designed to make it easier to develop if were aren't doing that anymore.
One day maybe can we just bake the website into the server, in assembly.
Coding agents benefit from high-level languages too. For example, finicky high-level type systems will catch more bugs, and coding agents can usually fix them. So you want the sort of language where, if it compiles, it probably works. That's not going to be true of assembly.