My company has some departments deep in this mode. We recently had a project where we made a little calculator page to help you decide if it made sense to rent one of our industrial equipment...
My company has some departments deep in this mode.
We recently had a project where we made a little calculator page to help you decide if it made sense to rent one of our industrial equipment products or go buy one outright. The numbers skew toward renting making more sense if you use the thing only 8 months of the year or less, or if you are in a location likely to damage the equipment (think forestry or working on electrical lines going through mountains).
When it came time for us to get the estimates of ownership costs over time, the department told us that it was not worth spending a couple of hours querying the data that we actually have on maintenance and damage costs by industry, opting to instead ask ChatGPT during the call and just sticking with what it said as gospel.
I checked later and it was off by a factor of two to three depending on industry, so the made up numbers actually encourage people to go buy a product from our competition rather than rent from us.
This in particular resonated with my workplace experience over the last couple of years:
This in particular resonated with my workplace experience over the last couple of years:
On several occasions, we’ve been exposed to folks that have been sort of lukewarm on our main offerings, but they really, really wanted to use AI to perform a natural language query on their data. And we thought “Okay, if you really want to see it, maybe we can caveat this appropriately and show you what it might look like.”
This was a terrible mistake. It backfired in the most predictable way imaginable – every lukewarm client that saw the chatbot in action, even with us telling them that it was not going to accomplish what they wanted, wanted to buy it immediately. Every other consideration, including millions of dollars that we could plausibly help them achieve by non-AI means, was swept aside. It was like a dark and terrible force seized control of their limbs, plunged their hands into their own chests, and presented their still-beating credit cards to us in grim supplication. We were so mortified by the inexplicable shift in energy that we (wisely) declined to take the money and ended the sales process, and soon thereafter removed Cortex from our list of demonstrations. It would have been too irresponsible to exploit this gap in their reasoning, and frankly, it was already irresponsible to have even run the demonstration – doctors don’t walk around showing off cool pills that they’d never prescribe.
Watching the total 180°, that shift from ice-cold to red-hot buying frenzy, was a deeply unsettling experience. It was personally uncomfortable to see people that clearly didn’t gel with us interpersonally suddenly dying to enter an ongoing relationship, but more broadly uncomfortable because for a brief moment I began to understand what is happening in sales meetings around the world.
It’s good writing and this consulting thing seems to be working out for him, but when I read one of his articles, I wonder how it is that Suresh is invariably reporting what’s going on inside the...
It’s good writing and this consulting thing seems to be working out for him, but when I read one of his articles, I wonder how it is that Suresh is invariably reporting what’s going on inside the worst companies.
It’s a great essay, though I wonder if the author has a slightly biased sample because they are a consultant. In my experience, there’s a decent number of successful companies that just take the...
It’s a great essay, though I wonder if the author has a slightly biased sample because they are a consultant. In my experience, there’s a decent number of successful companies that just take the “if it ain’t broke, don’t fix it” approach. Those companies are slow to adopt AI, cautious in their approach, and also much less likely to hire external consultants to fix internal problems. I think companies meaningfully spending on consultants have the same culture of aggressive change that would lead to the described approach to AI.
People have a hard time changing, and I think that’s the undercurrent in this entire piece.
But the broader reality is so much worse: people who have no background in the technology at all actually believe what they are saying.
Consider the perspective of someone with no technical background in AI. For nearly thirty years, their exposure to production-ready software is as follows. When it works, it works in extremely predictable ways, and when it doesn’t, it usually breaks in incredibly obvious ways. That’s the condition, after testing and QA and all the normal software best practicies.
“Prouduction-ready” LLMs and generative AI will never, ever meet that bar. You can tell that to a non-technical person, but those years of prior experience will make real persuasion difficult, even if they accept that these are random technologies at face value. That prior of consistent, predictable tech leads to these rollout strategies.
This was a terrible mistake. It backfired in the most predictable way imaginable – every lukewarm client that saw the chatbot in action, even with us telling them that it was not going to accomplish what they wanted, wanted to buy it immediately.
I work at a small consulting company, and while I haven’t seen these behaviors exhibited by our customers, I have seen it brainwash most of my team. At the start of the year everyone was rational...
I work at a small consulting company, and while I haven’t seen these behaviors exhibited by our customers, I have seen it brainwash most of my team. At the start of the year everyone was rational and reasonable. Some interested in AI, others unenthused. It’s now no longer acceptable to be outspoken against AI. I think the top down push is rooted in a mixture of fear and excitement. Fear that we’ll be put out of business by another consultancy that’s maximizing usage of AI. And excitement from the addictive feedback loop of AI agents.
I mean that, over the course of the engagement, these people have exhibited a pattern of behavior that has made it near-impossible to sell to them without incurring reputational and legal risk, and are furthermore crafting management environments that I can only describe as cultish, ineffective, and “please dear God, do not let it be on earth as it is on LinkedIn”.
My company has some departments deep in this mode.
We recently had a project where we made a little calculator page to help you decide if it made sense to rent one of our industrial equipment products or go buy one outright. The numbers skew toward renting making more sense if you use the thing only 8 months of the year or less, or if you are in a location likely to damage the equipment (think forestry or working on electrical lines going through mountains).
When it came time for us to get the estimates of ownership costs over time, the department told us that it was not worth spending a couple of hours querying the data that we actually have on maintenance and damage costs by industry, opting to instead ask ChatGPT during the call and just sticking with what it said as gospel.
I checked later and it was off by a factor of two to three depending on industry, so the made up numbers actually encourage people to go buy a product from our competition rather than rent from us.
And this is The Output of a 2026 LLM. There’s people who stopped using their brain for decisionmaking with GPT-4 already lmao
This in particular resonated with my workplace experience over the last couple of years:
It’s good writing and this consulting thing seems to be working out for him, but when I read one of his articles, I wonder how it is that Suresh is invariably reporting what’s going on inside the worst companies.
It’s a great essay, though I wonder if the author has a slightly biased sample because they are a consultant. In my experience, there’s a decent number of successful companies that just take the “if it ain’t broke, don’t fix it” approach. Those companies are slow to adopt AI, cautious in their approach, and also much less likely to hire external consultants to fix internal problems. I think companies meaningfully spending on consultants have the same culture of aggressive change that would lead to the described approach to AI.
People have a hard time changing, and I think that’s the undercurrent in this entire piece.
Consider the perspective of someone with no technical background in AI. For nearly thirty years, their exposure to production-ready software is as follows. When it works, it works in extremely predictable ways, and when it doesn’t, it usually breaks in incredibly obvious ways. That’s the condition, after testing and QA and all the normal software best practicies.
“Prouduction-ready” LLMs and generative AI will never, ever meet that bar. You can tell that to a non-technical person, but those years of prior experience will make real persuasion difficult, even if they accept that these are random technologies at face value. That prior of consistent, predictable tech leads to these rollout strategies.
I work at a small consulting company, and while I haven’t seen these behaviors exhibited by our customers, I have seen it brainwash most of my team. At the start of the year everyone was rational and reasonable. Some interested in AI, others unenthused. It’s now no longer acceptable to be outspoken against AI. I think the top down push is rooted in a mixture of fear and excitement. Fear that we’ll be put out of business by another consultancy that’s maximizing usage of AI. And excitement from the addictive feedback loop of AI agents.
Great post, loved this part in particular: