In all the hype and debate and excitement and frustration around AI, I think sometimes the fact that we've achieved something really cool gets lost. I want to talk about that a bit, which I'll try to do in a simple way, without too much math or computer science.

There's a surprising amount of intelligence encoded in applied language. To me that's the most remarkable insight in LLM technology. Except it's not really surprising at all. The whole point of language is to encode intelligence. Communication, at its core, is about sharing intelligence. LLMs agents are a way of turning the intelligence encoded in language, particularly written language, into useful work.

You might imagine that language is just the starting point when building LLMs. That it gets converted into computer stuff and then it's not language anymore. But in a way it's language all the way down. Distill language into math and then run gradient descent on it and the intelligence (for lack of a better term) bubbles out.

Forgive that hyperbole, but there is something a little bit magical about it, and the technology is entirely unmagical. The magic is in the language.

At the core the tech is pretty simple. First you go and get all of the language. Just completely idiotic amounts of applied language. Because there's a threshold: if you don't use enough language you don't get a useful result. The more language you use, assuming it's good quality, the more useful the result.

After you have all of the language, you convert it into tokens, which are essentially pieces of words represented as numbers. You also compute a compressed version of the relationships between the tokens. The frontier labs don't say how much this costs publicly anymore but it's safe to say hundreds of millions in compute are required for this process. Each time you do it.

Once you've determined the token relationships, you've pre trained an LLM. You have the initial weights. At this point your LLM can output credible prose pretty reliably when you run inference on the weights. Meaning that you can take a given set of language tokens (a prompt) and determine which token should come next, over and over again, until you have a response. The fluency of the prose can be uncanny, and there are latent tendencies towards useful output, but you can't really do much with it yet.

The next step, fine tuning, modifies the weights in order to nudge the inference towards useful things like instruction following, tool use and reasoning. Or a simulacrum of reasoning. Fine tuning is a massive, multi step, iterative process that turns your LLM into a useful tool that can do more than output believable prose.

Something that's really fascinating to me about fine tuning is that what you're training is still the language relationship weights, and what comes out during inference is still language. But now it's language that is more likely to result in useful output and behaviors. Language is the substrate that LLMs reason through. A model that's more likely to infer the words "but what if I look at it another way" is more likely to consider multiple possibilities. A model that reliably infers language around tool use in appropriate situations is an agent. The behavior is encoded in the language.

After tens of millions in fine tuning costs, assuming you got it right, you have the core of a LLM that can function as an AI agent. You can put it in a harness and ask it to do things in the language of your choice and it will actually be able to do them a shockingly high percentage of the time. It can even do things that weren't explicitly part of the training. That's something which has never happened before outside of science fiction. Until recently it was one of those computer sciencey things that was probably going to happen someday, after we were all dead.

It sucks that this tech revolution is being driven by the capital class, and that it's happening as a mad dash for market domination and golden IPOs. In a more perfect world technology derived from collective human intelligence would be a public good rather than a profit driver.

But nevertheless, the ability to imprint some part of human intelligence into an autonomous agent is a remarkable achievement. One that we can't yet see the full size of. And it's possible not because we typed instructions into an interface, but because we've spent centuries encoding our intelligence into language.

Hopefully the Tilderen aren't completely sick of the topic of AI writing.

To summarize my take: I think LLM agents are profoundly useful. And problematic. And I really really dislike AI writing when I encounter it outside of documentation. It's not only that it usually has a poor signal to noise ratio and a lot of annoying tropes, it also breaks a contract that has existed since writing was invented. And I dislike that it displaces human writing. Human writing is beautiful, we need more of it, not less.

By breaking a contract I mean that human writing implies an investment. The author invested time in creating it and has (sometimes) invested their reputation by publishing it. Whereas LLM prose requires minimal effort and thought. One great summary is:

"If you didn't take the time to write it, why should I take the time to read it?"

But it's not just about time or preference. We rely on signals of investment and competence to judge the value of information both personally and professionally. LLMs short circuit that completely. They incorporate signals of effort despite there being no effort involved. Signals that we could once rely on that someone has a deep understanding of a subject, or did extensive research, or is reasonably intelligent, or interesting, don't have any meaning when it's LLM prose.

Recently, while investigating Fable failure modes (while working in Claude Code), it outputted a summary that I think captures the situation really well: "My polish costs nothing".

Which is to say that LLM prose folds in signals of authority, competence, rigor and insight without any of those things needing to be present in the content. A fluent human is often worth listening to while a fluent LLM is just the default, the fluency carries no information. I think that's part of the reason for AI psychosis, I think it effectively uses, now outdated, expectations to hijack perception.

The reason that I sometimes take the time to investigate why an agent went wrong is partly to refine the scaffolding and partly to remind myself how dumb they are. Each generation is a little bit better, confidently right a little more than it's confidently wrong, so I make a point not to let myself fall into the trap of trusting their output too much.

I suppose I'm just posting to pass on the reminder.

Talking about software code, already a lot of it is written by ai and has a particular style.

Reminds me of when circuit boards used to be manufactured by human hands, they all had a unique style thats really fun to look at.

I wonder if in a few years my old software code will be like that, someone will look at it and go “oh, wow, this was written by hand”

Title, frankly. I’m looking for an old stock Humane Ai Pin (that stupid clothes pin gadget so bad it sunk the entire company that made it and then had the remaining employees move to the HP printer division, frankly a fate worse than death) to play around with. I recently learned that the sticker on the back covers four pins, which just expose standard USB, and it’s an Android device so in theory, it’s hackable.

I wanna try turning it into something more useful. It was such a cool little piece of hardware, although the software, pitch, whole idea really was of course threadbare to garbage. I guess I am still romanticising the Star Trek com badge experience.

Willing to pay up to 200 USD, excluding shipping to either the EU or continental US, whichever’s more convenient. Sorry if requests like this aren’t allowed, but I’m kinda drawing a blank. Even emailed the guy who designed it, but he doesn’t have any either.

Complete off the top of my head thing, no source/research, just anecdotes and pontificating.

As i've been doing some heavy editing today of a 300+ line SQL query, one thing I'm wondering about with these AI usage stats are how much of the "adoption" falls into things like intellisense suggestions.

There's two parts to this, with the first being just bad suggestions.

I've found them to be "okay" for something like F#, but for SQL, which has always required me to knife fight for an alias to begin with, they're just utter garbage.

I normally don't blow through my free github copilot suggestions in my month (I'm not in the code mines as much anyways), but I blew through it in a day of shitty SQL suggestions (and then just turned it off). This was last month, so not even while dealing with the current monster, and I'm left wondering how many people just have it vomit out useless stuff they change anyways.

The second part, is just the usual "you don't really know your tools".

While doing this query I considered turning it back on or using it. I have several CTE's for readability as this is a prototype but it necessitates an annoying pattern of taking the names of your columns, uplifting them to the next query select, and then summing and renaming them AGAIN such as SUM(COALESCE(a.example,0)) AS [example]

When you have 84 columns to do this with, it can be tempting to let AI notice the pattern and just do it. However it's not actually necessary, and now that they're clocking the tokens as we knew they would, I'm back to just using my multi line editor skills. Middle click select, some home/end to get a starting point, then ctrl+shift+arrows mixed with Alt+arrows and some copy paste and I'm mostly done. Few Ctrl + D's or straight up find and replaces and I'd edited 80 lines in maybe 30 seconds?

AI would've been a bit easier, and from what i've seen of MANY coders, something I suspect they only think is doable through AI. However much like how AI is getting people to use features they never even knew existed in their business tools, I wonder how much adoption on the coder side is the same. All these text editor helper tools many coders don't use (please keep all VIM manifestos under 400 pages) suddenly being automated out by the VASTLY more expensive AI.

And like some of the other AI solutions, as the money starts to hit the budgets, I think we'll see a lot more "Look you need to learn how to do this normally" (or "hey guess what we're removing from VS code!").

Not sure I have an overall point to this, but I'd be curious to hear what other people are seeing in their environments.

I currently work in an organization that is very AI forward. It is common for individuals to forward AI generated documents, meeting notes, or etc, with no critical thinking or review, in lieu of actual work.

This practice is insanely counter-productive, as it means that any good-faith attempt to interact with the individual pushing such documentation, really just pushes the burden of putting together said documentation onto the receiver, except now they also need to edit and verify the document they were forwarded.

I need a shorthand way to refer to this practice, that calls it out as a bad practice.

A few months ago I found an article that explained that it was bad manners to reply into a conversation anything akin to the phrase "I asked ChatGPT and it said X", for exactly the reason mentioned above. Can anyone find a link? I can't seem to find it.

This article (https://tombedor.dev/human-attention-and-human-effort/) seems to hit the nail on the head, though it does so so succintly and at such a surface level, I don't think it really gets the point across. The reason we use books as reference points for knowledge, is because they are difficult to make, and therefore we trust that the author put real work into ensuring their work was credible. If we knew they did not, their work would not be credit worthy. Neither is an unreviewed AI generated message. By this rule, the more obviously something is AI generated, the less likely it is worth reading.

I would love a law (like Conway's law is a law), that said something like: "It is never worthwhile to spend more time reading a document, than it took to write." that I could point people at when they send me AI slop, with explanations of the above.

Is anyone aware of such a thing or website?

It's been annoying me lately how often I see people declare that certain blog posts or articles are AI-generated due to the presence of certain "tell-tale signs," (emdashes being the #1 example). It annoys me even if I agree that the thing in question is probably is AI-generated—mainly because I really like emdashes and I use them in my own writing all the time, and it grinds my gears knowing that some percentage of the population will assume I used AI because "hurr durr emdash == AI."

I'm not necessarily talking about Tildes. I'm sure I've seen it on here, but the worst offender by far is Hacker News where often one of the top comments on a linked article that has an emdash somewhere in the body will be a snarky comment about how it's clearly AI generated.

Anyway, I blew off some steam this weekend by making this silly satirical website/art project called GENIUS AI Detector that makes me giggle, so I'm sharing it in the hopes that it brings a bit of joy to any fellow emdash users who are likewise depressed about the shadow that has been cast over one of my favorite punctuation marks.

I haven't been closely following the releases of new models and the research papers that sometimes go along with them. So I was wondering: is power consumption ever seriously talked about by OpenAI, Google, Anthropic, and others? Do we get some specific numbers of how much power their models actually consume to produce 100 tokens? The cost of training some of these models? Or we're not yet at that stage yet and nobody cares and at best we can just get really rough estimates based on "trust me bro" tweets by their respective CEO's?

Some time ago, I came across this post. GPT-OSS (2OB, 120B) are meager, yet energy efficient models and I was surprised that the power consumption is still larger than what I had estimated before.

For 120B to generate a 1000 tokens (which is really not a lot) it would take up around 83 Wh of electricity. For context my PC consumes ~100Wh when idling. So it's almost equivalent as leaving my PC on for an hour. Considering that proprietary models are TRILLIONS of parameters and probably not as energy efficient the true power consumption of these models is concerning. Of course, these big data centers do a lot of things to maximize the efficiency that this "test" fails to do. But even if you half the energy consumption it's still significant given their size and their pervasiveness in handling everyday, trivial tasks.

I haven't come across similar posts or studies for newer open source models, so if somebody has, please share.

Also, I don't seem to fully understand how does context size fit into all of this. The larger the context size the more power it would take to produce those 100 tokens?