post_below's recent activity
-
Comment on The magic is in the language in ~tech
-
Comment on Share your favourite dal recipe! in ~food
post_below LinkLooking at the recipe that @phoenixrises posted, it looks great. You can do it a little quicker though, and still end up with great dal. Here are a few things I've learned... Butter/ghee and cream...Looking at the recipe that @phoenixrises posted, it looks great.
You can do it a little quicker though, and still end up with great dal. Here are a few things I've learned...
Butter/ghee and cream are key. Mostly that's just about fat and viscosity, so if you want it vegan instead you can use coconut milk (full fat with cream) and a neutral oil like refined coconut or avocado. Some recipes call for adding the coconut milk towards the end, similar to how you might do it with cream. I like to add it earlier, it helps mellow out the coconut taste.
You don't actually have to toast whole spices, or make your own garam masala. It does make a difference but it's pretty subtle. A good pre-made garam masala blend and a few other jarred spices gets you at least 90% there. Add the spices at the end of the veggie saute step to toast them a little.
Speaking of the veggie saute step, since the linked recipe doesn't include veggies... Consider sauteeing, at the least, finely chopped or grated onions in the recipe's called for oil before adding the other ingredients.
You can pre-cook the lentils in an instant pot (pre-soak totally optional) to save time.
The truly important ingredients are black lentils, some form of tomato (paste, diced, strained, etc..), cream/coconut milk, butter/oil, garlic, ginger and garam masala. You can just kind of experiment with everything else until you dial it in. I like to be conversative with the spices for dal, too much can distract from the staple soul food goodness of a fundamentally simple dish.
Oh and mash the lentils a bit towards the end with a potato masher or immersion blender, it improves the emulsification, texture and mouthfeel. You want to leave the majority of the lentils whole though.
-
Comment on What JavaScript framework should I use? in ~comp
post_below Link ParentThe way I see it, the main selling points of frameworks are standardization and compatibility. They make it easier to hire a [framework] developer to replace the previous one and they come with...The way I see it, the main selling points of frameworks are standardization and compatibility. They make it easier to hire a [framework] developer to replace the previous one and they come with varying levels of built in patterns that help stop interchangeable developers from making messes. Once upon a time they also helped solve cross browser compatibility, but these days, for most use cases, that isn't near as much of an issue as it once was. The last selling point is also outdated: adding functionality that's hard to implement in, for example, vanilla css/js. It's been a long time since I've encountered something that's significantly easier to do with a framework than it is with modern css/js. YMMV of course.
It's true that you'll end up writing something like a framework if you build a vanilla UI, but whether or not that takes more time depends on various factors. It definitely doesn't have to. And unless you really mess things up, your bespoke UI is going to be lighter weight than the framework alternative.
I'm not sure I see the maintainability angle. Are you thinking because many people aren't familiar enough with vanilla and so have a hard time understanding or writing it in a logical way? It certainly doesn't demand any bad patterns by nature.
Something I think it's easy to forget is that html/css/js essentially is a framework. One that's been iterated for the use case (websites/apps) for 30 years.
Not that frameworks aren't useful in lots of cases but, as nothing more than language constructs, they don't add as much as I think people sometimes assume.
Also: coding agents. The time to do X in a framework has now become basically identical to the time to do X without the framework. The remaining difference is load time on a slow connection and resource usage on a cheap device. In that context frameworks are just adding weight. Provided of course that the coding agent is in the hands of someone who is fluent in the underlying languages. Otherwise it could potentially be better to use a framework you're more familiar with, or with more good idioms than bad in the training data.
Anyway, I appreciate the digression, but my curiousity was more specific to @delphi's situation. It seems like an ideal opportunity to hand roll the UI instead of spending that time learning a new framework. Especially when a frontier model is almost guaranteed to give you a solid answer to every version of "how do I do this thing in vanilla js" you're likely to encounter.
-
Comment on What JavaScript framework should I use? in ~comp
post_below LinkWhy do you need a framework? HTML/CSS/JS can do all the things, but it sounds like you probably know that, so I'm just curious.Why do you need a framework? HTML/CSS/JS can do all the things, but it sounds like you probably know that, so I'm just curious.
-
Comment on 52% of Americans think their personal data will be breached. They're probably right. in ~comp
post_below LinkThat means at least 48% of Americans don't know their data has already been breached.That means at least 48% of Americans don't know their data has already been breached.
-
Comment on Four top Google AI researchers form new startup in ~tech
post_below Link ParentContext: Jeff Dean facts (which I just learned about) are a riff on Chuck Norris jokes. Highlights:Context: Jeff Dean facts (which I just learned about) are a riff on Chuck Norris jokes.
Highlights:
- Jeff Dean's PIN is the last 4 digits of pi.
- There is no Ctrl key on Jeff Dean's keyboard. Jeff Dean is always in control.
-
Comment on Inside Google’s $200bn Wall Street finance machine for Anthropic in ~finance
post_below Link ParentAlong with the crippled chatbots we'd also have a huge crash in global markets. A lot of industries are now looking healthy from a market perspective only because of AI related...Along with the crippled chatbots we'd also have a huge crash in global markets. A lot of industries are now looking healthy from a market perspective only because of AI related spending/investment/fomo
Whether that would cause a long lasting recession is anyone's guess, markets don't seem to follow the same rules they once did. At the moment these companies are propping up markets to as significant a degree as investment banking has in the past.
A crash, and a recession, might be the right thing in the long run, but there would be a lot of collateral damage to everyday people.
-
Comment on The magic is in the language in ~tech
post_below Link ParentThe relationship between cognition and language deserves its own thread because there's a lot of debate. I think some version weak linguistic determinism is self evidently what's happening, but...The relationship between cognition and language deserves its own thread because there's a lot of debate. I think some version weak linguistic determinism is self evidently what's happening, but the relationship between language and thought is so complex, and so hard to study, that it will be an open question for a long time.
-
Comment on As Reddit stock falls, CEO questions value of Google’s AI Overviews in ~tech
post_below Link ParentNot only is it walled off from the rest of the internet, it's controlled by a for profit company on its way to an IPO that will eventually enshittify.- Exemplary
Not only is it walled off from the rest of the internet, it's controlled by a for profit company on its way to an IPO that will eventually enshittify.
-
Comment on The magic is in the language in ~tech
post_below Link ParentIndeed. And it really is wild how quickly it starts to become normal. I'd have to stop and think about it for a while to count all of the things that happen on a given workday that would have been...this tech works in a way that I would’ve called laughably implausible a decade ago
Indeed. And it really is wild how quickly it starts to become normal. I'd have to stop and think about it for a while to count all of the things that happen on a given workday that would have been laughably implausible not very long ago.
-
Comment on Big food versus the people in ~health
post_below Link ParentExactly, the business model is addiction and food was a great way to expand into a market where no one thought of the products as addicting. And because no one thought of the products as...Exactly, the business model is addiction and food was a great way to expand into a market where no one thought of the products as addicting.
And because no one thought of the products as addicting, influencing policy and regulation was comparitively easy. For a long time the idea that highly processed food and refined carbs were a serious health issue was something that pretty much no one wanted to hear. Big food just needed to keep it that way. That didn't really start to change until the last couple of decades and they're still working hard to make sure policy doesn't catch up with the science.
-
Comment on The magic is in the language in ~tech
post_below Link ParentBefore fine tuning (which includes RL), next token prediction is a reasonable mental model, but fine tuning changes things quite a lot. You're right that there's a lot of capability embedded in...Before fine tuning (which includes RL), next token prediction is a reasonable mental model, but fine tuning changes things quite a lot. You're right that there's a lot of capability embedded in the weights.
Even with just pre-training, you might be surprised by the response to your example from a large model because the training creates complex and layered associations that already look like something more than next token prediction. A large model with no RL or other post training steps might not get the answer right, but it would "know" that it was a math question and its "guess" probably wouldn't be terrible because even if it had never seen those particular numbers in a sequence, it has relationships between a lot of numbers, and the shape of a lot of math problems, in training.
-
Comment on Big food versus the people in ~health
post_below LinkAdditional note: The list of 9 companies in the article should actually be 8. Kellogg's is owned by Ferrero and Mars (Mars has North American cereals, Ferrero has everything else).Additional note: The list of 9 companies in the article should actually be 8. Kellogg's is owned by Ferrero and Mars (Mars has North American cereals, Ferrero has everything else).
-
Comment on The magic is in the language in ~tech
post_below Link ParentYeah extreme interpretations both ways. It must be something we don't understand!Yeah extreme interpretations both ways. It must be something we don't understand!
-
Comment on Big food versus the people in ~health
post_below Link[...] Similar to the story of big tobacco, oil, healthcare (in the US) and pretty much every other mature industry with enough money to influence the system.Bad diets kill millions of people globally every year, and lead to tens of billions of dollars in health costs, often placing the heaviest burden on communities who are least able to afford it. Children, with limited decision-making power, are particularly vulnerable to this.
Legislators and government officials have worked to rein in this public health crisis by enacting laws that require food companies to be transparent about their ingredients and limit the advertising of unhealthy foods.
In public, the world’s biggest and richest food companies such as Coca Cola, PepsiCola, and Mondelez say they want to be part of the solution. But behind closed doors, they have taken governments to court to delay, dilute, and derail public health laws, which the companies say violate their rights.
[...]
Of the cases brought by private companies where the plaintiff was identifiable, more than 1 in 3 came from just nine parent groups, led by Coca-Cola, PepsiCo, and Mondelez
Their actions are not only prolonging the public health crisis but also cost countries billions of dollars in both legal and healthcare costs. In addition, they have a chilling effect on policymakers that wish to better their citizens’ health but do not have the resources to engage in drawn-out legal fights with food companies.
Similar to the story of big tobacco, oil, healthcare (in the US) and pretty much every other mature industry with enough money to influence the system.
-
Big food versus the people
19 votes -
Comment on The magic is in the language in ~tech
post_below Link ParentLanguage! Or more specifically the signal encoded in the relationship between bits of language after it's been used by humans to convey meaning. It's extracting traces of intelligence from our...what sort of informational relationship does an LLM manifest?
Language! Or more specifically the signal encoded in the relationship between bits of language after it's been used by humans to convey meaning. It's extracting traces of intelligence from our (mostly written) use of language.
-
Comment on The magic is in the language in ~tech
post_below Link ParentThat's true about RL, that's most of the fine tuning step. But mostly what's happening during that step is that the weights are being nudged to increase the strength of patterns that were already...That's true about RL, that's most of the fine tuning step. But mostly what's happening during that step is that the weights are being nudged to increase the strength of patterns that were already there.
And those patterns are intelligence encoded by humans in the course of communication.
-
The magic is in the language
In all the hype and debate and excitement and frustration around AI, I think sometimes the fact that we've achieved something really cool gets lost. I want to talk about that a bit, which I'll try...
In all the hype and debate and excitement and frustration around AI, I think sometimes the fact that we've achieved something really cool gets lost. I want to talk about that a bit, which I'll try to do in a simple way, without too much math or computer science.
There's a surprising amount of intelligence encoded in applied language. To me that's the most remarkable insight in LLM technology. Except it's not really surprising at all. The whole point of language is to encode intelligence. Communication, at its core, is about sharing intelligence. LLMs agents are a way of turning the intelligence encoded in language, particularly written language, into useful work.
You might imagine that language is just the starting point when building LLMs. That it gets converted into computer stuff and then it's not language anymore. But in a way it's language all the way down. Distill language into math and then run gradient descent on it and the intelligence (for lack of a better term) bubbles out.
Forgive that hyperbole, but there is something a little bit magical about it, and the technology is entirely unmagical. The magic is in the language.
At the core the tech is pretty simple. First you go and get all of the language. Just completely idiotic amounts of applied language. Because there's a threshold: if you don't use enough language you don't get a useful result. The more language you use, assuming it's good quality, the more useful the result.
After you have all of the language, you convert it into tokens, which are essentially pieces of words represented as numbers. You also compute a compressed version of the relationships between the tokens. The frontier labs don't say how much this costs publicly anymore but it's safe to say hundreds of millions in compute are required for this process. Each time you do it.
Once you've determined the token relationships, you've pre trained an LLM. You have the initial weights. At this point your LLM can output credible prose pretty reliably when you run inference on the weights. Meaning that you can take a given set of language tokens (a prompt) and determine which token should come next, over and over again, until you have a response. The fluency of the prose can be uncanny, and there are latent tendencies towards useful output, but you can't really do much with it yet.
The next step, fine tuning, modifies the weights in order to nudge the inference towards useful things like instruction following, tool use and reasoning. Or a simulacrum of reasoning. Fine tuning is a massive, multi step, iterative process that turns your LLM into a useful tool that can do more than output believable prose.
Something that's really fascinating to me about fine tuning is that what you're training is still the language relationship weights, and what comes out during inference is still language. But now it's language that is more likely to result in useful output and behaviors. Language is the substrate that LLMs reason through. A model that's more likely to infer the words "but what if I look at it another way" is more likely to consider multiple possibilities. A model that reliably infers language around tool use in appropriate situations is an agent. The behavior is encoded in the language.
After tens of millions in fine tuning costs, assuming you got it right, you have the core of a LLM that can function as an AI agent. You can put it in a harness and ask it to do things in the language of your choice and it will actually be able to do them a shockingly high percentage of the time. It can even do things that weren't explicitly part of the training. That's something which has never happened before outside of science fiction. Until recently it was one of those computer sciencey things that was probably going to happen someday, after we were all dead.
It sucks that this tech revolution is being driven by the capital class, and that it's happening as a mad dash for market domination and golden IPOs. In a more perfect world technology derived from collective human intelligence would be a public good rather than a profit driver.
But nevertheless, the ability to imprint some part of human intelligence into an autonomous agent is a remarkable achievement. One that we can't yet see the full size of. And it's possible not because we typed instructions into an interface, but because we've spent centuries encoding our intelligence into language.
38 votes -
Comment on Better things are possible in ~talk
post_below Link ParentRenewable Energy in Germany Exceeds Fossil Fuels (First time ever)Renewable Energy in Germany Exceeds Fossil Fuels (First time ever)
It's an interesting topic.
Assuming I'm reading this right, I want to clarify a couple points. One of the key features of LLM agents is that they can be useful even outside of the direct context they were trained in/on. They're more useful, for some things, when you're closer to the center of the training distribution but they can do something that looks like generalizing, some of the time, outside of the distribution
The reason for this appears to be that there's is more information encoded in the token relationships than you might imagine. One way to look at it is that words have both independent meaning and meaning that changes based on the context. Which means both that a word has self contained meaning that exists independently of any training data, and that dynamic meaning can be built on the fly out of word relationships as you write them. Because this is encoded in the weights, when you're running inference, meaning can survive into space that isn't specifically in the training. Even when there is no reference (so to speak) the model can blaze a path through the dark built out of token relationships and the meaning they encode. The words and the reasoning are the same thing, so they can go uncharted places.
It starts to fall apart at some point, but with a large enough model it can make it pretty far into the dark before that happens. That's especially true when it can generate feedback and incorporate the results into its inference, which is one of the defining characteristics of agents. The answer doesn't have to be in training for a model to find it as long as there's a way to test and iterate. In domains like code, math, biology and physics, deterministic testing is often easily accessible.