skybrian's recent activity
-
Comment on Europa Universalis V review – even hardened grand strategy veterans may be startled by the intricacy of this historical simulation in ~games
-
Comment on OpenAI moves to complete potentially the largest theft in human history in ~tech
skybrian LinkThat's quite the headline. Is it justified? From the article: ... This is all speculative value - what stockholders think it's worth, and they could be wrong. But still. On the other hand: ... For...That's quite the headline. Is it justified? From the article:
the nonprofit will now only hold equity OpenAI claims is valued at approximately $130 billion [...] as opposed to its previous status of holding the bulk of the profit interests in a company valued at (when you include the nonprofit interests) well over $500 billion
...
Even if we thought the new control rights were as strong as the old, we would still be looking at a theft in excess of $250 billion, and a plausible case can be made for over $500 billion. I leave the full calculation to others.
This is all speculative value - what stockholders think it's worth, and they could be wrong. But still.
On the other hand:
the nonprofit, after the theft and excluding control rights, will have an on-paper valuation only slightly lower than the on-paper value of all of Anthropic.
...
If OpenAI can successfully go public at a $1 trillion valuation, then depending on how much of that are new shares they will be selling the nonprofit could be worth up to $260 billion.
For comparison, the Gates Foundation had about $77 billion at the beginning of the year.
-
OpenAI moves to complete potentially the largest theft in human history
20 votes -
Comment on Current studies may overestimate microplastics transferring from containers to food in ~food
skybrian Link ParentYeah, it's hardly the last word. They made some recommendations on how to do better studies.Yeah, it's hardly the last word. They made some recommendations on how to do better studies.
-
Comment on Signs of introspection in large language models in ~tech
skybrian (edited )Link ParentYeah, that's a good point. They're giving the LLM a big hint that it might have been manipulated and that's going to affect the vocabulary it uses to describe what happened. But they are also...Yeah, that's a good point. They're giving the LLM a big hint that it might have been manipulated and that's going to affect the vocabulary it uses to describe what happened.
But they are also directly manipulating its internal data structures and proved that it's looking at its internal data structures to answer the question, and not just guessing based on the text. (They tested it both with and without doing the manipulation.) This indicates that they've learned something about how LLM's work, though with caveats about how their understanding might be a bit off.
It's not just noticing low-probability text. It's noticing that and... something else.
-
Comment on Utah to open detention camp/involuntary treatment center for homeless people (gifted link) in ~society
skybrian Link ParentYes, but there are probably a lot more who aren’t like that. If you can’t get enough of them to accept free housing and medical care, maybe there is something wrong with it. If it’s a good place...Yes, but there are probably a lot more who aren’t like that. If you can’t get enough of them to accept free housing and medical care, maybe there is something wrong with it.
If it’s a good place to live, word should get around.
-
Comment on The last PCB you'll ever buy — rapid PCB prototyping with pre-made vias in ~tech
skybrian LinkGithub link: https://github.com/opulo-inc/viagridGithub link: https://github.com/opulo-inc/viagrid
Viagrid is a PCB template that allows for rapid PCB prototyping with 2-layer boards and factory-made vias. The blank boards have a standardized array of vias already in them; you just select which net they're connected to by cutting out the front and back layers of your board.
-
Comment on Offbeat Fridays – The thread where offbeat headlines become front page news in ~news
skybrian Link ParentMatt Levine commented on this last week: ...Matt Levine commented on this last week:
We have talked a few times about a guy in London who keeps snails in boxes to avoid taxes. The theory is that if a property is used for agriculture, it can avoid some local property taxes, and "snail farming" is the minimum amount of agriculture you can do to avoid taxes. This is an extremely funny theory that an extremely funny guy put into practice in a bunch of office buildings.
It does, however, have one flaw, which is that it is not true. Eventually the local property tax authorities will get around to suing you, and when they do, you will go to court and be like "lol snails" and the judge will be like "come on" and you'll have to pay the taxes. A reader pointed out to me a 2021 Queen's Bench case finding oh come on this is a sham:
...
The guy keeps doing snail farming, because (1) come on it's funny and (2) you never know if you might find a local council that won't sue you. But the theory behind it is suspect. Anyway. Nothing here is ever tax advice, is one lesson here. Another is that if you find a tax arbitrage that sounds like it might work, but doesn't work, but that involves breeding snails in office buildings, you might do it anyway for sheer love of the game.
-
Comment on New research shows attention lapses due to sleep deprivation coincide with a flushing of fluid from the brain in ~science
skybrian LinkFrom the article: ...From the article:
During sleep, the cerebrospinal fluid that cushions the brain helps to remove waste that has built up during the day. In a 2019 study, Lewis and colleagues showed that CSF flow during sleep follows a rhythmic pattern in and out of the brain, and that these flows are linked to changes in brain waves during sleep.
That finding led Lewis to wonder what might happen to CSF flow after sleep deprivation. To explore that question, she and her colleagues recruited 26 volunteers who were tested twice — once following a night of sleep deprivation in the lab, and once when they were well-rested.
...
Sleep-deprived participants performed much worse than well-rested participants on these tasks, as expected. Their response times were slower, and for some of the stimuli, the participants never registered the change at all.
During these momentary lapses of attention, the researchers identified several physiological changes that occurred at the same time. Most significantly, they found a flux of CSF out of the brain just as those lapses occurred. After each lapse, CSF flowed back into the brain.
“The results are suggesting that at the moment that attention fails, this fluid is actually being expelled outward away from the brain. And when attention recovers, it’s drawn back in,” Lewis says.
-
New research shows attention lapses due to sleep deprivation coincide with a flushing of fluid from the brain
33 votes -
Comment on US judges say Donald Trump administration must continue food aid during US shutdown in ~society
skybrian LinkFrom the article: ... But the practical implications seem unclear: Will SNAP benefits be paid in November? (CNN) ... ...From the article:
The Massachusetts judge said the administration must access the contingency funding to pay the benefits and had until Monday to report back to the court on whether they will authorise at least partial benefits for November.
Massachusetts US District Judge Indira Talwani wrote in her decision that the states who sued are likely to win in court on their claim that "Congress intended the funding of SNAP benefits, at a reduced rate if necessary, when appropriated funds prove insufficient".
Judge Talwani also wrote that the Trump administration "erred in concluding" that the USDA is blocked by law from tapping the emergency reserves in the contingency fund when there is a lapse in federal funding.
The USDA had said those reserves were insufficient to pay full benefits, which cost $8.5bn to $9bn each month. Agriculture Secretary Brooke Rollins had said she would only use the fund for an emergency such as a natural disaster.
...
Separately in Rhode Island, a lawsuit was brought by a number of US cities and NGOs who sued over what they called the "unlawful suspension" of the programme.
"There is no doubt and it is beyond argument that irreparable harm will begin to occur if it hasn't already occurred in the terror it has caused some people about the availability of funding for food, for their family," US District Judge John McConnell said.
But the practical implications seem unclear:
Will SNAP benefits be paid in November? (CNN)
It’s now unclear whether the administration will appeal the rulings. However, the Justice Department signaled during a hearing on Thursday that it would do so if US District Judge Indira Talwani in Boston issued an adverse ruling.
...
If only the contingency fund were tapped, the USDA would have to reduce benefits for all SNAP recipients, which it has never done, according to a declaration filed by the agency official who oversees the program. Also, state agencies would need to recode their systems to issue the smaller benefit amounts, which could result in payment errors.
...
For at least some states, the process could take anywhere from a few weeks to several months, Penn said, drawing upon informal conversations with state agencies.
Before states could even act, the USDA would also have to determine how to calculate and authorize the reduced allotments. This effort could be hampered if its employees have been furloughed during the shutdown or laid off or departed amid the Trump administration’s efforts to downsize the federal government, Plata-Nino said.
-
US judges say Donald Trump administration must continue food aid during US shutdown
43 votes -
Comment on Blogging in ~comp
skybrian LinkIt looks like Working Copy is an iOS app for GitHub with a built-in text editor. How do you like the editor?It looks like Working Copy is an iOS app for GitHub with a built-in text editor. How do you like the editor?
-
Comment on Signs of introspection in large language models in ~tech
skybrian (edited )Link ParentOne of the strange things about these models is that they’re non-human, but they speak English, and they use the same vocabulary that people use to talk about themselves. So to do the experiment,...One of the strange things about these models is that they’re non-human, but they speak English, and they use the same vocabulary that people use to talk about themselves. So to do the experiment, they had to prompt it with things like, “tell me what word you think about when…” as if it had thoughts about words.
Just using this vocabulary is implicitly making an analogy between whatever the LLM does and what humans do.
They also adopt that analogy in the discussion of the experiment and use scare quotes around words like “thoughts.” So, people have thoughts and an LLM has “thoughts.”
I appreciate the use of scare quotes, but maybe that’s not enough? How could they avoid making that assumption? I suppose they could invent an entirely separate technical vocabulary, but it would make it a lot harder to understand the research.
Using human vocabulary to talk about computers is nothing new. We commonly talk about how much memory a computer has, and nobody thinks twice about it. We understand that words have multiple uses.
On the other hand, we talk about a computer’s CPU and calling it the computer’s “brain” sounds weird. (I do remember articles directed kids or a non-technical audience that did that when computers were less familiar.)
So introducing technical vocabulary is an option. If it’s widely used, LLM’s will pick it up, too.
-
Comment on How Bill Gates is reframing the climate change debate in ~enviro
skybrian Link ParentSouth Korea has a surprisingly dramatic political history, but I don’t think anyone would deny that they’re a rich country now? That seems like an economic success story? Why wouldn’t you say it’s...South Korea has a surprisingly dramatic political history, but I don’t think anyone would deny that they’re a rich country now? That seems like an economic success story? Why wouldn’t you say it’s a success?
-
Comment on Weekly US politics news and updates thread - week of October 27 in ~society
skybrian LinkUS Senate votes to reject Trump’s global tariffs on more than 100 countries This seems symbolic since it would also need to pass the House and then would almost certainly be vetoed. Still, it’s...US Senate votes to reject Trump’s global tariffs on more than 100 countries
Four Republicans joined with all Democrats to vote 51-47 on a resolution to end the base-level tariffs that the president put into place via executive order.
It was the third time the Republicans have voted alongside Democrats on a tariff resolution this week, previously rallying to end tariffs targeting Brazil and Canada.
This seems symbolic since it would also need to pass the House and then would almost certainly be vetoed. Still, it’s nice to see the Senate rehearsing standing up to Trump.
-
Comment on Signs of introspection in large language models in ~tech
skybrian (edited )LinkFrom the article: I’ve said many times that making up plausible answers is most likely and asking an AI why it did something is a waste of time, so it will be interesting to read what they found…...From the article:
Have you ever asked an AI model what’s on its mind? Or to explain how it came up with its responses? Models will sometimes answer questions like these, but it’s hard to know what to make of their answers. Can AI systems really introspect—that is, can they consider their own thoughts? Or do they just make up plausible-sounding answers when they’re asked to do so?
I’ve said many times that making up plausible answers is most likely and asking an AI why it did something is a waste of time, so it will be interesting to read what they found…
Our new research provides evidence for some degree of introspective awareness in our current Claude models, as well as a degree of control over their own internal states. We stress that this introspective capability is still highly unreliable and limited in scope: we do not have evidence that current models can introspect in the same way, or to the same extent, that humans do. Nevertheless, these findings challenge some common intuitions about what language models are capable of—and since we found that the most capable models we tested (Claude Opus 4 and 4.1) performed the best on our tests of introspection, we think it’s likely that AI models’ introspective capabilities will continue to grow more sophisticated in the future.
How did they do it?
[…] we can use an experimental trick we call concept injection. First, we find neural activity patterns whose meanings we know, by recording the model’s activations in specific contexts. Then we inject these activity patterns into the model in an unrelated context, where we ask the model whether it notices this injection, and whether it can identify the injected concept.
…
It is important to note that this method often doesn’t work. Even using our best injection protocol, Claude Opus 4.1 only demonstrated this kind of awareness about 20% of the time. Often, it fails to detect injected concepts, or gets confused by them and starts to hallucinate (e.g. injecting a “dust” vector in one case caused the model to say “There’s something here, a tiny speck,” as if it could detect the dust physically). Below we show examples of these failure modes, alongside success cases. In general, models only detect concepts that are injected with a “sweet spot” strength—too weak and they don’t notice, too strong and they produce hallucinations or incoherent outputs.
They also write about forcing the chatbot to say “bread” when it made no sense (in which case it normally says it was an accident) versus making it say “bread” and also injecting the “bread” concept (in which case it sometimes confabulates a reason).
This behavior is striking because it suggests the model is checking its internal “intentions” to determine whether it produced an output. The model isn't just re-reading what it said and making a judgment. Instead, it’s referring back to its own prior neural activity—its internal representation of what it planned to do—and checking whether what came later made sense given those earlier thoughts. When we implant artificial evidence (through concept injection) that it did plan to say "bread," the model accepts the response as its own. While our experiment is conducted involves exposing the model to unusual perturbations, it suggests that the model uses similar introspective mechanisms in natural conditions.
…
We also found that models can control their own internal representations when instructed to do so. When we instructed models to think about a given word or concept, we found much higher corresponding neural activity than when we told the model not to think about it (though notably, the neural activity in both cases exceeds baseline levels–similar to how it’s difficult, when you are instructed “don’t think about a polar bear,” not to think about a polar bear!). This gap between the positive and negative instruction cases suggests that models possess a degree of deliberate control over their internal activity.
You can argue about whether this really counts as thinking, but it seems we’re a long way from “stochastic parrots?” Using developer tools, you can always make up a chat transcript, putting words in the AI character’s mouth, but it might notice!
They don’t know why it happens yet:
An interesting question is why such a mechanism would exist at all, since models never experience concept injection during training. It may have developed for some other purpose […]
-
Signs of introspection in large language models
24 votes -
Comment on Good News - A thread and a challenge in ~news
skybrian LinkOn Bluesky, this tag is a good source of this kind of news: https://bsky.app/hashtag/ShareGoodNewsTooOn Bluesky, this tag is a good source of this kind of news:
-
Comment on Utah to open detention camp/involuntary treatment center for homeless people (gifted link) in ~society
skybrian Link ParentSorry, that was unclear. What I am asking is why does it need to be involuntary? If it's voluntary then they need to maintain some minimum standard of care to get people to stay.Sorry, that was unclear. What I am asking is why does it need to be involuntary? If it's voluntary then they need to maintain some minimum standard of care to get people to stay.
Bret Deveaux wrote about it too: