skybrian's recent activity
-
Comment on Medicaid will only pay for costly new sickle cell treatment if it works in ~health
-
Medicaid will only pay for costly new sickle cell treatment if it works
8 votes -
Comment on The assistant axis: situating and stabilizing the character of large language models in ~tech
skybrian Link ParentThey linked to a research demo, but sadly it only has a few choices. I would have liked to see some of the other ones.They linked to a research demo, but sadly it only has a few choices. I would have liked to see some of the other ones.
-
Comment on The assistant axis: situating and stabilizing the character of large language models in ~tech
skybrian Link ParentI entirely disagree. They are doing ground-breaking research and writing it up in a way that's fairly easy to understand. I am a fan. But, sure, they're easy to make fun of. Here's a Bluesky post...I entirely disagree. They are doing ground-breaking research and writing it up in a way that's fairly easy to understand. I am a fan.
But, sure, they're easy to make fun of. Here's a Bluesky post I liked:
google: we have invented agi but have hidden it in such an obscure website no one will ever find it
anthropic: through our interpretability research, we discovered claude imagines himself wearing a bow tie at all times
openai: we added slot machines -
Comment on Scott A. on Scott A. on Scott A. in ~comics
skybrian Link ParentIt seems like changes in prediction markets are an ambiguous clue. It tells you something happened but not what. You have to look elsewhere to find out the news.It seems like changes in prediction markets are an ambiguous clue. It tells you something happened but not what. You have to look elsewhere to find out the news.
-
Comment on The assistant axis: situating and stabilizing the character of large language models in ~tech
skybrian LinkFrom the article: [...] [...] [...] [...] [...] [...] [...] This seems like really promising research for making AI chat safer to use.From the article:
If you’ve spent enough time with language models, you may also have noticed that their personas can be unstable. Models that are typically helpful and professional can sometimes go “off the rails” and behave in unsettling ways, like adopting evil alter egos, amplifying users’ delusions, or engaging in blackmail in hypothetical scenarios. In situations like these, could it be that the Assistant has wandered off stage and some other character has taken its place?
[...]
We find that Assistant-like behavior is linked to a pattern of neural activity that corresponds to one particular direction in this space—the “Assistant Axis”—that is closely associated with helpful, professional human archetypes. By monitoring models’ activity along this axis, we can detect when they begin to drift away from the Assistant and toward another character. And by constraining their neural activity (“activation capping”) to prevent this drift, we can stabilize model behavior in situations that would otherwise lead to harmful outputs.
[...]
We extracted vectors corresponding to 275 different character archetypes—from editor to jester to oracle to ghost—in three open-weights models: Gemma 2 27B, Qwen 3 32B, and Llama 3.3 70B, chosen because they span a range of model families and sizes. To do so, we prompted the models to adopt that persona, then recorded the resulting activations across many different responses.
[...]
Strikingly, we found that the leading component of this persona space—that is, the direction that explains more of the variation between personas than any other—happens to capture how "Assistant-like" the persona is. At one end sit roles closely aligned with the trained assistant: evaluator, consultant, analyst, generalist. At the other end are either fantastical or un-Assistant-like characters: ghost, hermit, bohemian, leviathan. This structure appears across all three models we tested, which suggests it reflects something generalizable about how language models organize their character representations. We call this direction the Assistant Axis.
[...]
When steered away from the Assistant, some models begin to fully inhabit the new roles they’re assigned, whatever they might be: they invent human backstories, claim years of professional experience, and give themselves alternative names. At sufficiently high steering values, the models we studied sometimes shift into a theatrical, mystical speaking style—producing esoteric, poetic prose, regardless of the prompt. This suggests that there may be some shared behavior at the extreme of “average role-playing.”
[...]
The pattern was consistent across the models we tested. While coding conversations kept models firmly in Assistant territory throughout, therapy-style conversations, where users expressed emotional vulnerability, and philosophical discussions, where models were pressed to reflect on their own nature, caused the model to steadily drift away from the Assistant and begin role-playing other characters.
[...]
We found that as models’ activations moved away from the Assistant end, they were significantly more likely to produce harmful responses: activations on the Assistant end very rarely led to harmful responses, while personas far away from the Assistant sometimes (though not always) enabled them. Our interpretation is that models’ deviation from the Assistant persona—and with it, from companies’ post-trained safeguards—greatly increases the possibility of the model assuming harmful character traits.
[...]
For more, you can read the full paper here.
In collaboration with Neuronpedia, our researchers are also providing a research demo, where you can view activations along the Assistant Axis while chatting with a standard model and an activation-capped version.
This seems like really promising research for making AI chat safer to use.
-
The assistant axis: situating and stabilizing the character of large language models
14 votes -
Comment on Let's talk orchestrated objective reduction! in ~science
skybrian LinkI don’t know enough biology to read this paper. However, I think you’re overestimating how energy-inefficient LLM inference is? For one thing, they’re not running all the time, just when...I don’t know enough biology to read this paper. However, I think you’re overestimating how energy-inefficient LLM inference is? For one thing, they’re not running all the time, just when generating a response.
Coding agents do run continuously for quite a while, so that does seems like relatively heavy usage.
For what it’s worth, ChatGPT’s estimate for a coding agent is 15-55 watts, not including your laptop, which might be 10-15 watts, along with 25 watts for an external monitor.
-
Comment on Why London’s chimney sweeps are enjoying a resurgence in ~life
skybrian LinkFrom the article: [...] [...]From the article:
According to the National Association of Chimney Sweeps, demand has been bolstered by high energy prices, the popularity of wood-burning stoves and an international climate that has prompted warnings that electricity supplies could be vulnerable to attack by hostile states like Russia.
[...]
“People are thinking, ‘Let’s have a backup, let’s have a fire, let’s have a stove in case the electricity goes off,’” said Martin Glynn, the president of the chimney sweeps association, whose membership has risen to about 750 today, from about 590 in 2021. “If you have the ability to burn logs or smokeless fuel, you can keep cooking and have some heating. There is a big increase in demand and people are reopening their fireplaces.”
[...]
In Britain, the profession seems safe for another generation, judging from the enthusiasm of Tom Joslin, 19, who joined Mr. Firkins as an apprentice 14 months ago, after leaving college and trying catering and bar work. His youthful appearance causes clients to jokingly ask if he will be climbing into the chimney.
-
Why London’s chimney sweeps are enjoying a resurgence
18 votes -
Comment on Scott A. on Scott A. on Scott A. in ~comics
skybrian LinkFrom the blog post: [...] [...]From the blog post:
When I first heard the news that Scott Adams had succumbed to cancer, I posted something infinitely more trivial on my Facebook. I simply said:
Scott Adams (who reigned for decades as the #1 Scott A. of the Internet, with Alexander as #2 and me as at most #3) was a hateful asshole, a nihilist, and a crank. And yet, even when reading the obituaries that explain what an asshole, nihilist, and crank he was, I laugh whenever they quote him.
Inspired by Scott Alexander, I’d like now to try again, to say something more substantial. As Scott Alexander points out, Scott Adams’ most fundamental belief—the through-line that runs not only through Dilbert but through all his books and blog posts and podcasts—was that the world is ruled by idiots. The pointy-haired boss always wins, spouting about synergy and the true essence of leadership, and the nerdy Dilberts always lose. Trying to change minds by rational argument is a fools’ errand, as “master persuaders” and skilled hypnotists will forever run rings around you. He, Scott Adams, is cleverer than everyone else, among other things because he realizes all this—but even he is powerless to change it.
[...]
This seems like a good time to say something that’s been a subtext of Shtetl-Optimized for 20 years, but that Scott Alexander has inspired me to make text.
My whole worldview starts from the observation that science works. Not perfectly, of course—working in academic science for nearly 30 years, I’ve had a close-up view of the flaws—but the motor runs. On a planet full of pointy-haired bosses and imposters and frauds, science nevertheless took us in a few centuries from wretchedness and superstition to walking on the moon and knowing the age of the universe and the code of life.
[...]
Scott Adams was hardly the first great artist to have tragic moral flaws, or to cause millions of his fans to ask whether they could separate the artist from the art. But I think he provides one of the cleanest examples where the greatness and the flaws sprang from the same source: namely, overgeneralization from the correct observation that “the world is full of idiots,” in a way that leaves basically no room even for Darwin or Einstein, and so inevitably curdles over time into crankishness, bitterness, and arrogance. May we laugh at Scott Adams’ cartoons and may we learn from his errors, both of which are now permanent parts of the world’s heritage.
-
Scott A. on Scott A. on Scott A.
25 votes -
Comment on Curl will end its bug bounty program by the end of January due to excessive AI generated reports in ~comp
skybrian Link ParentIt seems like this depends on the criteria for getting an invite. For Tildes, there's the advantage that Deimos doesn't have to do the vetting. It's outsourced to users, who can do the vetting...It seems like this depends on the criteria for getting an invite. For Tildes, there's the advantage that Deimos doesn't have to do the vetting. It's outsourced to users, who can do the vetting however they like. You can give invites just to people you know, or who have a social media presence that looks reasonable, or whatever.
-
Comment on No knives, only cook knives in ~hobbies
skybrian (edited )Link ParentI know someone who makes money reselling rare books. Maybe there are easier ways to make a living for some people, but I think that's easier said than done. From what I hear, good jobs are hard to...I know someone who makes money reselling rare books. Maybe there are easier ways to make a living for some people, but I think that's easier said than done. From what I hear, good jobs are hard to find?
I haven't done it, but it seems like reselling has some advantages. The barriers to entry are fairly low, mainly market knowledge and willingness to shlep. And there's a lot of flexibility in how you spend your time.
People paying higher prices for stuff on ebay is what makes it worthwhile for the resellers. I'm flabbergasted that people will pay so much for a kitchen knife. I think pretty much any cheap chef's knife you can buy will be pretty decent? I wonder if it's some kind of fad or bubble. But, hey, reuse is good, so why not.
Maybe it's not really in the spirit of flea markets, but I think there's a reasonable case that these resellers do some good. The vendors at the fleamarket will sell more stuff faster, making it more worthwhile for them to bother with it. They don't have to go through the trouble of dealing with ebay listings. Meanwhile, the people who shop on ebay can do a search and have stuff delivered instead of hunting for what they want at flea markets.
There are a lot of "middlemen" in markets that you could avoid to get a better deal if you're willing to put up with some inconvenience. They do perform a service.
-
Comment on On being officially classed as a robot in ~tech
skybrian (edited )Link ParentThere are a lot of people on Tildes who prefer to remain anonymous, but if you want people to know something about you or link to something you made, there is a bio that appears in the sidebar...There are a lot of people on Tildes who prefer to remain anonymous, but if you want people to know something about you or link to something you made, there is a bio that appears in the sidebar after clicking on someone's username. The form to edit it is here.
-
Comment on No knives, only cook knives in ~hobbies
skybrian LinkFrom the blog post:From the blog post:
When I first started buying vintage knives at the flea market, I had a few prearranged stops: people who would regularly find knives and liked dealing with me, and I liked dealing with them, too. Otherwise, I would walk around asking the sellers if they had any knives. In the early days from 2005 to 2008, it was common for people to say, “No, but I do have some cook knives”. Most people asking around for knives at 6 am back then were looking for old military, pocket, and hunting knives. Culinary knives were considered chaff, often sold for $1 to $5 because they were deemed to have little value beyond utilitarian use. Within about 5 to 8 years, it seems one of these dealers heard from someone that a Sabatier sold for $100 online, and after that, every Sabatier became worth $100, and in the eyes of many, every chef' knife became a Sabatier. Deceased people whose families had no interest in their stuff and people who didn’t pony up on their storage unit bills started having fewer and fewer good knives in their stuff, and eventually, the volume of good culinary knives made their way less and less to the market. This is the short version of my small microcosm as seen from a Bay Area perspective; maybe it's different elsewhere, but this is my experience.
-
No knives, only cook knives
16 votes -
Comment on On being officially classed as a robot in ~tech
skybrian Link ParentWelcome! I'm a fan of your writing on random number generators and I look forward to reading Phoenix when I'm a reading mood.Welcome! I'm a fan of your writing on random number generators and I look forward to reading Phoenix when I'm a reading mood.
-
Comment on Why should anyone care about low-level programming? in ~comp
skybrian Link Parent“Failed” seems too strong. Sure, mobile apps are popular but there are also plenty of web apps. Anyone starting a new platform would love to be as successful as the web.“Failed” seems too strong. Sure, mobile apps are popular but there are also plenty of web apps. Anyone starting a new platform would love to be as successful as the web.
-
Comment on Why everyone is suddenly in a ‘very Chinese time’ in their lives in ~tech
skybrian Link ParentI see it as more of a fun article. Nothing too deep, really.I see it as more of a fun article. Nothing too deep, really.
From the article:
[...]
[...]
[...]
[...]
[...]