Shades of Google dropping “don’t be evil” (yeah, yeah, now it’s do the right thing in a different document).
Shades of Google dropping “don’t be evil” (yeah, yeah, now it’s do the right thing in a different document).
Anthropic, the wildly successful AI company that has cast itself as the most safety-conscious of the top research labs, is dropping the central pledge of its flagship safety policy, company officials tell TIME.
In 2023, Anthropic committed to never train an AI system unless it could guarantee in advance that the company’s safety measures were adequate. For years, its leaders touted that promise—the central pillar of their Responsible Scaling Policy (RSP)—as evidence that they are a responsible company that would withstand market incentives to rush to develop a potentially dangerous technology.
This wouldn't have anything to do with Anthropic now working on AI models for the Pentagon that by necessity can not be non-harmful, and therefore infringe on some element of safety for somebody, no?
This wouldn't have anything to do with Anthropic now working on AI models for the Pentagon that by necessity can not be non-harmful, and therefore infringe on some element of safety for somebody, no?
I don't think it matters that much. A gun is an inherently dangerous object, a military is similarly so, so if your AI is working plans with a military, somebody's risk is already a negotiable....
I don't think it matters that much. A gun is an inherently dangerous object, a military is similarly so, so if your AI is working plans with a military, somebody's risk is already a negotiable.
More to your point about superintelligence, XKCD made a comic where the author is more worried about what certain people would be empowered to do with an autonomous fleet of kill drones that will follow their orders than if the drones decided to wipe out humanity or maximize paperclips without orders. There's a much richer history of malice on one side of that equation than the other.
Pete Hesgeth recently threatened to cut Anthropic from current and future DOD contracts unless they drop some of their safety measures. This is likely part of their response to that pressure....
Defense officials warned they could designate Anthropic a supply chain risk or use the Defense Production Act to essentially give the military more authority to use its products even if it doesn’t approve of how they are used.
The point I'm surprised ... no, not surprised, I guess, just--somehow--even more disappointed over... Snowden was less than 15 years ago. Today, the Pentagon is threatening to blacklist Anthropic,...
The point I'm surprised ... no, not surprised, I guess, just--somehow--even more disappointed over...
Snowden was less than 15 years ago.
Today, the Pentagon is threatening to blacklist Anthropic, explicitly, for not giving them full use of their AI, for A) fully autonomous, AI powered targeting & strike capabilities, and 2) unrestricted, fully autonomous AI powered mass surveillance of US citizens.
This is not a whistleblower thing, it's not a reporter "scoop", nothing.
The US Pentagon is flat-out openly stating that it will destroy an AI company if it can't use the AI for mass spying on the US public.
(Oh yeah ... and killing people w/o human oversight)
Underlying reasons aside, I really don't trust any of the AI companies as far as I can throw a data center and I was already sceptical on their ideas of AI safety. I don't consider much of their...
Underlying reasons aside, I really don't trust any of the AI companies as far as I can throw a data center and I was already sceptical on their ideas of AI safety. I don't consider much of their "research" to be anything more than AI fan fiction and have already had my rants on thier papers about AI introspection and AI blackmail.
I keep seeing pieces about how these companies "can't turn off their AIs" or how "they don't even understand how it works" and even how their LLMs are in the top percentiles of Maths Olympiads and Coders. I literally typed "What is 784×413 698×225 786×2÷15" into Google and the first AI generated answer was:
My crappy desk calculator shows 9 764 167 711 513,6. Do I trust the cheap legacy hardware that gives my the same answer over and over and goes into error if I use it incorrectly. Or the multi billion dollar AI that gives me 5,329,240,721,280 the second time and then this whole mess even though the correct answer shows up in the "thinking" as:
You could argue that I should prompt better or that it's up to me to verify the outputs. But this is an all powerful genius level everything machine that has already cost tens of thousands of people their incomes. It should be able to do basic maths.
What does this have to do with AI Safety?
Everything really.
Because I don't think the AI apocalypse will happen because of some major military operation where they turn off the AI safeguards and the machines go crazy. I think it'll happen because the machines are going to misinterpret some critical semantic point and give the wrong person the wrong information at the worst possible time and they make a bad decision with it.
If you can't interrupt a system while it's on bad rationalization pathways, then you have a bad system. If you don't understand how the system is reasoning or reaching outcomes, then you probably have no handle on the inputs and training data. If your system needs audits to identify errors in extended outputs, then maybe it should not be in live environments.
The math question is mainly one of syntax. It interpreted “413 698” as 413 * 698, which is fair enough. Using spaces instead of commas in arithematic syntax is highly irregular. Generally when you...
The math question is mainly one of syntax. It interpreted “413 698” as 413 * 698, which is fair enough. Using spaces instead of commas in arithematic syntax is highly irregular. Generally when you have two numerals next to each, multiplication is assumed. Eg, you parse 5x as 5 * x not “fifty x”.
Kind of apples and oranges isn’t it? That’s Google’s quick and dirty LLM designed to skim web results. If you ask a frontier model a question you will have a radically different experience. Also,...
I literally typed "What is 784×413 698×225 786×2÷15" into Google and the first AI generated answer was:
Kind of apples and oranges isn’t it? That’s Google’s quick and dirty LLM designed to skim web results. If you ask a frontier model a question you will have a radically different experience. Also, you gave it a pretty ambiguous question.
Shades of Google dropping “don’t be evil” (yeah, yeah, now it’s do the right thing in a different document).
Cool everything is accelerated in this era, even the speed at which we lose 'don't be evil'. I hate this timeline.
This wouldn't have anything to do with Anthropic now working on AI models for the Pentagon that by necessity can not be non-harmful, and therefore infringe on some element of safety for somebody, no?
Since they were just given an ultimatum by the Pentagon, I can't see how this wouldn't be connected.
I don’t think it’s that kind of safety they are taking about. This is more like, superintelligence risk.
I don't think it matters that much. A gun is an inherently dangerous object, a military is similarly so, so if your AI is working plans with a military, somebody's risk is already a negotiable.
More to your point about superintelligence, XKCD made a comic where the author is more worried about what certain people would be empowered to do with an autonomous fleet of kill drones that will follow their orders than if the drones decided to wipe out humanity or maximize paperclips without orders. There's a much richer history of malice on one side of that equation than the other.
Yeah, I just don’t think they ever had a policy of not training bots with the raw intelligence to commit atrocities with human assistance.
Pete Hesgeth recently threatened to cut Anthropic from current and future DOD contracts unless they drop some of their safety measures. This is likely part of their response to that pressure.
Money talks.
The point I'm surprised ... no, not surprised, I guess, just--somehow--even more disappointed over...
Snowden was less than 15 years ago.
Today, the Pentagon is threatening to blacklist Anthropic, explicitly, for not giving them full use of their AI, for A) fully autonomous, AI powered targeting & strike capabilities, and 2) unrestricted, fully autonomous AI powered mass surveillance of US citizens.
This is not a whistleblower thing, it's not a reporter "scoop", nothing.
The US Pentagon is flat-out openly stating that it will destroy an AI company if it can't use the AI for mass spying on the US public.
(Oh yeah ... and killing people w/o human oversight)
Good catch, that probably is a major factor.
Underlying reasons aside, I really don't trust any of the AI companies as far as I can throw a data center and I was already sceptical on their ideas of AI safety. I don't consider much of their "research" to be anything more than AI fan fiction and have already had my rants on thier papers about AI introspection and AI blackmail.
I keep seeing pieces about how these companies "can't turn off their AIs" or how "they don't even understand how it works" and even how their LLMs are in the top percentiles of Maths Olympiads and Coders. I literally typed "What is 784×413 698×225 786×2÷15" into Google and the first AI generated answer was:
My crappy desk calculator shows 9 764 167 711 513,6. Do I trust the cheap legacy hardware that gives my the same answer over and over and goes into error if I use it incorrectly. Or the multi billion dollar AI that gives me 5,329,240,721,280 the second time and then this whole mess even though the correct answer shows up in the "thinking" as:
You could argue that I should prompt better or that it's up to me to verify the outputs. But this is an all powerful genius level everything machine that has already cost tens of thousands of people their incomes. It should be able to do basic maths.
What does this have to do with AI Safety?
Everything really.
Because I don't think the AI apocalypse will happen because of some major military operation where they turn off the AI safeguards and the machines go crazy. I think it'll happen because the machines are going to misinterpret some critical semantic point and give the wrong person the wrong information at the worst possible time and they make a bad decision with it.
If you can't interrupt a system while it's on bad rationalization pathways, then you have a bad system. If you don't understand how the system is reasoning or reaching outcomes, then you probably have no handle on the inputs and training data. If your system needs audits to identify errors in extended outputs, then maybe it should not be in live environments.
The math question is mainly one of syntax. It interpreted “413 698” as 413 * 698, which is fair enough. Using spaces instead of commas in arithematic syntax is highly irregular. Generally when you have two numerals next to each, multiplication is assumed. Eg, you parse 5x as 5 * x not “fifty x”.
Kind of apples and oranges isn’t it? That’s Google’s quick and dirty LLM designed to skim web results. If you ask a frontier model a question you will have a radically different experience. Also, you gave it a pretty ambiguous question.
GPT 5.2 Thinking said:
Eg:
Interpreting your expression as:
784 \times 413698 \times 225786 \times 2 \div 15
Result (exact):
\frac{48,820,838,557,568}{5}
Decimal:
9,764,167,711,513.6