In effect they simply decide to just silently give you nonsense in a random field they don't like you exploring. I don't particularly like the model refusing to do something but if it at least...
In effect they simply decide to just silently give you nonsense in a random field they don't like you exploring.
I don't particularly like the model refusing to do something but if it at least tells you then fine.
This is unlikely to affect even most people who will be using Fable 5 but the problem is transparency and this making it impossible even as normal user to have any trust in an output of a tool whose accuracy was already problematic.
And they can decide to do this to absolutely any prompt.
The scenario is follows:
I pay a company to access their compute and violate their ToS. The sane outcome is followup proportionate reaction and refusal to allow the access.
The reaction they are saying they will do is to bill as usual and output nonsense. What I am missing that makes this legal?
“We’re changing Fable 5’s safeguards for frontier LLM development to make them visible,” Anthropic said in a statement to WIRED. “We made the wrong tradeoff and we apologize for not getting the balance right.”
…
Anthropic now says it’s changing course, and that Claude Fable 5’s safeguards for AI development will be visible to users. If the company suspects a user is trying to use Claude to build a highly capable AI it will alert them that it’s either refusing the request, or rerouting the user to a less capable model.
Anthropic reversed the policy after it received fierce backlash from the AI research community. Anthropic has already taken steps to limit competitors from using Claude to build closed and open source AI models, but critics say that quietly degrading the model’s performance for certain users went a step too far. Claude’s coding agent has become a favored tool among developers, including those working on open-source AI research projects, and researchers tell WIRED that the company’s latest policy could have led to a troubling future in which only a handful of leading AI labs could perform advanced AI research.
…
“These safeguards prevent foreign adversaries from using our most capable models in ways that pose severe safety risks. The US and its allies hold an edge in frontier chips and the highly optimized software that runs them at full potential,” the company said in a statement to WIRED. “These safeguards ensure Claude isn't used to erode that advantage—by optimizing chips developed by those adversaries, for example […] In deciding whether to make them visible or invisible we faced a choice. A hidden safeguard is harder to probe and work around. This means the safeguards can be targeted much more narrowly.”
Anthropic says that because this safeguard around AI development is now visible, it needs to cast a wider net, meaning more benign requests may trigger its safeguards. The company says it’s working to make its classifiers more precise as quickly as possible.
This was a pretty distressing policy, especially after seeing how bad they are at detecting intent in other areas of Fable’s safeguards. Who knows how many of my queries would have been mistagged...
This was a pretty distressing policy, especially after seeing how bad they are at detecting intent in other areas of Fable’s safeguards. Who knows how many of my queries would have been mistagged and therefore sabotaged without my knowing?
Definitely motivating me to experiment more with Qwen code so I can run it all local.
If they're aiming for the level of effectiveness of their next best model (Opus) on the limited tasks, then I don't think is too different from them keeping the best models for themselves (which...
If they're aiming for the level of effectiveness of their next best model (Opus) on the limited tasks, then I don't think is too different from them keeping the best models for themselves (which is a thing I have complex feelings about, but I can't imagine not doing that in their shoes if I thought there was a chance of AI being good enough to start doing recursive self improvement). Though they did fumble the communication hard by not emphasizing this if they are in fact aiming for Opus's level of effectiveness.
I do expect to see more of the top labs cautiously holding back their best stuff as AI gets better at programming.
In effect they simply decide to just silently give you nonsense in a random field they don't like you exploring.
I don't particularly like the model refusing to do something but if it at least tells you then fine.
This is unlikely to affect even most people who will be using Fable 5 but the problem is transparency and this making it impossible even as normal user to have any trust in an output of a tool whose accuracy was already problematic.
And they can decide to do this to absolutely any prompt.
The scenario is follows:
I pay a company to access their compute and violate their ToS. The sane outcome is followup proportionate reaction and refusal to allow the access.
The reaction they are saying they will do is to bill as usual and output nonsense. What I am missing that makes this legal?
This is America. Everything is legal if you have enough money.
That's The Golden Rule!
Whoever Has the Gold Makes the Rules
Anthropic Walks Back Policy That Could Have ‘Sabotaged’ AI Researchers Using Claude (Wired)
…
…
This was a pretty distressing policy, especially after seeing how bad they are at detecting intent in other areas of Fable’s safeguards. Who knows how many of my queries would have been mistagged and therefore sabotaged without my knowing?
Definitely motivating me to experiment more with Qwen code so I can run it all local.
If they're aiming for the level of effectiveness of their next best model (Opus) on the limited tasks, then I don't think is too different from them keeping the best models for themselves (which is a thing I have complex feelings about, but I can't imagine not doing that in their shoes if I thought there was a chance of AI being good enough to start doing recursive self improvement). Though they did fumble the communication hard by not emphasizing this if they are in fact aiming for Opus's level of effectiveness.
I do expect to see more of the top labs cautiously holding back their best stuff as AI gets better at programming.
https://lobste.rs/s/f2fwny/if_claude_fable_stops_helping_you_you_ll
What's old is new again.