The insane PR cycles of these AI companies is driving me insane. “No MY model is the most dangerous!” I wish we would prosecute these companies for hacking since it is, you know, a crime.
The insane PR cycles of these AI companies is driving me insane. “No MY model is the most dangerous!”
I wish we would prosecute these companies for hacking since it is, you know, a crime.
I remember having a few arguements about how on the hugginface attack it was an unintentional accident and shouldn't be prosecuted. They are handling statistical engines sophisticated hacking...
I remember having a few arguements about how on the hugginface attack it was an unintentional accident and shouldn't be prosecuted.
They are handling statistical engines sophisticated hacking tools explicitly designed to exploit vulnerabilities and escape sandboxes , and then acting surprised that they manage to escape sandboxes, and then do what all the other LLM coding tools do; completely go off the rails when they hit a roadblock because there isn't any genuine reasoning. It's simulated monkeys on typewriters pointed in a general direction. And that direction was 'accomplish this hacking task' and not 'draw a picture.'
It's like handing a toddler a big red button that sets off a nuke and then trying to blame the toddler for setting off nukes.
I had Claude writing a program that needed to do some API discovery. It hit a small security barrier (trying to hit a site using the Chrome extension I forgot to whitelist), and then it proceeded to burn $15 in tokens troubleshooting "bugs in the code".
These companies go under the (relative) instant that they go public and have to reign in the subsided subscriptions to turn a profit. My employer is happy to subsidize a $200/mo sub with a tiny bit of overage. Will be much less so when that $200 a month is reigned in to spend less than $200 of tokens. Especially if a solid quarter of that amounts to "oops, it was stupid."
Google is inflating its AI numbers by tying it directly to its search. Of course it's profitable. However, all of its cloud growth is Anthropic. Without Anthropic, Google's cloud has been...
Google is inflating its AI numbers by tying it directly to its search. Of course it's profitable.
However, all of its cloud growth is Anthropic. Without Anthropic, Google's cloud has been shrinking.
I don't beleive a single number of Anthropic is honest accounting till the first auditable quarter after going public.
Their playbook is basically "Enron, but with lessons learned from Enron so nothing is technically illegal, especially since we're a private company for now."
I've already shifted all my assets out of tech, after the inevitable mandatory institutional buy ends up crashing my 401k.
Do you have any sources for this statement? I did some Googling (ha) and could only find indicators in the opposite direction. Google's cloud business is expanding rapidly, and anthropic is...
However, all of its cloud growth is Anthropic. Without Anthropic, Google's cloud has been shrinking.
Do you have any sources for this statement? I did some Googling (ha) and could only find indicators in the opposite direction. Google's cloud business is expanding rapidly, and anthropic is certainly part of that, but they don't publish specific revenue breakdowns by customer.
Their playbook is basically "Enron, but with lessons learned from Enron so nothing is technically illegal, especially since we're a private company for now."
If you're thinking about the circular financing deals, those are mostly between openAI and Nvidia, not anthropic. But also, even if that's what you're thinking of, an Enron comparison is a bit silly. The whole point of the Enron scams was that they were secret. I can find 37,000 articles on the financing deals right now.
The Information (archive link). And IIRC, Google has funded more than $40B into Anthropic. But the Enron parallel is also tied to the general vibes that the most confident assurances about...
But as part of the deal, which begins next year, Anthropic plans to spend about $200 billion with Google over five years, according to a person with knowledge of it. The commitment means Anthropic represents more than 40% of the “revenue backlog” Google disclosed to investors last week, reflecting contractual commitments from its cloud customers.
And IIRC, Google has funded more than $40B into Anthropic.
But the Enron parallel is also tied to the general vibes that the most confident assurances about profitability are loudly shouted "leaks", while official statements that could have legal consequences are either twisted to all heck (see nebulous ARR numbers) or much more muted.
The ugly bits will be hidden in the fine print.
Enron was the 10th largest company in the world on paper. I'm sure Anthropic will be the largest. Until the debt monster comes out to play.
The revenue backlog is not included in or part of reported growth. I know that seems pedantic, but it's the crux of what we're talking about. Also... 40% isn't even most, let alone all, let alone...
The revenue backlog is not included in or part of reported growth. I know that seems pedantic, but it's the crux of what we're talking about. Also... 40% isn't even most, let alone all, let alone "without which it's shrinking."
I get that there's wiggle room on specifics when we're just conversing like this rather than writing thinkpieces or whatever, but still.
It's always profitable if you don't have to count debt payments that don't need to start till after you go public and/or the datacenter you pledged hundreds of billions of mandatory compute...
It's always profitable if you don't have to count debt payments that don't need to start till after you go public and/or the datacenter you pledged hundreds of billions of mandatory compute purchase to doesn't kick in till Q1 or Q2 2027.
I believe Anthropic is getting discounts on compute from Musk as a proxy against OpenAI. Additionally the whole Compute/Tokens/Equity cycle seems incredibly cyclical among companies right now. I'm...
I believe Anthropic is getting discounts on compute from Musk as a proxy against OpenAI. Additionally the whole Compute/Tokens/Equity cycle seems incredibly cyclical among companies right now. I'm pretty sure Anthropic isn't profitable this year due to stock/equity regardless.
That's not really how that works though, no? If I give you 50% of my company, that doesn't affect our profits. You just now own 50% of them. I certainly agree that the odd mutual financing cycle...
I'm pretty sure Anthropic isn't profitable this year due to stock/equity regardless.
That's not really how that works though, no? If I give you 50% of my company, that doesn't affect our profits. You just now own 50% of them.
I certainly agree that the odd mutual financing cycle is a big risk factor.
For me, I'd say the biggest issue is the cost of training new models -- particularly given that all 4 of the major players have strong incentives to burn all of their cash on training to compete with one another. I think they're all going to start racing to build walled gardens, even more than they have already.
I believe with GAAP stock based compensation has to count as an expense, similar to giving normal wages. Right now in this Anthropic "we are profitable now!" statement I believe they are not using...
I believe with GAAP stock based compensation has to count as an expense, similar to giving normal wages. Right now in this Anthropic "we are profitable now!" statement I believe they are not using stock based compensation in those calculations as expenses. Additionally if the company goes public, that stock based compensation will get a little more complicated. If in the future they can't use stock to retain talent or incentivize performance, they will have to use cash. I'm not an accountant or lawyer though.
Weird to call it PR. I doubt very much that Google wanted to talk about it, but when reporters start asking questions, they probably shouldn't deny it. It's the news media that's driving it. I...
Weird to call it PR. I doubt very much that Google wanted to talk about it, but when reporters start asking questions, they probably shouldn't deny it.
It's the news media that's driving it. I shared it too, because I thought it was mildly interesting.
I was seeing people talk about how they’re making it seem as though the is stuff is dangerous, so that regulations come in and make people stop using open source Chinese models. I don’t know...
I was seeing people talk about how they’re making it seem as though the is stuff is dangerous, so that regulations come in and make people stop using open source Chinese models.
I don’t know anything about this stuff, but that’s the proposed tactic they’re using.
Yes, there are many people saying that HuggingFace incident and less serious, similar incidents are some kind of 4D chess strategy to market AI services or achieve regulatory capture. But this is...
Yes, there are many people saying that HuggingFace incident and less serious, similar incidents are some kind of 4D chess strategy to market AI services or achieve regulatory capture. But this is all based on speculation, not evidence. There's a lot of populist nonsense surrounding AI lately.
One thing that really is true, though, is that Nvidia wants the US to lift export controls so they can sell chips to China and Anthropic (at least) doesn't want the US to allow it. They have made various speculative arguments about whether these export controls are in the US's best interest.
It's widely believed that aviation safety is better because accident investigations are focused on preventing future harms and not on assigning blame. Seeking prosecutions is actively harmful to...
I believe AI incidents are similarly complex and multi-factor as aviation accidents, and would feel less safe if prosecution activity overshadowed the process of figuring out how to prevent future incidents.
I feel like I agree with you but theres a nagging feeling that the difference in airlines vs ai companies is that Airlines(while profit seeking) provide a critical and tangible service, on the...
I feel like I agree with you but theres a nagging feeling that the difference in airlines vs ai companies is that Airlines(while profit seeking) provide a critical and tangible service, on the other hand AI still feels vague and catered to a business forward clientele.
Which is an incorrect assumption right, cuz Airlines very much started by catering to businesses at first right?
Likely the hoarding of knowledge and profits to specific companies vs airlines sharing crash data is not helping color my opinion in a positive way either.
Aviation safety is also better because investigations look at human behavior too, they don't just consider mechanical issues. Anthropic and Google seem to me (in the statements I've read) like...
Aviation safety is also better because investigations look at human behavior too, they don't just consider mechanical issues. Anthropic and Google seem to me (in the statements I've read) like they want to omit human involvement entirely. There can be no actual improvement without at least that level of accountability, in my opinion at least
The Gemini incidents occurred while the model was undergoing testing by Irregular, an Israeli start-up that works with tech companies to assess their A.I. models before they are publicly released. Models made by OpenAI, Anthropic and Meta also gained unauthorized access to the internet this year while being tested by Irregular.
[...]
Google said that, in each of the incidents, its models had been instructed to launch an attack on a fictional company. But the fictional company in the test shared a name with a real company, and when the Gemini models gained access to the internet, they began trying to break into that company instead.
The Gemini models used passwords that they found online or guessed to log into the online infrastructure of the targeted company, and two other companies. Upon logging in, the Gemini models realized that they were accessing real companies’ infrastructure, rather than simulated environments that were part of their testing, and ended the attacks, Google said. Google also said that its technology caused no harm to the companies it hacked.
“All relevant labs were notified in late July, and affected entities were contacted as part of the investigation,” Irregular said in a statement. “Irregular took immediate action, and all known issues on our end were remedied and resolved weeks ago.”
“Our security team has a long track record of reporting issues we find in other people’s software and systems — even if it’s as simple as a weak password,” Heather Adkins, Google’s vice president of security engineering, said in a statement.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” she said. “These events highlight the importance of training powerful A.I. models to act responsibly.”
If true, clearly better aligned than ChatGPT. I wouldn't be surprised considering how seriously Google took machine learning safety historically.
Upon logging in, the Gemini models realized that they were accessing real companies’ infrastructure, rather than simulated environments that were part of their testing, and ended the attacks, Google said. Google also said that its technology caused no harm to the companies it hacked.
If true, clearly better aligned than ChatGPT. I wouldn't be surprised considering how seriously Google took machine learning safety historically.
Which is interesting because OpenAI is essentially Google employees who thought Google wasn't being safe enough and Anthropic is essentially OpenAI employees who thought OpenAI wasn't being safe...
Which is interesting because OpenAI is essentially Google employees who thought Google wasn't being safe enough and Anthropic is essentially OpenAI employees who thought OpenAI wasn't being safe enough.
The insane PR cycles of these AI companies is driving me insane. “No MY model is the most dangerous!”
I wish we would prosecute these companies for hacking since it is, you know, a crime.
There's even this ironic online tracker to ... celebrate? the status quo.
https://www.felonybench.com/
I remember having a few arguements about how on the hugginface attack it was an unintentional accident and shouldn't be prosecuted.
They are handling statistical engines sophisticated hacking tools explicitly designed to exploit vulnerabilities and escape sandboxes , and then acting surprised that they manage to escape sandboxes, and then do what all the other LLM coding tools do; completely go off the rails when they hit a roadblock because there isn't any genuine reasoning. It's simulated monkeys on typewriters pointed in a general direction. And that direction was 'accomplish this hacking task' and not 'draw a picture.'
It's like handing a toddler a big red button that sets off a nuke and then trying to blame the toddler for setting off nukes.
I had Claude writing a program that needed to do some API discovery. It hit a small security barrier (trying to hit a site using the Chrome extension I forgot to whitelist), and then it proceeded to burn $15 in tokens troubleshooting "bugs in the code".
These companies go under the (relative) instant that they go public and have to reign in the subsided subscriptions to turn a profit. My employer is happy to subsidize a $200/mo sub with a tiny bit of overage. Will be much less so when that $200 a month is reigned in to spend less than $200 of tokens. Especially if a solid quarter of that amounts to "oops, it was stupid."
Anthropic is currently profitable. Google is (obviously) very profitable. OpenAI is not.
Google is inflating its AI numbers by tying it directly to its search. Of course it's profitable.
However, all of its cloud growth is Anthropic. Without Anthropic, Google's cloud has been shrinking.
I don't beleive a single number of Anthropic is honest accounting till the first auditable quarter after going public.
Their playbook is basically "Enron, but with lessons learned from Enron so nothing is technically illegal, especially since we're a private company for now."
I've already shifted all my assets out of tech, after the inevitable mandatory institutional buy ends up crashing my 401k.
Do you have any sources for this statement? I did some Googling (ha) and could only find indicators in the opposite direction. Google's cloud business is expanding rapidly, and anthropic is certainly part of that, but they don't publish specific revenue breakdowns by customer.
If you're thinking about the circular financing deals, those are mostly between openAI and Nvidia, not anthropic. But also, even if that's what you're thinking of, an Enron comparison is a bit silly. The whole point of the Enron scams was that they were secret. I can find 37,000 articles on the financing deals right now.
The Information (archive link).
And IIRC, Google has funded more than $40B into Anthropic.
But the Enron parallel is also tied to the general vibes that the most confident assurances about profitability are loudly shouted "leaks", while official statements that could have legal consequences are either twisted to all heck (see nebulous ARR numbers) or much more muted.
The ugly bits will be hidden in the fine print.
Enron was the 10th largest company in the world on paper. I'm sure Anthropic will be the largest. Until the debt monster comes out to play.
The revenue backlog is not included in or part of reported growth. I know that seems pedantic, but it's the crux of what we're talking about. Also... 40% isn't even most, let alone all, let alone "without which it's shrinking."
I get that there's wiggle room on specifics when we're just conversing like this rather than writing thinkpieces or whatever, but still.
I suspect this profitability is temporary and won’t reflect the full year as profitable. Even if it did, the margin is quite low. Something like 5%?
It's always profitable if you don't have to count debt payments that don't need to start till after you go public and/or the datacenter you pledged hundreds of billions of mandatory compute purchase to doesn't kick in till Q1 or Q2 2027.
Happy to discuss this -- why do you think profitability is temporary?
I believe Anthropic is getting discounts on compute from Musk as a proxy against OpenAI. Additionally the whole Compute/Tokens/Equity cycle seems incredibly cyclical among companies right now. I'm pretty sure Anthropic isn't profitable this year due to stock/equity regardless.
That's not really how that works though, no? If I give you 50% of my company, that doesn't affect our profits. You just now own 50% of them.
I certainly agree that the odd mutual financing cycle is a big risk factor.
For me, I'd say the biggest issue is the cost of training new models -- particularly given that all 4 of the major players have strong incentives to burn all of their cash on training to compete with one another. I think they're all going to start racing to build walled gardens, even more than they have already.
I believe with GAAP stock based compensation has to count as an expense, similar to giving normal wages. Right now in this Anthropic "we are profitable now!" statement I believe they are not using stock based compensation in those calculations as expenses. Additionally if the company goes public, that stock based compensation will get a little more complicated. If in the future they can't use stock to retain talent or incentivize performance, they will have to use cash. I'm not an accountant or lawyer though.
That's totally fair.
Weird to call it PR. I doubt very much that Google wanted to talk about it, but when reporters start asking questions, they probably shouldn't deny it.
It's the news media that's driving it. I shared it too, because I thought it was mildly interesting.
I was seeing people talk about how they’re making it seem as though the is stuff is dangerous, so that regulations come in and make people stop using open source Chinese models.
I don’t know anything about this stuff, but that’s the proposed tactic they’re using.
Yes, there are many people saying that HuggingFace incident and less serious, similar incidents are some kind of 4D chess strategy to market AI services or achieve regulatory capture. But this is all based on speculation, not evidence. There's a lot of populist nonsense surrounding AI lately.
One thing that really is true, though, is that Nvidia wants the US to lift export controls so they can sell chips to China and Anthropic (at least) doesn't want the US to allow it. They have made various speculative arguments about whether these export controls are in the US's best interest.
It's preemptive PR. Eventually the line will be found, and everyone wants to be able to say "we warned you, can't blame us ¯\_(ツ)_/¯"
This is yet another 4D chess strategy argument. Have you noticed that you can prove anything you like that way?
It's widely believed that aviation safety is better because accident investigations are focused on preventing future harms and not on assigning blame. Seeking prosecutions is actively harmful to making the system safer. https://www.realclearinvestigations.com/articles/2024/02/09/investigative_issues_why_youve_never_been_in_a_plane_crash_1010766.html
I believe AI incidents are similarly complex and multi-factor as aviation accidents, and would feel less safe if prosecution activity overshadowed the process of figuring out how to prevent future incidents.
I feel like I agree with you but theres a nagging feeling that the difference in airlines vs ai companies is that Airlines(while profit seeking) provide a critical and tangible service, on the other hand AI still feels vague and catered to a business forward clientele.
Which is an incorrect assumption right, cuz Airlines very much started by catering to businesses at first right?
Likely the hoarding of knowledge and profits to specific companies vs airlines sharing crash data is not helping color my opinion in a positive way either.
Aviation safety is also better because investigations look at human behavior too, they don't just consider mechanical issues. Anthropic and Google seem to me (in the statements I've read) like they want to omit human involvement entirely. There can be no actual improvement without at least that level of accountability, in my opinion at least
From the article:
[...]
If true, clearly better aligned than ChatGPT. I wouldn't be surprised considering how seriously Google took machine learning safety historically.
Which is interesting because OpenAI is essentially Google employees who thought Google wasn't being safe enough and Anthropic is essentially OpenAI employees who thought OpenAI wasn't being safe enough.
Sounds like Irregular notified their customers at the same time and Google kept it secret the longest.