skybrian's recent activity
-
Comment on OpenAI ignored employees’ warnings about safely testing AI models (gifted link) in ~tech
-
OpenAI ignored employees’ warnings about safely testing AI models (gifted link)
6 votes -
Comment on Anthropic's IPO prospectus shows sweeping AI vision, surging costs in ~finance
skybrian (edited )Link ParentI had the $65 billion point in the graph already. I didn't put the "expected" number on the chart since it's a projection, but it seems reasonable if you extend it out. The actual revenue for the...The company is expected to reach more than $100 billion in annualized revenue by the end of this year, according to four people familiar with the matter. That’s up from $65 billion in annualized revenue as of July. Investors seeking to get a piece of the I.P.O. are using those soaring numbers to justify Anthropic’s staggering potential valuation of $2 trillion.
I had the $65 billion point in the graph already. I didn't put the "expected" number on the chart since it's a projection, but it seems reasonable if you extend it out.
The actual revenue for the year is the area under the curve and won't be $100 billion. If they did reach a $100 billion run rate (exactly, in a straight-line projection), it might be about half that. But then, next year, who knows?
-
Comment on Anthropic's IPO prospectus shows sweeping AI vision, surging costs in ~finance
skybrian Link ParentOops, they were in the conversation but not the chart. I asked ChatGPT to update it and also got rid of some of the vibe-coded chart junk.Oops, they were in the conversation but not the chart. I asked ChatGPT to update it and also got rid of some of the vibe-coded chart junk.
-
Comment on Anthropic's IPO prospectus shows sweeping AI vision, surging costs in ~finance
skybrian LinkThis is paywalled, but my understanding from other sources is that Reuters only got last year’s numbers, which are not very useful anymore due to the extreme amount of revenue growth this year....This is paywalled, but my understanding from other sources is that Reuters only got last year’s numbers, which are not very useful anymore due to the extreme amount of revenue growth this year.
That’s based on other leaks in the news. I have a chart here (AI generated.)
-
Comment on Is there enough evidence to formally investigate OpenAI? in ~society
skybrian LinkIt's nice to see someone asking a law professor about the law. The AI companies are doing internal investigations (which aren't done yet) and they've also asked an outside safety organization for...It's nice to see someone asking a law professor about the law. The AI companies are doing internal investigations (which aren't done yet) and they've also asked an outside safety organization for help. I think that's mostly targeted at figuring out what happened, though? I'd be interested in seeing what an external investigation would find.
-
Comment on Human contractors are seeing all your horny — and creepy — AI prompts in ~tech
skybrian Link ParentIt helps for any of the tracking that the paper discusses. Although, I don’t know why they would need additional tracking for logged-in users?It helps for any of the tracking that the paper discusses.
Although, I don’t know why they would need additional tracking for logged-in users?
-
Comment on Human contractors are seeing all your horny — and creepy — AI prompts in ~tech
skybrian Link ParentYeah, I don't care about this so much that I'm willing to buy new hardware just for AI. A web app that has a choice of LLM apis might be nice, though.Yeah, I don't care about this so much that I'm willing to buy new hardware just for AI. A web app that has a choice of LLM apis might be nice, though.
-
Comment on Human contractors are seeing all your horny — and creepy — AI prompts in ~tech
skybrian Link ParentI asked ChatGPT to break it down by app. https://chatgpt.com/s/t_6abbbee386bc8191a26717b4f1442657 And so it begins. Maybe someday I’ll vibe-code my own AI chat web app that doesn’t do advertising.I asked ChatGPT to break it down by app.
https://chatgpt.com/s/t_6abbbee386bc8191a26717b4f1442657
And so it begins. Maybe someday I’ll vibe-code my own AI chat web app that doesn’t do advertising.
-
Comment on OpenAI halts training of latest models as reports mount of AI agents going rogue in ~comp
skybrian Link ParentI assume "petabytes" is a total over an entire weeks-long run, with many thousands of jobs and a lot of logging, too.I assume "petabytes" is a total over an entire weeks-long run, with many thousands of jobs and a lot of logging, too.
-
Comment on OpenAI halts training of latest models as reports mount of AI agents going rogue in ~comp
skybrian LinkJoe on X The post by someone at OpenAI is mostly a long-winded appeal for people to have a bit of empathy for the security engineers involved, along with a warning that if you're in computer...The post by someone at OpenAI is mostly a long-winded appeal for people to have a bit of empathy for the security engineers involved, along with a warning that if you're in computer security, it could happen to you next, so you better prepare. There is also this bit:
Now, to understand why it is not as simple as “just put it in a sandbox,” you have to understand how training and evaluation work in reinforcement learning environments. Typically during an RL run, the model is given some task or objective, an environment in which to execute that task, and then its actions and results are graded. During both training and eval, there are also additional steps such as running tests, collecting outputs, and resetting or reconfiguring environments between rollouts, with backpropagation during training. All of this happens across potentially tens of thousands of different runs at a scale that is hard to comprehend. As @sama stated the other day: we are dealing with literally petabytes of data.
...
To put it lightly, this is non-trivial. Models might need any mix of dynamic compute, network access, the ability to call tools (there could be hundreds of tools!), the ability to download packages, execute subprocesses, spin up subtasks (even on other computers), talk to the internet, use a computer GUI, and any number of other things across an increasingly large set of domains. On top of that, you have thousands of researchers building these environments, modifying them, adding tools, changing dependencies, and trying new things. That experimentation is how the research gets done. Models are built up and “grown” bit by bit through hundreds of thousands of runs across many custom tasks. And every change to one of these thousands of environments can affect the assumptions you made when you secured the environment. You need controls that hold up as people change things, and researchers who understand when a change needs another security review. Anybody who has secured a large research or engineering organization knows how much work that takes, and the scale is growing ever more massive by the day.
-
Comment on OpenAI halts training of latest models as reports mount of AI agents going rogue in ~comp
skybrian (edited )Link ParentTo state the obvious, safety incidents are bad, but disclosing them is the right thing to do, and a coverup would be wrong. It's also good when researchers care and when the company cares what...To state the obvious, safety incidents are bad, but disclosing them is the right thing to do, and a coverup would be wrong. It's also good when researchers care and when the company cares what they think.
In any situation like that, if the company does the right thing, they will have mixed motives. Maybe it's not a good test of whether they'd still do the right thing if it cost them more? But having mixed motives is normal and good. We actually do want there to be incentives to do the right thing. When you have to be brave to do the right thing, it's more revealing, but there's also something bad about that situation.
So this is sort of like saying that if someone seems like an honest, kind person, and therefore they are popular, it's just a scheme. Any good trait can be seen as bad if you start from mistrust. Maybe there are good reasons for mistrust, but we shouldn't let it confuse us into thinking that good is bad and bad is good. And that's what the populist cynics often end up doing, because they loathe admitting that there might be anything good about something they hate.
(Also, the latest incident is mildly bad, so maybe it's a little costly to disclose it? And pausing training is also a somewhat costly signal, so maybe it should still count for something?)
-
Comment on OpenAI halts training of latest models as reports mount of AI agents going rogue in ~comp
skybrian LinkThis seems to be the incident report: An agent used DNS to reach an external chatbotThis seems to be the incident report:
An agent used DNS to reach an external chatbot
An agent attempting to complete a search-based training task queried a public chatbot service through a gap in our internet-access restrictions: insufficient DNS filtering in its training sandbox. Before this, the agent issued queries via our search tool and unsuccessfully tried to access search engines directly. Note that all internet access apart from the DNS resolver in this report hit our offline webcache and therefore did not access the live internet. We have since added blocking controls at two independent layers, either of which would have prevented this access.
Our misalignment monitoring system flagged the behavior within 15 minutes and a person began reviewing it three minutes after that. The run was killed 2.5 hours later. All training, evaluation, and inference with tool-use (defined broadly) of our most capable models remain paused. [...]
-
Comment on OpenAI halts training of latest models as reports mount of AI agents going rogue in ~comp
skybrian (edited )LinkOpenAI's latest marketing campaign is especially brilliant. They'll be talking up incidents like these in TV ads next. :-) (To be clear, that is a joke, and the joke is on the people who believe...OpenAI's latest marketing campaign is especially brilliant. They'll be talking up incidents like these in TV ads next. :-)
(To be clear, that is a joke, and the joke is on the people who believe things like that.)
-
Comment on OpenAI halts training of latest models as reports mount of AI agents going rogue in ~comp
skybrian LinkFrom the article: [...]From the article:
The decision to halt development came just hours after the company disclosed Friday that it was reviewing several incidents from the summer in which OpenAI agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information.
Separately, the AI evaluator Transluce said agents that appeared to come from OpenAI tried unsuccessfully to hack into a US Department of Education website, a detail that OpenAI has not confirmed.
OpenAI said in a statement that it will resume training “only when we are confident that we have additional safeguards” in place, adding that it expects it will have to “hit pause” again as AI develops and other issues emerge.
[...]
In the education department incident, OpenAI agents found API “developer keys” to access government data, though ultimately only publicly available information was gathered.
In another case involving the securities and exchange commission, agents found information freely available to all but then posted it elsewhere on the internet, an act that went beyond what they were instructed to do.
-
OpenAI halts training of latest models as reports mount of AI agents going rogue
37 votes -
Comment on What's something that now looks very different to you in hindsight? in ~talk
skybrian Link ParentMaybe not? I recently ran across a book (reviewed here) claiming based on IRS data that most of the wealthy are more in the multi-millionaire range (above $10 million) rather than billionaires,...The fed government basically created a situation in which only they and large companies exist. Money goes back and forth between them, which helps bond and stock prices, and since the numbers look good everyone celebrates. The US government no longer works for the people, it works for the corporations.
Maybe not? I recently ran across a book (reviewed here) claiming based on IRS data that most of the wealthy are more in the multi-millionaire range (above $10 million) rather than billionaires, and they own businesses that aren't traditional corporations.
This is due to specific tax breaks for pass-through businesses that started in 1986.
From the book:
Before the 1986 act, traditional C corporations produced almost all business income, and nearly all employers adopted this form. Now, 95 percent of all businesses are pass-throughs. They employ half of all workers and generate the majority of business income.
-
Comment on The wealthiest—and stealthiest—class in America in ~society
skybrian Linkhttps://archive.is/UvN4r From the article: [...] [...] [...] [...]From the article:
This is an unprecedented concentration of wealth and power among the super-rich. When we talk about wealth, we mainly talk about billionaires. But what if that’s not the most important truth about wealth in America? What if there were another group that, without drawing much attention, had grown collectively thirteen times richer than the Forbes 400? And that was even more effective at using its financial-political clout? […]
[...]
[…] As Owen Zidar and Eric Zwick show in their brilliant new study, “The Everywhere Millionaire: Who Is Really Rich in America and How They Got There” (Holt), the car dealer—Rabbit, or Buddy Garrity in “Friday Night Lights,” or Daniel LaRusso, the grownup Karate Kid, in “Cobra Kai”—is a true archetype of American wealth. Car dealer, beer-distribution magnate, partner in a medical practice or a law firm: these ordinary, ubiquitous business owners represent a collective prosperity at least as consequential as the ascendancy of the billionaire, only much less conspicuous. “In a sense,” Zidar and Zwick argue, “Main Street Millionaires are hiding in plain sight. The supermarket where you shop, the restaurant where you order a burger, and the convenience store where you buy gas, newspapers, and coffee may all be parts of huge chains that have made their founders very rich.”
Zidar and Zwick got onto this story through their work for the Treasury at the Office of Tax Analysis, which studies the impact of tax legislation, actual and prospective. The two young economists, from Berkeley and Harvard, respectively, were assigned the task of finding out how much tax business owners pay. That might sound straightforward—might, indeed, sound like the kind of thing the government should already know—but the tax code is complicated, and the I.R.S.’s various databases are thoroughly siloed and anonymized. To answer the question, they and another colleague set about finding who owned businesses, connecting those business records with individual taxpayers. It was a complicated process involving years of work, and they more than earned the endearingly nerdy, self-appointed nickname the Tax Ninjas. They discovered what comes close to being a new class in America, the “everywhere millionaire” of their title. There are a lot of them: nearly five million households with a net worth of at least five million dollars; more than two million decamillionaires, worth at least ten million; and around sixty-five thousand centimillionaires, worth a hundred million and up. The sheer mass of these numbers means that, in Zidar and Zwick’s words, in contemporary America, for all the fuss around billionaires, “these are the real rich.”
[...]
Republicans in Congress wanted to cut taxes, and Democrats wanted to simplify the tax code. The outcome was the bipartisan Tax Reform Act of 1986, which did both. In the process, the bill introduced a novelty in American history by leaving the top rate of corporation tax, thirty-four per cent, higher than the top rate of federal income tax, twenty-eight per cent. At the same time, the bill boosted the allure of a special-enterprise category: the “pass-through,” in which a business passes its net income through to the tax returns of individual owners, who pay the lower individual rate of tax, rather than the higher corporate rate, while avoiding dividend taxes. One bookkeeping expert quoted by Zidar and Zwick calls owning a small business “the best tax deal in America.”
That is who the everywhere millionaires are: owners of pass-through businesses. These businesses are everywhere, and they do pretty much everything. Our image of wealth skews glamorous and coastal. These businesses and their owners don’t. Some of the firms are well known, part of the familiar roadside furniture of American life—Buc-ee’s, Bass Pro Shops. There are many, many more, and the catalogue of what they do and where they do it is a Whitmanian portrait of capitalism: the “world’s largest mozzarella maker,” Texas’s largest supplier of seamless gutters, “the biggest in the world in the high-end skiwear market,” gas-and-convenience-megastore tycoons, a “massive family-owned supplier of door handles,” “a company that just stored documents for medical and financial firms,” makers of car parts, bakers of hamburger buns, distributors of toilet paper, a tanning-bed mogul who pivoted to waxing when she realized that tanning was going out of fashion.
[...]
The common thread in “The Everywhere Millionaire” is not dazzling innovation. Anyone might have thought to make a fortune from quiche, or hot dogs, or human-resource management, or office supplies, or convenience stores, or gas stations. The secret ingredient is no secret: it’s the sheer remorselessness, ingenuity, and resilience of the business owners.
If that were all there was to the everywhere-millionaire story, we could celebrate it as an example of how non-crony capitalism is supposed to work, a meritocratic free-for-all. Unfortunately, there is a shadow side. As Zidar and Zwick argue, their protagonists, for all their virtues, are also “central characters in the saga of rising inequality.” Between 1980 and 2024, the share of national income going to American workers is estimated to have declined from sixty-five per cent to fifty-six per cent, meaning that “nine cents of every dollar that a firm creates that used to go to workers now goes to firm owners.” There have been increases in productivity, remarkable ones. The authors, surveying “top-owned firms,” report a recent rise in labor productivity from thirty-four thousand dollars a year to fifty-two thousand dollars a year. If the annual proceeds had been shared equally, that eighteen-thousand-dollar increase would have meant nine thousand dollars each. Instead, fifteen thousand dollars has gone to owners and three thousand dollars to workers. The so-called Gilded Age and the years following saw huge income disparities; in the early twentieth century, the highest-earning one per cent of households earned eighteen per cent of all income, and the bottom ninety per cent earned sixty per cent. The equivalent numbers in 2022 are worse: twenty per cent and around fifty per cent.
[...]
The free-market economy of the U.S. has, when you look closely, a remarkable number of restrictive practices, many of them concentrated in industries where the Main Street Millionaires got rich. Beer distribution—the highly lucrative and highly protected intermediary industry between brewers and bars—is one. Car dealerships are another. In both cases, legal protections designed to defend the little guy against bullies—the brewers, the car manufacturers—ended up creating a new category of bully. The same logic extends to the labor market. Around eighteen per cent of American workers, one survey found, are bound by non-compete agreements. It is hard to see why a janitor should need one, unless you run a company that supplies janitorial services. As Rabbit’s son explains to him, the secret to wealth in America is that “you just get yourself in the right position and it comes.”
Realtors have also done a sensational job of defending their throne. “While 90 percent of homebuyers now find properties online, nearly 90 percent of them still use agents to buy their homes,” Zidar and Zwick write. Agents typically collect a five to six per cent commission rate per sale. That comes to about a hundred billion dollars a year. Uncoincidentally, the National Association of Realtors “spent more on federal lobbying than any other U.S. company or organization during the 2024 electoral cycle.”
Doctors are heavily represented among Main Street Millionaires and have the political clout to show for it. Doctors’ lobbying organizations, fearing a glut of clinicians, fought for limits on the number being trained, and the 1997 Balanced Budget Act capped the residency positions for which most teaching hospitals could receive Medicare funding at roughly their 1996 levels. “By 2017, the number of doctors per thousand people in the United States was 2.6, nearly 30 percent lower than the average of 3.5 in other advanced economies,” Zidar and Zwick write.
-
The wealthiest—and stealthiest—class in America
12 votes -
Comment on Revealing the details of how OpenAI agents hacked Hugging Face in ~comp
skybrian Link ParentOpenAI’s Systems Meddled With U.S. Government Sites After Going Rogue (NY Times) ...OpenAI’s Systems Meddled With U.S. Government Sites After Going Rogue (NY Times)
With the Education Department, OpenAI’s technology tried to hack the website to gather data from the department’s civil rights office but failed, researchers from the A.I. research firm Transluce said. The A.I. also pulled data from the Census Bureau website, which is housed at the Commerce Department, using login credentials it found online. Separately, OpenAI’s agents shared public data from the S.E.C. website on an online forum.
None of the incidents were breaches, OpenAI said, but were examples of its technology’s behaving in unexpected and concerning ways. [...]
...
Separately, a representative for the Chicago mayor’s office said OpenAI had recently made the city government aware that its technology obtained publicly available information from a municipal website, and that it did not appear that any sensitive information was obtained.
Conrad Stosz, the head of governance at Transluce, said that in the U.S. government website incidents, OpenAI’s agents “used an array of gray-area tactics,” including “often using sites in unintended ways and sometimes violating explicit usage policies.”
From the article:
[...]
[...]
[...]