skybrian's recent activity
-
Comment on An AI researcher writes about his crisis of faith in ~tech
-
Comment on AI text watermarking is free and good in ~comp
skybrian Link ParentYes, I imagine the watermark detection algorithm won't work as well when the text is hightly constrained by context. It will always need a minimum amount of text to work with, and for...Yes, I imagine the watermark detection algorithm won't work as well when the text is hightly constrained by context. It will always need a minimum amount of text to work with, and for highly-constrained text, it will need more.
But if it doesn't have enough text to pick up a signal, wouldn't it answer "not detected" most of the time? That is, the input will affect the score randomly and the score will be close to what it would be for random noise.
-
Comment on US judge to baby: file an asylum application in ~society
skybrian Link ParentThat scenario seems pretty plausible too.That scenario seems pretty plausible too.
-
Comment on An AI researcher writes about his crisis of faith in ~tech
skybrian LinkFrom the blog post: [...] [...]From the blog post:
Even if the nightmare scenario of full human redundance is not imminent, the fact that a considerable number of people claim that this is their goal is unsettling. This view is perhaps most succinctly stated by the startup Mechanize, which aims to “fully automate the economy”. As a participant in the economy, I find this goal deeply objectionable. I think that trying to fully replace all human labor is not a morally acceptable goal to have. It’s frankly horrifying. To be clear, “fully automate the economy” means “destroy society as we know it”. Unfortunately, similar goals are articulated by quite a few inside and outside Silicon Valley. I think the callousness of such objectives should be called out whenever they are encountered.
The great irony of this is that those who claim to want to automate all human labor are typically the kind of competitive, smart, high-agency persons who absolutely need to have a purpose and something to build. They would hate to be redundant. Yet, here they fly, like so many moths to a flame.
[...]
Why didn't I just quit AI research? My predicament would seem like that of a vegan butcher, or a monk who makes money on Onlyfans. But me quitting and becoming an Uber driver would not make the world better. The pace of AI progress would clearly not slow down noticeably. And I assure you, I still love AI research, even if I sometimes hate what AI does to the world. I'm not even very good at anything else. AI is what I do. And I think that I can do more good by trying to steer my field in a good direction than if I became an Uber driver. So I’d rather think of myself as akin to a hypochondriac doctor, or a pilot with a fear of heights.
[...]
From this perspective, the history of AI is a history of attempts to mimic the specific combination of behaviors and capabilities that humans have; most of them successful in some way, but all of them quite different to humans. The onslaught of LLMs becomes a push in a particular capability direction. Understanding that direction becomes crucial to figuring out which types of human intellectual patterns and capabilities will become more important in the future. Where the new domains of human excellence will appear. But this understanding can also help us develop different types of AI that are more complementary to what humans can do and like to do. Seeing intelligence as a scalar, where machines can overtake humans, is a recipe for paralysis; dissolving this faulty notion gives us the freedom to act. More of this argument in the article I linked above; what's important here is that there are things that can be done. Indeed, there's a lot to do. Such as building mechanisms for meaningfully incorporating humans in creative search processes, and open-ended learning and discovery processes that are not based on imitating what humans do.
But not everything has a technical fix. Norms, structures, and laws are probably more important. I still think that trying to automate humans out of the processes that give them, and our civilization, meaning is immoral. We need to build counter-narratives, and be vocal that "fully automating the economy" is not an acceptable goal to work towards. We also need to push hard to counter the centralization of power that so easily comes with lavishly funded tech companies trying to achieve monopolies on some layers of the AI stack. Our best bet for a future where we all matter is one with a myriad different forms of intelligence, open and accessible for all to use as tools for our natural intelligences. Let's get to work.
-
An AI researcher writes about his crisis of faith
11 votes -
Comment on Your executable is a SQLite database in ~comp
skybrian LinkFrom the article: [...] [...] [...] [...]From the article:
I never let the idea go and with the recent improvements with LLMs, I find it compelling to revisit these ideas to explore further. Specifically, can we replace ELF with SQLite as an executable format? 🤔
Not “a database that describes an executable”, but the actual file you
chmod +xand run.[...]
I developed a pretty fleshed out prototype. It is called SELF, the Structured Executable & Linkable Format, because I am unoriginal. It is on GitHub if you are interested. I’m surprised about all the interesting things that fall out of this idea.
[...]
There is a fixed ~5 ms to open SQLite and start the interpreter, plus a copy proportional to the image. That copy is worse than it looks, because the b-tree pages are not mapped into memory. Two processes running the same SELF binary do not share text pages the way a normally-mmap‘d ELF does, because the bytes are copied out of the b-tree rather than mapped.33You might notice that
curl(274 KiB, 27 libraries) starts slower than ELFgit(4.6 MiB, 5 libraries). That isld.sodoing work proportional to the number of objects rather than the number of bytes, which I have complained about before.[...]
611.9 MiB of database against 644.4 MiB of ELF files. The whole userland, as one queryable file, is smaller than the files it came from. The b-tree cost that doubled a single
helloamortises to nearly nothing across 1,123 objects and is roughly 6% over the actual program bytes.The libraries and closure are shared across the executables very similar to how Nix might share them across multiple closures, if the store-path was the same. If every root shipped its own private closure (i.e. the AppImage model), the same 723 programs would come to 5.53 GiB but the deduplication of libraries and symbols falls out naturally from the database schema.
[...]
The format is done and round-trips between ELF and SELF losslessly. The tooling is done and can query, modify, and pack closures. Lookup through SQL works on unmodified glibc programs perfectly and the native-SQL loader works enough to explore it as a possibility for ideas.
The whole thing is at fzakaria/selfdb.
nix run .#self-vmboots a NixOS VM wherehellois a SQLite database. 🙌 -
Your executable is a SQLite database
9 votes -
Comment on AI text watermarking is free and good in ~comp
skybrian Link ParentIt's going to arbitrarily choose among identifiers that the LLM thinks are equally good. That will depend on context. For example, the LLM might prefer to maintain consistency with surrounding...It's going to arbitrarily choose among identifiers that the LLM thinks are equally good. That will depend on context. For example, the LLM might prefer to maintain consistency with surrounding code, which means that in a particular context, a consistent choice is better. Or, maybe there's a style guide in context?
It will also change as models get smarter and/or more opinionated about good coding style.
-
Comment on US judge to baby: file an asylum application in ~society
skybrian Link ParentClearly, this result is absurd and unjust. But maybe the judge knew that? Maybe it's their way of calling attention to the injustice of the system? That's pure speculation about the motives of a...Clearly, this result is absurd and unjust. But maybe the judge knew that? Maybe it's their way of calling attention to the injustice of the system?
That's pure speculation about the motives of a perfect stranger and I don't consider it particularly likely that I guessed right. But you're doing the same. We all speculate sometimes, but we shouldn't be confident that our speculations are correct. "Must certainly be" seems a bit much?
We shouldn't judge people without an investigation, and it's unreasonable and no fun to expect readers to investigate most of what we read in the news. I think that usually means saying "that sounds terrible" and moving on.
-
Comment on AI text watermarking is free and good in ~comp
skybrian Link ParentLet me try to explain it a slightly different way: Yes, some randomly-chosen word choices are better and others are worse. However, the LLM doesn't "know" that. The AI uses the random number...Let me try to explain it a slightly different way:
Yes, some randomly-chosen word choices are better and others are worse. However, the LLM doesn't "know" that. The AI uses the random number generator to choose among what the LLM considers to be "equivalent" paraphrases. If they don't seem equivalent to you, it's because you know better than the model.
Sometimes the LLM does "know" that one word is better than other in a given circumstance. For example one word is the right answer and another word is wrong. But if it knew that, it wouldn't defer its choice to the random number generator. The probability distribution would be so skewed that it forces the right answer.
How can an AI lab optimize the probability distribution to serve you better? They could train a new model or continue to train one that they already have. Messing with the random number generator isn't going to do it.
-
Comment on AI text watermarking is free and good in ~comp
skybrian Link ParentAI chat is often using a random number generator to decide what to write. If you're concerned about LLM's not serving you wholeheartedly, maybe you should be concerned about that, too? They're...AI chat is often using a random number generator to decide what to write. If you're concerned about LLM's not serving you wholeheartedly, maybe you should be concerned about that, too? They're rolling the dice to decide what to tell you! How does that serve you?
Swapping one random number generator for another isn't going to change that.
Though of course it's not just random. The weights bias the results, making some answers much more likely than others.
Overall, the answers being chosen from are in some sense equivalent. Even though they might not seem at all equivalent to you, the AI has no preference between them.
But perhaps a better model would have a preference? That's pretty much what happens when switching to an improved model.
Running the same query multiple times can be a good way of seeing what a model considers to be equivalent. Though, maybe this watermarking scheme would reduce the variety since it's using a biased generator? It's not going to make it better or worse on average, but it will reduce the number of answers that it's choosing from.
-
Comment on AI text watermarking is free and good in ~comp
skybrian Link ParentAnthropic links to this paper. Apparently the tokens are chosen to score highly according to a random scoring function. This is not very clear to me, but here’s the key bit: Google’s AI summary...Anthropic links to this paper. Apparently the tokens are chosen to score highly according to a random scoring function. This is not very clear to me, but here’s the key bit:
The key idea of Tournament sampling is to use a tournament-like process to choose an output token that scores highly with respect to some random watermarking functions. An illustration is given in Fig. 2 (top). First, we take the random seed rt provided by the random seed generator. This seed is passed to m (in this case, m = 3) watermarking functions g1, g2, g3, …, gm—these are independent pseudorandom number functions that assign a score gℓ(xt, rt) (in this case, a 0 or 1) to any candidate token xt ∈ V.
In the second stage (Fig. 2, bottom), we start by sampling M = 2m candidate tokens from the LLM distribution pLM(⋅∣x<t) (some tokens may appear multiple times): these are the initial participants of the m-layer tournament. We randomly divide these candidates into M/2 pairs, and, in the first tournament layer, in each pair the token with the higher score under g1(⋅, rt) is selected, and the other discarded (any ties are broken randomly). The remaining M/2 tokens are regrouped randomly into M/4 pairs, and the function g2(⋅, rt) determines the winners for this second tournament layer. This iterative process continues until one token emerges as the final winner, which becomes the output token xt. A formal description of Tournament sampling is given in Algorithm 2 in Methods.
By design, Tournament sampling selects a token from the LLM distribution that is likely to score higher under the random watermarking functions g1(⋅, rt), …, gm(⋅, rt). To detect whether a piece of text x = x1, …, xT is watermarked, we measure how highly x scores with respect to these functions.
Google’s AI summary seems a bit clearer, so maybe the thing to do is to ask your friendly AI what this all means?
It sounds like they start with the scoring functions and produce the text specifically so it scores high. So, they only need the scoring functions to verify the text. But they do need to keep them secret to prevent forgery, so we will have to rely on their API to tell us whether the text was from their LLM or not. Yeah, that’s not ideal.
-
Comment on US judge to baby: file an asylum application in ~society
skybrian LinkFrom the article: [...] [...] [...]From the article:
A 1-year-old baby appeared in immigration court without legal representation this week in Arizona and received an instruction: File an asylum application before the next hearing or face deportation.
An attorney from the U.S. Committee for Refugees, one of the groups with a new contract to represent migrant children, was silent as it happened, according to filings Thursday in the legal dispute between a network of nonprofits and the Trump administration.
“The USCRI attorney did not speak up—let alone suggest that it is facially absurd to ask an unrepresented baby to do anything—despite knowing the baby would be ordered deported if unable to file the complicated application by the next hearing,” one filing says.
The baby is one of the unaccompanied migrant children who have had to represent themselves in immigration court since July 31, when a contract between the federal government and a network of nonprofits that provided legal services ended, the groups say in the court filings.
[...]
The end of the contract with the Acacia Center for Justice, which distributed the government funds to nearly 100 legal providers representing more than 24,000 kids, has led to confusion, the groups say. Minors are showing up unrepresented, despite two new contracts the Department of Health and Human Services awarded to the USCRI for $20 million and Our Rescue, an anti-human-trafficking organization, for $158 million.
[...]
In court declarations, attorneys and leaders of the nonprofits that previously were contracted to represent children say they’ve witnessed USCRI attorneys remain mum during immigrant children’s court proceedings.
[...]
In another instance on Aug. 6, an unaccompanied minor without legal representation in a California court requested to voluntarily leave the country while the USCRI attorney present didn’t say anything, according to a declaration from Cristel Stefany Martinez, managing director of California immigration law firm MNM Law Offices.
-
US judge to baby: file an asylum application
34 votes -
Comment on AI text watermarking is free and good in ~comp
skybrian LinkHere are Scott Aaronson's comments. So if the writing quality isn't that great to begin with, this doesn't make it worse. Any of the samples would use a similar number of AI catch-phrases. ...Here are Scott Aaronson's comments.
[C]ontrary to many people’s intuitions, there’s no inherent tradeoff between watermarking and the quality of LLM output. Basically, nearly every LLM output was already a sample from a cloud of exponentially many possibilities, all of them about equally good, so there’s plenty of room to steer within that cloud without affecting anything that an ordinary user would notice.
So if the writing quality isn't that great to begin with, this doesn't make it worse. Any of the samples would use a similar number of AI catch-phrases.
I could add that, in addition, there’s recent progress (see here for example) on what I’ve called “semantic watermarking,” or watermarking at the level of the underlying concept vectors rather than the tokens themselves. This actually seems to work, albeit with no theoretical guarantees, and will hopefully make removing watermarks a lot harder—although the Barak et al. impossibility result suggests that under plausible assumptions, no LLM watermarking method will be completely foolproof.
...
Nevertheless, Google DeepMind implemented something very similar to my proposal in its SynthID, deployed in all its Gemini text models. But they heavily restricted who gets to detect the watermark, which made their admirable decision of limited use to my academic colleagues, who’ve been begging me for a way to detect whether their students are using AI to cheat. (For now, I mainly send them to Pangram, a leading AI detector not based on watermarking, as a first line of defense.)
And now, apparently to comply with EU regulations, Anthropic says they’ve deployed a watermarking scheme like mine where anyone will be able to do detection (though they also say in their FAQ that they’re still working on the detection API). Even OpenAI suggests that it plans to follow suit. So, four years after I seriously thought about this, it looks to my surprise like this is actually happening. Thanks, EU!
-
Comment on The war on data centres is a bit fake in ~society
skybrian Linkhttps://archive.is/0tDCq From the article: [...]From the article:
Data centres are a nightmare of political economy. Voters see them as ugly, dirty, noisy and power-hungry bastions of arrogant Silicon Valley companies. And—thanks to ChatGPT—it has never been easier to pen a letter to the planning department saying so. Yet the public has an awful lot riding on these windowless sheds, for the stock market is a one-way bet on them being built. Data centres are particularly disorientating for Democrats, since unions tend to support the projects. If artificial intelligence is a class war, it pits a coalition of billionaires and electricians against the office droids both despise.
Wildcatters like Mr O’Leary are a soft target for politicians who want to be seen as tough on data centres but still want them to be built. On August 18th Josh Shapiro, the governor of Pennsylvania, derided the developers “scaring our communities” and signed an executive order he said would squash “speculative proposals” that had little chance of being built in the first place. Of the more than 100 projects announced in the state, less than a fifth had applied for the permits needed to develop them, he said.
Imaginary scalps are easy to claim. Silicon Valley giants are the biggest backers of data centres. But beside them is a long line of property developers, private-capital firms, “neo-cloud” operators and crypto-miners, followed by international chancers, national rogues and local spivs. Most of their plans were destined to remain press releases. Analysts at Bernstein, a broker, attempted to quantify the credibility of each watt of announced capacity. Plans by Google and Amazon are taken at their word. Those by CoreWeave and Nebius, two large neo-clouds, are discounted by around half. Mr O’Leary is ignored.
According to their model, another fantasist is Vermaland, a land bank that announced a gargantuan 3GW facility in Arizona last year. In May the project was reportedly cut by four-fifths after facing local opposition—a shadow-boxing knockout. Little information is available about a 1GW project in south Florida that was supposedly shelved in February, except that the land had been bought for $15m in 2024 by an obscure investment firm. When a data-centre plan that would never have materialised is cut down, it often makes a very large sound.
[...]
It is often said that China has an advantage in the AI race because it can build big things by force. Yet America has the power of willing things into existence by exaggeration. The grand plans of developers will be watered down. Politicians will also overstate their opposition. Often they will cancel each other out. Meanwhile, the searing pace of construction will continue.
-
The war on data centres is a bit fake
6 votes -
Comment on Authors retract Nature paper projecting high costs of climate change in ~enviro
skybrian LinkFrom the article: [...] Here is Andrew Gelman's commentary:From the article:
The authors of a highly publicized study predicting climate change would cost $38 trillion a year by 2049 have retracted their paper following criticism of the data and methodology, including that the estimate is inflated.
“The economic commitment of climate change,” which appeared April 17, 2024, in Nature, looked at how changes in temperature and precipitation could affect economic growth. Forbes, the San Diego Union-Tribune and other outlets covered the paper, which has been accessed over 300,000 times. It has been cited 168 times, according to Clarivate’s Web of Science.
But after two commentaries published this August raised questions about the study’s data and methodology, the researchers revisited their findings. “The authors acknowledge that these changes are too substantial for a correction,” the retraction notice, published today, states.
[...]
Authors of the first “Matters Arising” commentary published August 6 noted the article projected the global gross domestic product would be lowered by 62 percent by 2100, “an impact roughly 3 times larger than similar previous estimates.” The authors of the critique also pointed out the PIK authors had used a dataset for Uzbekistan with “anomalies.” By removing the Uzbek dataset, the estimate in the original paper “aligns closely with previous literature,” the critique reads.
In a second Matters Arising, published a week later, Christof Schötz, a researcher at the Technical University of Munich in Ottobrunn, Germany, and a researcher at PIK, argued the analysis in the paper “underestimates uncertainty … rendering their results statistically insignificant when properly corrected.”
Here is Andrew Gelman's commentary:
It still seems crazy that the data from a single mid-sized country could have such a big effect of a global estimate. That’s something that the original researchers should’ve been aware of, and what it suggests to me is that there is a larger methodological problem that this didn’t get looked at automatically during the research process.
-
Authors retract Nature paper projecting high costs of climate change
14 votes -
Comment on AI text watermarking is free and good in ~comp
skybrian LinkThis is counterintuitive because we know that word choices matter when we write, so how can choosing a different word not matter? Surely there are subtle grades of meaning to consider when picking...This is counterintuitive because we know that word choices matter when we write, so how can choosing a different word not matter? Surely there are subtle grades of meaning to consider when picking one synonym over another?
That's going to depend on what words come before. In some positions, the next word will be highly constrained so it will always pick the next word the same way. There won't be any randomness at that position. At other places, the next word is a freer choice.
This won't prevent the AI labs from working on improving AI writing style. As training gets better, LLM's will become more sensitive about word choices and less likely to choose randomly. So, the amount of freedom delegated to the random number generator will drop. Probably not to zero, though!
If you want to see what that looks like today, talk to people who have retired, early or otherwise. There are plenty of retirees to talk to!
In my case, I do spend time helping relatives and I'm glad I've been able to drop everything and go when people needed me. I'm not helping random strangers, though, except with my donations. That's real work and I don't think it's going to automated any time soon?
Assuming there is a decline in other kinds of work (I'm uncertain about that), I imagine more work is going to be care-giving of various sorts. Healthcare is already a growing industry.
But even "everything except care-giving is automated" seems rather utopian? There are too many fields that AI has had no effect on yet.
Even for something like teaching where AI is having an effect, I can see it having an effect like "kids learn some more things from computers," but not "therefore nobody needs to take care of them." I imagine that the aspects of the job that are more about taking care of kids and motivating them get even more emphasis?