bitshift's recent activity
-
Comment on 1Password wades into a right-wing mess after funding a Linux project in ~tech
-
Comment on 1Password wades into a right-wing mess after funding a Linux project in ~tech
bitshift Link ParentI'm in a similar boat. In terms of usability and security, 1Password is unfortunately the best existing solution for me. In an ideal world, there would be an open standard for syncing credentials...I'm in a similar boat. In terms of usability and security, 1Password is unfortunately the best existing solution for me.
In an ideal world, there would be an open standard for syncing credentials and legal requirements to ensure companies play nicely in that ecosystem. We shouldn't be in a position where we're forced to choose from a very small handful of providers, of which only one or two are actually secure. But there are reasons why that's hard and why we don't live in that ideal world.
So we're forced into an optimization problem. You can't have too many hard lines, because at a certain point you basically couldn't use technology at all. The best you can do is use the limited wiggle room you have to minimize how much you have to hold your nose. (And also, not judge anyone, yourself or others, for giving these companies money.)
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentThat's what I'm saying about code: it's more repetitive than prose. You often have to quote snippets of existing code verbatim, and you often do this multiple times in the same program. If...That's what I'm saying about code: it's more repetitive than prose. You often have to quote snippets of existing code verbatim, and you often do this multiple times in the same program.
If "AbstractWidgetFactoryManager" is long enough to contain a watermarked token, typing that into your code once isn't going to trigger the detector—but in a verbose language, you might have to repeat it multiple times. And there might be multiple such identifiers[1], especially if you're interfacing with a library written by AI.
I think it comes down to:
- What's the shortest length of quote that could still carry some watermark with it?
- How many times do you need to quote the AI before the watermark adds up to become detectable?
[1] Or combinations of identifiers. Boilerplate like
new FooBarManager(new BazQuuxConfig())might need to be repeated verbatim, and the combination could contain a watermark if the AI had a say in naming the config class.Obviously this is going to be more of an issue in languages with lots of boilerplate, which is why I'm picking on Java. As a contrast, I would think APL code would be very resistant to watermarking.
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentYeah, but I'm talking about the case where there's already a choice between equally-good next tokens: Here, "equally likely" is based on all the context you mentioned: surrounding code, style...Yeah, but I'm talking about the case where there's already a choice between equally-good next tokens:
Maybe the LLM says "Manager" and "Repository" are equally likely
Here, "equally likely" is based on all the context you mentioned: surrounding code, style guides, all of it. And what I'm saying is, if those two tokens are equally likely after taking all that into consideration (which is plausible), then the watermark algorithm has an opportunity to bias towards one of them.
The bias is determined by an RNG, which is seeded with the last several immediately prior tokens (not the LLM context). So if the last several tokens were "AbstractWidgetFactory", those go into the RNG and bias the coin flip. And in my example, "Manager" wins the biased coin flip, so the overall choice is "AbstractWidgetFactoryManager".
And from there, the choice is set in stone. From now on, it's no longer 50/50; "AbstractWidgetFactory" now has a 99.99% chance of being followed by "Manager". So there's no more wiggle room. It doesn't matter if you're a human, an AI, or a space alien: if you want your code to compile, "AbstractWidgetFactory" must be followed by "Manager".
But the watermark detector doesn't have access to the LLM probability distribution; it cannot even in principle recalculate it, because it doesn't know the exact original prompt. The only thing the detector knows is the RNG seed, which comes from hashing the immediately preceding tokens.
So as I understand it, the watermark detector runs a sliding window over its input, calculating the RNG seed at each position. It doesn't know the LLM percentages, but it can look at a single token and tell you if the current RNG "likes" it or not. For human text, the RNG will only like the next token 50% of the time. But for AI text, the RNG will like it slightly more often, say 51% of the time.
So what happens when the detector reads human-written code with that identifier in it?
AbstractWidgetFactoryManager awfm = new AbstractWidgetFactoryManager(123);- It's going to run its sliding window over "AbstractWidgetFactory", seed its RNG with that, and the RNG looks at what immediately follows: "Manager". Oh, the RNG likes that!
- Then it advances the sliding window over some other tokens. The RNG likes half of them.
- Then the sliding window is "AbstractWidgetFactory" again, the same seed. And what follows? "Manager", of course! The RNG likes that, just the same as it had liked it before, because same seed.
- Then it reads some more tokens. Again, the RNG likes half of them on average.
So the detector adds all those "likes" after reading the entire file, and it discovers that the RNG liked 55% of the tokens. So the file as a whole is flagged as written by AI, because the average "like" rate is too high. But AI didn't write it! The AI wrote a fragment of text that was long enough to contain a watermark, and by necessity, the human author was forced to repeat that fragment of text multiple times in their own program.
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentI'm curious how this will affect stuff like class and function names. A long name like AbstractWidgetFactory consists of multiple tokens, which might be long enough to start to see watermark...I'm curious how this will affect stuff like class and function names.
A long name like
AbstractWidgetFactoryconsists of multiple tokens, which might be long enough to start to see watermark effects. What do you call the class that coordinates them? Maybe the LLM says "Manager" and "Repository" are equally likely, but given the last few tokens, the watermark algorithm biases towardsAbstractWidgetFactoryManager.Now you've got a watermarked sequence of tokens that you have to repeat verbatim everywhere, regardless of whether you're AI or not:
AbstractWidgetFactoryManager awfm = new AbstractWidgetFactoryManager(123);Code is more constrained than prose, but it's also more repetitive, which might allow token sequences to leak into nearby code. It's not a problem for prose, because we don't quote each other with every other sentence.
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentWell, that's not the output that users receive. If you want to get technical about it, the output the user receives is a sampling of a distribution. Digging into the paper linked to from the...The probability distribution is the output of the LLM.
Well, that's not the output that users receive. If you want to get technical about it, the output the user receives is a sampling of a distribution.
Digging into the paper linked to from the article:
Generative watermarking (Fig. 1, bottom) works by carefully modifying the next-token sampling procedure to inject subtle, context-specific modifications into the generated text distribution.
Are they modifying the sampling procedure? The distribution? Some of this is just quibbling over definitions, but fortunately they supply multiple mathematical definitions about the degree to which output texts are distorted:
Choosing the level of non-distortion involves a trade-off; weaker levels of non-distortion can reduce text quality and diversity, whereas stronger levels of non-distortion can reduce detectability and increase computational complexity (Supplementary Information section G.3). For our experiments, we configure SynthID-Text to be single-sequence non-distortionary; this preserves text quality and provides good detectability, while having some reduction to inter-response diversity.
My takeaways from that:
- There are different settings. They ran their experiment on "low" and their users didn't notice.
- Even on "low", it affected diversity of responses. Which @skybrian kind of mentioned already: you can generate multiple times to get second opinions, but due to the biased sampling you'll get less new information per generation.
- It might not be set on "low" for you. Your provider can adjust the dial to whatever feels right to them—which might not be aligned with your preferences.
- Your provider might not even be using this algorithm! They might be watermarking using red/green lists, which, as I understand it (I could be wrong), bias generation in a way that is less mathematically ideal than SynthID.
I will say this: if we have to have watermarking, I do believe Scott Aaronson and friends did their homework here, and I'd rather have watermarking that preserves subtle mathematical properties (assuming that vendors have it at the lowest setting). But it's a matter of principle to me. Going back to the doctor analogy, even if you could mathematically prove to me that the pharmaceutical company's visit did not reduce my quality of care, that little doubt is still going to stick around in the back of my mind: did my doctor prescribe this because it's best for me, or for someone else?
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentBecause in an ideal world, the probability distribution of the dice is optimized to serve me. I draw a distinction between the variance versus what causes it. Humans are random, too! If I go to...They're rolling the dice to decide what to tell you! How does that serve you?
Because in an ideal world, the probability distribution of the dice is optimized to serve me.
I draw a distinction between the variance versus what causes it. Humans are random, too! If I go to the doctor, there's lots of factors I can't control: my name reminds them of a previous case, they didn't sleep well last night, a colleague told them a funny story over lunch, etc… any of which could subtly affect the treatment I receive. That's variance. But it's inherent to the process: if I see my doctor yawn, I'm not gonna go "how dare you get a bad night's sleep" because everyone's day has random events. They might make mistakes, but as long as they're trying their best to uphold their oath, I'm happy.
Now, suppose on my next visit, I see new brochures for a pharmaceutical company. Alongside all the existing variance, there's an additional random event: whether this company wined and dined my doctor. I can't personally tell if it caused my own medical care to change. Maybe they prescribe a certain medication 1% more often, but you'd never know unless you did statistics across thousands of patients. And maybe it's a decent medication—it doesn't always work, but it's about as effective as the physical therapy they would have otherwise prescribed. No net harm.
But I would still feel uneasy. Even though the "before" and "after" are in some sense equivalent, the doctor still allowed an additional influence on their practice, whose purpose was not exclusively to make me healthier.
-
Comment on How to write a short story in five sentences in ~creative
bitshift LinkThis five-sentence structure feels a lot like a form of poetry to me. You have to craft a story carefully, without fluff. And it sounds much more prescriptive than most story structures: there are...This five-sentence structure feels a lot like a form of poetry to me. You have to craft a story carefully, without fluff. And it sounds much more prescriptive than most story structures: there are constraints, e.g., "a character must reveal a secret in Sentence 3," or "Sentence 5 must evoke one of the five senses."
I like it.
The author said it was from a week-long course—I'm curious what else they learned that week!
-
Comment on AI text watermarking is free and good in ~comp
bitshift LinkFrom the "what you don't know can't hurt you" department: I have a visceral reaction against this. First of all, "has no effect" is not the same thing as "users can't tell". It's like saying the...From the "what you don't know can't hurt you" department:
This has no practical impact on outputs. Humans cannot tell the difference, at all.
I have a visceral reaction against this. First of all, "has no effect" is not the same thing as "users can't tell". It's like saying the drug trial was a success because users couldn't tell whether they got the placebo. That's cool, but isn't it more important to know if the drug harms its users?
But even if watermarking didn't degrade the LLM's output at all, I would still be deeply bothered by it. It's the same way I feel about printer watermarks. I'm not an activist or a counterfeiter, so why should I care? Because when I pay money for a tool, I expect it to serve me wholeheartedly. And if it goes out of its way to do something not purely in my best interests, that violates my expectation of what a tool ought to be.
-
Comment on Less than a year on, Microsoft tells Mico to pipe down in ~tech
bitshift Link ParentI'm trying to remember why people mock Clippy. How much of it was stuff inherent to Clippy, such as getting in the way and being unhelpful—versus people's tendency to mock any character, no matter...I'm trying to remember why people mock Clippy. How much of it was stuff inherent to Clippy, such as getting in the way and being unhelpful—versus people's tendency to mock any character, no matter who it is?
I have fond memories of being bored while writing essays, clicking on Clippy, and triggering the little bonus animations.
-
Comment on Any cool coding languages? in ~comp
bitshift LinkIf you're looking for a language that would be interesting to learn and easy to port, I've heard lots of good things about Forth. It's a very different paradigm from most programming languages...If you're looking for a language that would be interesting to learn and easy to port, I've heard lots of good things about Forth. It's a very different paradigm from most programming languages (including the ones you mentioned), so it might be a good choice if you want to expand your mind. And it has a reputation of being relatively easy to port; it's easy enough that a traditional rite of passage in the Forth community is to write your own Forth interpreter from scratch.
(Disclaimer: I have not yet written my own Forth. Someday…)
I don't know what the best learning resource is, but I'm immensely fond of Dave Gauer's introduction to Forth. It's a ton of text, but his writing is engaging, and I love his whimsical illustrations. He does an amazing job of explaining why one should care about this little oddball language from the 70s.
-
Comment on What happened: OpenAI and Hugging Face in ~comp
bitshift Link ParentOr as it appears, a short version of five much longer posts, the last two of which have estimated reading times of over an hour. (In contrast, Firefox reader says this short version is 18–22...This is the short version of a much longer post.
Or as it appears, a short version of five much longer posts, the last two of which have estimated reading times of over an hour. (In contrast, Firefox reader says this short version is 18–22 minutes.)
A while back, I subscribed to Zvi in my feed reader because I wanted to be more informed, but I never end up actually reading his posts because they're gigantic. I don't want to be judgemental because hey, I don't have to read it if I don't like it. But it makes me wonder: who is reading his posts? Are most of his subscribers aggressively skimming or using AI summaries?
-
Comment on I wanted a better Gemini web proxy, so I built a single-file PHP gateway in ~comp
bitshift Link ParentIt might be one of those situations where no matter what they had said, it would have sounded bad. Realistically, the name was never going to change, so you can't have a proper apology: "Hey,...It might be one of those situations where no matter what they had said, it would have sounded bad.
Realistically, the name was never going to change, so you can't have a proper apology: "Hey, sorry for taking the name, but actually not sorry because we want it more than you do." And maybe they really did feel bad about stepping on someone's toes, and they wanted to acknowledge that. But sometimes a token gesture is worse than nothing.
-
Comment on I wanted a better Gemini web proxy, so I built a single-file PHP gateway in ~comp
bitshift Link ParentBefore I got to your second paragraph, I also was thinking of the Go! language. (Wiki article for folks who haven't seen it.)Before I got to your second paragraph, I also was thinking of the Go! language. (Wiki article for folks who haven't seen it.)
-
Comment on An emotional day, and a request about pets in ~life.pets
bitshift Link ParentThere are plenty of gray areas. Maybe it's a smart rule because the executives considered all possible rules and chose the one that would result in the least suffering—but at the same time it's...There are plenty of gray areas. Maybe it's a smart rule because the executives considered all possible rules and chose the one that would result in the least suffering—but at the same time it's also a dumb rule because you could eliminate even more suffering if you could trust your employees with case-by-case discretion (which might not be possible). Or you mentioned that good employees might be coerced into following bad rules. It's also possible for good executives to be coerced into making bad rules: "This rule will cause suffering, but it'll look bad if we don't impose it, and the board and/or the public will have our heads."
It's hard to say how the blame should be divvied up. On one hand, banality of evil, "Just following orders" shouldn't be a get-out-of-jail-free card. But on the other hand, it's hard to get angry at someone who is also being mistreated by the system.
Kudos to you for being more charitable that I was.
-
Comment on The magic is in the language in ~tech
bitshift Link ParentI feel like this underestimates folks from past eras, especially as late as 1950. You don't need to understand how something works to accept it. The Mechanical Turk actually was a demon trapped in...Like if you showed a random person from any time before around 1950 a cell phone running ChatGPT in interactive mode, the only possible explanation virtually anyone would be able to come up with is that you captured a demon inside a smooth rock.
I feel like this underestimates folks from past eras, especially as late as 1950. You don't need to understand how something works to accept it.
The Mechanical Turk actually was a demon trapped in a smooth rock—well, technically a human trapped in a wooden cabinet. Close enough. But it was presented as technology: nothing but gears inside, no strings leading elsewhere, use this lodestone to see that it's not being controlled magnetically, etc. And it sounds like audiences in the 1700s and 1800s accepted that the intelligence was mechanical? There were definitely skeptics, but I get the sense that it was not obviously a hoax. Technology was a plausible explanation for the wondrous even back then.
Then again, I do see this quote on Wikipedia:
[o]ne old lady, in particular, who had not forgotten the tales she had been told in her youth ... went and hid herself in a window seat, as distant as she could from the evil spirit, which she firmly believed possessed the machine.
So maybe out of an abundance of caution, don't wave around your smartphone in front of just anyone the next time you visit the 1780s—get to know them first.
-
Comment on An emotional day, and a request about pets in ~life.pets
bitshift LinkIt's not the biggest baddest evil in the universe right now, and I don't know why I feel conviction in this one instance—probably because animals, I dunno. But worshipping rules, to the exclusion...But the shelter wouldn't let us.
It's not the biggest baddest evil in the universe right now, and I don't know why I feel conviction in this one instance—probably because animals, I dunno. But worshipping rules, to the exclusion of helping people, is an act of evil. (And that's coming from a computer/math nerd. I like rules.)
Sorry you had to go through that double whammy. Wishing you a better week to come, and tell your cat I said ps ps ps.
-
Comment on Discovering cryptographic weaknesses with Claude in ~comp
bitshift LinkRelated: Matthew Green reacting to Anthropic's results. No huge surprises. His understanding seems to roughly line up with what I had gotten: that these results are real and academically...Related: Matthew Green reacting to Anthropic's results. No huge surprises. His understanding seems to roughly line up with what I had gotten: that these results are real and academically interesting, but they don't impact real-world security.
I was happy to see a professional cryptographer react to the results, though, and I hope to see more. When Anthropic said things like this in their blog post:
Researchers at Anthropic then spent several hundred hours learning enough cryptography research to validate the model’s claim
…I got a teensy bit nervous. Spending a month or two doing a cryptography crash course is better than nothing, but what would have been even better is if, from the outset, they had partnered with someone who was already a cryptography researcher.
-
Comment on I don't really like the "Human-made" badging trend in ~tech
bitshift Link ParentIt has crossed my mind that with the advent of LLMs, it is now cheaper to run large text-based psyop campaigns. If you were concerned about the rapid development of AI by American companies, one...It has crossed my mind that with the advent of LLMs, it is now cheaper to run large text-based psyop campaigns. If you were concerned about the rapid development of AI by American companies, one thing you might be tempted to do is to sow discord in tech communities. Plant a bunch of personas, stir up some anti-AI sentiment. Maybe encourage a few software projects to fork needlessly. Start a few boycotts. (For maximum effectiveness, don't forget to work the other side, too! Convince a few startups that the secret to their success was AI. Place some accounts to parrot the narrative to CTOs of larger companies. Plant the idea of massive layoffs + AI = massive profit.)
But again, Occam's razor. I don't seriously believe that conspiracy theory because in all likelihood, folks just enjoy black-and-white narratives too much, and that naturally leads to polarization—and explains why conspiracy theories are so seductive, too.
-
Comment on What programming/technical projects have you been working on? in ~comp
bitshift Link ParentI'm curious to hear more about what you're thinking in terms of UX. My impression is that Discourse is the de facto default for "modern" forums these days, and to the limited extent that I've used...I'm curious to hear more about what you're thinking in terms of UX. My impression is that Discourse is the de facto default for "modern" forums these days, and to the limited extent that I've used it, it feels okay. But it sounds like you have a more specific vision for UX that's different from theirs.
True, fair enough!
When I wrote that, one of the things on my mind was stuff like passkeys being locked down to devices/accounts—which seems like the sort of thing that Apple/Google/etc would be excited about, and not for altruistic reasons. I presume 1Password can export passkeys though. It just requires diligence in knowing where your keys are stored.