bitshift's recent activity
-
Comment on The story of Mel - New to me in ~comp
-
Comment on Anthropic's Claude AI blocks video game cloning and reverse-engineering — sweeping new restrictions also hit legitimate developers and basic troubleshooting tasks in ~tech
bitshift Link ParentWhen I first read this, I misread it as, "More generally, the lesson learned is that any provider of online services cannot be trusted." Which, y'know, has its own truth.More generally, the lesson learned by any provider of online services is that users cannot be trusted.
When I first read this, I misread it as, "More generally, the lesson learned is that any provider of online services cannot be trusted."
Which, y'know, has its own truth.
-
Comment on Popular fanfic site "Archive of Our Own" (AO3) down for 2-4 weeks in ~tech
bitshift Link ParentI chuckled. (I assume this is good natured.) But on a more serious note, let's not forget that fans fight for our rights—even if you personally don't read or write fanfic! I like Maciej...I chuckled. (I assume this is good natured.) But on a more serious note, let's not forget that fans fight for our rights—even if you personally don't read or write fanfic!
I like Maciej Cegłowski's "Fan is A Tool-Using Animal", and the second half of that talk does a good job of spelling out why fandom is important to culture in general. Headings include:
- Fans Fight Censorship
- Fans Fight For Privacy
- Fans Improve Our Culture
Definitely worth skimming that part of the transcript if you have a few minutes.
-
Comment on 4 × 6 in ~games
bitshift Link ParentThe game itself is fine, though some of the text feels like an LLM wrote it: The phrasing just feels off to me. "solver's line"? 100% agree. It's not a bad game, but it's definitely not original....The game itself is fine, though some of the text feels like an LLM wrote it:
Like colors with Like. Pull from the right. Unscramble the board.
Matched the solver's line exactly, 2 under par.
The phrasing just feels off to me. "solver's line"?
My biggest criticism for this one is that it feels like one of those test-tube color sort mobile games
100% agree. It's not a bad game, but it's definitely not original. Feels sort of like going to McDonald's: yeah, it's a burger, it tastes exactly like you'd expect, but unless you're craving junk food or nothing else is open, there's nothing compelling that will bring you back.
-
Comment on The internet runs on cats and porn. It’s about to get cattier and pornier. in ~tech
bitshift Link ParentWe coined some words with the old TLDs. When I see ".org" it doesn't trigger the concept "organization" in my brain, in the same way that a capitalized "Smith" doesn't reliably evoke the concept...the shortenings were necessary due to technical constraints that we don't have today, but like, .txt feels so much more right than .text, doesn't it? Or .exe instead of .executible? It's just so iconic.
We coined some words with the old TLDs. When I see ".org" it doesn't trigger the concept "organization" in my brain, in the same way that a capitalized "Smith" doesn't reliably evoke the concept of metalworking. It produces its own concept now.
".txt" is the only file extension I will naturally translate to "text file" when reading out loud. But all the others are their own words to me. Calling something an "EXE file" doesn't even have the same meaning as "executable": it's a different, more precise term. Sort of like "lavender" versus "purple", or "hatchback" versus "car".
-
Comment on 1Password wades into a right-wing mess after funding a Linux project in ~tech
bitshift Link ParentTrue, fair enough! When I wrote that, one of the things on my mind was stuff like passkeys being locked down to devices/accounts—which seems like the sort of thing that Apple/Google/etc would be...True, fair enough!
When I wrote that, one of the things on my mind was stuff like passkeys being locked down to devices/accounts—which seems like the sort of thing that Apple/Google/etc would be excited about, and not for altruistic reasons. I presume 1Password can export passkeys though. It just requires diligence in knowing where your keys are stored.
-
Comment on 1Password wades into a right-wing mess after funding a Linux project in ~tech
bitshift Link ParentI'm in a similar boat. In terms of usability and security, 1Password is unfortunately the best existing solution for me. In an ideal world, there would be an open standard for syncing credentials...I'm in a similar boat. In terms of usability and security, 1Password is unfortunately the best existing solution for me.
In an ideal world, there would be an open standard for syncing credentials and legal requirements to ensure companies play nicely in that ecosystem. We shouldn't be in a position where we're forced to choose from a very small handful of providers, of which only one or two are actually secure. But there are reasons why that's hard and why we don't live in that ideal world.
So we're forced into an optimization problem. You can't have too many hard lines, because at a certain point you basically couldn't use technology at all. The best you can do is use the limited wiggle room you have to minimize how much you have to hold your nose. (And also, not judge anyone, yourself or others, for giving these companies money.)
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentThat's what I'm saying about code: it's more repetitive than prose. You often have to quote snippets of existing code verbatim, and you often do this multiple times in the same program. If...That's what I'm saying about code: it's more repetitive than prose. You often have to quote snippets of existing code verbatim, and you often do this multiple times in the same program.
If "AbstractWidgetFactoryManager" is long enough to contain a watermarked token, typing that into your code once isn't going to trigger the detector—but in a verbose language, you might have to repeat it multiple times. And there might be multiple such identifiers[1], especially if you're interfacing with a library written by AI.
I think it comes down to:
- What's the shortest length of quote that could still carry some watermark with it?
- How many times do you need to quote the AI before the watermark adds up to become detectable?
[1] Or combinations of identifiers. Boilerplate like
new FooBarManager(new BazQuuxConfig())might need to be repeated verbatim, and the combination could contain a watermark if the AI had a say in naming the config class.Obviously this is going to be more of an issue in languages with lots of boilerplate, which is why I'm picking on Java. As a contrast, I would think APL code would be very resistant to watermarking.
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentYeah, but I'm talking about the case where there's already a choice between equally-good next tokens: Here, "equally likely" is based on all the context you mentioned: surrounding code, style...Yeah, but I'm talking about the case where there's already a choice between equally-good next tokens:
Maybe the LLM says "Manager" and "Repository" are equally likely
Here, "equally likely" is based on all the context you mentioned: surrounding code, style guides, all of it. And what I'm saying is, if those two tokens are equally likely after taking all that into consideration (which is plausible), then the watermark algorithm has an opportunity to bias towards one of them.
The bias is determined by an RNG, which is seeded with the last several immediately prior tokens (not the LLM context). So if the last several tokens were "AbstractWidgetFactory", those go into the RNG and bias the coin flip. And in my example, "Manager" wins the biased coin flip, so the overall choice is "AbstractWidgetFactoryManager".
And from there, the choice is set in stone. From now on, it's no longer 50/50; "AbstractWidgetFactory" now has a 99.99% chance of being followed by "Manager". So there's no more wiggle room. It doesn't matter if you're a human, an AI, or a space alien: if you want your code to compile, "AbstractWidgetFactory" must be followed by "Manager".
But the watermark detector doesn't have access to the LLM probability distribution; it cannot even in principle recalculate it, because it doesn't know the exact original prompt. The only thing the detector knows is the RNG seed, which comes from hashing the immediately preceding tokens.
So as I understand it, the watermark detector runs a sliding window over its input, calculating the RNG seed at each position. It doesn't know the LLM percentages, but it can look at a single token and tell you if the current RNG "likes" it or not. For human text, the RNG will only like the next token 50% of the time. But for AI text, the RNG will like it slightly more often, say 51% of the time.
So what happens when the detector reads human-written code with that identifier in it?
AbstractWidgetFactoryManager awfm = new AbstractWidgetFactoryManager(123);- It's going to run its sliding window over "AbstractWidgetFactory", seed its RNG with that, and the RNG looks at what immediately follows: "Manager". Oh, the RNG likes that!
- Then it advances the sliding window over some other tokens. The RNG likes half of them.
- Then the sliding window is "AbstractWidgetFactory" again, the same seed. And what follows? "Manager", of course! The RNG likes that, just the same as it had liked it before, because same seed.
- Then it reads some more tokens. Again, the RNG likes half of them on average.
So the detector adds all those "likes" after reading the entire file, and it discovers that the RNG liked 55% of the tokens. So the file as a whole is flagged as written by AI, because the average "like" rate is too high. But AI didn't write it! The AI wrote a fragment of text that was long enough to contain a watermark, and by necessity, the human author was forced to repeat that fragment of text multiple times in their own program.
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentI'm curious how this will affect stuff like class and function names. A long name like AbstractWidgetFactory consists of multiple tokens, which might be long enough to start to see watermark...I'm curious how this will affect stuff like class and function names.
A long name like
AbstractWidgetFactoryconsists of multiple tokens, which might be long enough to start to see watermark effects. What do you call the class that coordinates them? Maybe the LLM says "Manager" and "Repository" are equally likely, but given the last few tokens, the watermark algorithm biases towardsAbstractWidgetFactoryManager.Now you've got a watermarked sequence of tokens that you have to repeat verbatim everywhere, regardless of whether you're AI or not:
AbstractWidgetFactoryManager awfm = new AbstractWidgetFactoryManager(123);Code is more constrained than prose, but it's also more repetitive, which might allow token sequences to leak into nearby code. It's not a problem for prose, because we don't quote each other with every other sentence.
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentWell, that's not the output that users receive. If you want to get technical about it, the output the user receives is a sampling of a distribution. Digging into the paper linked to from the...The probability distribution is the output of the LLM.
Well, that's not the output that users receive. If you want to get technical about it, the output the user receives is a sampling of a distribution.
Digging into the paper linked to from the article:
Generative watermarking (Fig. 1, bottom) works by carefully modifying the next-token sampling procedure to inject subtle, context-specific modifications into the generated text distribution.
Are they modifying the sampling procedure? The distribution? Some of this is just quibbling over definitions, but fortunately they supply multiple mathematical definitions about the degree to which output texts are distorted:
Choosing the level of non-distortion involves a trade-off; weaker levels of non-distortion can reduce text quality and diversity, whereas stronger levels of non-distortion can reduce detectability and increase computational complexity (Supplementary Information section G.3). For our experiments, we configure SynthID-Text to be single-sequence non-distortionary; this preserves text quality and provides good detectability, while having some reduction to inter-response diversity.
My takeaways from that:
- There are different settings. They ran their experiment on "low" and their users didn't notice.
- Even on "low", it affected diversity of responses. Which @skybrian kind of mentioned already: you can generate multiple times to get second opinions, but due to the biased sampling you'll get less new information per generation.
- It might not be set on "low" for you. Your provider can adjust the dial to whatever feels right to them—which might not be aligned with your preferences.
- Your provider might not even be using this algorithm! They might be watermarking using red/green lists, which, as I understand it (I could be wrong), bias generation in a way that is less mathematically ideal than SynthID.
I will say this: if we have to have watermarking, I do believe Scott Aaronson and friends did their homework here, and I'd rather have watermarking that preserves subtle mathematical properties (assuming that vendors have it at the lowest setting). But it's a matter of principle to me. Going back to the doctor analogy, even if you could mathematically prove to me that the pharmaceutical company's visit did not reduce my quality of care, that little doubt is still going to stick around in the back of my mind: did my doctor prescribe this because it's best for me, or for someone else?
-
Comment on AI text watermarking is free and good in ~comp
bitshift Link ParentBecause in an ideal world, the probability distribution of the dice is optimized to serve me. I draw a distinction between the variance versus what causes it. Humans are random, too! If I go to...They're rolling the dice to decide what to tell you! How does that serve you?
Because in an ideal world, the probability distribution of the dice is optimized to serve me.
I draw a distinction between the variance versus what causes it. Humans are random, too! If I go to the doctor, there's lots of factors I can't control: my name reminds them of a previous case, they didn't sleep well last night, a colleague told them a funny story over lunch, etc… any of which could subtly affect the treatment I receive. That's variance. But it's inherent to the process: if I see my doctor yawn, I'm not gonna go "how dare you get a bad night's sleep" because everyone's day has random events. They might make mistakes, but as long as they're trying their best to uphold their oath, I'm happy.
Now, suppose on my next visit, I see new brochures for a pharmaceutical company. Alongside all the existing variance, there's an additional random event: whether this company wined and dined my doctor. I can't personally tell if it caused my own medical care to change. Maybe they prescribe a certain medication 1% more often, but you'd never know unless you did statistics across thousands of patients. And maybe it's a decent medication—it doesn't always work, but it's about as effective as the physical therapy they would have otherwise prescribed. No net harm.
But I would still feel uneasy. Even though the "before" and "after" are in some sense equivalent, the doctor still allowed an additional influence on their practice, whose purpose was not exclusively to make me healthier.
-
Comment on How to write a short story in five sentences in ~books
bitshift LinkThis five-sentence structure feels a lot like a form of poetry to me. You have to craft a story carefully, without fluff. And it sounds much more prescriptive than most story structures: there are...This five-sentence structure feels a lot like a form of poetry to me. You have to craft a story carefully, without fluff. And it sounds much more prescriptive than most story structures: there are constraints, e.g., "a character must reveal a secret in Sentence 3," or "Sentence 5 must evoke one of the five senses."
I like it.
The author said it was from a week-long course—I'm curious what else they learned that week!
-
Comment on AI text watermarking is free and good in ~comp
bitshift LinkFrom the "what you don't know can't hurt you" department: I have a visceral reaction against this. First of all, "has no effect" is not the same thing as "users can't tell". It's like saying the...From the "what you don't know can't hurt you" department:
This has no practical impact on outputs. Humans cannot tell the difference, at all.
I have a visceral reaction against this. First of all, "has no effect" is not the same thing as "users can't tell". It's like saying the drug trial was a success because users couldn't tell whether they got the placebo. That's cool, but isn't it more important to know if the drug harms its users?
But even if watermarking didn't degrade the LLM's output at all, I would still be deeply bothered by it. It's the same way I feel about printer watermarks. I'm not an activist or a counterfeiter, so why should I care? Because when I pay money for a tool, I expect it to serve me wholeheartedly. And if it goes out of its way to do something not purely in my best interests, that violates my expectation of what a tool ought to be.
-
Comment on Less than a year on, Microsoft tells Mico to pipe down in ~tech
bitshift Link ParentI'm trying to remember why people mock Clippy. How much of it was stuff inherent to Clippy, such as getting in the way and being unhelpful—versus people's tendency to mock any character, no matter...I'm trying to remember why people mock Clippy. How much of it was stuff inherent to Clippy, such as getting in the way and being unhelpful—versus people's tendency to mock any character, no matter who it is?
I have fond memories of being bored while writing essays, clicking on Clippy, and triggering the little bonus animations.
-
Comment on Any cool coding languages? in ~comp
bitshift LinkIf you're looking for a language that would be interesting to learn and easy to port, I've heard lots of good things about Forth. It's a very different paradigm from most programming languages...If you're looking for a language that would be interesting to learn and easy to port, I've heard lots of good things about Forth. It's a very different paradigm from most programming languages (including the ones you mentioned), so it might be a good choice if you want to expand your mind. And it has a reputation of being relatively easy to port; it's easy enough that a traditional rite of passage in the Forth community is to write your own Forth interpreter from scratch.
(Disclaimer: I have not yet written my own Forth. Someday…)
I don't know what the best learning resource is, but I'm immensely fond of Dave Gauer's introduction to Forth. It's a ton of text, but his writing is engaging, and I love his whimsical illustrations. He does an amazing job of explaining why one should care about this little oddball language from the 70s.
-
Comment on What happened: OpenAI and Hugging Face in ~comp
bitshift Link ParentOr as it appears, a short version of five much longer posts, the last two of which have estimated reading times of over an hour. (In contrast, Firefox reader says this short version is 18–22...This is the short version of a much longer post.
Or as it appears, a short version of five much longer posts, the last two of which have estimated reading times of over an hour. (In contrast, Firefox reader says this short version is 18–22 minutes.)
A while back, I subscribed to Zvi in my feed reader because I wanted to be more informed, but I never end up actually reading his posts because they're gigantic. I don't want to be judgemental because hey, I don't have to read it if I don't like it. But it makes me wonder: who is reading his posts? Are most of his subscribers aggressively skimming or using AI summaries?
-
Comment on I wanted a better Gemini web proxy, so I built a single-file PHP gateway in ~comp
bitshift Link ParentIt might be one of those situations where no matter what they had said, it would have sounded bad. Realistically, the name was never going to change, so you can't have a proper apology: "Hey,...It might be one of those situations where no matter what they had said, it would have sounded bad.
Realistically, the name was never going to change, so you can't have a proper apology: "Hey, sorry for taking the name, but actually not sorry because we want it more than you do." And maybe they really did feel bad about stepping on someone's toes, and they wanted to acknowledge that. But sometimes a token gesture is worse than nothing.
-
Comment on I wanted a better Gemini web proxy, so I built a single-file PHP gateway in ~comp
bitshift Link ParentBefore I got to your second paragraph, I also was thinking of the Go! language. (Wiki article for folks who haven't seen it.)Before I got to your second paragraph, I also was thinking of the Go! language. (Wiki article for folks who haven't seen it.)
-
Comment on An emotional day, and a request about pets in ~life.pets
bitshift Link ParentThere are plenty of gray areas. Maybe it's a smart rule because the executives considered all possible rules and chose the one that would result in the least suffering—but at the same time it's...There are plenty of gray areas. Maybe it's a smart rule because the executives considered all possible rules and chose the one that would result in the least suffering—but at the same time it's also a dumb rule because you could eliminate even more suffering if you could trust your employees with case-by-case discretion (which might not be possible). Or you mentioned that good employees might be coerced into following bad rules. It's also possible for good executives to be coerced into making bad rules: "This rule will cause suffering, but it'll look bad if we don't impose it, and the board and/or the public will have our heads."
It's hard to say how the blame should be divvied up. On one hand, banality of evil, "Just following orders" shouldn't be a get-out-of-jail-free card. But on the other hand, it's hard to get angry at someone who is also being mistreated by the system.
Kudos to you for being more charitable that I was.
Possibly of interest: Bryan Cantrill's talk on Oral Tradition in Software Engineering (2016). He talks about the story of Mel for a bit, about 11 minutes into the recording.