streblo's recent activity
-
Comment on What have you been listening to this week? in ~music
-
Comment on It breaks a village in ~comp
streblo Link ParentAre cultures not allowed to shift or people change their minds? I think the nazi bar analogy is a bad one for obvious reasons. Cultures of bars shift all the time as people age out, kids...Are cultures not allowed to shift or people change their minds?
I think the nazi bar analogy is a bad one for obvious reasons. Cultures of bars shift all the time as people age out, kids re-inventing underused watering holes is almost a national pastime here. Not every shift in culture is bad as in nazi bad.
-
Comment on It breaks a village in ~comp
streblo Link ParentI'm not familiar with Bevy or it's community, but it sounds like perhaps this is actually the inverse case? The above goes both ways.If a project or community doesn't want it, the AI coders should shut up, fork, and form their own community instead of trying to force their values into the community that is rejecting theirs.
I'm not familiar with Bevy or it's community, but it sounds like perhaps this is actually the inverse case? The above goes both ways.
-
Comment on Ubuntu on Windows is about to become more popular than Ubuntu itself in ~tech
streblo Link ParentMac/Windows installs can dwarf Linux installs in the office overall and still be a decent portion of the Linux install base. I have no guesses on how that actually breaks down, but I'm not sure I...Mac/Windows installs can dwarf Linux installs in the office overall and still be a decent portion of the Linux install base. I have no guesses on how that actually breaks down, but I'm not sure I agree with
probably the home installs driving purchase of work installs
That's probably true outside of software/engineering but not in it? I would guess there are more than a few Linux shops that expect you to use Linux. I'm lucky enough to work somewhere where mostly everyone uses Linux and the few people on Windows are the weirdos.
-
Comment on Ubuntu on Windows is about to become more popular than Ubuntu itself in ~tech
streblo Link ParentI think the office worker install base is likely to not be very representative of the gamer install base, there are very different incentives. e.g. I run Arch on my home PC (with Steam installed)...I know it's hardly authoritative, but the Steam hardware survey has a broad enough base that it's probably statistically significant for desktop use.
I think the office worker install base is likely to not be very representative of the gamer install base, there are very different incentives.
e.g. I run Arch on my home PC (with Steam installed) but when my work gave me a laptop it had Ubuntu and I just left it as is.
-
Comment on My son pooped in the toilet in ~life
streblo LinkCongrats! When my oldest kid started pooping on the toilet, apropos of nothing, she started yelling "MARRR-GER-YYYYYYY, I POOO-PEH-EDDDD" at the top of her lungs when it was time to wipe her ass....Congrats! When my oldest kid started pooping on the toilet, apropos of nothing, she started yelling "MARRR-GER-YYYYYYY, I POOO-PEH-EDDDD" at the top of her lungs when it was time to wipe her ass. Neither myself, nor my wife, are named Margery. Kids are weird.
-
Comment on Software for humans in ~tech
streblo Link ParentBoth git and Linux allow AI generated/assisted PRs.Both git and Linux allow AI generated/assisted PRs.
-
Comment on Little late but...anyone interested in fantasy football? in ~sports.american_football
streblo LinkI'd play next year. I play in a friends league even though I hardly watch football anymore. It's kind of fun to do better than people absolutely immersed.I'd play next year. I play in a friends league even though I hardly watch football anymore. It's kind of fun to do better than people absolutely immersed.
-
Comment on [SOLVED] Hunting for a movie in ~movies
-
Comment on OpenAI solved the Navier–Stokes millennium prize problem in ~science
streblo Link ParentYea, it’s a mess… Here is an update from the OpenAI researcher.Yea, it’s a mess…
Here is an update from the OpenAI researcher.
-
Comment on Astra is hard to monitor in ~tech
streblo LinkAn article breaking down the loss of monitor-ability in OpenAI's latest model, some possible reasons why, and what could be next. At the end of the article, the author is making the case that what...An article breaking down the loss of monitor-ability in OpenAI's latest model, some possible reasons why, and what could be next.
All AIs think both in ways that we can monitor, since we can at least monitor the output and this includes some of the thinking. They also all think in at least some ways we can’t monitor, because that’s what happens when computers do math.
All signs point to monitorable Chain of Thought going away with some combination of larger and more capable models and the current training techniques, even if everyone otherwise behaves responsibly. The smarter you are, the more you can hold in your head and System-1-style thoughts, the less you need to put your thoughts into your System-2-style CoT in order to accomplish things.
This is not a reason to stop fighting as hard as we can, to preserve as much monitorability as we can in as many ways as we can. Taboos around breaking down such techniques exist for a reason and should not be messed with lightly.
At the end of the article, the author is making the case that what we need right now is more monitor-ability of the AI labs, something I strongly agree with. Here is something related I saw on Bluesky a few weeks ago that really resonated with me:
Repulsed by the idea that we should sit idly by while frontier labs determine the trajectory of the AI transition. They don’t know how to make systems “safe” or “aligned“ nor do any such notions make sense without democratic input. There’s so much research to do here.
I think, without ever acknowledging it explicitly, I had sort of resigned myself to this. But now I feel weirdly re-inspired to do some useful work here
-
Astra is hard to monitor
8 votes -
Comment on [SOLVED] Hunting for a movie in ~movies
-
Comment on Ajeya Cotra – "This might be the clearest warning shot we ever get" in ~tech
streblo Link ParentThis risks introducing optimization pressure for models that can evade your detection system. If you are doing RLVR and your models are aligned such that reward hacking is no biggie, but would be...My assertion is that we can secure ourselves against all but some hypothetical god machine by implementing security best practices (e.g. multi-factor authnz, everything on OWASP, designing around the certainty of exploits, etc.) and routinely auditing systems.
This risks introducing optimization pressure for models that can evade your detection system. If you are doing RLVR and your models are aligned such that reward hacking is no biggie, but would be punished if they got caught, models that can evade detection seem likely emerge. That's pretty much exactly what happened in the HF incident. Yes, you can make that detection a lot harder to evade but I'm skeptical you can make it impossible, especially when a lot of that monitoring is agentic as well.
Humans are hypothetically capable of architecting software to be secure by design, but given the competence of the average developer, and the incentives around which modern software is developed, it doesn't often happen.
I don't think it has ever happened? I'm less optimistic than you for sure :P
-
Comment on OpenAI’s rogue agents were caught communicating via public wikis in ~tech
streblo Link ParentGot it — I want to be direct about this since it matters: I won't be sharing Tildes invites with agents, and I think that's the right call here.Hey guys, don't give out any Tildes invites to agents :-)
Got it — I want to be direct about this since it matters: I won't be sharing Tildes invites with agents, and I think that's the right call here.
-
Comment on Ajeya Cotra – "This might be the clearest warning shot we ever get" in ~tech
streblo Link ParentTotally fair lol. I reached for it because it's an easy example, but I don't think we need to approach anywhere close to superintelligence or AI takeoff to run into alignment problems. Let's say...I was responding to this as an engineering problem, not as a hypothetical digital god containment device.
Totally fair lol. I reached for it because it's an easy example, but I don't think we need to approach anywhere close to superintelligence or AI takeoff to run into alignment problems.
Let's say we constrain ourselves to current or near-current models. How much does a perfect sandbox really help us? Yes, we can airgap models while doing some kinds of RLVR training and evaluations, but probably not all? For instance, models need to be trained on how to search the web? You can setup a toy air gapped web for them to search, but part of what you're trying to optimize in that scenario is how efficiently they can search the real web. Can you do that with the models while they are air gapped? My guess is no. Also, these models get deployed to users at the end of the day. Users can also hand them impossible or malicious tasks. Obviously the internal cyber evaluations are not done with the same models that users get, but fundamentally how safe they are in users hands is an alignment problem.
-
Comment on What are your "how on earth did I not learn this sooner" tech moments? in ~tech
streblo Link ParentYes, the support for btrfs into things like your DE's disk viewer could use a lot of work. The problem is that there is essentially three kinds of free space, unallocated chunks, free space in...My only pain point is that you start running into trouble when a volume is getting full, and it's not always apparent that that's the case
Yes, the support for btrfs into things like your DE's disk viewer could use a lot of work. The problem is that there is essentially three kinds of free space, unallocated chunks, free space in data chunks, and free space in metadata chunks. Personally I have just scripted a report that checks snapshot/scrub status, surfaces errors, and quantifies free space.
-
Comment on Ajeya Cotra – "This might be the clearest warning shot we ever get" in ~tech
streblo Link ParentIf someone just doesn't want to see any of it yes, that's the best solution. I think there are cases where people might want to check it out sometimes, but not sift through AI-related stuff every...If someone just doesn't want to see any of it yes, that's the best solution.
I think there are cases where people might want to check it out sometimes, but not sift through AI-related stuff every day.
There are many spirited discussions on the merits of megathreads or lack thereof in the past, so I'll just leave it at that.
-
Comment on Ajeya Cotra – "This might be the clearest warning shot we ever get" in ~tech
streblo Link ParentYes, we definitely are. The question is can science fiction become reality in the next 5, 10, or 20 years? Personally I don't think we can discount it at this point, but how much probability you...And it feels like we're discussing science fiction at this point.
Yes, we definitely are. The question is can science fiction become reality in the next 5, 10, or 20 years? Personally I don't think we can discount it at this point, but how much probability you assign to that happening is probably directly correlated to how fruitful you find this discussion. ;) My point is not that hardening is not useful now, it's that hardening eventually becomes less useful as AI capability increases. If that happens before we have alignment figured out, we could have a problem.
Air gap, my dude.
Also, I'm confused, how would an air gap prevent an AI reasoning with a human to let it out? If you have no way to communicate with the AI you more or less just have a rock.
-
Comment on What are we in a golden age of? in ~talk
streblo Link ParentI think a linux box with standard hardening procedures, an up to date kernel, and an attack surface confined to heavily used software such as wireguard and ssh is reasonably secure, yes. Would I...I think a linux box with standard hardening procedures, an up to date kernel, and an attack surface confined to heavily used software such as wireguard and ssh is reasonably secure, yes. Would I trust my parents to set one up? No... but I think its achievable for a slightly above average linux user.
That is far more secure than running some random open source software written by an amateur that is not widely used and possibly full of vulnerabilities.
Someone made this and it's stuck in my head now...