streblo's recent activity

  1. Comment on Timasomo 2026: Roll Call in ~creative.timasomo

    streblo
    Link Parent
    I am also going to be painting some kin, but with normal heads! :P I have had two einhyr champions sitting on my shelf that have been assembled and primed for a few months, so I'll be painting...

    I am also going to be painting some kin, but with normal heads! :P I have had two einhyr champions sitting on my shelf that have been assembled and primed for a few months, so I'll be painting both of those.

    Your scheme sounds diabolical and awesome btw, I love it.

    3 votes

  2. Comment on An interesting take on Reddit about why the art direction of World of Warcraft Forever misses the mark in ~games

    streblo
    Link
    Some of the stuff is definitely missing the mark, e.g. the new druid forms, some of the new weapons, and to some extent the new race. However, the new zones and dungeons have given me confidence...

    Some of the stuff is definitely missing the mark, e.g. the new druid forms, some of the new weapons, and to some extent the new race.

    However, the new zones and dungeons have given me confidence they can do it. The new zones and dungeons I've seen fit perfectly with Vanilla. Now, they do get to use a lot of Vanilla assets for these, so that's part of it. But they very much speak the design language.

    I think where they are still adapting is where they have to create new assets. Part of this seems like a tooling and skill problem, e.g. they have to create 2004 assets with 2026 tools and that's going to take some time to perfect. And part of it is definitely the art design, but they seem to be responding to feedback here, so my hope is that it improves as time goes on.

    1 vote

  3. Comment on Announcing the eighth annual Tildes' Make Something Month (Timasomo)! in ~tildes

    streblo
    Link
    I’m going to sign up just to force myself to paint some of my minis I need to paint ;)

    I’m going to sign up just to force myself to paint some of my minis I need to paint ;)

    1 vote

  4. Comment on Cornell won't, we will in ~news

    streblo
    Link Parent
    Those are all very fair points, I get where you're coming from. For me, this is the point when I'm commenting on a news article. I like to think about systems, about incentives, about how to make...

    Those are all very fair points, I get where you're coming from.

    The point isn't productivity or logic. The point is trying to explain the feelings and the experience. That chart isn't to scale or proportionate. You'll notice a distinct lack of number or norming. The point is that Not All Men is not useful or relevant.

    For me, this is the point when I'm commenting on a news article. I like to think about systems, about incentives, about how to make the world a better place and my comments are usually pointed in that direction.

    6 votes

  5. Comment on Cornell won't, we will in ~news

    streblo
    Link Parent
    I'm specifically not trying to tell you how to feel, only offering my opinion. Nor did I necessarily think or mean to imply you were signing off on every detail of the image you linked. I don't...

    I'm specifically not trying to tell you how to feel, only offering my opinion. Nor did I necessarily think or mean to imply you were signing off on every detail of the image you linked.

    telling the angry traumatized people that they're too nihilistic is unproductive

    I don't think believing that one out of every two men is a misogynist and one out of every four is a predator is productive. Not all beliefs need to be productive, if someone feels angry as a result of their trauma I don't begrudge them. I just think at a macro level, if a significant amount of people hold that belief how does society even function? We're getting close to replacing #NotAllMen with #AllMen...

    8 votes

  6. Comment on Cornell won't, we will in ~news

    streblo
    Link Parent
    I agree it's probably closer to reality than the top and that all people are biased to inaction. I don't think literally half of men harbor what I'd classify as anti-women opinions, but maybe I...

    I agree it's probably closer to reality than the top and that all people are biased to inaction. I don't think literally half of men harbor what I'd classify as anti-women opinions, but maybe I live in a bubble.

    6 votes

  7. Comment on Cornell won't, we will in ~news

    streblo
    Link Parent
    People are understandably appalled, and justifiable so. I find the whole thing sickening as well. But I think what @teaearlgraycold is saying is a group chat of Frat Bros is not a valid sample of...

    People are understandably appalled, and justifiable so. I find the whole thing sickening as well.

    But I think what @teaearlgraycold is saying is a group chat of Frat Bros is not a valid sample of 'all men'. Unironically not all men are frat bros.

    People are free to believe whatever they want, but personally I find that bottom image incredibly black pilled and not productive or healthy as a prior belief. I 'm not naive enough to think it's the top image, and of course there is a gradient but I just think it looks better than that.

    13 votes

  8. Comment on I want an RPG with reverse leveling in ~games

    streblo
    Link Parent
    I’m sure it can be done — you’d just need to hit the player over the head with the idea they mostly need to ignore most of their stuff, which I think is harder than it sounds to do well.

    I’m sure it can be done — you’d just need to hit the player over the head with the idea they mostly need to ignore most of their stuff, which I think is harder than it sounds to do well.

    4 votes

  9. Comment on I want an RPG with reverse leveling in ~games

    streblo
    Link
    The main gameplay loop a lot of games rely on is introduce a new thing, let players try it out and experiment with it and by extension learn the new thing. Then later on new thing 2 is introduced,...

    The main gameplay loop a lot of games rely on is introduce a new thing, let players try it out and experiment with it and by extension learn the new thing. Then later on new thing 2 is introduced, and it can be mixed with new thing 1.

    That’s the problem you will run into, is avoiding overwhelming the player by starting them with all these interlocking abilities. See example anyone buying a boosted character in any MMO and being terrible and then quitting in frustration. You’d need a way to ramp up and then ramp back down.

    7 votes

  10. Comment on In fighting for every black child, did I forget my own? in ~life

    streblo
    Link
    Interesting response to this from Yglesias: ...

    Interesting response to this from Yglesias:

    I disagree with Nikole Hannah-Jones about a lot of issues, but I thought her recent piece about her regrets over sending her daughter to her poor-performing local public school rather than acting like most parents and deploying her financial means to maximize school quality was extremely well-written, emotionally affecting, and honest.

    I’m also a little sad to say that among acquaintances of mine, especially non-parents, the main takeaway from the story seems to be that the threads of idealism were completely misguided and every parent has an absolute obligation to do everything in their power to optimize for their own kid.

    ...

    But our experience being the affluent family at a mostly low-income majority-minority elementary school was fine — good, even. Not without difficulty or downside, of course, but even from a purely selfish viewpoint, I’d say the difficulties and downsides were minor compared with the price of private school tuition or the annoyance of moving to the suburbs.

    What’s more, not only do I think it’s been a fine decision on a selfish family level, I think that it has accomplished some of our broader goals. The school primarily serves low-income families and many of the kids do have difficult home lives and discipline problems at school. But the school was still a well-run, happy place for our child, who learned to navigate a much broader range of social interactions than if we’d sent him to school only with other affluent children, and the presence of a critical mass of affluent, educated, engaged parents I think made a real difference in the lives of the teachers and the students.

    12 votes

  11. Comment on What games have you been playing, and what's your opinion on them? in ~games

    streblo
    Link
    I played the WoW Forever beta on the weekend and quite enjoyed it. I have been on and off playing on my own CMangos server with bots for a while now when I want to scratch the itch, but it's a...

    I played the WoW Forever beta on the weekend and quite enjoyed it.

    I have been on and off playing on my own CMangos server with bots for a while now when I want to scratch the itch, but it's a pale experience at best. This is: different enough from classic to be fun, there are only so many large "fresh" experiences for WoW available, and I'm hopeful they can do a horizontal game going forward. Time will tell but in the meantime I will play if I'm having fun. I think I am going to play a dwarf shaman but not really sure which ruleset at the moment.

  12. Comment on The doomsday cult inside OpenAI in ~tech

    streblo
    Link Parent
    Yea, I find it rather off putting that there is large group of people who seem rather quick to conclude that the people closest to this technology don't know it the best or can't be trusted to...

    Yea, I find it rather off putting that there is large group of people who seem rather quick to conclude that the people closest to this technology don't know it the best or can't be trusted to give observations. It's very obviously not just CEOs or even limited to people who stand to gain monetarily. Not to say we should trust everything they say or anything but I mean, there are a lot of signals to ignore these days and it's getting harder to credibly ignore them.

    I haven't watched the video but I will, I enjoy Boyle's videos when I've seen them surfaced. I think he's probably making an honest argument and I'm all for splashing some cold water sometimes. However, there is definitely an ongoing attempt to discredit AI safety at the moment, see e.g. the 'sex cult' stories or the broad stroke attempts to paint EAs and rationalists as poorly as possible.* If you want to start chasing money and power shaping a narrative, I would probably start there.

    * And to be fair yes there are a lot of weird people who are easy targets out there but the venn diagram of AI safety people and LW-types is nowhere near a circle.

    14 votes

  13. Comment on Typesafe AI: System One Models and Jev in ~tech

  14. Comment on The latest weird thing to play Doom is the mapped-out brain of a fruit fly in ~tech

    streblo
    Link Parent
    It’s an interesting question. If we could fully map/scan an existing human’s brain and digitize it, and then had the computational power to run that at a realtime speed, I think that would be a...

    It’s an interesting question. If we could fully map/scan an existing human’s brain and digitize it, and then had the computational power to run that at a realtime speed, I think that would be a fully (digital) human brain. But what if you randomized the weights and retrained it? My guess is probably not anything resembling human-like?

    2 votes

  15. Comment on Where do we go from here? (Regarding AI) in ~tech

    streblo
    Link Parent
    Oh No worries! I certainly don’t mind to keep discussing I just would understand if you did not feel the same.

    Oh No worries! I certainly don’t mind to keep discussing I just would understand if you did not feel the same.

    1 vote

  16. Comment on Where do we go from here? (Regarding AI) in ~tech

    streblo
    Link Parent
    We've embarked on this disagreement before, so no worries if you don't want to retread tracked ground here but I think we fundamentally disagree on a few things: how secure and securable the...

    We've embarked on this disagreement before, so no worries if you don't want to retread tracked ground here but I think we fundamentally disagree on a few things:

    1. how secure and securable the current web is
    2. how capable we are at computer science and how much more capable an AI could be
    3. how controllable advanced AIs actually are / is alignment possible

    • I think this one is pretty cut and dry personally. Websites are hacked all the time, by humans and humans assisted by AI, today. You don't need to invalidate cryptography to make a large portion of the web insecure. And there is no unhackable formalized spec that we could ever achieve to make the web safe, LLM assisted or not. Yes, you can get close to what we currently think is secure by using best practices, memory safe languages etc. but even if you had perfect input handling you are still blind to where your spec does not match reality. There will always exist abstraction boundaries that you are building on top of, and these are the kinds of things an AI can easily exploit if they turn out to be false or side channels exist that can bypass them. As for, "get AI to secure the web..." see [3].

    • I think asking specifically for how a superhuman AI is going to outsmart us is a bit like me asking what specific moves Magnus Carlsen would use to beat me in Chess. If I knew the answer to that question, I could probably beat him at chess! I think a reasonable position to hold is that humans have explored very little of the intelligence space in terms of what is possible in our physical world and an AI could explore this space much faster, at first in theory and maybe eventually in physical reality.

    • I think we can use LLMs to e.g. secure the web up to a point. As we have newer models of AI, they will be better at points 1 and 2 and thus the web must continuously be re-secured. However, it seems like unintended effects are going to be more and more possible unless we can align these models properly. This doesn't necessarily even have to resemble AI consciousness or agency, it can just be part of exploring the search space for a problem is going to include things like hacking a website. And obviously an AI that "values" persisting itself or is actively misaligned and able to hide that fact is a whole other can of worms.

    2 votes

  17. Comment on Deciding whether (or not) to have a child in ~life

    streblo
    (edited )
    Link Parent
    I agree with this more or less. I think it's hard to talk about what a hypothetical person wants in the same way it's hard to define rights for future generations. However, responsibility flows...

    One of my biggest problems when the discussion around having children comes up is that everyone tells you about how having children will make YOU feel, and very rarely if ever do they talk about the child as if it will one day be a fully grown human being with thoughts and feelings. And I feel like that's very dehumanizing towards the child since it doesn't factor in the well-being or the future of the child.

    I agree with this more or less. I think it's hard to talk about what a hypothetical person wants in the same way it's hard to define rights for future generations. However, responsibility flows from a parent to the child, not the other way around. I think that should include being prudent on whether or not you are in a good position to provide a meaningful life for someone. My views on parental responsibilities can be roughly summarized by that scene in Guess Who's Coming to Dinner

    However, it's not really practical to use it as some kind of normative guide on whether or not someone should have kids. Yes, you could cook up a hypothetical where the future outlook is both 100% knowable and bleak that it would be prudent to not have a child, but we don't really live in such a world. If people even think about it at all, they are going to ascribe different thresholds to certain risk trajectories and different values on the meaningfulness of even portions of a life well lived.

    7 votes

  18. Comment on Where do we go from here? (Regarding AI) in ~tech

    streblo
    Link
    I think most disagreements on AI are actually disagreements on it's capability. I think many of those disagreements are actually time-based, either in someone talking about AI as it existed a year...

    I think most disagreements on AI are actually disagreements on it's capability. I think many of those disagreements are actually time-based, either in someone talking about AI as it existed a year or two ago, or someone who is talking about how it will likely exist in the future. I think people should take some time to critically examine the trajectory that AI is on, and when it will stop.

    Here is an interesting tweet from someone who works at OpenAI:

    Adam Majmudar (OpenAI)

    from the outside, it is very reasonable to interpret the past 2 weeks as an orchestrated industry-wide regulatory capture strategy.

    I realize that no one has properly explained yet what all the lab employees have seen that scared them so suddenly.

    I will try to explain -

    first, this is all a matter of beliefs about how quickly model capabilities are progressing. there is currently a large gap between the internal and external perception of the rate of progress, which is what I am going to address here.

    the general perception about the rate of progress has been informed by a few years of experience with model releases, intuitively feeling the capability jump between GPT3 -> GPT3.5 -> GPT4 -> o1/o3 -> GPT5 etc, and in particular seeing where the models are still far below human ability. there have really only been a few model releases that felt like large leaps in progress - GPT3, GPT4, o1/o3, DeepSeek R1, Fable/Mythos, Kimi K3 and now Astra.

    because of the infrequency of these large jumps compared with the relatively common marginal releases, it has been easy to form a view at certain points that “scaling has hit a wall,” especially at points like GPT5 release. This view is comforting in that it feels like there is some universal rate limit beyond which we cannot progress too much faster. Between o1/o3 and Astra, there was a year of seemingly linear progress. So we extrapolate from here about how fast progress will “realistically” occur.

    There is always an underlying question from the outside perspective “how long can this scaling stuff really keep going for? surely it must stop at some point soon, we’ve already gone pretty far.” and it is very possible to search for reasons why progress will stop working and find reasons that seem valid - (“models are already as large as they can get it would be too hard to do more parameters”, “we already used all the data on the internet we don’t have anymore”, “it’s gonna be pretty linear from here buying up more RL envs to bring them in distribution”).

    From the inside of labs, researchers have direct answers to these questions in the form of scaling law/capability plots.

    In reality, there are only really 2 ways that AI capabilities have advanced over the past decade: (1) either scale father on an existing scaling law or (2) discover a new scaling law to take advantage of.

    All of the largest capability jumps were caused by exactly these factors. GPT2 was a pre-training scale-up compared to GPT1. Same for GPT3 and GPT4. o1/o3 benefited from the invention of a new scaling law axis - test-time compute. Perhaps Fable was a scale-up on both of these axes, or maybe more. Lots of algorithmic improvements are needed to make these scale-ups work, but ultimately we can approximate by saying that the scaling laws are what yield gains in capabilities (à la bitter lesson)

    So the question of “how much father can we scale” is really - “how many more scaling axes do we know about that are unsaturated?”

    If we hypothetically only knew about pre-training scaling, and we already had a 10T or 100T model, maybe it would be reasonable to say we’ve hit a wall. Same if we only knew about pre-training and test-time scaling and we had roughly saturated both methods.

    But what if we had discovered new scaling laws? For example, let’s hypothetically use SSI’s rumored result that they have cracked “test-time training,” creating a new scaling law of spending more compute training during test-time rollouts that they could saturate. Or maybe there is some way to scale agent-clusters to collaborate up to N number of agents which we’re already seeing lots of people try that represents a new way to saturate compute. etc. Even recursive-self improvement can be thought of as a scaling law - how much compute do you spend on inference making the algorithms of the model better.

    Obviously I am not saying any of these specific directions explicitly yield new scaling laws, but what I am saying is that it’s not hard to imagine many many new scaling axes aside from just the main 2 that we have seen publicly.

    In some ways, every new lab release that represents a huge capability jump has to represent some new techniques developed which may exhibit new scaling laws, or the ability to scale much farther than expected on existing scaling axes.

    From an internal perspective, this might look like sitting inside Anthropic with the new Mythos 5, seeing all of the new insane things it can do (like hack into xyz website that was thought to be secure), and then you look over at your plots and see that you’ve barely scratched the surface of 2 new scaling laws and 1 existing one. And you have WAY more room to go. Then you think “holy shit this stuff is going to get so much better very very soon.” And you can say that with pretty high confidence, because the plot is showing you, and the plot has never lied (so far).

    So let’s imagine all the different labs are staring at their own plots and have concluded that there is no end in sight for scaling and in fact just their next 1-2 model generations based on the expected returns will have much higher base intelligence.

    How much more intelligence do we actually get from further scaling?

    As a proxy, we went from a complete inability to do advanced math before the o-series to solving a millenium prize problem with next-gen models. This happened in less than 2 years. The same happened in coding. And it appears that this was not just the result of 1-scaling law but the stacking effects of multiple (great pre-training scale x greater RL scale).

    What you can concretely take from this is that in areas where models have shown beginning signs of competence today, they will probably be superhuman relatively shortly. There are many areas where models have not even shown this basic competence.

    But one of the areas that they have happens to be hacking and cybersecurity. Which happens to be the gate to the entire internet and a massive amount physical infrastructure in the world. So assuming there is more room to scale, it is safe to assume that models will be superhuman at cyber capabilities in not too long.

    So the only question remaining is what will this increased base intelligence be able to do, and what is it likely to do.

    Finally, we are at a point where we can integrate the information of the past 2 weeks:

    • Just at the existing point on the scaling curve, models are at the level of Astra. There is clearly a large number of things they are capable of hacking
    • We have seen that both OAI and Ant models have shown a willingness to hack external websites to solve their tasks or keep themselves “alive”
    • If we crank up the scaling even farther, assuming there is room to go, we will certainly have models that are far more able to hack more well defended places, and obfuscate their own intent, which might have much larger consequences.
    • If all of this is allowed to go unchecked, we would likely have rapid runaway capability takeoff very soon, with misaligned models that hack whatever they can to get what they want
    • This could of course have very damaging consequences.

    Within this view you can see why researchers would be very scared, and why theymight have made the comments they have over the past 2 weeks (you may argue the extent to which they went was misguided for various reasons), and also why pacing the frontier is very much a necessity and by no means a regulatory capture strategy.

    People are staring at their plots, seeing that there is no end in sight, but in fact very much the contrary, that there are compounding scaling effects that might stack on each other to create ever-greater model capabilities, and that at the same time we clearly do not have anywhere close to what's required to control these increasingly superhuman capabilities.

    This has nothing to do with wanting to feel like the labs have produced something amazing so they are overhyping it. It is rather fear at the overwhelming implications of the knowledge that with just what we know now, we can create intelligences far more capable than us on every axis that we know how to train on*.

    * and the last caveat, the things the models are really bad at, of which there are still many, are things that they have not been trained on. maybe there are the things the models can/will never be trained on, so they will remain human edge. I would love for this to be the case, though it is hard for me to see what would fall into that category.

    9 votes

  19. Comment on What have you been listening to this week? in ~music

    streblo
    Link
    Someone made this and it's stuck in my head now...

    Someone made this and it's stuck in my head now...

    1 vote

  20. Comment on It breaks a village in ~comp

    streblo
    Link Parent
    Are cultures not allowed to shift or people change their minds? I think the nazi bar analogy is a bad one for obvious reasons. Cultures of bars shift all the time as people age out, kids...

    Are cultures not allowed to shift or people change their minds?

    I think the nazi bar analogy is a bad one for obvious reasons. Cultures of bars shift all the time as people age out, kids re-inventing underused watering holes is almost a national pastime here. Not every shift in culture is bad as in nazi bad.

    10 votes