22 votes

Rewriting Bun in Rust

17 comments

  1. [8]
    teaearlgraycold
    Link
    I wish they had gone more into the amount of human code review they employed. Just a ballpark idea of how much of the code has had a human look at it. I’m not a bun rewrite hater - and I suspect...

    I wish they had gone more into the amount of human code review they employed. Just a ballpark idea of how much of the code has had a human look at it. I’m not a bun rewrite hater - and I suspect those people are mainly insecure and afraid of change - but I do think you should anchor the project in some of the old ways just to gain and share confidence.

    10 votes
    1. [6]
      DeaconBlue
      Link Parent
      They'll never do that, this whole thing is functionally a marketing stunt by Anthropic to sell Claude.

      I wish they had gone more into the amount of human code review they employed. Just a ballpark idea of how much of the code has had a human look at it.

      They'll never do that, this whole thing is functionally a marketing stunt by Anthropic to sell Claude.

      21 votes
      1. teaearlgraycold
        Link Parent
        Honestly that tells me it didn't work super well. If they could say "we combed over a randomized sampling of 10% of the files with our best engineers and only found one minor issue" that would be...

        Honestly that tells me it didn't work super well. If they could say "we combed over a randomized sampling of 10% of the files with our best engineers and only found one minor issue" that would be a huge win for them.

        10 votes
      2. [4]
        skybrian
        Link Parent
        It's overdetermined. Some other reasons to use your own company's product are that it's free to use for employees and already approved and set up, you want to become familiar with your company's...

        It's overdetermined. Some other reasons to use your own company's product are that it's free to use for employees and already approved and set up, you want to become familiar with your company's products, and you can help out by giving feedback. There are also the reasons stated in the article for doing the project.

        Like what is he going to do, use OpenAI?

        So it's definitely not an unbiased decision and yes, it might be good marketing, too. But saying things are "a marketing stunt," is mostly evidence that people like to choose the most cynical reason for doing a thing.

        7 votes
        1. [3]
          Diff
          Link Parent
          It's not even all that cynical. It's quite a big deal to merge in a whole rewrite with no human review on such a short time scale. It trashes every single open pull request that people were...

          It's not even all that cynical. It's quite a big deal to merge in a whole rewrite with no human review on such a short time scale. It trashes every single open pull request that people were working on, and eliminates any human experience with the codebase, and it was all done practically overnight. Not even a piecemeal transition period, no "Hey, we have this rewrite teed up, come help us shake out the new codebase before the full swap in X time," no announcements actually of any kind. It took some time for even this blog post to surface.

          Nobody's expecting him to have carefully considered all alternatives before gulping down this rewrite. The rewrite itself is what's outlandish, and is causing people to anecdotally drop Bun due to the loss of faith in the project. At least in the short term, it's a net loss for Bun. It's really not that cynical to think that this ill-advised venture could have been encouraged by their new owner who it's been a net positive for.

          8 votes
          1. [2]
            skybrian
            Link Parent
            I don’t know how many outside contributions they actually get. Do you know if they had a lot of pull requests they wanted to keep? In theory they could have a coding agent rewrite them too.

            I don’t know how many outside contributions they actually get. Do you know if they had a lot of pull requests they wanted to keep?

            In theory they could have a coding agent rewrite them too.

            2 votes
            1. Diff
              Link Parent
              I scrolled back a few months and I see quite a few pull requests from people who aren't Members or Collaborators or robobun, yes.

              I scrolled back a few months and I see quite a few pull requests from people who aren't Members or Collaborators or robobun, yes.

              2 votes
    2. post_below
      Link Parent
      The confidence is key, given the mixed sentiment about AI coding agents. The rewrite alienates a lot of developers. I have no idea exactly how many people/orgs have stopped using Bun as a result...

      The confidence is key, given the mixed sentiment about AI coding agents. The rewrite alienates a lot of developers. I have no idea exactly how many people/orgs have stopped using Bun as a result (or are in the process of migrating) but it's definitely not none.

      Something more than "we did vibed security review" and "we setup fuzzing" might go a long way to convincing people they can trust this massive bet on vibecoding. It's a huge rewrite and the timeline makes it clear that the initial rewrite was nearly 100% vibecoded. What have they done since to take ownership over the codebase? How much will humans be in the loop going forward? Will the project be largely maintained by agents? If so it's going to get progressively uglier under the hood unless new generations of models are much better than what we have now. Personally I'd never consider building a project on top of a runtime like that, the risk/reward is all wrong.

      5 votes
  2. skybrian
    Link
    From the article: [...] [...] [...] [...] [...] [...] [...] [...]

    From the article:

    Bun started as a line-for-line port of esbuild's JavaScript & TypeScript transpiler from Go to Zig. I wrote my first line of Zig on April 16, 2021. I bet on Zig after seeing the single-page Zig Language Reference on Hacker News and getting really excited about the low-level control and care for performance.

    [...]

    For Bun, correctly handling the lifetimes of garbage-collected values and manually-managed values has been a major source of stability issues - most often small memory leaks and occasionally, crashes. Every memory allocation has to be meticulously reviewed. Where do these bytes get freed? How do we ensure it only gets freed once? Did we check for JavaScript exceptions properly? Is this garbage-collected pointer visible to the conservative stack scanner? Is this garbage collected memory or manually managed memory?

    [...]

    A large percentage of bugs from that list are use-after-free, double-free, and "forgot to free" in an error path. In safe Rust, these are compiler errors and RAII-like automatic cleanup with Drop. Compiler errors are a better feedback loop than a style guide.

    Historically, rewrites are a terrible idea. Excluding comments, Bun is 535,496 lines of Zig. A rewrite in another language would take a small team of engineers a full year. It would mean freezing bugfixes, security fixes or feature development for that time. The least risky approach to getting something shippable would be a mechanical port from Zig to Rust, with the minimal number of behavioral changes, using the exact same test suite we already use for testing Bun.

    Fortunately, Bun's own test suite is written in TypeScript which means it doesn't depend on the runtime's programming language.

    [...]

    Since merging the Rust port, we've completed 11 rounds of security review from Claude Code Security and addressed the findings.

    We've also added 24/7 coverage-guided fuzzing of every parser in Bun — JavaScript, TypeScript, JSX, CSS, JSON5, JSONC, TOML, YAML, Markdown, INI, Bun Shell scripts, semver ranges, .patch files, and CSS colors. The fuzzer automatically sends the bugs it finds to Claude to submit a PR reproducing & fixing, and humans review the PRs. So far, it's executed our parsers 100 billion times which has led to around 15 PRs.

    [...]

    So far, Bun v1.4.0 fixes 128 bugs that reproduce in v1.3.14. These range from memory leaks to crashes to miscolored help text.

    [...]

    In Bun v1.3.14, every build leaks about 3 MB, forever — tools like dev servers that bundle on every request eventually run out of memory. In Bun v1.4.0, memory levels off:

    [...]

    The initial changes in the Rust rewrite reduced binary size by 3.8 MB on Windows, 5.5 MB on macOS, and 6.8 MB on Linux. This is largely because we used too much comptime in our Zig code.

    [...]

    Claude Code v2.1.181 (released June 17th) and later use the Rust port of Bun. Startup got 10% faster on Linux but otherwise, barely anyone noticed. Boring is good.

    [...]

    Bun v1.3.14 was the last version of Bun written in Zig. Bun v1.4.0 will be the first version of Bun written in Rust. It's available in canary now - please report any issues you find:

    5 votes
  3. [6]
    Bauke
    Link
    They're not doing a major version bump for something this large? This is a masterclass in project mismanagement. I'm shocked they're going through with it.

    Bun v1.3.14 was the last version of Bun written in Zig. Bun v1.4.0 will be the first version of Bun written in Rust.

    They're not doing a major version bump for something this large? This is a masterclass in project mismanagement. I'm shocked they're going through with it.

    5 votes
    1. [5]
      Wes
      Link Parent
      The intention may be to show that it's a drop-in replacement, and that there's no breaking changes. If it were v2.0.0, many may assume a migration of their project is required. Semantic versioning...

      The intention may be to show that it's a drop-in replacement, and that there's no breaking changes. If it were v2.0.0, many may assume a migration of their project is required.

      Semantic versioning is pretty common in the JS ecosystem. Version numbers are often more about compatibility than the degree of changes in the source code itself.

      That said, I do personally agree that a 2.x build would feel like a cleaner break here.

      8 votes
      1. [4]
        Bauke
        Link Parent
        I personally feel that changing the programming language of a project is a breaking change, never mind doing a complete rewrite, even if the subsequent version is a drop-in replacement with no...

        I personally feel that changing the programming language of a project is a breaking change, never mind doing a complete rewrite, even if the subsequent version is a drop-in replacement with no changes needed on the end user side.

        And of course we're assuming that it is indeed a drop-in replacement with no immediate issues. I imagine the reality of when this rolls out fully it is not going to be that smooth.

        5 votes
        1. [3]
          stu2b50
          Link Parent
          I mean the metric for breaking change is simply as a user, do you need to change your integration to be compatible with the new version or not? And the answer here is: no.

          I mean the metric for breaking change is simply as a user, do you need to change your integration to be compatible with the new version or not? And the answer here is: no.

          7 votes
          1. [2]
            Diff
            Link Parent
            If you as a user are building it from source for whatever reason, requiring a whole new toolchain seems to qualify.

            If you as a user are building it from source for whatever reason, requiring a whole new toolchain seems to qualify.

            1. skybrian
              (edited )
              Link Parent
              That's not the normal way to install Bun and only contributors are expected to do that. (The instructions about how to compile Bun are on the "contributing" page.) It seems unlikely that the Bun...

              That's not the normal way to install Bun and only contributors are expected to do that. (The instructions about how to compile Bun are on the "contributing" page.)

              It seems unlikely that the Bun team made any compatibility promises that cover building from source. Internal changes like that aren't normally covered by compatibility guarantees.

              If it were a Zig library rather than an executable, that would be different, because library API's are inherently tied to a specific language. An executable’s “api” is the command line interface, and for Bun there is also its JavaScript API, as well as an experimental foreign function interface that’s not Zig-specific.

              6 votes
  4. [2]
    thecakeisalime
    Link
    There was a previous discussion on this. I guess this is the follow up blog post promised in the PR.

    There was a previous discussion on this. I guess this is the follow up blog post promised in the PR.

    2 votes
    1. tauon
      Link Parent
      It’s really notable, IMO, that the follow-up took so long to write and/or publish. Another indicator that this is a gigantic Anthropic marketing operation, if you ask me. (Not that that’s the only...

      It’s really notable, IMO, that the follow-up took so long to write and/or publish.
      Another indicator that this is a gigantic Anthropic marketing operation, if you ask me. (Not that that’s the only reason for the port/rewrite or taking a while to write a blog post, but probably a major factor in both nonetheless.)

      4 votes