For the past year and a half, the team building Roc's compiler has been rewriting our 300,000 lines of Rust code into Zig, for reasons I'll recap below. We recently passed an exciting milestone: feature parity with the original compiler!
Since the Bun project recently shared an experience report of their rewrite in the other direction (from Zig to Rust, although that's only the tip of the iceberg of differences between our rewrites), this seems like a nice time to reflect on how our move from Rust to Zig is going.
[...]
To be clear, this is a milestone but not a formal release. (We aim to land version 0.1.0 later this year.) That said, it's a wonderful milestone to have reached, and I'm extremely grateful to all the people who came together to make this happen! I want to thank some in particular who have been especially helpful in getting the language and compiler to this point:
[...]
Speaking of time: our 487-day rewrite took 476 days longer than Bun's 11-day rewrite from their ~500K lines of Zig into Rust. There are many reasons for this difference which have nothing to do with Rust or Zig, including the fact that theirs was a direct port whereas we'd decided to rewrite because of how much we were going to change. The techniques they used wouldn't have worked in our case.
The laundry list of changes we made also means comparing our original Rust code base and new Zig code base won't be apples-to-apples. Still, we've reached a nice point to reflect on how the rewrite has gone, both in terms of what new features it has unlocked for Roc programmers, as well as how our experiences with Rust and Zig have compared.
[...]
I've talked in depth about our reasons for going with Zig elsewhere—in writing, on podcasts, and so on—and we only seriously considered Rust and Zig, because those were the only systems languages our team knew well enough. The biggest considerations on our minds when deciding between Rust and Zig were:
Build times. Our cargo build times were a major pain point, even for incremental builds, and getting worse as our code base grew. We expected build times in a Zig rewrite to be much faster.
Memory control. We use a variety of different memory allocators throughout compilation, especially arenas, and struct-of-arrays layouts all over the place. Rust's ecosystem consistently assumes one global allocator, including soa_rs. Zig's whole ecosystem assumes granular allocators, and struct-of-arrays support is standard.
Ecosystem relevance. Rust's ecosystem is much bigger than Zig's overall…but almost no packages in either ecosystem are relevant to our particular needs. For the niche things we wanted to get off the shelf—such as a faster way to emit LLVM bitcode than wrapping LLVM's C++ library—more of that code existed in Zig than in Rust.
Memory-unsafety assistance. Rust is designed to isolate memory-unsafe code inside rare unsafe blocks, and use things like miri or Valgrind to vet those. Memory-unsafe code wasn't rare for us, though (more on this later) and we ended up with about 1,200 uses of unsafe (out of our 300K lines of Rust code; compare to about 40,000 uses of unsafe in rust's 3.5M lines, and remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job). Zig has more features than Rust for making memory-unsafe code work correctly, and that was the area where we wanted the most help.
[...]
You might be wondering how the Rust-based compiler had any memory corruption bugs at all, let alone more than double the total count of the Zig-based one. Is it because of that pesky Unsafe Rust again?
Actually, no. None of those 21 memory corruption bugs occurred in the compiler's logic itself, which is a testament to Rust's borrow-checker working as intended. The reason we had memory corruption bugs in our Rust-based compiler is that it's a compiler.
Compilers emit machine instructions. When a machine executes those instructions, they can cause memory corruption, resulting in memory corruption bug reports from the people who experienced them. Regardless of which process had the bug—the compiler or compiled program—in both cases the processor only did the bad thing because the compiler told it to. And in both cases the fix is the same: the compiler's code must change, since that code was what caused the memory corruption.
[...]
So while our decision to remain on stable 0.16.0 (plus how many of our contributors run Mac laptops with ARM processors; -fincremental only works on x86-64 CPUs right now) means we haven't yet reaped the anticipated build-time rewards of choosing Zig for the rewrite, we certainly have something to look forward to in the next stable Zig release!
From the article:
[...]
[...]
[...]
[...]
[...]