17 votes

Taking the warts off C, with Andrew Kelley, creator of the Zig programming language

41 comments

  1. [5]
    mtset
    Link
    Some context - I am a longtime and very committed Rust programmer who has worked on multiple large Rust projects, and co-authored a book on the subject. Forgive any bias, but please do take it...

    Some context - I am a longtime and very committed Rust programmer who has worked on multiple large Rust projects, and co-authored a book on the subject. Forgive any bias, but please do take it into account.

    The current state of Zig reminds me of the early days of Rust, where things broke on every release and there was constant churn in features. That's not necessarily a bad thing - Rust's present stability is largely an artifact of that willingness to try out new paths, evaluate them, and discard them if necessary - but Zig's team and community lack some of the amazing commitment to community and accessibility that Rust has enjoyed since pre-1.0.

    Zig does have in-code docs which export to HTML like Rust, but they only work, so far as I can tell, on the in-progress self-hosted compiler.

    Zig does not have, and there doesn't seem to be a plan for, a package manager and build wrangler like Cargo. Cargo's extremely approachable interface is a huge part of what makes Rust so easy to work with, and being able to easily share well-encapsulated algorithms, data structures, and language extensions (to which Zig is even better suited!) is critical to building a useful ecosystem.

    It's been said that part of the problem with C is that people are incentivized to build a new solution to a problem every time it comes up because it's such a pain to share and use libraries, and that only very large or very clever code gets reused widely. In Rust, on the other hand, it's almost trivial to use, say, a data structure that someone else wrote that's more optimized for your use case, so esoteric and use-case-specific data structures are often built, packaged, and widely reused, which leads to highly optimized software with minimal effort.

    Documentation, distribution, and community-mindedness are critical to a language that will be used in such difficult and constrained environments. I believe that Zig can become a great language - to C what Rust is to C++ - but I doubt that can happen if its team and community don't change direction within the early stage of the language's development.

    16 votes
    1. [2]
      JRandomHacker
      Link Parent
      There will be an official Zig package manager - it's just been pushed back a bit while the self-hosted compiler gets built out. There's also a fair bit of discussion explicitly looking at cargo as...

      There will be an official Zig package manager - it's just been pushed back a bit while the self-hosted compiler gets built out. There's also a fair bit of discussion explicitly looking at cargo as a model worth emulating and/or ripping off

      12 votes
      1. mtset
        Link Parent
        Oh nice, I'm glad to see that's planned! I'm excited to see where it goes.

        Oh nice, I'm glad to see that's planned! I'm excited to see where it goes.

        7 votes
    2. [2]
      Apos
      (edited )
      Link Parent
      Just learned that there's already a working unofficial package manager called Gyro: https://github.com/mattnite/gyro. The creator did a showcase a while ago when it was still called zkg:...

      Just learned that there's already a working unofficial package manager called Gyro: https://github.com/mattnite/gyro. The creator did a showcase a while ago when it was still called zkg: https://www.youtube.com/watch?v=1yiCgMHDu4k (In the video chat, andrewrok is the Zig creator.). It's inspired by Cargo it seems.

      3 votes
      1. mtset
        Link Parent
        Neat! Thanks for sharing.

        Neat! Thanks for sharing.

  2. [2]
    Ember
    Link
    Some direct comparisons to C and Rust:

    Some direct comparisons to C and Rust:

    [...] Let’s say you’re writing a desktop application C code. You’re going to make OS calls, you’re writing blocking imperative code. And that’s a well-understood concept.

    If you’re writing Go code, you’re doing an event loop, always. You’re not doing the thing that you’re doing in C. It would be very difficult to have a Go library that you call from C. Because Go depends on a hefty run time to do all the event loop stuff. So those are distinct. Rust supports both but there’s modes. They’re different codebases. In Rust, you can write the kind of C code where you have blocking imperative code and you make OS calls. Or you can do async stuff with Rust where you depend on Tokio, or maybe there’s another one. It’s very pluggable.

    But then you’re getting the Go thing, where it’s the event-based thing. But it’s a different codebase. The thing that’s interesting about Zig is that, like Rust, Zig supports both of these use cases, but with the same codebase. You can have a library that both can be compiled for the Go use case and for the C use case. We call it colorblind async functions.

    9 votes
    1. Apos
      Link Parent
      There's also this a bit later: 😎

      There's also this a bit later:

      Yeah, I claim that Zig is faster than C and I stand by that assertion. I can give like micro benchmarks of examples of messing around with integers and you can see why the code that’s generated is better, but I would also make the argument that based on the principles of the language, the conventions that we have, the organization of the standard library, there’s also the results that in practice, Zig applications tend to be fast. And I would say faster than C and I would even say faster than Rust.

      😎

      9 votes
  3. skybrian
    Link
    Zig is very promising and I expect the standard libraries will work in both sync and async mode. However, I also expect that only in simple cases will a library "just work" both ways. To make it...

    Zig is very promising and I expect the standard libraries will work in both sync and async mode.

    However, I also expect that only in simple cases will a library "just work" both ways. To make it work, you'll need to test the code both ways. This is similar to how if you have C code with platform-specific ifdefs, you need to actually compile it and run tests for each platform. If you just develop on Linux then the Windows code never gets tested.

    When there are a lot of compile-time if statements that are based on a lot of different parameters, there are going to be a lot of combinations and it seems unlikely that they will all work. This might result in problems for the library ecosystem once there is a package manager.

    It seems like keeping track of the platform compatibility matrix in all the different libraries will be pretty important. And not just platform compatibility but feature compatibility, like sync/async.

    7 votes
  4. Apos
    Link
    I listened to the audio version and followed the transcript, but just realized this is also available in video form: https://www.youtube.com/watch?v=gn3YsZ6HUHw.

    I listened to the audio version and followed the transcript, but just realized this is also available in video form: https://www.youtube.com/watch?v=gn3YsZ6HUHw.

    4 votes
  5. [5]
    streblo
    Link
    I've never really looked at Zig, but how does a colorblind async work? If I call a synchronous function from an async one can the whole thing just get suspended? If I'm holding a mutex in my sync...

    I've never really looked at Zig, but how does a colorblind async work?

    If I call a synchronous function from an async one can the whole thing just get suspended? If I'm holding a mutex in my sync thread can I deadlock?

    3 votes
    1. [4]
      skybrian
      Link Parent
      Currently in the standard library, there is a global variable that is either "evented" or "blocking". In functions in the standard library, the code either does a regular system call (in blocking...

      Currently in the standard library, there is a global variable that is either "evented" or "blocking". In functions in the standard library, the code either does a regular system call (in blocking mode) or runs the call in a separate thread (in evented mode).

      So, I guess the idea is that other functions either rely on the standard library or do their own switch based on this flag.

      Functions are classified as either containing suspend points or not and this is inferred recursively, and you can't tell just by looking at a function whether it suspends. It might suspend or not depending on how the program is compiled.

      The trick is that using the "async" keyword on a function that doesn't have any suspend points will just run it to completion. If not in "evented" mode that means the normal system calls will be used and it will block in the normal way. Using await just gets the return value.

      5 votes
      1. [3]
        streblo
        Link Parent
        Thanks for the clarifications, that makes sense. So the global variable would be a compile-time constant where blocking vs non-blocking calls get compiled in or out depending on what 'colour'...

        Thanks for the clarifications, that makes sense. So the global variable would be a compile-time constant where blocking vs non-blocking calls get compiled in or out depending on what 'colour' you've choosen? And I guess calling 'sync' code from async within the same translation unit could result in it being compiled as async but calling library sync code from async will just block until it returns.

        2 votes
        1. [2]
          skybrian
          Link Parent
          I’m not sure what you mean by translation unit, and it’s unclear to me how zig does separate compilation. It looks like zig lets you build executables and C libraries (shared or static). I don’t...

          I’m not sure what you mean by translation unit, and it’s unclear to me how zig does separate compilation. It looks like zig lets you build executables and C libraries (shared or static). I don’t see anything about doing separate compilation for building a large zig program, other than compiling each part as a separate C library and linking them. (There is a warning about always using the same build flags for different compilation units.)

          I think the inference goes bottom-up, transitively. Suppose X calls Y, which calls Z. If the compiler determines that Z can suspend, X and Y can also suspend. If Z is in a different C library then I don’t know what happens.

          I was talking as if I know the language but I’ve only skimmed the docs, so I might have gotten it wrong.

          2 votes
          1. skybrian
            Link Parent
            I asked about separate compilation on Reddit and they say that the async calling convention (allowing a function to suspend using await) only works within a single shared library or executable.

            I asked about separate compilation on Reddit and they say that the async calling convention (allowing a function to suspend using await) only works within a single shared library or executable.

            3 votes
  6. [13]
    petrichor
    Link
    /offtopic For those versed in C: why do the kinds of safety checks Andrew Kelley is talking about not exist in it? Is it a standardization issue? What stops someone from making a C compiler (or...

    /offtopic

    For those versed in C: why do the kinds of safety checks Andrew Kelley is talking about not exist in it? Is it a standardization issue? What stops someone from making a C compiler (or standard library) that checks for out of bounds on arrays, or any / all of the other classic C bugs?

    3 votes
    1. wirelyre
      Link Parent
      The C memory model as standardized makes these checks quite difficult — fixing it would require a new ABI, among other things. For example, it's legal to cast any valid object pointer to any other...

      The C memory model as standardized makes these checks quite difficult — fixing it would require a new ABI, among other things.

      For example, it's legal to cast any valid object pointer to any other (appropriately aligned) object pointer type, regardless of whether you can legally use that new pointer. So you'd have to store type information within memory, and then check validity a lot.

      There are other complications. For example, the pointer to a structure type can be cast directly into a pointer to its first member. And if you copy a structure byte by byte to another (well-aligned) location, the copy needs to work too.

      The upshot is that all C structures, including arrays, are very closely tied to memory layout. If you want to make a library with these kinds of safety checks, it's almost easier to put a new language on top.

      9 votes
    2. [10]
      streblo
      Link Parent
      One example is in a language like rust, things like Vec have a bounds checks on access to prevent things like buffer overflow attacks or unintended memory corruption -- your program will just...

      One example is in a language like rust, things like Vec have a bounds checks on access to prevent things like buffer overflow attacks or unintended memory corruption -- your program will just abort instead. (I'm pretty sure that's how it works, rust programmers feel free to correct me). C doesn't do any of this, so you need to do it yourself. It's possible to implement your own array implementation with bounds checking if you want. From the podcast it sounds like the Zig std library also does this by default, but it allows you to compile it without the safety checks as well, which presumably do incur some performance hit.

      4 votes
      1. [9]
        petrichor
        Link Parent
        So what stops the C compiler from checking these and similar?

        So what stops the C compiler from checking these and similar?

        2 votes
        1. skybrian
          Link Parent
          I did a search and it’s been tried a few times by researchers, but apparently the performance overhead is substantial. The modern approach is to use Address Sanitizer which they say causes a 2x...

          I did a search and it’s been tried a few times by researchers, but apparently the performance overhead is substantial.

          The modern approach is to use Address Sanitizer which they say causes a 2x slowdown, but that’s often fast enough for testing. (Also check out Google’s other sanitizers.)

          3 votes
        2. [7]
          streblo
          Link Parent
          I think it’s also important to have languages that don’t abstract these sorts of things away. Don’t get me wrong things like Rust and Zig are cool and I’m sure they will supplant a lot of uses...

          I think it’s also important to have languages that don’t abstract these sorts of things away. Don’t get me wrong things like Rust and Zig are cool and I’m sure they will supplant a lot of uses cases as better tools for most jobs but sometimes you might not want to pay for them. If you want bounds checking in your C project it’s trivial to test your own array implementation.

          2 votes
          1. [6]
            skybrian
            Link Parent
            There is an ecosystem argument though. Presumably in Zig’s ecosystem (once it has one), code will usually be written to not trigger bounds checks when they are turned on, and if it does it will be...

            There is an ecosystem argument though. Presumably in Zig’s ecosystem (once it has one), code will usually be written to not trigger bounds checks when they are turned on, and if it does it will be treated as a bug. This makes it easier to reuse other people’s code and also compile with bounds checking turned on.

            So the question is what do you want typical library code to look like?

            Also, it’s helpful if the language is designed with this kind of runtime checking in mind, even if it’s not always on. The language and the tools coevolve.

            This is somewhat undercut by easy linking with C, though, and Zig’s package manager not even existing yet, but it seems like a good goal.

            4 votes
            1. [5]
              streblo
              Link Parent
              That’s a fair distinction — I’m not trying to make a case for the C world staying the way it is more so having the option to do things in a C fashion. From listening to the podcast I’m actually...

              That’s a fair distinction — I’m not trying to make a case for the C world staying the way it is more so having the option to do things in a C fashion. From listening to the podcast I’m actually quite intrigued by Zig; I like the batteries-not-included aspect of the language for a C replacement.

              3 votes
              1. [4]
                FlippantGod
                Link Parent
                Agreed, I've been perfectly happy without cargo in Zig. Frankly, Zig build is just so, so nice. My next experiment will be including and compiling a big, messy c++ project that requires many build...

                Agreed, I've been perfectly happy without cargo in Zig. Frankly, Zig build is just so, so nice. My next experiment will be including and compiling a big, messy c++ project that requires many build steps across multiple utils as a stress test.

                1 vote
                1. [3]
                  streblo
                  Link Parent
                  It might be Stockholm syndrome at this point, but I don't mind working with CMake. What's so nice about zig build?

                  It might be Stockholm syndrome at this point, but I don't mind working with CMake. What's so nice about zig build?

                  2 votes
                  1. [2]
                    FlippantGod
                    Link Parent
                    You just write some Zig code :) Also, conditional compilation and comptime makes many things much neater, imo, for an actual maintainability benefit.

                    You just write some Zig code :)

                    Also, conditional compilation and comptime makes many things much neater, imo, for an actual maintainability benefit.

                    1 vote
                    1. streblo
                      Link Parent
                      Yea that sounds a lot nicer than learning C++, CMake, and template meta-programming. :P

                      Yea that sounds a lot nicer than learning C++, CMake, and template meta-programming. :P

    3. Moonchild
      Link Parent
      Such compilers have been devised. TinyCC is one.

      Such compilers have been devised. TinyCC is one.

      2 votes
  7. [9]
    Micycle_the_Bichael
    Link
    Off topic: Can someone explain to me the appeal/use of Zig? Like when/why should I use Zig over other languages? I’m sure there are times and reasons but I’m not sure what they are. Normally there...

    Off topic:

    Can someone explain to me the appeal/use of Zig? Like when/why should I use Zig over other languages? I’m sure there are times and reasons but I’m not sure what they are. Normally there is a hook of some sort for each language I’ve learned that made me want to learn it (with the exception of C#) and usually that reason is plastered everywhere online. I’m struggling to find that with Zig but the language is popular so I’m sure I’m missing something. I’d love to know what it is. I love learning new coding paradigms and ways to approach and break down problems. Help me fall in love with this language please :((

    3 votes
    1. [2]
      wirelyre
      Link Parent
      Zig is a better C. The syntax is strongly inspired by C but it's been tweaked, really expertly in fact. Writing types is easier; writing expressions is easier. Also it's 100% compatible with C —...

      Zig is a better C.

      The syntax is strongly inspired by C but it's been tweaked, really expertly in fact. Writing types is easier; writing expressions is easier.

      Also it's 100% compatible with C — you can import C headers into Zig and export Zig functions as a C library. (Zig-exclusive features aside.)

      For anything you'd already use C for, I'd strongly consider using Zig instead.

      6 votes
      1. Micycle_the_Bichael
        Link Parent
        Ah yeah, that seems to be my issue. Everything I’m reading about Zig is with its relation to C. I never touch any C code so that’s not a huge selling point for me. Which, to be clear, isn’t a...

        Ah yeah, that seems to be my issue. Everything I’m reading about Zig is with its relation to C. I never touch any C code so that’s not a huge selling point for me. Which, to be clear, isn’t a knock on the language. Can’t expect everything to be designed around me :P a lot of smart people are working on it so I’ll probably poke around a bit eventually and see if there’s anything worth stealing for my own needs eventually.

        And again, to be clear, when I say “steal” I mean it in the most joking sense. Like how I “stole” the concept of interfaces from Golang as inspiration for tools at work of “for a tool to fit under this project it must implement XYZ features”

        2 votes
    2. [6]
      Apos
      Link Parent
      I think this article does a good job: https://ziglang.org/learn/why_zig_rust_d_cpp/. If you agree with points like no hidden control flow, then it's a good selling point since other languages...

      I think this article does a good job: https://ziglang.org/learn/why_zig_rust_d_cpp/. If you agree with points like no hidden control flow, then it's a good selling point since other languages won't give you that.

      There's also the fact that the build system is done with real Zig. It's just a build.zig file such as: https://github.com/andrewrk/tetris/blob/master/build.zig. It's so nice coming from C# where I have to go through msbuild and the .csproj XML (It's gotten better with .net Core, but I still have to learn something separate and it's incredibly annoying to read.). Likewise the upcoming package manage should work in the same way.

      The language has a scope and people working on it don't want to add multiple ways of doing the same thing. Coming from C#, I think it's a breath of fresh air. C# is becoming more and more like an ugly beast. I still like C#, but the language could use some trimming. (I think that's one thing Beef lang aims to achieve?)

      tbh, you can ignore most of what I wrote above. What really sold me on Zig is their comptime idea. I'm sure a lot of languages will eventually steal that. The idea is that you can run Zig code directly at build time and get the result. That means the code doesn't need to be executed at runtime. I much prefer that to a preprocessor / macros. It's much easier to work with since it's real Zig code. You don't have to switch to a different language to use it.

      Any easy example would be this:

      const std = @import("std");
      
      pub fn main() !void {
          const string = [_]u8{'a', 'b', 'c'};
      
          std.debug.print("{s} {s}!", .{string, "world"});
      }
      

      This will print abc world!. Since everything is comptime, it literally compiles down to:

      std.debug.print("abc world!", .{});
      

      No allocation was needed.

      5 votes
      1. [4]
        streblo
        Link Parent
        C++ has constexpr, which is similarly powerful. I wrote a compile time generation of a mandelbrot svg with C++ constexpr, it was kinda fun. When I played with it there was no support for the...

        What really sold me on Zig is their comptime idea. I'm sure a lot of languages will eventually steal that.

        C++ has constexpr, which is similarly powerful. I wrote a compile time generation of a mandelbrot svg with C++ constexpr, it was kinda fun. When I played with it there was no support for the standard library but apparently C++ 20 now allows constexpr allocations so you can actually use the standard library in constexpr.

        1 vote
        1. [3]
          Apos
          Link Parent
          comptime is a fair bit more powerful; it involves fully emulating the target architecture, and can deal with types as first-class values (which don't exist at runtime). They are analogous in that...

          comptime is a fair bit more powerful; it involves fully emulating the target architecture, and can deal with types as first-class values (which don't exist at runtime). They are analogous in that they both happen at compile time.

          1 vote
          1. [2]
            streblo
            Link Parent
            Wow, that sounds pretty neat. So kind of like compile-time reflection?

            and can deal with types as first-class values (which don't exist at runtime)

            Wow, that sounds pretty neat. So kind of like compile-time reflection?

            1 vote
            1. Apos
              Link Parent
              Reflection would be a use-case. The comptime generics don't need reflection for example. Here is an example from this page: const std = @import("std"); fn List(comptime T: type) type { return...

              Reflection would be a use-case. The comptime generics don't need reflection for example.

              Here is an example from this page:

              const std = @import("std");
              
              fn List(comptime T: type) type {
                  return struct {
                      items: []T,
                      len: usize,
                  };
              }
              
              pub fn main() void {
                  var buffer: [10]i32 = undefined;
                  var list = List(i32){
                      .items = &buffer,
                      .len = 0,
                  };
              
                  std.debug.print("{d}\n", .{list.items.len});
              }
              

              That creates a list of type i32 with a length of 10 elements.

              Something that heavily uses reflection would be std.fmt.format. It's what generates std.debug.print("abc world!", .{}); in my example above.

              3 votes
      2. Moonchild
        Link Parent
        Done in D and common lisp for decades now.

        comptime

        Done in D and common lisp for decades now.

  8. [4]
    space_cowboy
    Link
    I'd really like to see the linux kernel rewritten in zig; that would be cool.

    I'd really like to see the linux kernel rewritten in zig; that would be cool.

    1 vote
    1. [3]
      teaearlgraycold
      Link Parent
      I honestly wouldn't expect The Kernel to ever get rewritten in another language. In 100 years it'll either be abandoned or still written in C.

      I honestly wouldn't expect The Kernel to ever get rewritten in another language. In 100 years it'll either be abandoned or still written in C.

      3 votes
  9. wirelyre
    Link
    I played with Zig a while ago and I tripped on the compile-time stuff. When you do generics by constructing types in functions, you basically do compile-time reflection. It works fine but the...

    I played with Zig a while ago and I tripped on the compile-time stuff.

    When you do generics by constructing types in functions, you basically do compile-time reflection. It works fine but the values are all anonymous ad-hoc "structures" and kinda hard to deal with.

    I remember getting frustrated with the io library because I couldn't tell which values satisfied which interfaces. (Or even what the interfaces were, but I see the stdlib documentation is getting much better.) I think that was because they are all again basically untyped ad-hoc structures.

    Without a higher-kind interface/typeclass/trait system I can't see myself writing generic Zig code. My brain isn't big enough to do that without making a lot of mistakes.

    1 vote