8 votes

Your executable is a SQLite database

3 comments

  1. skybrian
    Link
    From the article: [...] [...] [...] [...]

    From the article:

    I never let the idea go and with the recent improvements with LLMs, I find it compelling to revisit these ideas to explore further. Specifically, can we replace ELF with SQLite as an executable format? 🤔

    Not “a database that describes an executable”, but the actual file you chmod +x and run.

    [...]

    I developed a pretty fleshed out prototype. It is called SELF, the Structured Executable & Linkable Format, because I am unoriginal. It is on GitHub if you are interested. I’m surprised about all the interesting things that fall out of this idea.

    [...]

    There is a fixed ~5 ms to open SQLite and start the interpreter, plus a copy proportional to the image. That copy is worse than it looks, because the b-tree pages are not mapped into memory. Two processes running the same SELF binary do not share text pages the way a normally-mmap‘d ELF does, because the bytes are copied out of the b-tree rather than mapped.33You might notice that curl (274 KiB, 27 libraries) starts slower than ELF git (4.6 MiB, 5 libraries). That is ld.so doing work proportional to the number of objects rather than the number of bytes, which I have complained about before.

    [...]

    611.9 MiB of database against 644.4 MiB of ELF files. The whole userland, as one queryable file, is smaller than the files it came from. The b-tree cost that doubled a single hello amortises to nearly nothing across 1,123 objects and is roughly 6% over the actual program bytes.

    The libraries and closure are shared across the executables very similar to how Nix might share them across multiple closures, if the store-path was the same. If every root shipped its own private closure (i.e. the AppImage model), the same 723 programs would come to 5.53 GiB but the deduplication of libraries and symbols falls out naturally from the database schema.

    [...]

    The format is done and round-trips between ELF and SELF losslessly. The tooling is done and can query, modify, and pack closures. Lookup through SQL works on unmodified glibc programs perfectly and the native-SQL loader works enough to explore it as a possibility for ideas.

    The whole thing is at fzakaria/selfdb. nix run .#self-vm boots a NixOS VM where hello is a SQLite database. 🙌

    2 votes
  2. bme
    Link
    Ridiculous work. So good. Honestly love it, and very on brand. I would absolutely love to see a persistent sqlite process, write lock held by a nix Daemon, readers able to enjoy lighting fast...

    Ridiculous work. So good. Honestly love it, and very on brand. I would absolutely love to see a persistent sqlite process, write lock held by a nix Daemon, readers able to enjoy lighting fast access to the database. It does seem like not being able to share memory is lame but if statically linked distros exist then it's at least practical if not efficient.

    2 votes
  3. json
    Link
    All this mention of 'self' reminded me of the Smalltalk variant of the same name. https://en.wikipedia.org/wiki/Self_(programming_language) It would be fun to see a Self interpreter as a S'ELF...

    All this mention of 'self' reminded me of the Smalltalk variant of the same name.

    https://en.wikipedia.org/wiki/Self_(programming_language)

    It would be fun to see a Self interpreter as a S'ELF executable.