6 votes

NixOS Configuration for a VPS

Since I took so long to reply to Tips to use NixOS on a server? by @simao, I decided to create a new topic to share my configs. Hopefully this is informative for anyone looking to do similar things - I'll also gladly take critiques, since my setup is probably not perfect.

First, I will share the output of 'lsblk' on my VPS:

NAME      MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
vda       253:0    0   180G  0 disk  
├─vda1    253:1    0   512M  0 part  /boot
└─vda2    253:2    0 179.5G  0 part  
  └─crypt 254:0    0 179.5G  0 crypt 

That is, I use an unencrypted /boot partition, vda1, with GRUB 2 to prompt for a passphrase during boot, to unlock the LUKS encrypted vda2. I prefer to use ZFS as my file system for the encrypted drive, and LUKS rather than ZFS encryption. This is an MBR drive, since that's what my VPS provider uses, though UEFI would look the same. The particular way I do this also requires access through the provider's tools, and not ssh or similar. The hardware-configuration.nix file reflects this:

Click to view the hardware configuration file
# Do not modify this file!  It was generated by ‘nixos-generate-config’
# and may be overwritten by future invocations.  Please make changes
# to /etc/nixos/configuration.nix instead.
{ config, lib, pkgs, modulesPath, ... }:

{
  imports =
    [ (modulesPath + "/profiles/qemu-guest.nix")
    ];

  boot.initrd.availableKernelModules = [ "aes_x86_64" "ata_piix" "cryptd" "uhci_hcd" "virtio_pci" "sr_mod" "virtio_blk" ];
  boot.initrd.kernelModules = [ ];
  boot.kernelModules = [ ];
  boot.extraModulePackages = [ ];

  fileSystems."/" =
    { device = "rpool/root/nixos";
      fsType = "zfs";
    };

  fileSystems."/home" =
    { device = "rpool/home";
      fsType = "zfs";
    };

  fileSystems."/boot" =
    { device = "/dev/disk/by-uuid/294de4f1-72e2-4377-b565-b3d4eaaa37b6";
      fsType = "ext4";
    };

  swapDevices = [ ];

}
I disobey the warning at the top to add `"aes_x86_64"` and `"cryptd"` to the available kernel modules, to speed up encryption. The `configuration.nix` follows:
Click to view the configuration file
# Edit this configuration file to define what should be installed on
# your system.  Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).

{ config, lib, pkgs, ... }:

{
  imports =
    [ # Include the results of the hardware scan.
      ./hardware-configuration.nix
    ];

  # Hardware stuff
  # add the following to hardware-configuration.nix - speeds up encryption
  #boot.initrd.availableKernelModules ++ [ "aes_x86_64" "cryptd" ];
  boot.initrd.luks.devices.crypt = {
    # Change this if moving to another machine!
    device = "/dev/disk/by-uuid/86090289-1c1f-4935-abce-a1aeee1b6125";
  };
  boot.kernelParams = [ "zfs.zfs_arc_max=536870912" ]; # sets zfs arc cache max target in bytes
  boot.supportedFilesystems = [ "zfs" ];
  nix.maxJobs = lib.mkDefault 6; # number of cpu cores

  # Use the GRUB 2 boot loader.
  boot.loader.grub.enable = true;
  boot.loader.grub.version = 2;
  # boot.loader.grub.efiSupport = true;
  # boot.loader.grub.efiInstallAsRemovable = true;
  # boot.loader.efi.efiSysMountPoint = "/boot/efi";
  # Define on which hard drive you want to install Grub.
  boot.loader.grub.device = "/dev/vda"; # or "nodev" for efi only
  boot.loader.grub.enableCryptodisk = true;
  boot.loader.grub.zfsSupport = true;

  networking.hostName = "m"; # Define your hostname.
  # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.

  # The global useDHCP flag is deprecated, therefore explicitly set to false here.
  # Per-interface useDHCP will be mandatory in the future, so this generated config
  # replicates the default behaviour.
  networking.useDHCP = false;
  networking.interfaces.ens3.useDHCP = true;
  networking.hostId = "aoeu"; # set this to the first eight characters of /etc/machine-id for zfs
  networking.nat = {
    enable = true;
    externalInterface = "ens3"; # this may not be the interface name
    internalInterfaces = [ "wg0" ];
  };
  networking.firewall = {
    enable = true;
    allowedTCPPorts = [ 53 25565 ]; # open 53 for DNS and 25565 for Minecraft
    allowedUDPPorts = [ 53 51820 ]; # open 53 for DNS and 51820 for Wireguard - change the Wireguard port
  };
  networking.wg-quick.interfaces = {
    wg0 = {
      address = [ "10.0.0.1/24" "fdc9:281f:04d7:9ee9::1/64" ];
      listenPort = 51820;
      privateKeyFile = "/root/wireguard-keys/privatekey"; # fill this file with the server's private key and make it so only root has read/write access

      postUp = ''
        ${pkgs.iptables}/bin/iptables -A FORWARD -i wg0 -j ACCEPT
        ${pkgs.iptables}/bin/iptables -t nat -A POSTROUTING -s 10.0.0.1/24 -o ens3 -j MASQUERADE
        ${pkgs.iptables}/bin/ip6tables -A FORWARD -i wg0 -j ACCEPT
        ${pkgs.iptables}/bin/ip6tables -t nat -A POSTROUTING -s fdc9:281f:04d7:9ee9::1/64 -o ens3 -j MASQUERADE
      '';

      preDown = ''
        ${pkgs.iptables}/bin/iptables -D FORWARD -i wg0 -j ACCEPT
        ${pkgs.iptables}/bin/iptables -t nat -D POSTROUTING -s 10.0.0.1/24 -o ens3 -j MASQUERADE
        ${pkgs.iptables}/bin/ip6tables -D FORWARD -i wg0 -j ACCEPT
        ${pkgs.iptables}/bin/ip6tables -t nat -D POSTROUTING -s fdc9:281f:04d7:9ee9::1/64 -o ens3 -j MASQUERADE
      '';

      peers = [
        { # peer0
          publicKey = "{client public key}"; # replace this with the client's public key
          presharedKeyFile = "/root/wireguard-keys/preshared_from_peer0_key"; # fill this file with the preshared key and make it so only root has read/write access
          allowedIPs = [ "10.0.0.2/32" "fdc9:281f:04d7:9ee9::2/128" ];
        }
      ];
    };
  };

  # Configure network proxy if necessary
  # networking.proxy.default = "http://user:password@proxy:port/";
  # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";

  nixpkgs.config = {
    allowUnfree = true; # don't set this if you want to ensure only free software
  };

  # Select internationalisation properties.
  i18n.defaultLocale = "en_US.UTF-8";
  console = {
    font = "Lat2-Terminus16";
    keyMap = "us";
  };

  # Set your time zone.
  time.timeZone = "America/New_York"; # set this to the same timezone your server is located in

  # List packages installed in system profile. To search, run:
  # $ nix search wget
  environment = {
    systemPackages = with pkgs; let
      nvimcust = neovim.override { # lazy minimal neovim config
        viAlias = true;
        vimAlias = true;
        withPython = true;
        configure = {
          packages.myPlugins = with pkgs.vimPlugins; {
            start = [ deoplete-nvim ];
            opt = [];
          };
          customRC = ''
            if filereadable($HOME . "/.config/nvim/init.vim")
              source ~/.config/nvim/init.vim
            endif

            set number

            set expandtab

            filetype plugin on
            syntax on

            let g:deoplete#enable_at_startup = 1
          '';
        };
      };
    in
    [
      jdk8
      nvimcust
      p7zip
      wget
      wireguard
    ];
  };

  # Some programs need SUID wrappers, can be configured further or are
  # started in user sessions.
  # programs.mtr.enable = true;
  # programs.gnupg.agent = {
  #   enable = true;
  #   enableSSHSupport = true;
  #   pinentryFlavor = "gnome3";
  # };

  # List services that you want to enable:

  # Enable the OpenSSH daemon.
  services = {
    dnsmasq = {
      enable = true;
      # this allows DNS requests from wg0 to be forwarded to the DNS server on this machine
      extraConfig = ''
        interface=wg0
      '';
    };
    fail2ban = {
      enable = true;
    };
    openssh = {
      enable = true;
      permitRootLogin = "no";
    };
    zfs = {
      autoScrub = {
        enable = true;
        interval = "monthly";
      };
    };
  };

  # Set sudo to request root password for all users
  # this should be changed for a multi-user server
  security.sudo.extraConfig = ''
    Defaults rootpw
  '';

  # Define a user account. Don't forget to set a password with ‘passwd’.
  users.users = {
    vpsadmin = { # admin account that has a password
      isNormalUser = true;
      home = "/home/vpsadmin";
      extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user.
      shell = pkgs.zsh;
    };
    mcserver = { # passwordless user to run a service - in this instance minecraft
      isNormalUser = true;
      home = "/home/mcserver";
      extraGroups = [];
      shell = pkgs.zsh;
    };
  };

  systemd = {
    services = {
      mcserverrun = { # this service runs a systemd sandboxed modded minecraft server as user mcserver
        enable = true;
        description = "Start and keep minecraft server running";
        wants = [ "network.target" ];
        after = [ "network.target" ];
        serviceConfig = {
          User = "mcserver";
          NoNewPrivileges = true;
          PrivateTmp = true;
          ProtectSystem = "strict";
          PrivateDevices = true;
          ReadWritePaths = "/home/mcserver/Eternal_current";
          WorkingDirectory = "/home/mcserver/Eternal_current";
          ExecStart = "${pkgs.jdk8}/bin/java -Xms11520M -Xmx11520M -server -XX:+AggressiveOpts -XX:ParallelGCThreads=3 -XX:+UseConcMarkSweepGC -XX:+UnlockExperimentalVMOptions -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:MaxGCPauseMillis=10 -XX:GCPauseIntervalMillis=50 -XX:+UseFastAccessorMethods -XX:+OptimizeStringConcat -XX:NewSize=84m -XX:+UseAdaptiveGCBoundary -XX:NewRatio=3 -jar forge-1.12.2-14.23.5.2847-universal.jar nogui";
          Restart = "always";
          RestartSec = 12;
        };
        wantedBy = [ "multi-user.target" ];
      };
      mcserverscheduledrestart = { # this service restarts the minecraft server on a schedule
        enable = true;
        description = "restart mcserverrun service";
        serviceConfig = {
          Type = "oneshot";
          ExecStart = "${pkgs.systemd}/bin/systemctl try-restart mcserverrun.service";
        };
      };
    };
    timers = {
      mcserverscheduledrestart = { # this timer triggers the service of the same name
        enable = true;
        description = "restart mcserverrun service daily";
        timerConfig = {
          OnCalendar = "*-*-* 6:00:00";
        };
        wantedBy = [ "timers.target" ];
      };
    };
  };

  # This value determines the NixOS release from which the default
  # settings for stateful data, like file locations and database versions
  # on your system were taken. It‘s perfectly fine and recommended to leave
  # this value at the release version of the first install of this system.
  # Before changing this value read the documentation for this option
  # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
  system.stateVersion = "20.09"; # Did you read the comment?

}
You'll notice that this server acts as a Wireguard endpoint and as a Minecraft server. I described the first part on the [NixOS wiki page for Wireguard](https://nixos.wiki/wiki/Wireguard) under the section that mentions dnsmasq. The second part is done using NixOS's systemd support, which can be a bit confusing at first but is easy enough once you know how it works.

Edit: Also, the provider I use is ExtraVM, who has been excellent.

3 comments

  1. [3]
    simao
    Link
    Thanks for these, it's helpful to see how other people do it. Inside those user's directories, do you create other directories manually? Or do you manage that outside of nix entirely?

    Thanks for these, it's helpful to see how other people do it.

    Inside those user's directories, do you create other directories manually? Or do you manage that outside of nix entirely?

    4 votes
    1. Wulfsta
      (edited )
      Link Parent
      Usually I will su to root then to that user and do everything as that user. I find this to be the easiest way to ensure that ownership is correct. If I needed to deploy many of these I'd likely...

      Usually I will su to root then to that user and do everything as that user. I find this to be the easiest way to ensure that ownership is correct. If I needed to deploy many of these I'd likely script all of that instead - but I have no need to deploy many servers.

    2. spit-evil-olive-tips
      Link Parent
      As a rule of thumb, NixOS won't touch user home directories, beyond creating them if requested. It's possible to use home-manager as sort of a separate "instance" of Nix to manage dotfiles etc.

      As a rule of thumb, NixOS won't touch user home directories, beyond creating them if requested.

      It's possible to use home-manager as sort of a separate "instance" of Nix to manage dotfiles etc.