• Activity
  • Votes
  • Comments
  • New
  • All activity
  • Showing only topics with the tag "linux". Back to normal view
    1. A few easy linux commands, and a real-world example on how to use them in a pinch

      This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would...

      This below is a summary of some real-world performance investigation I recently went through. The tools I used are installed on all linux systems, but I know some people don't know them and would straight up jump to heavyweight log analysis services and what not, or writing their own solution.

      Let's say you have request log sampling in a bunch of log files that contain lines like these:

      127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000
      127.0.0.1 [2021-05-27 23:51:22.019] "GET /pl/player/123456/changelog/ HTTP/1.1" 200 16524 TLSv1.2 MISS-CLUSTER SessionID:(null) Cache:

      You might recognize Fastly logs there (IP anonymized). Now, there's a lot you might care about in this log file, but in my case, I wanted to get a breakdown of hits vs misses by URL.

      So, first step, let's concatenate all the log files with cat *.log > all.txt, so we can work off a single file.

      Then, let's split the file in two: hits and misses. There are a few different values for them, the majority are covered by either HIT-CLUSTER or MISS-CLUSTER. We can do this by just grepping for them like so:

      grep HIT-CLUSTER all.txt > hits.txt; grep MISS-CLUSTER all.txt > misses.txt
      

      However, we only care about url and whether it's a hit or a miss. So let's clean up those hits and misses with cut. The way cut works, it takes a delimiter (-d) and cuts the input based on that; you then give it a range of "fields" (-f) that you want.

      In our case, if we cut based on spaces, we end up with for example: 127.0.0.1 [2021-05-27 23:28:34.460] "GET /static/images/flags/2/54@3x.webp HTTP/2" 200 1806 TLSv1.3 HIT-CLUSTER SessionID:(null) Cache:max-age=31536000.

      We care about the 5th value only. So let's do: cut -d" " -f5 to get that. We will also sort the result, because future operations will require us to work on a sorted list of values.

      cut -d" " -f5 hits.txt | sort > hits-sorted.txt; cut -d" " -f5 misses.txt | sort > misses-sorted.txt
      

      Now we can start doing some neat stuff. wc (wordcount) is an awesome utility, it lets you count characters, words or lines very easily. wc -l counts lines in an input, since we're operating with one value per line we can easily count our hits and misses already:

      $ wc -l hits-sorted.txt misses-sorted.txt
        132523 hits-sorted.txt
        220779 misses-sorted.txt
        353302 total
      

      220779 / 132523 is a 1:1.66 ratio of hits to misses. That's not great…

      Alright, now I'm also interested in how many unique URLs are hit versus missed. uniq tool deduplicates immediate sequences, so the input has to be sorted in order to deduplicate our entire file. We already did that. We can now count our urls with uniq < hits-sorted.txt | wc -l; uniq < misses-sorted.txt | wc -l. We get 49778 and 201178, respectively. It's to be expected that most of our cache misses would be in "rarer" urls; this gives us a 1:4 ratio of cached to uncached URL.

      Let's say we want to dig down further into which URLs are most often hitting the cache, specifically. We can add -c to uniq in order to get a duplicate count in front of our URLs. To get the top ones at the top, we can then use sort, in reverse sort mode (-r), and it also needs to be numeric sort, not alphabetic (-n). head lets us get the top 10.

      $ uniq -c < hits-sorted.txt | sort -nr | head
          815 /static/app/webfonts/fa-solid-900.woff2?d720146f1999
          793 /static/app/images/1.png
          786 /static/app/fonts/nunito-v9-latin-ext_latin-regular.woff2?d720146f1999
          760 /static/CACHE/js/output.cee5c4089626.js
          758 /static/images/crest/3/light/notfound.png
          757 /static/CACHE/css/output.4f2b59394c83.css
          756 /static/app/webfonts/fa-regular-400.woff2?d720146f1999
          754 /static/app/css/images/loading.gif?d720146f1999
          750 /static/app/css/images/prev.png?d720146f1999
          745 /static/app/css/images/next.png?d720146f1999
      

      And same for misses:

      $ uniq -c < misses-sorted.txt | sort -nr | head
           56 /
           14 /player/237678/
           13 /players/
           12 /teams/
           11 /players/top/
      <snip>
      

      So far this tells us static files are most often hit, and for misses it also tells us… something, but we can't quite track it down yet (and we won't, not in this post). We're not adjusting for how often the page is hit as a whole, this is still just high-level analysis.

      One last thing I want to show you! Let's take everything we learned and analyze those URLs by prefix instead. We can cut our URLs again by slash with cut -d"/". If we want the first prefix, we can do -f1-2, or -f1-3 for the first two prefixes. Let's look!

      cut -d'/' -f1-2 < hits-sorted.txt | uniq -c | sort -nr | head
       100189 /static
         5948 /es
         3069 /player
         2480 /fr
         2476 /es-mx
         2295 /pt-br
         2094 /tr
         1939 /it
         1692 /ru
         1626 /de
      
      cut -d'/' -f1-2 < misses-sorted.txt | uniq -c | sort -nr | head
        66132 /static
        18578 /es
        17448 /player
        17064 /tr
        11379 /fr
         9624 /pt-br
         8730 /es-mx
         7993 /ru
         7689 /zh-hant
         7441 /it
      

      This gives us hit-miss ratios by prefix. Neat, huh?

      13 votes
    2. Share your linux desktop/setup

      I've put quite a bit of work into my i3 set up recently and I'm curious if the people here are interested in that kind of thing. I'd be interested in looking through configs to get ideas, and...

      I've put quite a bit of work into my i3 set up recently and I'm curious if the people here are interested in that kind of thing.

      I'd be interested in looking through configs to get ideas, and sharing screenshots and such.

      Here is what my desktop looks like right now. Let me know what you think.

      26 votes
    3. What is the difference between Linux distros? Why do you use the one you use?

      I still mainly use Windows, although I've dual-booted Linux a few times and I have Linux Mint on an old laptop right now. One thing I've never understood about Linux is all the different...

      I still mainly use Windows, although I've dual-booted Linux a few times and I have Linux Mint on an old laptop right now. One thing I've never understood about Linux is all the different distributions - their different reputations and why they have them. What is the mechanical difference between using one distribution of Linux and another? Or are the differences usually not mechanical?

      For example, Ubuntu and Debian seem to be large families, meaning that a lot of other distributions are based on them (using packages built for them in their package managers at least) as well as being popular distros on their own. But what's different between the two of them, and between each and the other distros based on them? (and what's similar? I gather they all use the Linux kernel at least!)

      I also know that people are quite opinionated on their choice of distro, I wondered what reasons people had for their choice. What things are easier or harder for you in your distro of choice? Is it mainly day-to-day tasks that are important or more how the OS works underneath? How much difference does your preferred distro make?

      For myself, I've only used Kubuntu (though not much) and Linux Mint, which was mainly for UI reasons, and particularly for the latter, ease of use for someone used to Windows (at least that was what I found years ago when I first looked into it).

      Though I doubt I'll ever fully move away from Windows I would like / need to have access to a Linux OS, so maybe this will help me to know what is important to look for. But I also hope it'll be a useful and interesting discussion topic. Also, there are some previous discussions on the latter question so I'd be more interested in learning about the main topic.

      also, please do add more tags

      29 votes
    4. Anyone using a lightweight browser with Linux?

      I've got a crappy Chromebook running GalliumOS (Xubuntu) and Chromium is slow as molasses. I tried a few other browsers like Otter and Falkon. They're alright for most sites -- not Tildes, but...

      I've got a crappy Chromebook running GalliumOS (Xubuntu) and Chromium is slow as molasses. I tried a few other browsers like Otter and Falkon. They're alright for most sites -- not Tildes, but this seems consistent with QT5 browsers.

      Anyway, outside of text browsers, anybody have any light weight browser suggestions?

      14 votes
    5. Does reformatting an ext4 partition fix bad sectors, and what are they anyway?

      My Linux desktop is having a bit of difficulty with bad sectors. Lately I've had to boot into recovery and run an fsck a few times to try to fix a problem where the OS drops into read-only mode at...

      My Linux desktop is having a bit of difficulty with bad sectors. Lately I've had to boot into recovery and run an fsck a few times to try to fix a problem where the OS drops into read-only mode at the drop of a hat. Today I tried copying some files from one directory to another and got the following error message:
      cp: error reading "foo/bar": Input/output error

      I've just booted into a live USB and run fsck /dev/sda1 -c and it fixed a load of bad sectors, but the above error message is still happening.

      A bit of googling tells me that this is down to bad sectors on the SSD, and I'm not really sure what that means. Is anybody able to enlighten me? And as a follow-up question, would reformatting the hard drive resolve the problem, or are there any other things I can try to fix it?

      9 votes
    6. NixOS Configuration for a VPS

      Since I took so long to reply to Tips to use NixOS on a server? by @simao, I decided to create a new topic to share my configs. Hopefully this is informative for anyone looking to do similar...

      Since I took so long to reply to Tips to use NixOS on a server? by @simao, I decided to create a new topic to share my configs. Hopefully this is informative for anyone looking to do similar things - I'll also gladly take critiques, since my setup is probably not perfect.

      First, I will share the output of 'lsblk' on my VPS:

      NAME      MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
      vda       253:0    0   180G  0 disk  
      ├─vda1    253:1    0   512M  0 part  /boot
      └─vda2    253:2    0 179.5G  0 part  
        └─crypt 254:0    0 179.5G  0 crypt 
      

      That is, I use an unencrypted /boot partition, vda1, with GRUB 2 to prompt for a passphrase during boot, to unlock the LUKS encrypted vda2. I prefer to use ZFS as my file system for the encrypted drive, and LUKS rather than ZFS encryption. This is an MBR drive, since that's what my VPS provider uses, though UEFI would look the same. The particular way I do this also requires access through the provider's tools, and not ssh or similar. The hardware-configuration.nix file reflects this:

      Click to view the hardware configuration file
      # Do not modify this file!  It was generated by ‘nixos-generate-config’
      # and may be overwritten by future invocations.  Please make changes
      # to /etc/nixos/configuration.nix instead.
      { config, lib, pkgs, modulesPath, ... }:
      
      {
        imports =
          [ (modulesPath + "/profiles/qemu-guest.nix")
          ];
      
        boot.initrd.availableKernelModules = [ "aes_x86_64" "ata_piix" "cryptd" "uhci_hcd" "virtio_pci" "sr_mod" "virtio_blk" ];
        boot.initrd.kernelModules = [ ];
        boot.kernelModules = [ ];
        boot.extraModulePackages = [ ];
      
        fileSystems."/" =
          { device = "rpool/root/nixos";
            fsType = "zfs";
          };
      
        fileSystems."/home" =
          { device = "rpool/home";
            fsType = "zfs";
          };
      
        fileSystems."/boot" =
          { device = "/dev/disk/by-uuid/294de4f1-72e2-4377-b565-b3d4eaaa37b6";
            fsType = "ext4";
          };
      
        swapDevices = [ ];
      
      }
      
      I disobey the warning at the top to add `"aes_x86_64"` and `"cryptd"` to the available kernel modules, to speed up encryption. The `configuration.nix` follows:
      Click to view the configuration file
      # Edit this configuration file to define what should be installed on
      # your system.  Help is available in the configuration.nix(5) man page
      # and in the NixOS manual (accessible by running ‘nixos-help’).
      
      { config, lib, pkgs, ... }:
      
      {
        imports =
          [ # Include the results of the hardware scan.
            ./hardware-configuration.nix
          ];
      
        # Hardware stuff
        # add the following to hardware-configuration.nix - speeds up encryption
        #boot.initrd.availableKernelModules ++ [ "aes_x86_64" "cryptd" ];
        boot.initrd.luks.devices.crypt = {
          # Change this if moving to another machine!
          device = "/dev/disk/by-uuid/86090289-1c1f-4935-abce-a1aeee1b6125";
        };
        boot.kernelParams = [ "zfs.zfs_arc_max=536870912" ]; # sets zfs arc cache max target in bytes
        boot.supportedFilesystems = [ "zfs" ];
        nix.maxJobs = lib.mkDefault 6; # number of cpu cores
      
        # Use the GRUB 2 boot loader.
        boot.loader.grub.enable = true;
        boot.loader.grub.version = 2;
        # boot.loader.grub.efiSupport = true;
        # boot.loader.grub.efiInstallAsRemovable = true;
        # boot.loader.efi.efiSysMountPoint = "/boot/efi";
        # Define on which hard drive you want to install Grub.
        boot.loader.grub.device = "/dev/vda"; # or "nodev" for efi only
        boot.loader.grub.enableCryptodisk = true;
        boot.loader.grub.zfsSupport = true;
      
        networking.hostName = "m"; # Define your hostname.
        # networking.wireless.enable = true;  # Enables wireless support via wpa_supplicant.
      
        # The global useDHCP flag is deprecated, therefore explicitly set to false here.
        # Per-interface useDHCP will be mandatory in the future, so this generated config
        # replicates the default behaviour.
        networking.useDHCP = false;
        networking.interfaces.ens3.useDHCP = true;
        networking.hostId = "aoeu"; # set this to the first eight characters of /etc/machine-id for zfs
        networking.nat = {
          enable = true;
          externalInterface = "ens3"; # this may not be the interface name
          internalInterfaces = [ "wg0" ];
        };
        networking.firewall = {
          enable = true;
          allowedTCPPorts = [ 53 25565 ]; # open 53 for DNS and 25565 for Minecraft
          allowedUDPPorts = [ 53 51820 ]; # open 53 for DNS and 51820 for Wireguard - change the Wireguard port
        };
        networking.wg-quick.interfaces = {
          wg0 = {
            address = [ "10.0.0.1/24" "fdc9:281f:04d7:9ee9::1/64" ];
            listenPort = 51820;
            privateKeyFile = "/root/wireguard-keys/privatekey"; # fill this file with the server's private key and make it so only root has read/write access
      
            postUp = ''
              ${pkgs.iptables}/bin/iptables -A FORWARD -i wg0 -j ACCEPT
              ${pkgs.iptables}/bin/iptables -t nat -A POSTROUTING -s 10.0.0.1/24 -o ens3 -j MASQUERADE
              ${pkgs.iptables}/bin/ip6tables -A FORWARD -i wg0 -j ACCEPT
              ${pkgs.iptables}/bin/ip6tables -t nat -A POSTROUTING -s fdc9:281f:04d7:9ee9::1/64 -o ens3 -j MASQUERADE
            '';
      
            preDown = ''
              ${pkgs.iptables}/bin/iptables -D FORWARD -i wg0 -j ACCEPT
              ${pkgs.iptables}/bin/iptables -t nat -D POSTROUTING -s 10.0.0.1/24 -o ens3 -j MASQUERADE
              ${pkgs.iptables}/bin/ip6tables -D FORWARD -i wg0 -j ACCEPT
              ${pkgs.iptables}/bin/ip6tables -t nat -D POSTROUTING -s fdc9:281f:04d7:9ee9::1/64 -o ens3 -j MASQUERADE
            '';
      
            peers = [
              { # peer0
                publicKey = "{client public key}"; # replace this with the client's public key
                presharedKeyFile = "/root/wireguard-keys/preshared_from_peer0_key"; # fill this file with the preshared key and make it so only root has read/write access
                allowedIPs = [ "10.0.0.2/32" "fdc9:281f:04d7:9ee9::2/128" ];
              }
            ];
          };
        };
      
        # Configure network proxy if necessary
        # networking.proxy.default = "http://user:password@proxy:port/";
        # networking.proxy.noProxy = "127.0.0.1,localhost,internal.domain";
      
        nixpkgs.config = {
          allowUnfree = true; # don't set this if you want to ensure only free software
        };
      
        # Select internationalisation properties.
        i18n.defaultLocale = "en_US.UTF-8";
        console = {
          font = "Lat2-Terminus16";
          keyMap = "us";
        };
      
        # Set your time zone.
        time.timeZone = "America/New_York"; # set this to the same timezone your server is located in
      
        # List packages installed in system profile. To search, run:
        # $ nix search wget
        environment = {
          systemPackages = with pkgs; let
            nvimcust = neovim.override { # lazy minimal neovim config
              viAlias = true;
              vimAlias = true;
              withPython = true;
              configure = {
                packages.myPlugins = with pkgs.vimPlugins; {
                  start = [ deoplete-nvim ];
                  opt = [];
                };
                customRC = ''
                  if filereadable($HOME . "/.config/nvim/init.vim")
                    source ~/.config/nvim/init.vim
                  endif
      
                  set number
      
                  set expandtab
      
                  filetype plugin on
                  syntax on
      
                  let g:deoplete#enable_at_startup = 1
                '';
              };
            };
          in
          [
            jdk8
            nvimcust
            p7zip
            wget
            wireguard
          ];
        };
      
        # Some programs need SUID wrappers, can be configured further or are
        # started in user sessions.
        # programs.mtr.enable = true;
        # programs.gnupg.agent = {
        #   enable = true;
        #   enableSSHSupport = true;
        #   pinentryFlavor = "gnome3";
        # };
      
        # List services that you want to enable:
      
        # Enable the OpenSSH daemon.
        services = {
          dnsmasq = {
            enable = true;
            # this allows DNS requests from wg0 to be forwarded to the DNS server on this machine
            extraConfig = ''
              interface=wg0
            '';
          };
          fail2ban = {
            enable = true;
          };
          openssh = {
            enable = true;
            permitRootLogin = "no";
          };
          zfs = {
            autoScrub = {
              enable = true;
              interval = "monthly";
            };
          };
        };
      
        # Set sudo to request root password for all users
        # this should be changed for a multi-user server
        security.sudo.extraConfig = ''
          Defaults rootpw
        '';
      
        # Define a user account. Don't forget to set a password with ‘passwd’.
        users.users = {
          vpsadmin = { # admin account that has a password
            isNormalUser = true;
            home = "/home/vpsadmin";
            extraGroups = [ "wheel" ]; # Enable ‘sudo’ for the user.
            shell = pkgs.zsh;
          };
          mcserver = { # passwordless user to run a service - in this instance minecraft
            isNormalUser = true;
            home = "/home/mcserver";
            extraGroups = [];
            shell = pkgs.zsh;
          };
        };
      
        systemd = {
          services = {
            mcserverrun = { # this service runs a systemd sandboxed modded minecraft server as user mcserver
              enable = true;
              description = "Start and keep minecraft server running";
              wants = [ "network.target" ];
              after = [ "network.target" ];
              serviceConfig = {
                User = "mcserver";
                NoNewPrivileges = true;
                PrivateTmp = true;
                ProtectSystem = "strict";
                PrivateDevices = true;
                ReadWritePaths = "/home/mcserver/Eternal_current";
                WorkingDirectory = "/home/mcserver/Eternal_current";
                ExecStart = "${pkgs.jdk8}/bin/java -Xms11520M -Xmx11520M -server -XX:+AggressiveOpts -XX:ParallelGCThreads=3 -XX:+UseConcMarkSweepGC -XX:+UnlockExperimentalVMOptions -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:MaxGCPauseMillis=10 -XX:GCPauseIntervalMillis=50 -XX:+UseFastAccessorMethods -XX:+OptimizeStringConcat -XX:NewSize=84m -XX:+UseAdaptiveGCBoundary -XX:NewRatio=3 -jar forge-1.12.2-14.23.5.2847-universal.jar nogui";
                Restart = "always";
                RestartSec = 12;
              };
              wantedBy = [ "multi-user.target" ];
            };
            mcserverscheduledrestart = { # this service restarts the minecraft server on a schedule
              enable = true;
              description = "restart mcserverrun service";
              serviceConfig = {
                Type = "oneshot";
                ExecStart = "${pkgs.systemd}/bin/systemctl try-restart mcserverrun.service";
              };
            };
          };
          timers = {
            mcserverscheduledrestart = { # this timer triggers the service of the same name
              enable = true;
              description = "restart mcserverrun service daily";
              timerConfig = {
                OnCalendar = "*-*-* 6:00:00";
              };
              wantedBy = [ "timers.target" ];
            };
          };
        };
      
        # This value determines the NixOS release from which the default
        # settings for stateful data, like file locations and database versions
        # on your system were taken. It‘s perfectly fine and recommended to leave
        # this value at the release version of the first install of this system.
        # Before changing this value read the documentation for this option
        # (e.g. man configuration.nix or on https://nixos.org/nixos/options.html).
        system.stateVersion = "20.09"; # Did you read the comment?
      
      }
      
      You'll notice that this server acts as a Wireguard endpoint and as a Minecraft server. I described the first part on the [NixOS wiki page for Wireguard](https://nixos.wiki/wiki/Wireguard) under the section that mentions dnsmasq. The second part is done using NixOS's systemd support, which can be a bit confusing at first but is easy enough once you know how it works.

      Edit: Also, the provider I use is ExtraVM, who has been excellent.

      6 votes
    7. Tips to use NixOS on a server?

      I see some people using NixOs on their servers. I would like to try it out to self host some services and learn about NixOs. I use hetzner and they have an NixOs iso available so I can just use...

      I see some people using NixOs on their servers. I would like to try it out to self host some services and learn about NixOs.

      I use hetzner and they have an NixOs iso available so I can just use that to install NixOs. But how do people manage remote instances of NixOs? They would just use ansible or something like it, to run nix on the host, or is there a better way?

      Thanks

      11 votes