26 votes

What are the best practices regarding personal files and encryption?

Over the past year I have done a lot to shore up my digital privacy and security. One of the last tasks I have to tackle is locking down the many personal files I have on my computer that have potentially compromising information in them (e.g. bank statements). Right now they are simply sitting on my hard drive, unencrypted. Theft of my device or a breach in access through the network would allow a frightening level of access to many of my records.

As such, what are my options for keeping certain files behind an encryption "shield"? Also, what are the potential tradeoffs for doing so? In researching the topic online I've read plenty of horror stories about people losing archives or whole drives due to encryption-related errors/mistakes. How can I protect against this scenario? Losing the files would be almost as bad as having them compromised!

I'm running Linux, but I'm far from tech-savvy, so I would either need a solution to be straightforward or I'd have to learn a lot to make sense of a more complicated solution. I'm willing to learn mainly because it's not an option for me to continue with my current, insecure setup. I do use a cloud-based password manager that allows for uploading of files, and I trust it enough with my passwords that I would trust it with my files, though I would like to avoid that situation if possible.

With all this in mind, what's a good solution for me to protect my personal files?

16 comments

  1. [8]
    spit-evil-olive-tips Link
    Full-disk encryption - this means every time you boot your computer, you enter a password (separate from any user account password). If I steal your hard drive, I can't read any of the contents...
    • Full-disk encryption - this means every time you boot your computer, you enter a password (separate from any user account password). If I steal your hard drive, I can't read any of the contents without knowing the password. This is fast enough on any modern CPU that it causes essentially zero slowdown. Most Linux distros have this as an option when you install, though you should double-check that it's LUKS and not eCryptFS for true full-disk encryption. LUKS also allows multiple keys, so you can have your normal everyday password/passphrase you use for boot, and a second key with an incredibly long passphrase that you write on a piece of paper and store in a safety deposit box, or wherever you keep crucial documents like your passport.

    • Encrypted backups - we had a long thread on this fairly recently but the search function is failing me so I can't find it. Variety of solutions here too, this ensures that you have backups of your important files if you were to have your primary machine crash. The passphrase for these backups should be different from your disk encryption passphrase.

    • Sync between machines - you didn't mention this explicitly, but if you have documents you want replicated between machines (not as a backup, but say between laptop, desktop, and phone) use Syncthing instead of a service like Dropbox or Google Drive. Syncthing allows limited "store old versions" functionality but it is not a replacement for true offline, separate-location backups.

    • Password manager - you mention you're happy with the cloud-based one you've got now but I'll also throw out that offline / self-hosted options are available and tend to be more secure than something like LastPass. I use pass(1) in combination with syncthing, so I get the convenience of having all my passwords on every device I care about, without the data (even encrypted) ever being stored on someone else's server in the cloud.

    24 votes
    1. elfox Link Parent
      Yeah, I agree. If you want to go simple just do full disk encryption. Since you have Linux you can just encrypt your /home too. I know you didn't really ask about redundancy, but in that area it...

      Yeah, I agree. If you want to go simple just do full disk encryption. Since you have Linux you can just encrypt your /home too.

      I know you didn't really ask about redundancy, but in that area it just depends how important your stuff is to you. For my personal stuff I usually have just one step past a single point of failure. And honestly, I still have quite a few single points of failure. Anything super important I push off site onto a cloud resource. But that is very few items, and most of the time that's because I want easy access to them or I am sharing them.

      7 votes
    2. [6]
      kfwyre Link Parent
      This is a great rundown, thank you. I do have lots of questions though, if you have the patience for them. I don't have great background knowledge of the topic, so some of this is me simply trying...

      This is a great rundown, thank you. I do have lots of questions though, if you have the patience for them. I don't have great background knowledge of the topic, so some of this is me simply trying to get my mind around some of the probably obvious, big picture stuff.

      Full-disk encryption

      Is this the kind of thing I could roll out now in my current distribution, or would I have to do a reinstall? Does the potential slowdown impact gaming? Do I have a greater risk of losing the whole drive?

      Encrypted backups

      Currently I have SpiderOak keeping my backups in the cloud (which are encrypted). If I were to encrypt my hard drive, what does a recovery scenario look like? Download encrypted files from SO, then decrypt again on my local machine? Furthermore, if I'm in a recovery scenario and I need only a few specific files, how would I find and decrypt those if they were encrypted using full-disk encryption? Would I have to have the entire disk image to decrypt? Would I have to dump everything and then sift through them unencrypted?

      I don't know if that question makes sense, so I guess what I'm getting at is this: if I encrypt my disk and then look at it without the encryption, would I still see the same directory structure and number of files in each folder but just with encrypted names and filetypes, or would it just be just a large unrecognizable single chunk of data?

      Sync between machines

      I already use Syncthing to sync files between two different computers and my phone. If I enable encryption on one, wouldn't that cause Syncthing to sync encrypted versions that would be useless on the other devices? Can it even selectively sync certain files if full-disk encryption is enabled? Is there a way to set up concurrent encryption on all endpoints of Syncthing (including Windows/Android) so that the files remain usable outside of my main Linux machine?

      Password manager

      I'm currently using Bitwarden, and convenience is king for me right now. I've looked into other, more secure non-cloud options, but they all have much more of a learning curve. While I would eventually like to shore up my technical knowledge to make me comfortable with one of those alternates, for now I'm okay with what I've got. After all, it was far better than the password reuse system I had in place before.

      7 votes
      1. [5]
        spit-evil-olive-tips Link Parent
        Some background info for understanding full-disk encryption - think of everything in terms of layers. At the bottom layer (for our purposes) is your SSD or hard drive. In Linux terms it's a block...

        Some background info for understanding full-disk encryption - think of everything in terms of layers. At the bottom layer (for our purposes) is your SSD or hard drive. In Linux terms it's a block device. It has small, fixed size blocks (512 bytes, or 4KB on some newer drives) and each block gets a number. You can ask "write these 512 bytes to sector #1234" or "what's stored at sector #1234" but that's about it. At this layer there's no concept of files, filenames, directories, etc.

        Layered on top of the block device is part of the OS called the filesystem. You ask the filesystem "what's stored in the /home/kfwyre/Documents/ directory?" and it knows what blocks of the disk to go read to look the directory listing, including filenames, sizes, modification times, and all that. And for each of those documents, it knows what blocks of the disk they're actually stored at. For 100-byte files it'll know that only uses up a partial block, so it reads all 512 bytes but ignores the extra 412 bytes[1]. For a large file, it knows it needs to split it into multiple blocks, so it'll allocate a single span of "block #1000 to #2000", or it'll store a list of "blocks 1234, 3456, 7890..."[2].

        1: If the filesystem is smart, and modern filesystems are very smart, it won't actually waste those 412 bytes, but cram another small file in there, or cram the file's metadata in alongside the contents itself, or something along those lines.

        2: If you're old enough to remember the good ol' days when your computer would get slow and you'd defragment your hard drive and it'd get faster, this is what was happening under the covers. You'd have a file scattered across your hard drive, and reading that file would send the filesystem on a wild goose chase all over the actual disk. Defrag would take the blocks of the file and move them into one long contiguous segment on the disk.

        The full disk encryption I'm talking about (LUKS) is yet another layer. It gets sandwiched so it's sitting on top of the disk block device, but below the filesystem, and it acts like just another block device.

        So the filesystem says "here, write this data to sector #1234", and the data the filesystem is working with is unencrypted. Normally that'd go straight to the disk, but because LUKS is sitting in the way, it encrypts the data then writes that encrypted data to sector #1234 of the actual hard drive. Likewise, when the filesystem asks for sector #1234 to be read, LUKS reads the encrypted data off the disk, decrypts it, then hands the decrypted data up to the filesystem.

        This is why you'd need to wipe+reinstall to start using FDE - right now all your data is sitting on the disk unencrypted, and the moment you put that LUKS layer in, it'll expect everything to be encrypted.

        For your questions about whether apps like Syncthing and Spideroak will deal with encrypted data - nope, they won't. They're layers on top of the filesystem.

        What will happen, is say you've got Syncthing running on two machines, both with FDE. Syncthing reads the file from one machine, which goes through LUKS to decrypt it, then it sends it over the network to the other machine. That connection is encrypted by Syncthing, so someone eavesdropping to their conversation wouldn't be able to get your file. Then the other machine takes the file, writes it to disk, which goes through LUKS to encrypt it on-disk. The two machines will be using different encryption keys (even if they have the same FDE passphrase) so if I stole both of those disks, I wouldn't even be able to tell they have the same file on them.

        Similarly, Spideroak reads a file from your disk, LUKS decrypts it as it goes, then Spideroak encrypts it with a different encryption key and uploads it to their servers. If your home is hit by a meteor and you have to recover from your Spideroak backups, it's OK that your encrypted drive was destroyed, because you just need your Spideroak credentials and you can download your backed up files, unencrypted (though if your new machine also has FDE, they get encrypted the moment they're written to that new disk).

        This also answers your question here:

        if I encrypt my disk and then look at it without the encryption, would I still see the same directory structure and number of files in each folder but just with encrypted names and filetypes, or would it just be just a large unrecognizable single chunk of data?

        Full disk encryption means everything is encrypted, even directory listings, filenames, etc. This is because the filesystem layer has to write a block to disk saying "here's the contents of the /home/kfwyre/Documents directory" and since that goes through LUKS too, it gets encrypted like everything else. If you went block-by-block over your encrypted drive, you'd have a small unencrypted segment at the beginning, containing your /boot partition - the bare minimum Linux needs to start up and show you an enter disk encryption passphrase: prompt. The rest of the drive would be absolute gibberish, indistinguishable from random noise.

        As far as performance goes, it shouldn't be noticeable, even for gaming. The reason is that as fast as SSDs have gotten, CPUs are still faster, and they have special instructions that accelerate AES, the encryption algorithm used for this. On my desktop (a ~4 year old Haswell i5):

        $ cryptsetup benchmark
        ...
                aes-xts   256b  2094.6 MiB/s  2094.8 MiB/s
        

        There's other algorithms listed in the benchmark, but that's the algorithm I'm using, and the default if you were to set this up. My 4-year-old CPU can encrypt and decrypt data at 2 GB/sec. My SSD isn't anywhere close to that fast, so reading data off the disk isn't slowed down at all by the encryption (though it might increase my CPU usage by 1-2%).

        25 votes
        1. [4]
          kfwyre (edited ) Link Parent
          Wow, thank you for taking the time to write that out and explain it so well. That was genuinely helpful and gave me some much needed perspective that I was missing. I had no idea the encryption...

          Wow, thank you for taking the time to write that out and explain it so well. That was genuinely helpful and gave me some much needed perspective that I was missing.

          I had no idea the encryption would be effectively transparent to me and my programs while my computer is in use. This was exactly the kind of solution I was looking for, especially because it is compatible with the other systems I have in place (e.g. Syncthing and SpiderOak).

          A final question about implementation:

          Can a system using LUKS read unencrypted drives? I'm assuming yes, but I don't want to make a fatal mistake here. Currently my machine has most of my data on a large external hard drive, and I have a second hard drive of the same size acting as an on-site backup. If I were to wipe and reinstall, would I simply wipe one of the drives, enable encryption on it, and then move my files into that encrypted drive from the unencrypted one?

          EDIT: It is going to be a while before I am able to fully wipe my Linux installation given some of the stuff going on in my life at the moment. In lieu of full-disc encryption, is there a way I can simply encrypt a single directory? Is a password-protected archive good enough?

          8 votes
          1. [3]
            spit-evil-olive-tips Link Parent
            Yeah, I love that FDE is something you can forget about for the most part. I use it on every single desktop and laptop I own - no reason to ever not use it (well, one reason is if you have a...

            Yeah, I love that FDE is something you can forget about for the most part. I use it on every single desktop and laptop I own - no reason to ever not use it (well, one reason is if you have a server you want to boot without user interaction, discussed here).

            You can absolutely have one system with both encrypted and non-encrypted drives. My explanation above, as detailed as it was, was actually hand-wavy in many ways. Both at the lower levels (things like wear leveling and erase blocks on SSDs) as well as at the upper levels. On Linux you've got something called the VFS (Virtual File System) that makes it possible to have many different filesystems in use all at once.

            On the desktop I'm typing this on, it's got an SSD, with a LUKS-encrypted partition, formatted with a filesystem called XFS, and mounted at /. There's a separate partition on the SSD, which is not encrypted, formatted with a different filesystem (VFAT) and mounted at /boot. Linux, because of the VFS, knows that if I ask for a directory listing of /var/whatever, that lives on my root partition and it should ask XFS to go figure that out. But if I need something under /boot, it will go through VFAT - not just differing in encrypted vs. not-encrypted but also in the actual code that lays out data on disk. Both of those are on the same SSD, but there's no reason they need to be. My /home directory could easily be an ext4 filesystem on a different SSD.

            The VFAT /boot directory is necessary because my UEFI BIOS understands the VFAT filesystem, but not XFS, so when it first starts up it knows how to find Linux (technically, another piece of software called Grub) and say "you take over booting from here".

            If I wanted to copy some files to my girlfriend's computer (which runs Windows), I could plug in an external drive, format it with the NTFS filesystem (because both Linux and Windows understand it) and then mount it at /mnt/external. Linux's VFS handles it so that if I copy from /home/spit-evil-olive-tips/Downloads/somefile to /mnt/external/somefile that it knows it's copying from my SSD, decrypting the file using LUKS as it goes, and then writing that to the external drive. When I disconnect that and plug it in to her computer, Windows recognizes the NTFS filesystem and can read the files perfectly. If I had LUKS-encrypted the drive first, it would appear as random gibberish to Windows and wouldn't even get a drive letter - but the same thing would happen if I left it unencrypted, but formatted it with XFS or ext4 or another Linux-only filesystem. The data wouldn't be encrypted but Windows wouldn't have the Linux-specific XFS code needed to read the data on disk.

            The VFS can handle network drives, too - I've got a Linux server in my apartment, with a bunch of disks running a filesystem called ZFS, and a piece of software called Samba. My desktop has /mnt/crap in its VFS, but it knows that that's not a local disk at all, but a network filesystem on another machine, so when I ask to read a file from there it actually reaches out to my home server over the network. Then Samba running on the server gets that request, does a local read of the /mnt/crap filesystem using ZFS, and sends the content back over the network. Because of the VFS, I can open a document on my encrypted local SSD the same as I do a file on my Samba share on a different computer.

            For your question about just encrypting one directory to start, the best option would be something called eCryptFS. It's not suitable for full-disk encryption, and I wouldn't recommend it even for full-home-directory encryption (it limits filename length to something smaller than most filesystems allow, which can break programs in unexpected ways) but you can certainly create ~/crypt-test/ and play around with it. The Arch Linux wiki has a good article on it. Even if you don't run Arch, their wiki is an amazing resource for these more advanced Linux topics.

            Another option is something called a loopback device. This is a little trickier than eCryptFS, so I wouldn't use it for your main encrypted document archive...but it's incredibly useful for playing around / testing. Hold my beer and watch this:

            daario ~> dd if=/dev/zero of=test bs=1M count=100
            100+0 records in
            100+0 records out
            104857600 bytes (105 MB, 100 MiB) copied, 0.0319706 s, 3.3 GB/s
            daario ~> sudo losetup loop0 test
            daario ~> ls -l /dev/loop0
            brw-rw---- 1 root disk 7, 0 Oct  1 22:06 /dev/loop0
            daario ~> sudo mkfs.ext4 /dev/loop0
            daario ~> sudo mount /dev/loop0 /mnt/test/
            daario ~> df -h /mnt/test/
            Filesystem      Size  Used Avail Use% Mounted on
            /dev/loop0       93M  1.6M   85M   2% /mnt/test
            

            I just created a blank 100mb file in my home directory. Then I told Linux "take this 100mb file, and pretend it's a 100mb hard drive named loop0". I now have a block device called /dev/loop0, but rather than the blocks living directly on a disk partition, those blocks live in a normal everyday file, which in turn lives in my home directory, LUKS-encrypted, on my SSD (layers upon layers upon layers). I can format that block device with any filesystem I want (ext4 in this case) and mount it somewhere in the VFS, say at /mnt/test.

            13 votes
            1. [2]
              kfwyre Link Parent
              If your first comment was the assignment, then the second was some one-on-one tutoring. This last one is an enrichment activity that pushes higher-level learning. Across all three you've given me...

              If your first comment was the assignment, then the second was some one-on-one tutoring. This last one is an enrichment activity that pushes higher-level learning. Across all three you've given me a mini-class in encryption! I've got a lot to digest and learn. It's time for me to do some homework.

              Also, I don't know if you see yourself this way, but to me you're a modern day wizard.

              5 votes
              1. UntouchedWagons Link Parent
                Just a note, it is technically possible to convert your existing install to one that uses LUKS but it'd be a bit fidly. I've been experimenting with trying to convert a partition-based Fedora...

                Just a note, it is technically possible to convert your existing install to one that uses LUKS but it'd be a bit fidly. I've been experimenting with trying to convert a partition-based Fedora install to one that uses LVM, so far I've been unsuccessful.

                1 vote
  2. [3]
    Diff (edited ) Link
    Lots of distros will ask in the installer if you want your home directory encrypted. Elementary OS did, works flawlessly. Totally forgot I had encryption enabled til I tried to pull a file out of...

    Lots of distros will ask in the installer if you want your home directory encrypted. Elementary OS did, works flawlessly. Totally forgot I had encryption enabled til I tried to pull a file out of it from a LiveCD. Still managed to do it, but without Elementary automatically handling it all, it was a slight pain to run all the little decryption utils myself. Just had to read the docs, but I had to read the docs.

    But yeah that kind of automated home directory encryption is pretty great.

    I hear KDE has some encrypted folder integration into its shell recently, too. I think they're calling it "Vault"? Maybe check that out if you're a KDE kinda guy. Looked pretty need NEAT from their demo video.

    5 votes
    1. [3]
      Comment deleted by author
      Link Parent
      1. [2]
        spit-evil-olive-tips (edited ) Link Parent
        It's possible (though I haven't tried it myself) to embed dropbear SSHD inside an initramfs, so that you can SSH in and type the FDE passphrase at boot. In theory, yes, if only your /home is...

        It's possible (though I haven't tried it myself) to embed dropbear SSHD inside an initramfs, so that you can SSH in and type the FDE passphrase at boot.

        In theory, yes, if only your /home is encrypted, you could SSH in, type the passphrase there, and then mount /home. The caveat is that your ~/.ssh/authorized_keys file will still be encrypted when you SSH in, so you'd have to login using password authentication.

        An additional option would be a /mnt/private or something where you keep sensitive files. That could bypass the SSH authentication problem and allow you to lock down your SSHD config to allow key-based auth only.

        There's also the overkill option, which is upgrade to server hardware supporting IPMI. That would let you connect to the management interface of your machine and then get a VNC-like console during boot to type the passphrase in.

        4 votes
        1. [2]
          Comment deleted by author
          Link Parent
          1. spit-evil-olive-tips Link Parent
            Yeah, you'd either need to SSH in as root (which would allow key-based auth) or as your normal user, but with password auth only. Either case would require softening sshd from the recommended...

            Yeah, you'd either need to SSH in as root (which would allow key-based auth) or as your normal user, but with password auth only. Either case would require softening sshd from the recommended hardened config (no root login, key auth only).

            Trying for an uptime high score and just dealing with the pain when you do reboot does seem like a simpler option.

  3. [3]
    Nitta Link
    I'd think about the balance between security and accessibility. It's like asking yourself a question: Would I rather like to forget password than someone else captures the data, or it's more...

    I'd think about the balance between security and accessibility. It's like asking yourself a question: Would I rather like to forget password than someone else captures the data, or it's more important to have free access even at the cost of someone else gaining it?

    These two principles are kinda the opposite. The more you encrypt and password protect, the more likely it is to accidentally lose the backup or archive by forgetting passwords over time. I'm even curious if it's a real thing or just my anecdotal observations.

    3 votes
    1. Hubi Link Parent
      I agree that there needs to be a balance between usability and security. A simple solution would be to use a password manager - that way you'd only have to remember one tough combination to access...

      I agree that there needs to be a balance between usability and security. A simple solution would be to use a password manager - that way you'd only have to remember one tough combination to access all your stored keys. Or maybe just write it on a piece of paper.

      I'd encourage everyone to use encryption for personal files, there is really no downside to it.

    2. kfwyre Link Parent
      I think my hesitation came from my own misconceptions about the "face" of encryption and how I would have to interact with it. I was picturing it being this big, impenetrable vault that I could...

      I think my hesitation came from my own misconceptions about the "face" of encryption and how I would have to interact with it. I was picturing it being this big, impenetrable vault that I could never really see inside. I thought I would be dumping stuff into a black box, hoping it's safe, and hoping that I can get it out eventually if needed.

      Knowing now that I can operate inside the encryption is a game-changer, as I can still do everything I need to and be protected at the same time. Given what I've learned the question is no longer to choose between encryption or convenience, but instead to choose how long I'm willing to go without encryption, since wiping and reinstalling is going to be a somewhat laborious task for me.

  4. SleepyGary Link
    In addition to what is being said here, enable Multi-factor authentication whenever possible. If you have a password manager, even a cloud based on like LastPass, if you have a MFA enabled then...

    In addition to what is being said here, enable Multi-factor authentication whenever possible.

    If you have a password manager, even a cloud based on like LastPass, if you have a MFA enabled then all is not lost if someone gains access because they would also need your phone or desktop app, which hopefully you've password protected with a password different than your password manager master password.

    Do not let applications and sites remember your password or even remember device for X days so you can avoid entering the MFA code, have it ask every time, at least for sensitive content like your email or financial logins.

    1 vote
  5. nic Link
    If you have a mac, using an encrypted disc image lets you sandbox files nicely, just back it up as always.

    If you have a mac, using an encrypted disc image lets you sandbox files nicely, just back it up as always.

  6. Comment removed by site admin
    Link