14
votes
How do you keep your digital files organized?
Mine are a bit of a mess, with decades of accumulation and lack of maintenance resulting in a forest of poorly organized folders and completely inconsistent filename schemes.
I'm interested in hearing about things like folder hierarchies/taxonomies, naming conventions, and techniques for efficient access. This goes for everything from personal documents to projects to pictures to media collections.
Mine is a bit weird, I try to keep everything in a single directory so it's easier to migrate computers, and the scheme is slightly strange (but in my defense I story a strange collection of information).
This formatting is kinda cozy and oldschool. And the folders remind mine
I think I might steal "Journal" and "Writing" as a top level categories. "Documents" feels too formal for a lot of what's in mine, and right now it has everything from to-do lists to old e-mail forwards to financial statements to creative writing. Those all don't exactly need to sit side by side.
I think I'll follow your lead and break stuff up "Journal," (everyday documents), "Documents," (formal stuff and record-keeping) and "Writing" (creative stuff). Thanks!
Heh. That implies I've been able to organize it, which I haven't, since it dates back to 1994 and I'm a hoarder.
Broadly scoped it's a bit like this...
\movies\film-name (year)\film.mp4 (and support files)
\television\series-name (year)\season 00-99\ep# - title.mp4 (and support files)
\music\artist name\year - album name\track - title.mp3
\clips\subject\author\video title (for youtube, ted, feynman lectures, etc)
\concerts\artist\year - venue (for musical shows which I have a fondness for)
\docs\school, work, taxes, writing, notes, etc - this is a bit slapdash but it's smallish so no problem
\pics\ with various subfolders, including a rather massive but old wallpaper gallery
That's the 'organized' bits. There's an entire tens-of-terabyte bit bucket I'm still harvesting and importing into that format. The kind of directory structure full of folders named 'sort this shit' that also contain five even larger folders titled 'sort this shit too' which also contain large folders called 'incoming 1' through 'incoming 12'. There's a couple hundred thousand albums untagged from a.b.mp3.* that are encoded in such old versions of the mp3 standard they segfault most modern player programs - and some of that music is so rare I can't even find info on the artists. Hundreds of music collections that are just random mixtapes (like the ENTIRE summer mix series 2008-2015) or time-life collections and greatest decades compilations. There are also some shows and films in that heap still in ISO format waiting for me to grind them up properly into something fit for Kodi to scan.
That knot of data is intractable for a computer to organize... in fact I'm pretty sure it'd stump an AGI. The only way to do it is folder by folder. Every once in a while when I'm feeling ambitious I take a chunk out of it.
Any/all new data is filed away properly into the system, thanks largely to programs like tagscanner and tinymediamanager which make that stuff into near one-click operations. I didn't get better at organizing. The tools merely improved to the point where they were effective.
It's funny, I used to have a massive library of software back in the warez days. Now I just stick to open source for almost everything and rarely do I run into a need for a commercial program. I deleted all of that old stuff, except for a /games/ folder full of dos-era through ps1-era roms/isos/installers. There's something like 4000 dos era games in there I haven't looked at in a very long time, except when one of my virus scanners picks up something ancient in a pointless signature scan - as if those viruses could even execute in a modern environment.
Your "sort this shit" folders are reminiscent of mine. I was inspired to make this thread while I was running a backup and saw
/Personal/Personal/Documents/Writing/Documentscome up as a directory. Yikes.I also have
/Photos,/Pictures,/Photos Archive, and/Pictures Archive. Oh, and/Pictures Archive/PhotosThere are undoubtedly a lot of duplicate files between all of them.I've got my files and folders organized/setup like this:
Well you get the gist ;)
In the most general sense, I sort folders from most to least permanent and biggest to smallest, with a priority on making new folders as the materials in a given folder are very different from each other. A couple of examples: Personal>Taxes>2018>1099's. I'll always have personal files, I'll probably always have files regarding my personal taxes, I'll need to deal with 2018 for a while, and I have enough 1099s to have them deserve a file to keep them separate from other documents. Or take Business>Clients>[Clientname]>Contracts: Business is big and fairly permanent, I'll always have clients, I want that client's documents separate from my other client's files, and I want their contract separate from other working docs.
Big to small, permanent to impermanent, split when too different.
I prefer rather flat structure that makes content reachable. As there's usually some old or less needed content that can crowd folders, it goes to
[other]then. For example for current and recent work projects it's just/projects/<project_name>, the older ones go to/projects/[other]/<year>/<project_name>.Speaking of top folders, there are also just a few:
Interests: folders with personally valuable content per hobby/collection/topic
Projects
Software: mostly a loose collection of rare/critical installables and scripts
Books: categorized into folders by topic
Documents: loose pile of recent ones, and the rest are in folders by topic, like health, work, ownership
Other: to throw content here to sort later, or temporarily
Media: a LAN shared folder to play or watch content from on PC or tablet
Yeah, I definitely need to flatten things out. In the past I made a lot of subdirectories thinking that would help with organization, but it's clear now that really deep paths just mean I lose things. I no longer have a good mental map of where things are because there are so many different places they could be.
I keep it simple, and it's easy to move between computers when they inevitably die, or I actually choose to upgrade.
Most things are within two levels of my home directory.
Code has one directory, most other things end up in Documents, or a category that suits them, like my artwork goes into Pictures.
If I am looking for something in particular, but can't quite remember where it is, then the silver-searcher (ag) can run through everything incredibly quickly.
The key folders in my home directory look like:
Some application keeps creating ~/Desktop, to my displeasure, and although it sticks out like a sore thumb it is inconvenient to call one's downloads folder anything other than ~/Downloads. Perhaps at some point I will try and set up all the necessary configurations and environment variables to use ~/dl instead.
Try setting XDG_DESKTOP_DIR and XDG_DOWNLOAD_DIR in
~/.config/user-dirs.dirs, then runxdg-user-dirs-update(might need to install some xdg software).Thanks. I've given that a try - hopefully this should fix my issues.
Funnily enough, one of my current projects is an app that helps users sort, manage, and organize huge amounts of files. It's still in the very early development stage, but after seeing the interest here, I'll probably post a version of it once it's presentable.
Nearly every program I've used for that sort of thing is abysmal. Commercial ones attempt to foist the company's products/standards and tend to have a very childish, small-minded scope... the 'bare minimum' they can then hype up on a product page and all of those crapware sites like majorgeeks/sourceforge to sucker people into paying them money. Most people who go looking for this kind of software end up burned by a couple of these and drop the whole thing in disgust. Search engines seem primed to return nothing but garbage results in this area, and have for years.
A lot of the tools make the mistake of forcing people to file things away according to the programmer's preferences. This has always been my chief issue with iTunes - the wad of scattered directories and mislabelled files it calls a 'music collection' is an affront to sanity on every level, and whoever came up with that should be euthanized. I like metadata in my files just fine, but I also like my file names and hierarchy to reflect all the relevant organizational data. Programs that just rely on the metadata and ignore the file names get a hard pass from me. That was what WinFS was going to be for, but I guess that's dead now too.
I find the ones that target niches do the best jobs within those niches. I have to give the gold star to tiny media manager When it comes to easily managing and updating a library of films and television shows (along with associated subtitles, thumbnails, xml info files, tagging) there's nothing else out there that's near the same league. There were several other more narrow open source tools for this, but they've all been absorbed into tmm at this point and are basically dead projects. Anyone who has a Kodi/XBMC library at home should be using this tool to preseed all the data into the file structure so Kodi can read it all locally without having to look it up online.
A lot of people still swear by Mediamonkey for managing music collections. I find it to be a total failure at that task for any decently large collection (mine is several terabytes and way beyond MM's capability). Most of MM's 'great' plugins are crapware (their audio-ID sucks). I prefer tagscanner since it's more powerful and also more basic. Tagscanner does one simple thing incredibly well, and once it's done, you'll never need to edit the tags again. The ability to apply all actions in part or in bulk is what attracted me to it. Quite the timesaver.
I'd love to find some tool that could actually listen to my music and tag it based on that, but every single tool in that area is garbage for anyone who has more than top40 music in their collection. If you want to fiddle with this stuff, MusicBrainz' Picard is as good as any commercial product I've tried (and it still sucks). Frankly there's no chance of improving this kind of tool without a hell of a lot more cratediggers using it for years and populating the database with orders of magnitude more sound samples and accurate release information. There just isn't enough metadata out there yet for this tech to work.
My favorite tool for file name management is the 1-4-a-renamer. This is some geocities-era old-school emailware. Frankly, it beats unix tools at this job, no small feat. Given the complexity of the file sorting/naming tasks it's capable of the UI walks you through how to use it rather well. When you want your brain fried, turn on 'expert mode' and take those options for a walk. It's only ever taken me 1-3 iterations to accomplish any kind of file name cleanup using this tool no matter how hellish the original files were. It has the kind of crazy yet useful options you can only get from decades of getting fan suggestions in your email and implementing them.
If you're looking for inspiration on how to present your project I'd say there's plenty to draw from here. This remains a woefully under-served segment of the application market. Make a good tool and it will get used because there are so few good tools out there.
First of all, thanks for such a informative reply. Secondly, my design philosophy as a whole is pretty much opposed to most of the flaws you pointed out. I hate programs that force preferences onto users. I want full control over my computer and my programs, and I usually can't have that, which is why I make so many of my own programs. I also am a bit of a perfectionist, and can't leave anything unfinished (unless I get distracted doing something else). I'll still try to keep this advice in mind one way or the other.
I'm a big fan of "everything in the same folder, sort by last modified". It's not for everything or everyone, but I've been using this strategy for 90% of my emails and documents for years now, and I almost never lose anything.
At one point I had loads of hidden directories cluttering my $HOME, so I set aside some time to get rid of almost all of them. Now it's much tidier: (I'll steal @pleure's presentation style)
I don't actually use
~/dumpthat much, most stuff gets sorted (eventually) somewhere under/data:Of which I've symlinked to $HOME those which I use the most. As for what naming conventions I try to stick by, mostly using short and lowercase directory names. It's not the most ordered hierarchy, but it's generally very easy to find stuff.
I keep all my data is one folder named "bitorrent", from way back when I used to use bitorrent sync to sync it between all my machines (nowadays I use sshfs and my unlimited Gdrive storage courtesy of university to keep important stuff synced up). It has literally zero organization; there's a "Music" folder that has all my music, mostly sorted by which game it belongs to (I listen to VGM), but has a bunch of folders like zreal or Z Internal that only make sense once you know the context (the former is for vocal music, all either pop or historical stuff, the latter because it was once stored on my Sansa Clip+'s internal storage). LaTeX is the name of the folder that used to hold all my school work, because I used to use LaTeX to type up everything. Games holds my collection of emulated games and their save files. And then there's tons of other scattered files and folders for specific times. It's very messy...