TL;DR: NASA's public Planetary Data System is at risk of being shut down. Anyone have any ideas for backing it up?

Hi everyone,

Bit of a long-shot here, but I wanted to try on high-quality tildes before jumping back into the cesspool of reddit. I'm posting it in ~science rather than ~space as I figure interest in backing up public data is broader than just the space community.

I work regularly with NASA's Planetary Data System, or PDS. It's a massive (~3.5petabytes!!) archive of off-world scientific data (largely but not all imaging data). PDS is integral for scientific research - public and private - around the world, and is maintained, for free, by NASA (with support of a number of Academic institutions).

The current state of affairs for NASA is grim:

• NASA Lays Off ISS Workers at Marshall Space Flight Center
• More layoffs at JPL
• NASA is sinking its flagship science center during the government shutdown — and may be breaking the law in the process, critics say

And as a result, I (and many of my industry friends) have become increasingly concerned that PDS will be taken down as NASA is increasingly torn down for spare parts and irreparably damaged. This administration seems bent on destroying all forms of recording-keeping and public science, so who knows how long PDS will be kept up. Once it's down, it'll be a nightmare to try and collect it all again from various sources. I suspect we'll permanently lose decades worth of data - PDS includes information going all the way back to the Apollo missions!

As such, we've been pushing to back-up as much of PDS as we can, but have absolutely no hope of downloading it all within the next year or two, nevermind in a few months if the current cuts impact us soon.

If you or someone you know would be interested in helping figure out how we can back-up PDS before it's too late, please let me know here or in a DM. I've already tried reaching out to the Internet Archive, but did not hear anything back from them.

Edit: to clarify, the larger problem is download speeds - we've topped out at 20mb/s with 8 connections.

Hey Tilderinos.
I've been looking into buying several flash drives since my largest flash drive is a 32GB sandisk, and I use or interact with all the 3 major OSes, I use Linux on my desktop and on a secondary laptop, I use MacOS on a Macbook and everyone else I know uses Windows(So I'll need an exFAT drive for them).
My recent experience with flash drives though makes me more willing to trust my data to a system's RAM than to a flash drive. At least RAM wouldn't lure me into a false sense of security then spontaneously fail, I know that my data isn't going to last a reboot.
I've got 3 sandisk cruzer blades fail on me, once was an error on my part where I accidentally hit it with my knee while plugged into a device(device unharmed, the drive is dead), one time I upgraded the SSD on my SteamDeck and flashed the steamdeck recovery to an 8GB stick, it worked fine while restoring and it still can be read... it's stuck on read only and, Gparted, Windows formatter, Rufus, Mac's disk utility nor mkfs can make it reusable, I assume it entered read only because it tripped some "whoops I'm dying" thing like some SSDs have(from what I know). The last one is effectively a resistor that connects to a USB port, it heats up, SOMETIMES appears on PC, Linux can open it, copying things into it via Nautilus works albeit very slowly, then when I try to open the root of the flash drive it is stuck perpetually trying to load, when I unplug it and plug it back in again, I can see the folders but entering any of them immediately goes into the permanent loading state, mkdir, cd and ls can work on the drive... intermittently, but I'm treating it as dead.

This leaves me with only 2 other drives, my largest drive, is a 32GB Sandisk Ultra, and my smallest drive which has been more reliable than the Cruzer Blades is a 4GB Sony... USM4GP thing, which I have no idea what year it was from and a quick googling didn't bring up any release date, but it had faster read/write than the cruzer blades, and it's been my main drive for things like installing an OS/burning an image into it yet it lasted all these years.

I'm trying to get a USB-C drive specifically since all my laptops that are in use and my PC have a USB-C port, but all I see in local big retailers are Sandisk, Sandisk and more Sandisk, Amazon however has some that aren't Sandisk.

Does anyone know a USB-C flash drive that is genuinely reliable? Was it specifically Cruzer Blades that is garbage? I've not had a Sandisk SD card fail on me yet, should I just avoid Sandisk for anything but SD cards? Should I just say fuck it and buy a bunch of enclosures and NVMe drives?

I've read that flash drives get bottom of the barrel NAND chips that can't be used on SSDs, too. I know that flash drives aren't meant for long term storage/backup but a drive that old shouldn't be still going on that strongly against new ones.

I've been looking at PNY Elite V3 with USB-C as a connector that I've seen a few listings on Amazon but it's 1TB price is almost the same price as a 1TB NVMe SSD(Though not factoring in the enclosure).

I wanted some help with a homelab server I am in the beginning stages of designing. I am looking for a flexible and scalable media and cloud system for home use, and I thought this community would be a good place to source feedback and recommendations before taking any real next steps! I really want to check that I am approaching the architecture correctly and not making any bad assumptions. I am open to all feedback, so please let me know what you think!
I already run a simple home server and I have typical homelab FOSS apps, such as jellyfin, navidrome and audiobookshelf, but I am also interested in migrating away from cloud storage using nextcloud, immich, etc. In an ideal world, this setup would also allow me to leave windows on my main machine and use a windows vm for business related work that can’t be done on Linux. I will likely be the one primarily using the services, however I could expect up to 10 - 20 users eventually.

High level setup is with two machines:

• Proxmox Server
• TrueNAS Scale server
  • JBOD with either 90 bay or 45 bay storage
• 10G switch

This might be a stupid setup right off the bat, which is why I wanted to discuss it with you all! I have read a ton about using TrueNAS as a WM within Proxmox, but I just like the idea of different machines handling different tasks. The idea here would be to set up the TrueNAS server so it can be optimized for managing the storage pool to allow for easy growth. While the Proxmox server can handle all the VMs and connecting users, with higher IO, etc.

TrueNAS System Specs:

• AMD ryzen CPU and motherboard
• 64 or 128GB ram
• Mirror 500GB M.2 NVMe OS Drives
• GPU if necessary, but hopefully not needed
• Dual 10gb pcie card if the motherboard doesnt already come with them
• An hba for the JBOD something like the LSI SAS 9305-16e
• SLOG and L2ARC as necessary?

JBOD enclosure

• While I am interested in a 90-bay enclosure, I would only realistically be starting with two vdevs which is why I think a 45 bay enclosure wouldn’t be an issue.
• Im tentatively planning for an 11 wide Raidz2 vdev configuration. This would hopefully scale to 8 vdevs with 2 hot spares or 4 vdevs with 1 hot spare.
• All drives would be HDDs

Proxmox Server Specs:

I am less familiar with the specs I will need for a good Proxmox server, but here is what I am thinking.

• AMD epyc and motherboard if I can get my hands on a less expensive one. Otherwise I was thinking a higher end AMD ryzen cpu
• 128 or 256GB ram
• Mirror 500GB M.2 NVMe OS Drives
• Somewhere between 2 and 8 TBs of SSD storage. Depending on the number of drives, I think this would be a single drive, mirror or raidz1.
  • This storage will be used for all the vm configuration and storage, except for something like Nextcloud where the main storage will go onto the TrueNAS mount.
  • I would also use this for temporal storage such as downloading a file before transferring it to the TrueNAS mount.
• A dedicated GPU primarily for transcoding media streams, but also for testing and experimenting with different AI models.
• Dual 10gb pcie card

Questions:

• I know Proxmox can do zfs right out of the box so I know I don’t need the TrueNAS server, but splitting it this way just seems more flexible. Is this a realistic setup or would it just be better to let Proxmox do everything?
  • Does anyone have experience creating NFS shares in TrueNAS for mounting in Proxmox? I would be interested in thoughts on performance, and stability among any other insights.
• Do any of the system specs I listed seem out of line? Where and how do you think things should be scaled up or down?
• If I ever did expand to a second JBOD shelf, assuming the first one was full first, is it be possible to create new vdevs that spanned across the shelfs without losing data?
• Is SLOG and/or L2ARC necessary for this setup? What capacity and configuration would be best?
• What else have I missed?

Lastly, a quick blurb:

I have been building PCs for a while and undertook building a home server a few years ago. I loved the experience of learning Linux (the server is running Ubuntu), picking up docker, and learning more about the FOSS community has been a joy! Part of this project is to learn along the way but also have a setup that I can build towards over time! Proxmox, TrueNAS and zfs would all be new to me so I really see it as an opportunity to explore. I want a solid media and cloud server setup, while also giving myself the freedom to explore new operating systems and general hypervisor functionality.

Hi all,

I've got (probably) a few thousand family photographs that I plan on scanning/digitizing. These photographs are organized into dozens or hundreds of envelopes with month/year and sometimes event description written on them. I'm on the fence between using a service to do it or DIYing it with a scanning machine.

The way I see it is -

Service pros:

1. I don't have to do it myself

Service cons:

1. I may have no control over how the digitized photos are tagged or organized (date tagged, filename)
2. Risk of photographs being lost/damaged
3. $$$$

DIY pros:

1. I can tag and organize the photos exactly how I want
2. Much less expensive

DIY cons:

1. I have little spare time and this project could be extremely time consuming.

I would love to hear if anyone here has experience doing this and what techniques or pitfalls you may have discovered along the way.

I think most people would agree that the internet and technology in general have absolutely gone to shit over the past decade or so. There is no corner of the internet nor of the software world that hasn't been affected by enshittification. Everything exists to serve you ads. Everyone wants to extract as much money from you as possible. Every website is in a race for the bottom as they try to find the lowest effort content that makes them the most money. Every piece of software is pushed out half-baked and/or stripped down to the bare minimum with the rest paywalled or with the devs pinky promising to fix it 5 updates down the road.

Every social medium is just bots. The front page of Reddit is easily 35% easily detectable bots at least and who knows what the rest is comprised of. And it's probably the one that's doing the best at the moment, Facebook, Twitter, Instagram, Tiktok, all of them are just bots and propaganda and engagement farming the whole way down. And the worst thing is, they're complicit. Hell, they're actively encouraging it and trying to find ways to make it worse. And I have no doubt Reddit will bend the knee soon enough too (they just banned /r/whitepeopletwitter because Musk made a tweet critical of the sub).

There's probably some element of rose-tinted glasses here, but the old internet was just so much better looking back. Like, early 2000's to maybe 2012, 2013 or so, that was the peak. No colossal data harvesting schemes feeding into algorithms designed to keep you engaged on their site 24/7 for the purpose of shilling you advertisements and selling your data, no mass propaganda, no Dead Internet Theory (which can hardly be considered a theory anymore). Yeah there was shit content, there was tons of it, but I can deal with shit content and petty forum drama and whatnot; what I can't deal with is all the multi-billion dollar corporations trying to shape the entire landscape of the Web into the perfectly minmaxxed cash-generating machine that does as little as possible for as much data and advertising as possible.

Modern software isn't much better. Windows and MacOS are filled with anti-user features, telemetry you just can't turn off, Windows will often just install shit on your computer without telling you. They turn your computer into a walled garden, where you can do what you want as long as you play by their rules, but without giving you any real control over what your computer does. Yeah you can delete system files and brick your laptop if you feel like it, but anyone who's ever tried to permanently disable Windows updates will know that in the end you're not the one calling the shots: Microsoft are. And... Like, that's insane, right? It's running on my fucking computer, it's my CPU doing the work, I want to know what the hell it's doing and not just the parts it lets me see, and if I want it to do something different then I should be able to make it so.

I hate it all. I'm tired. I want out.

These are my problems. Here's what I've done about it so far.

• Obsessive privacy on the web. No Google services. Firefox with as much telemetry turned off as possible. Protonmail and ProtonVPN for everything (and I'm considering getting out of those too with the pro-Trump stances they've been taking recently). As minimal an online footprint as I can get, I make as few accounts as possible and I don't use shared or even slightly related usernames (my username here is an exception as it's my Reddit username, and no, it's not my real name), I delete accounts whenever I can and I GDPR request the services afterward. Virtual cards for online payments as much as possible. Will probably make a Javascript whitelist at some point too. Is all of this overkill? Yes. Why do I bother? Because fuck them.

• As little social media presence as possible. Real life necessitates some amount of social media interaction of course, I have Facebook and Instagram but use them exclusively for messaging. I often see people excluding Reddit from social media but I don't fully agree, even if it's not exactly in the category it still targets a lot of the same psychological weak points in us, encouraging doom scrolling and shaping our opinions through echo chambers and propaganda (it's always important to remember that echo chambers and propaganda you agree with are still echo chambers and propaganda). I still use Reddit admittedly, but I've tried to minimise my usage as much as possible and I'm shopping for alternatives.

• Free and Open Source software as much as possible. I'm all in on GNU these days. Yes, it's a massive pain in the ass. My job unfortunately requires some Windows-only software so I'm running a dual partition but I'm trying to get as much of my computer usage onto Linux as possible (I use Arch btw). Like I said above, it's my computer, if I can't control what it's computing then it stops being my computer, it's at best shared between me and all the developers of the proprietary software I have installed on it.

That's my rant. It's been a long time coming.

There are still things I'm looking to change, especially with how I use the internet. Getting rid of Reddit is the next big step for me, I think. I just can't be bothered with it anymore, but there is still something about it that I love, every time I look through a small niche topic community, or an interesting new hobby sub I've never seen before with years of cool posts for me to go through. And yeah, I do still enjoy browsing through /r/all even when it's 80% shit and objectively bad for my mental health. But at this point the overwhelming mass of utter shit is just not worth digging through anymore. I'm tired.

Tildes is really cool. It reminds me of the old internet, the ideal usage of the Web. I open the site, I see a link to an interesting article, I read it, I give it a like, I read and/or contribute to the discussion in a comments section. I want more of this.

If anyone has any links to cool sites that I should check out I'd greatly appreciate it.