-
13 votes
-
Request for help: Backing up NASA public databases
TL;DR: NASA's public Planetary Data System is at risk of being shut down. Anyone have any ideas for backing it up? Hi everyone, Bit of a long-shot here, but I wanted to try on high-quality tildes...
TL;DR: NASA's public Planetary Data System is at risk of being shut down. Anyone have any ideas for backing it up?
Hi everyone,
Bit of a long-shot here, but I wanted to try on high-quality tildes before jumping back into the cesspool of reddit. I'm posting it in ~science rather than ~space as I figure interest in backing up public data is broader than just the space community.
I work regularly with NASA's Planetary Data System, or PDS. It's a massive (~3.5petabytes!!) archive of off-world scientific data (largely but not all imaging data). PDS is integral for scientific research - public and private - around the world, and is maintained, for free, by NASA (with support of a number of Academic institutions).
The current state of affairs for NASA is grim:
- NASA Lays Off ISS Workers at Marshall Space Flight Center
- More layoffs at JPL
- NASA is sinking its flagship science center during the government shutdown — and may be breaking the law in the process, critics say
And as a result, I (and many of my industry friends) have become increasingly concerned that PDS will be taken down as NASA is increasingly torn down for spare parts and irreparably damaged. This administration seems bent on destroying all forms of recording-keeping and public science, so who knows how long PDS will be kept up. Once it's down, it'll be a nightmare to try and collect it all again from various sources. I suspect we'll permanently lose decades worth of data - PDS includes information going all the way back to the Apollo missions!
As such, we've been pushing to back-up as much of PDS as we can, but have absolutely no hope of downloading it all within the next year or two, nevermind in a few months if the current cuts impact us soon.
If you or someone you know would be interested in helping figure out how we can back-up PDS before it's too late, please let me know here or in a DM. I've already tried reaching out to the Internet Archive, but did not hear anything back from them.
Edit: to clarify, the larger problem is download speeds - we've topped out at 20mb/s with 8 connections.
61 votes -
Technicolor's last frame: the collapse and liquidation of a Hollywood legend
24 votes -
The Internet Archive’s microfiche digitization livestream
15 votes -
The mystery of Winston Churchill's dead platypus was unsolved - until now
8 votes -
A command line, playable version of the NYT Connections game archive
12 votes -
The Internet Archive is now an official US federal documents library
51 votes -
The Danish government deputized private detectorists to unearth artifacts buried in farm fields. Their finds are revealing the country's past in extraordinary detail.
9 votes -
These catchy old songs aren't as think as you drunk they are
9 votes -
More than 80,000 manuscripts from the Vatican Library to be restored and digitized
20 votes -
Can It Run Doom? An archive of all known ports.
28 votes -
Inside the Svalbard vault that holds digital back-ups of some of humanity's great works of art, history and technology
14 votes -
Title of work deciphered in sealed Herculaneum scroll via digital unwrapping
15 votes -
Fourteen thousand World War I poems digitised
20 votes -
The Vatican secret archive
8 votes -
Yugologo, an archive of business logos from the former Yugoslavia
37 votes -
Intern finds only known surviving copy of 'The Heart of Lincoln,' a silent film thought to be lost to history
35 votes -
The BBC’s library of classic sci-fi sounds is now available to sample
27 votes -
Distributed Denial of Secrets
15 votes -
San Francisco unveils marble bust of Aaron Swartz, hero of open-access internet
48 votes -
Prince estate blocks release of Netflix documentary by Oscar-winning director Ezra Edelman
22 votes -
Fadedpage: an archive of ebooks in the Canadian public domain
9 votes -
Shakespeare’s sister: how using digital archives revealed hidden insights into world famous playwright’s unknown sibling
12 votes -
Scientists and archivists worry Epic Games' control of the 3D model market will 'destroy' cultural heritage
35 votes -
Big changes are coming to ArchiveBox!
10 votes -
Inside Iron Mountain: It’s time to talk about hard drives
23 votes -
Does anyone have experience with tools for locally archiving the web, like Archivebox for example?
I found myself on the Archivebox website earlier today. After reading some of it, that's the kind of program I could use. The ephemerous nature of the web is bothersome, so much content is lost...
I found myself on the Archivebox website earlier today. After reading some of it, that's the kind of program I could use. The ephemerous nature of the web is bothersome, so much content is lost for one reason or another. Archivebox seems to be one of the most popular tools, and it can automatically mirror my locally downloaded website to archive.org, which is great. It seems complex though, maybe more complex than I usually tolerate these days. Which is why I am asking if anyone has personal experience with Archivebox or other similar programs. Do you find them useful and reliable? Have you ever found in your local storage a webpage that you really liked, which was gone from the web? How's your setup?
Thank ;)
19 votes -
Tapedeck.org is a digital archive that features hundreds of cassette tape designs
13 votes -
Google will now link to The Internet Archive to add more context to Search results
37 votes -
The Internet Archive lost their latest appeal. Here’s what that means for you.
27 votes -
ROMhacking.net moves to news only, database and file archive released to Internet Archive
34 votes -
PSA: Internet Archive “glitch” deletes years of user data and accounts
34 votes -
Corrupt Winamp skin investigation leads to treasure trove of hidden content
23 votes -
The critical window of shadow libraries
16 votes -
Stephen Hawking Archive made available to historians and researchers
17 votes -
Internet Archive forced to remove 500,000 books after publishers’ court win
59 votes -
GlobalUsefulNativeTrees, a database documenting 14,014 tree species, supports synergies between biodiversity recovery and local livelihoods in landscape restoration
31 votes -
Over 1000 new asteroids and other objects found through examining data from Hubble's archives
7 votes -
The Museum of Science and Industry abruptly closed for a day last week to allow it to move “military artifacts from archival storage”
26 votes -
How to make a time capsule
5 votes -
I ported thousands of apps to Windows 95
23 votes -
Marvin Gaye: Never-before heard music resurfaces in Belgium
10 votes -
A university librarian asks: How do we rescue the past?
14 votes -
A new archive of modern American political history
2 votes -
Surprising detail in New York bank records helped a historian bust a longstanding myth about Irish immigrants
15 votes -
I found Frank Herbert’s Dune script. It’s hard to imagine a weirder film version of Dune than the one David Lynch released in 1984, but Frank Herbert found a way. Dune: Part Two is better.
32 votes -
Millions of research papers at risk of disappearing from the Internet: An analysis of DOIs suggests that digital preservation is not keeping up with burgeoning scholarly knowledge
26 votes -
Packages seized by the Royal Navy from a Faroese cargo ship bound for Denmark during the Napoleonic Wars opened – previously hidden away in the National Archives
9 votes -
Google’s retiring of Internet archiving tool draws ire of China researchers
18 votes -
An archive of Wikipedia from Thursday, December 20, 2001
18 votes