I don't see why Lynne is singling out the Archive Team for "consent" when scraping publicly available data or their refusal to remove any such data, especially when it's done for public interest purposes (to preserve history). There are already plenty of unknown actors (state intelligence, private intelligence, internet hoarders, etc...) who do not advertise their scraping activities. While it's unfair to have one's data stored indefinitely and used for unknown purposes, it's not like the Fediverse was ever going to guarantee the privacy of their data in the first place. Fundamentally, it should be assumed that every single bit of data made available through federation will become public, as Lain pointed out in their blog post, and this expectation should be extended to any platform on the internet.

The problem is that so many websites shut down with little to no warning that the Archive Team has almost no choice but to scrape whatever they can without being able to check what it is.

Places like Sporum, Minecraft Forums or Tumblr shut down parts of their website or all of it with little warning, there is lots of content on them that is worth archiving but there isn't enough time to moderate all of it and find out what should be archived and what not.

Ideally, website owners should submit the appropriate data to the archive on their own, then the team won't have to archive anything. Or atleast help them to not backup private data and only the relevant parts.

Historical archiving is important and too much of the web is lost each day.

Making something public is not the same as consenting to having it stored for all eternity

No, it is exactly, totally, completely, inevitably, irrefutably the same. Or else, nothing is public, ever. To publish something is to put it out there for everybody to see. Something can't be partially public. If something is published, people will do whatever they want with it, unless they are caught doing so AND what they are doing is illegal, and even then it will essentially remain public because there is no way of wiping it from the entire world full of devices, backups, caches, archives, and whatnot. It will remain in hard disks as unassociated data waiting to be discovered. It'll be sitting on a pile of computer parts somewhere in Nigeria. It will have been seen and will sit in people's minds. This is the phenomenon as it is, not a social construct around it. You can't have something appear on other people's screens (including bots) and be sure it is not replicated and stored. A robots.txt can not be more than a kind request. If a URI is accessible without authentication, the data is public. There is no way to enforce the inverse. There may not be any grey areas between private and public, for that is not possible. You can't be forgotten, as it is up to other people, and even not really up to them in many cases (you can't deliberately forget things, and really deleting stuff on computers is hard). You may build legal frameworks around it, but it will serve nothing. You will always run the very likely danger of being archived. And law can only help when those archives are published and you encounter them and you sue and you're lucky enough to win. Even then, there is no possible way to ensure that the data is got rid of forever.

What I'm trying to convey is that, well, there may be a right to have public stuff be half-private---which I don't think there is, but let's brush my opinion aside for this---, but there is no way to enforce it. Which means, there is no reliable way in which this can be guaranteed, which in turn means that this is futile. If you want stuff to be private, keep it private. If you will publish stuff, think well, twice, thrice. In the olden days we did not have this problem because it was a hassle to publish stuff, but now if you sit wrong your phone in your arse pocket will publish stuff for you. And most of the blame is on apps that default to publicly-viewable posts. They should default to private instead.

edit: I want to also add: this sort of suggestion is placing people in the peril of security/privacy theatre, which is really dangerous. Just like you can't un-eat things, you can't unsee things or make people unsee things.

Genuine question here: what is there to be gained by archiving this instance?

I say this not to be patronizing in the slightest, and this isn't a dig at the users of the instance nor the archivers themselves. I guess it's more a question about what merits archiving in the first place. I know there are people out there who believe that we should save everything simply because we can, but outside of a sort of loss aversion, what positive or constructive usecase will anyone feasibly have with this data?

I can think of a number of negative and destructive ones. Bad actors can plumb it for identifying information or ammunition for doxxing. I can also picture any number of companies who traffic in user data tossing this into their already robust data sets (though, to be honest, the major ones probably already had it anyway).

I probably have a bit of a blind spot in this area, and I'm also probably a bit biased in that I deleted my own Mastodon account just this week, but I guess I don't understand why we would need to save the instance and its posts in the first place. The people on it who wanted to save their data probably already did (as I did with mine), and I can't see the data at large being terribly meaningful or important to someone that wasn't on the instance (just like I can't imagine what a stranger would want to do with my posts).

To approach this from some different perspectives (and I should note that this is all my speculation, as I have not done any research on the matter).

I think there is a question here of how technology, and the increasing prevalence of digital interactions, changes the availability of resources for historical research on the lives of the general population.

With physical artifacts, while there are differences in the probabilities of different forms of texts surviving, we end up with some subset of quotidian texts preserved by chance, while many are lost. Some receipts, some private journals, some letters, some notes, will survive, often with little thought to preservation: enough, hopefully, to gain an understanding of an era and culture. Thus, for example, we have the graffiti of Pompeii, cuneiform receipts, and so on, or miscellaneous letters that have survived through a few generations.

With many digital forms of expression, however, this chance of survival without regard to intentional preservation seems often to be replaced with something very different: more of a binary choice between everything being preserved, through comparatively minimal but conscious efforts of archiving, or everything being lost. Wanting to gain consent for this archiving would limit its usefulness as a record of culture at the time. Even having people opt out of the archiving would. Both would present a sanitized view that would present peoples and cultures only as they wished to be remembered. Yet we don't have the chance-based preservation to temper this record, and keep it from being overly invasive. Neither do we have the difficulty of access and searching that would make many problematic uses infeasible for physical records.

There is, perhaps, a balance to be struck here between individual privacy and the pursuit of knowledge. One option that could provide a compromise, but would require very long-term planning and stable infrastructure and governance, would be to archive without regard to consent, but, absent opt-in consent, make material accessible only after long periods of time, perhaps 150 or 300 years, enough to ensure that the material cannot be used to directly invade the privacy of living people.

This is, to some extent, what is often done with authors and others in the public eye, regardless of their wishes, where there are extant remnants of their communications and lives that are sufficiently sought after or cared for to exist in such detail. I am reminded of the recent TLS article on the continuing publication of TS Eliot's letters, something that the article points out was very much against his wishes, at least as they were expressed in 1938 (though a later letter to the editor points out some complexities here of consent and coherence): the TLS, for some odd reason, seems to contain a multitude of titles for the same article, and I would point to the "No posthumous privacy" title being quite apt. We often value these posthumous perspectives, though invasive, and I do think that there would be something lost without them.

I have what I once thought to be a minority opinion but increasingly find it echoed by others.

I don't believe the person posting info publicly owns that info anymore.

Once you put something into a public space, then that becomes something that the audience you posted it to has a claim to. The person who posted it still has rights under this understanding of mine, right of attribution, right to commercialisation, to an extent rights over derivatives. But to delete? Fuck no on a fuck no sandwich.

I hit this fairly regularly with my behaviour around certain game modding scenes. I do not believe in the 'right' of a mod developer to a) remove their mod from public access, or b) restrict subsequent modifications to that mod. So I quite often rehost these where ones I have are removed. I am a rather polarised figure in some of these communities as a result - certain mod creators don't like me at ALL, people who developed 'mods of mods' before finding their dependency removed and people who like playing removed mods like me a lot.

I have also hit it in my own work. Left pad impacted the company I worked for at the time, and in several cases there have been free but not properly FLOSS softwares that have stopped being updated where we had the physical but not legal ability to keep them updated and useful to us (and others!). So in at least 2 cases we just did it anyway and covered our tracks.

This applies to things that aren't code just as much - my experience is with bug trackers and development forums that close down, especially of the mailing list variety. You'd better believe they get archived, and if an author asked to have their posts removed they would be flat out laughed at (never happened that I know of).

If you haven't already, you should read The Cathedral and the Bazaar. It's focused on the open source software models, but it can be applied the same way to public discussion. The public OWNS the public space. Just as you can be filmed on the street without the expectation of privacy but not in your home, if you post on a public site you don't and shouldn't own jack.