43
votes
We unleashed Facebook and Instagram’s algorithms on blank accounts. They served up sexism and misogyny.
Link information
This data is scraped automatically and may be incorrect.
- Authors
- Josh Taylor
- Published
- Jul 20 2024
- Word count
- 1314 words
I’m going to speculate a bit here because I don’t use these platforms and really don’t know how they work nowadays… but if you scroll quickly past some posts but spend more time with others on your screen, isn’t that considered algorithmic feedback? If the people running this experiment were swiping through benign content and Office memes, but paused to screenshot and tally every time a scantily clad woman appeared, wouldn’t that in effect be telling the app that those posts are more “engaging” to the user, therefore more like them will be recommended over time?
I feel like, as a control, another batch of blank accounts should be created for the purpose of “determining whether the platforms serve up increasingly more Office memes over time” or whatever. Then all other posts would be deliberately swiped through quickly but Office memes would receive the screenshot and tally treatment, or whatever specific actions the researchers were taking with the sexist or misogynistic content they identified in the first round. That would determine if the platforms really are pushing this content on people automatically, or if there are other variables that weren’t accounted for.
That said, I wouldn’t be shocked if the original thesis was proven correct. Sex sells. That’s nothing new. I’m a straight male… if you put an image of an attractive woman in front of me, I’m gonna look at it. It’s not an indicator of sexism or misogyny in me. It’s an instinctive, biological response to sexual, visual stimuli. I think the algorithms know this, and capitalize on it. I doubt anyone at Meta is actively trying to propagate Andrew Tate’s worldview, but I fully believe they know that lots of people like looking at thots for no other reason than dopamine. So of course these feeds end up full of thots.
Not even all the examples in the article were "sexy" rather than "sexist" so I don't understand where your angle on that came from?
I'm queer and quite attracted to women and I don't understand the "I'm going to look at them thus the feed is fine" when again, it's the sexism that's the problem. Fill your feed full of thirst traps if you like, but that isn't really what the article was talking about.
Plus "thot?" Come on. Can we not bring the misogyny back home?
11/12 examples were, that’s the vast majority of them. I assume if a user sends some kinds of positive feedback signal for the first eleven, the recommendation engine serves up the occasional twelfth since it considers that “related content” according to other existing (misogynistic) associative data.
Again, 11/12 examples shown in the article were exactly that. I think the point is that thirst trap content is inherently sexist, and I don’t actually disagree with that. Still, if those pics showed up on my phone while I was mindlessly swiping through a feed, I’m sure I would unconsciously hesitate in my swiping for a moment. If that’s all it takes for the algorithm to think I want more of that, then I guess my feed would look more like that over time too, even though I wasn’t intentionally seeking it out.
But like I said I’m just speculating about how the recommendations work; I read that TikTok uses “time watched” as an engagement metric, I don’t know about Facebook and Instagram. Seems like they’re all tripping over each other to copy functionality though. I don’t use any of these platforms.
Apologies for that, apparently there’s more coded in that word than I realized.
The single image of the Insta recommended feed is in contrast with the memes and other images that are discussed throughout. I parsed it as 3 images, with 2 being misogynist, and 2 featuring sexy women. It's still the misogyny that's the problem.
Definition for anyone confused
noun
, Slang: Disparaging and Offensive.
Acronym for t(hat) h(o) o(ver) t(here)
Thanks for the definition, I used it in ignorance. The term “thirst trap” you included is closer to what I had in mind anyway. I’ll be more mindful about “thot” in the future, that’s not the kind of language I should be slinging around carelessly.
Anyway, I counted all of those thumbnails separately as the first image seemed to be a collage meant to exemplify what the study was about.
I appreciate the willingness to learn. I counted it as one since it was a pic of the singular page. But the narrative in the article is far more about the broader context, which is why I was frustrated about the fixation on "of course I'll look at a sexy woman."
Fun bit of (slightly off-topic) trivia -- I was at a linguistics workshop on social meaning in 2017 that involved a lot of discussion of slurs. One of the studies shown was a psycholinguistics study where they did certain types of EEG or eye-tracking or both while showing people words from different categories to gauge their reactions. There were real slurs and profanity, and real inoffensive words, but as control there were also fake gibberish -- one category that was just random and one that was meaningless words that still follow the rules of English phonology (something like "blick" as opposed to "ahfve"). These were generated programmatically.
The researchers found that one of the words in their "fake meaningless words that still follow the rules of English phonology" was bizarrely patterning just like the profanity. It turned out that one of their generated words was "thot", which was relatively new mainstream slang at the time and thus hadn't percolated into demographics of the PhD students who'd designed the study but was recognizable to the undergrads who were the subjects. I was an undergrad at the time and was well aware of the word "thot", so it cracked me up sooner than most when they presented this, since the existence of the word was news to most of the linguists in the room.
funnily enough "blick" is a word in london slang for a pistol and blik is a word in philosophy (invented by RM Hare), so when our teacher was mentioning it our class was laughing at it being used out of context
Huh, today I learned! There was a art supplies store called "Blick" near me when I was in undergrad, but I suspect it was not named with either of those in mind.
Dick Blick (and Blick's Catalog) which is mostly branded BLICK now is named after, well, Dick Blick. The guy who started it. (Illinois facts today, I'm full of them.)
Yep, they should assume the site is a black box and that all feedback and input is influential.
They(let's be real, this is over the top) should have automated it with consistent timings, and detected the content through computer vision*. Done that on 50 or so accounts with different profile characteristics. And then come back with results.This isn't rigorous enough. Even if what we're trying to confirm is something we already kind of know is a thing.
*collected is a better word
While it makes complete sense to try to have the data collection as neutral as possible, this will also not capture what a real new account will experience. A real account will give some information (useful or not) through its usage, which will cause algorithmic changes. Using an account in a "neutral" way doesn't really remove input from the algorithm, it just gives it different input, which will lead to different results.
For example the algorithm might be branching out into more and more obscure and extreme areas if nothing shown so far seems to spike the slightest reaction. That doesn't really tell us if a less robotic user would be shown the same content.
Realistically it may be worth trying several different patterns of (content independent) information and comparing them to avoid this -- one account where you swipe immediately on every other video, for instance. Testing with a variety of these would make whatever patterns you identify in your findings more robust.
Computer vision would not be effective for this task whatsoever, and it's a gross misunderstanding of what they were actually looking for to claim it would. While it may be theoretically possible to train a computer vision model to detect specifically sexist memes, it would be incredibly difficult to do so without integrating some sort of language model, and whether it were or not the final result would be worth publishing in an actual research paper -- this is not something within the skillset of investigators at The Guardian Australia. Even if it were, it arguably would still be insufficient to actually identify some of the types of content the article discusses being "funneled" toward, since the nuances of detecting manosphere-adjacent content is going to include even more subtle determinations than that.
The authors don't actually detail how long they were waiting on each item or whether they were swiping past them quickly, so we don't actually know that they weren't using consistent timing (for instance, making note of each specific thing as they came across it). Since this isn't conducted by experienced academic researchers and is an investigation by journalists, it's unsurprising that they don't report every detail of how they conducted the testing, and it doesn't mean that we can immediately assume the worst and discard the results, merely that in future formal academic research we should make sure to specifically target potential confounding factors like this.
For a journalistic investigation, this seems more than rigorous -- especially given that it confirms existing research that has been done in a more formal academic context.
Well, hold on, I didn't mean to automate the whole thing. I'm well aware computers are no magic black box.
If at a larger scale, it could help filter out things or classify in a general sense. Considering that once we've automatically "looked" at each post, we could actually evaluate each screenshot in our own time, without our activity interfering with the algorithm.
More about automating the acquisition in the most controlled manner possible. Not automating the research itself.
Even doing so on a couple of accounts, they'd still have to look at each screenshot. But we'd have eliminated the potential variables.
I get that, I got lost in the thought experiment :)
I think the lower-tech version would just be to time how long you spend on each one to make sure it's the same and download every single one. That way you're sure to avoid timing issues and you'll have a record of everything you viewed (ideally one that can be viewed separately from the app in the future if it's possible to truly download the videos).
Couldn't you just time it and screen record, then go back and tally from the recording?
That could also work! I think it would depend which is easier to execute for the people doing it.
It also just occurred to me that you'd have to decide whether to equalize time by proportion of the video or absolute number of seconds. Since videos are different lengths, you can't do both, but both are probably factors in the underlying algo.
I'm a human with primitive emotional responses wired into my brain. If I see a traffic accident where someone gets killed, I'm going to look at it. This is not a reasonable justification to start showing me traffic accidents that I haven't subscribed to, liked, or otherwise interacted with.
The whole point here is that social media platforms are abusing our primitive wiring to make us worse off than we started out, and that they should stop doing that.
I think that's a problem. Why facilitate delivery and thus supply of content that causes people to pause in consternation when they see it? There should be an explicit "more of this", "less of this" feedback.
What's odd is that there is a more/less prompt, but it only seems to be offered for my friends' posts, not the advertising or the random stuff it puts in my feed.
And my feed is such trash. "Sponsored" advertising, sure, no way around that. But posts from my friends are heavily outnumbered by random accounts reposting sports and celebrity stuff which I have zero interest in and have never clicked on. I hide these posts and check the "hide all from" option, but it's an unending river of crap that still outnumbers my friends' posts.
Edit: i double-checked the see more/less and find it is available on the randos. You do have to click the ... menu same as when you want to hide it. However, it's still never offered within the feed for randos, only friends, and if you click the "see less" it says you'll see less for a while. How about permanently, Facebook?
I totally agree! And maybe there is. I am running my mouth off about a hypothetical algorithm function, but I don’t actually know how it works.
Of course, these algorithms are all famously opaque, so nobody really knows. That in itself is a huge problem.
I can think of two reasons.
The first is that a pause can mean slight interest, but not enough to go out of your way to take an action on it. Maybe presenting more related content will grow that interest enough to engage, like the point of advertising.
The second is that it’s something the user is interested in, but doesn’t explicitly engage with for whatever reason - embarrassment, denial, politically/socially “correct” response, whatever. “We don’t know if you like these, wink wink, so we’ll keep them in your feed until you tell us”.
In either case, if the person is indeed having a negative reaction (pausing in “consternation”), they’ll either leave active feedback eventually (or reduce feed usage) or don’t care enough. Until then, views cost very little, so tune based on broader trends and any signal you can get no matter how small and hope for the best.
That's plausible, but still not right. We should have not allowed unmoderated spaces that aimlessly tries to make an addict out of you.
See how hard is to criticize the situation? If somebody were intentionally spreading male supremacy propaganda, they would be asked to stop. If somebody does it by accident, that's just a force of nature.
"Not ideal but such are men, closet lechers. Wink, wink."