I mean I'm aware that "AI" doesn't randomize well, and I think that the output is evidence of a problem with the training set. But also that this is evidence of societal bias, bias in the...
Exemplary
I mean I'm aware that "AI" doesn't randomize well, and I think that the output is evidence of a problem with the training set. But also that this is evidence of societal bias, bias in the programmers and thus bias built into in the model.
If women (particularly Asian women) are being portrayed naked by default (or at a much higher frequency) that is a problem. They screwed up their training data. But the response to that being "you just have to put in different words" is unhelpful. They should absolutely be criticized and be expected to retrain their model IMO.
We know there's racial bias as well but if every search for a Black man shows a minstrel show character we'd be worried about that too, right? It's concerning when every search for "doctor" or "autistic" gives you white men.
I find it confusing that the response here is "well thats just how it is because of the training material." Yeah it is, but that's not ok.
No one's saying it's ok. If I'm reading the room right, the response is "it's not OK, but it's definitely not malicious, and it's also a problem we're aware of and working towards solving. In the...
No one's saying it's ok. If I'm reading the room right, the response is "it's not OK, but it's definitely not malicious, and it's also a problem we're aware of and working towards solving. In the meantime, here's a workaround". Holding software back because you've found one major bug is not usually the way to go[1], so long as it's not safety critical software or a showstopper bug. You get it into many hands, thus find many bugs fast, then solve those bugs. If you yoinked/delayed the software for every major bug, you'd see it for a week before it's yoinked, then 6 months of crickets while Dave furiously types to fix his fuckup. Then it's another week of public testing and now it's John's time to fix his fuckup for the next 6 months. Bit over the top, but still mostly accurate. Maybe even more accurate for AI, where "bugs" can be quite nebulous and might require a longer societal debate to nail down. See the OP, where a single individual might not even know they're dealing with biased treatment (aka a bug), because they only see output for themselves.
Also, I'm curious how you think programmer bias gets baked into the model? Like, is that because they approve flawed training data, or because of more active intervention of their part that biases the model?
[1] a positive, not normative statement. That's just the way the software industry does business nowadays, for better or worse. That that has consequences is another way more general can of worms.
I think you're viewing this more as a software issue rather than considering it a reflection of biaseses our society is still coming to terms it. Being provided a work around and referring to the...
Exemplary
I think you're viewing this more as a software issue rather than considering it a reflection of biaseses our society is still coming to terms it. Being provided a work around and referring to the issue as a "bug" is dehumanizing because it fails to acknowledge that these biaseses have been included from the beginning without even second thought. It's not a malicious act necessarily, but it tells me as a woman that the teams creating these algorithms are absolutely naive to these issues.
The disparities are worse for Asian women in particular, who have often been flattened into two-dimensional characters lacking agency or written as sex objects, whether stereotypically submissive or cunning.
This biases is so strong in media that no doubt it influenced these algorithms. But I think what has I and others upset about this, is "why is this the default?" Like really, why did no one stop to think that this would be an issue? It took me 5 minutes to read into it and a dose of empathy to make me want to avoid it.
In Hollywood, it can more subtly manifest in objectified portrayals like that of “A Benihana Christmas,” or the running gag on “Scrubs” about Dr. Kelso fetishizing Asian women. It proliferates through the widespread appropriation of “Me so horny,” the line famously uttered by a Vietnamese sex worker in Stanley Kubrick’s “Full Metal Jacket” that men have repeated while catcalling Asian women since the film premiered in 1987.
People are missing the fact that if those who are creating these AI image creators are ignorant to these issues, they will get baked into the code base and exemplified within them. They expose our biaseses that we continue to hold in society and how we are still unwilling to grapple them. I get the whole idea that it doesn't make sense to scrap all the work and start anew, but it also tells me that the people working on these things don't care enough to prevent them from arising. Instead the AI bubble is too big of a money making machine to care about how their portrayals of women may affect how they are viewed in real life. Instead they are inconvenience to the model and their very existence requires a "workaround" to get a more accurate reflection of who they really are. Honestly, how would that make you feel that the default to generate an image akin to your own identity in real life requires a damn "workaround" to avoid over sexualization. It disgusts me and it inherently bothers me to my core that so many others don't get it or don't care to get it.
I'm not going to engage too thoroughly as I don't think there's much I can contribute on many of the aspects you're raising, not that I even disagree to begin with. One point I want to raise more...
Exemplary
I'm not going to engage too thoroughly as I don't think there's much I can contribute on many of the aspects you're raising, not that I even disagree to begin with. One point I want to raise more generally, and yours is just the comment that sparks that thought for me; it's not specific to your comment, so don't take it as an attack or anything:
Keep in mind that "the AI bros" are often not a homogeneous group, nor are they all organized in any way. It's entirely possible for an AI engineer at a corporation to train a specific model, but tell their manager "hey, yeah, sure... but this thing is going to be drawing asian women in the nude". The manager greenlights the model's release anyway; he doesn't have much of a choice, as fixing the issues would take years of either research or data curation, while less scrupulous companies are always around. He thus orders any mitigations to be put in place that can be achieved within a set budget. Meanwhile another AI researcher at a public university is working on making it randomize better, e.g. by explicitly deprioritizing outputs we don't want, but that's basic research and will take years. And independently of that, enthusiasts online figure out what kinda prompts make the shitty model produce reasonable outputs. (This example is only loosely inspired by your post, so if it seems like a strawman, that's why. It's not supposed to be a faithful representation of any of your arguments.)
Now we could paint this entire group with one broad stroke as "AI bros produced a shitty model and now they're telling me a fix will be ready in maybe 3 years and also, here's a workaround, that's all you're going to get because you're a minority we don't care about." But that wouldn't be fair to the engineer's awareness of what is going to happen and him raising concerns; it wouldn't be fair to the researcher who's doing his level best to actually fix the problem; nor the enthusiast who -within the scope of his abilities- is also contributing to getting the most fair results possible out of the model. The only one it's arguably doing justice is the manager who gave the green light, but that bloke also has a bottom line to pay attention to, and shelving a project for years in a very temperamental business environment probably isn't great. Whether you think some sacrifice is appropriate here is subjective, I think, but I think the impact would be limited given there's no shortage of shitty AI startups.
What I'm saying here is, it can be frustrating for those trying to actually make responsible AI to hear these broad strokes narratives that misattribute the malicious actions. A lot of the people at the core of the field are very aware of their responsibility and the impact of their actions, and also aware of the issues around bias. They're trying really damn hard, and painting the entire field as naive buffoons helps no one, except the naive buffoons that do exist in the field. Also, regarding fixes and changes that laypeople propose: xkcd never misses.
You bring up some excellent points and I completely understand. I work in a similar environment and have my own frustrations with it. I guess more so it is the frustration that this is the default...
You bring up some excellent points and I completely understand. I work in a similar environment and have my own frustrations with it.
I guess more so it is the frustration that this is the default and that more effort is needed to avoid that. It sorta reminds me of that scene in Rick and Morty where Rick keeps dying and each new universe is filled with Nazis. This leads to Rick screaming "Goddamnit when did this shit become the default!" It's just all so troubling, but I appreciate your comment making me think.
Your explanation represents how things are, and it embodies the ethical problem described in Ursula K. LeGuin's short story, "The Ones Who Walk Away From Omelas" [PDF warning]. Can you knowingly...
Your explanation represents how things are, and it embodies the ethical problem described in Ursula K. LeGuin's short story, "The Ones Who Walk Away From Omelas" [PDF warning].
Can you knowingly abide the suffering of a minority of people in order to bring the vision of AI abundance to fruition? Can you continue to ignore the people who tell you they are being directly and preventably harmed?
Repairing AI function is not a trolley problem. There is no comparably clear utilitarian answer where we can say, "this technology helps billions, will save or improve many lives, and only a few might be harmed (even if they don't get a choice)."
We have direct evidence that AI bias is causing prolonged injuries and social harm to many people, particularly when seen across all minorities who are not the statistical average Internet example. This might ultimately be a very large number, when you count the insurance, loan, and rental denials, policing errors, rejected job applications, stereotype-driven query responses, misdiagnoses, and so on.
Continuing to offer the technology while refusing to respond to the issues raised here is engineering malpractice, in addition to the ethical and moral wrongs.
[And I say this as a more-or-less female technologist.]
Do we really? Maybe I'm emotionally stunted, but I wouldn't describe someone who wants to use an AI for face filtering and realizes they don't like the result as significantly harmed.
We have direct evidence that AI bias is causing prolonged injuries and social harm to many people
Do we really? Maybe I'm emotionally stunted, but I wouldn't describe someone who wants to use an AI for face filtering and realizes they don't like the result as significantly harmed.
There are more substantial impacts, but I think those are all models that exist solely behind closed doors. Anytime you combine actually high-stakes situations like legal or financial issues....
There are more substantial impacts, but I think those are all models that exist solely behind closed doors. Anytime you combine actually high-stakes situations like legal or financial issues. Think job applications, insurance applications, medical decision making. The case of discrimination is more difficult to make, as those models exist behind closed doors so can't be freely evaluated. It's also going on mostly independently of the current hype around generative AI, most of that is by now just plain old statistics.
Though I'm not terribly convinced we (collectively) have a good grasp of the amount of discrimination going on there. Not every error is discrimination, and not every discrimation is "erroneous" (statistically speaking), so it's difficult to measure even under transparent conditions. And with the individual case studies we're getting, I think people will absolutely turn on their confirmation bias and see or not see discrimination, depending on their general outlook.
A very concrete one is a chat bot that replaced live support for people with eating disorders. It gave them information on how to restrict calories. It's just direct harm caused by poor...
A very concrete one is a chat bot that replaced live support for people with eating disorders. It gave them information on how to restrict calories. It's just direct harm caused by poor implementation of a product that was probably mis-sold to them.
I actually agree with that. But I think that the focus on specifically biases in image generation models is misplaced, firstly because I have doubts about the harm and secondly because the...
I actually agree with that. But I think that the focus on specifically biases in image generation models is misplaced, firstly because I have doubts about the harm and secondly because the mechanisms are simply different than in LLMs and the real parallels are not that strong.
The fact that people seem to mostly focus on Stable Diffusion not because it's particularly bad but because it's the only one that's open and therefore easy and cheap to examine also isn't great.
They're referring to uses of 'AI' outside image generation As evidenced by this part of their comment. If 'AI' can't even generate a picture that represents anything slightly deviating from the...
They're referring to uses of 'AI' outside image generation
This might ultimately be a very large number, when you count the insurance, loan, and rental denials, policing errors, rejected job applications, stereotype-driven query responses, misdiagnoses, and so on.
As evidenced by this part of their comment. If 'AI' can't even generate a picture that represents anything slightly deviating from the perceived norm, do you really want it helping your doctor?
True! Sorry. I don't think those two things are similar enough to compare and I think that placing the focus on removing bias from image generation models instead of talking about those things is...
True! Sorry.
If 'AI' can't even generate a picture that represents anything slightly deviating from the perceived norm, do you really want it helping your doctor?
I don't think those two things are similar enough to compare and I think that placing the focus on removing bias from image generation models instead of talking about those things is misguided.
That said, as someone with an uncommon chronic disease that many doctors know almost nothing about, I definitely 100% do want it helping my doctor, even though I acknowledge that many potential issues exist. I think that preliminary research actually shows that in this particular case an AI might have fewer biases than most doctors and since the doctor is still always present to veto it, the potential damage is smaller than with products related to insurance, loans, rental denials etc., where there's a higher chance that an AI would be used unsupervised.
I'm just curious though. What, exactly, should an AI generative model do when given a prompt? Should it start running the user through a bias checklist? Admonish the user? Should it be hardcoded...
I'm just curious though.
What, exactly, should an AI generative model do when given a prompt? Should it start running the user through a bias checklist? Admonish the user? Should it be hardcoded with a list of "acceptable" results and only respond through them? Is it the responsibility of the AI to refuse to fulfill requests that seem "adult" in nature?
Because what the AI actually does absent a specific criteria is pick randomly, weighted by what's in its dataset. Which can be changed as the user likes. Anyone who has specific needs for their AI dataset can build something that suits their needs. And people do.
Infinite diversity in infinite combination. Humanity is the very definition of that statement. A wonderful array of choices, each unique, each wonderful in his or her own way.
But details matter. If an human artist is posing a human model, the artist will get into specifics with the model. Who the model is, what to wear, how to pose, facial expressions, everything.
It's not racist, or sexist, or really any kind of -ist, for the computer to default to the most likely responses based in its dataset if it's not given details to narrow the response with. If the user has specifics in mind, tell the computer! So it can use that data to parse its response. The same as the artist would parse her prospective models and details if she had specifics in mind.
A Korean artist working in Korea will have to get very specific, and take some extra steps, to get a non-Korean model in to pose. Same for an Australian artist, an American, a French, and so on. It would be silly for those artists, if they put out a casting call, to be upset that the people who live in their city are who show up for consideration. A Korean in Seoul being angry that "I didn't mean I wanted a Korean to come pose for this picture" would be a silly complaint.
Especially since all they'd have to do is ... be specific about what and who they'd like to come pose for them. Sure maybe there might not be that many non-Koreans in Seoul at that time, but at least they'd have better odds of some of those perhaps coming in to meet with the artist for consideration.
And it occurs to me that non-Koreans, seeing a generic "I want a model to pose" call in Seoul, might assume the artist would prefer a Korean. After all, if they didn't specify, a woman who's black and six foot four could very reasonably assume she's probably not who a random Korean artist wants to pose for a painting. How should this lovely non-Korean woman be expected to somehow know that artist would very much like to meet with her, if the artist wasn't specific in the casting request?
Assumptions go both ways, see. And if details matter, details should be given.
It's not -ist to need to give the computer specifics. And anyone who doesn't like a certain model (dataset) can generate their own. On Stable Diffusion dataset websites new models and modifications of models are posted in the hundreds, every day. By people who had some very specific thing they wanted put into a dataset, and proceeded to work on.
A lot of what they post is just mindboggling to me, but they wanted it and got it (and then decided to share it). Anyone can do the same. There are even websites that will do all the computer stuff; just upload the pictures with details and it'll give you back a model after it runs the numbers.
So I see all these articles by "reporters" eager to play "hah! gotcha!" with AI and it's puzzling. If they feel they're not represented, they can build models that do represent whoever and whatever they want, however they want, whenever they want. Which then leads to the assumption they might be complaining sort of just to complain. Outrage gets clicks.
Again, why does a Asian women have to put in more inputs to a avoid a sexualized output when a white male does not? Do you really not see the unfairness and the issue here? It is racist. It is...
Exemplary
It's not racist, or sexist, or really any kind of -ist, for the computer to default to the most likely responses based in its dataset if it's not given details to narrow the response with. If the user has specifics in mind, tell the computer!
Again, why does a Asian women have to put in more inputs to a avoid a sexualized output when a white male does not? Do you really not see the unfairness and the issue here?
It is racist. It is misogynistic.
Why does an Asian womam need to put more of a quadratic equation into the prompt to avoid sexualization while a white male can avoid that with just 2+2?
The article I linked in my post refers to a Asian hate crime shooting that occurred in Atlanta a few years ago.
In reading about how the White man charged with killing eight people, six of whom were Asian women, at spas in the Atlanta area blamed his “sexual addiction” and viewed the spas as a “temptation” he aimed to “eliminate,” film scholar Celine Parreñas Shimizu took note of the tragedy as “part of a long historical trajectory of locating that perverse sexuality on Asian women’s bodies.” On-screen portrayals are just a single factor lending to dehumanizing perceptions of Asian women, but an undeniable one.
These type of biaseses have impacts and failing to stop their perpetuation leads to this hate. There is a clear biases in the data sets being used in these models and AI models like this are just going to further stress these problems. A fair model set would have non-sexualized outputs by just typing "Asian woman" akin to entering "White Man" without no extra work needed.
It's so frustrating as a minority to hear from the same people that "AI is the future" but also "you minorities need to go through extra hoops to use the default AI experience and have it be the...
It's so frustrating as a minority to hear from the same people that "AI is the future" but also "you minorities need to go through extra hoops to use the default AI experience and have it be the same as everyone else".
This hurts to read, but it is the unfortunate reality of the dawn of AI. When I read this type of commentary I am instantly empathize that this is wrong and is hurting people. What gets me fuming...
This hurts to read, but it is the unfortunate reality of the dawn of AI. When I read this type of commentary I am instantly empathize that this is wrong and is hurting people. What gets me fuming is the absolute smugness of a "just deal with it" attitude that is given as a response to these concerns. It's not that I view AI itself as the boogie man, but more so its developers that are ushering it to be such. At a certain point this pure ignorance tiptoes into malicious territory when it comes to impacting something pivotal.
A while ago there was some meta commentary that Tildes can be too nice at times. I'm over it with this topic and it's time to rip the bandaid off. If you're a supporter of AI who clearly doesn't understand the issues of this and can't be bothered to read or empathize with it, you're a fucking monster who is continuing to uphold the ills we have yet to overcome as a society. Do better; we already should have at this point.
Fwiw I was the one who gave you the Exemplary tag :) I don't very often but thank you for being up and down this thread, it's very appreciated! I read through this thread again and this is what I...
Fwiw I was the one who gave you the Exemplary tag :) I don't very often but thank you for being up and down this thread, it's very appreciated!
I read through this thread again and this is what I keep coming back to that really annoys me:
Which then leads to the assumption they might be complaining sort of just to complain. Outrage gets clicks.
The implication that a minority literally saying that "hey maybe this is a weird thing happening" is "just outrage" is infuriating. I've actually noticed more and more even here on Tildes that there's a push back on furthering diversity and shutting out voices that maybe there's still a lot of racism out there.
I hope my posts in this thread haven’t come off as brushing these problems off. My intention was discuss why/how things wound up as they are (which I think is an important step in effecting...
I hope my posts in this thread haven’t come off as brushing these problems off. My intention was discuss why/how things wound up as they are (which I think is an important step in effecting change, e.g. identifying sources of bias to avoid in training datasets for example), but that can sometimes unintentionally read as excusing things.
Hey, sorry if I came off a bit angry in some of my comments, but I didn't mean for any to be targeting towards any one user. @pheonixrises may have similar experiences to my own on this site, but...
Hey, sorry if I came off a bit angry in some of my comments, but I didn't mean for any to be targeting towards any one user. @pheonixrises may have similar experiences to my own on this site, but I've noticed there's not a lot of minority voices on here. Especially women.
I've experienced several threads being locked due to the majority male voice outright silencing female experiences on here. It has even led to some outright disturbing posts that enter 4Chan Territory that fortunately lead in those users being banned. But it is so dismaying to constantly have to be the lone female voice to share those experiences and deal with the continual ignorance. I'm frankly quite tired of this in 2024 for fucks sake.
Women's issues are valid! Minority issues are valid! Like why are these questioned and dismissed as if they are not realities? It makes me wonder how many of these male users actually care to be aware of these things or is it simply not worth the time or "profitable enough." I live this reality every day, just about every minority does. It really serves as a way to further dehumanize us all while our rights are being eroded away. It really exemplifies a portion of the population doesn't care and I think I'm just frustrated with that. It came out in my comment, but I am tired of being dismissed.
Sorry if I have caused any distress to you or others.
At least for me, I'm lucky enough to have a slightly more noticeable Asian presence here on Tildes, but even then it's still very clear that it's a minority. Furthermore as a model minority I feel...
At least for me, I'm lucky enough to have a slightly more noticeable Asian presence here on Tildes, but even then it's still very clear that it's a minority. Furthermore as a model minority I feel like we have a more complex relationship with the "default" perspective.
I honestly don't know how anyone survives as a woman online. It's already really annoying as an Asian American guy, and I just avoid commenting to avoid the headache. Props to you for speaking up, I hope you know it's appreciated!
Very rarely have minority rights been viewed as profitable enough to be handled by the free market. I've heard libertarians who believe that a) businesses should be allowed to discriminate freely...
Very rarely have minority rights been viewed as profitable enough to be handled by the free market. I've heard libertarians who believe that a) businesses should be allowed to discriminate freely and b) no significant number of businesses would lose the money they'd lose for discriminating against Black folks, or Muslim folks, or queer folks, disabled folks, etc.
When that absolutely flies in the face of our history and our present. People don't act "rationally" but they think they do. And the number of people and institutions willing to cut their own noses off to be able to spite the face of someone they consider lesser than them is absolutely wild to me.
All this to say I appreciate your voice and am here to help amplify it when I can.
THIS. So many people working AI, particularly men, brush off reproduction of real-world biases by machine learning models as just an inevitable outcome of the code, something they could not...
People are missing the fact that if those who are creating these AI image creators are ignorant to these issues, they will get baked into the code base and exemplified within them. They expose our biaseses that we continue to hold in society and how we are still unwilling to grapple them. I get the whole idea that it doesn't make sense to scrap all the work and start anew, but it also tells me that the people working on these things don't care enough to prevent them from arising. Instead the AI bubble is too big of a money making machine to care about how their portrayals of women may affect how they are viewed in real life. Instead they are inconvenience to the model and their very existence requires a "workaround" to get a more accurate reflection of who they really are.
THIS. So many people working AI, particularly men, brush off reproduction of real-world biases by machine learning models as just an inevitable outcome of the code, something they could not possibly have prevented or influenced. But, like, they're the ones who ultimately curated the training data -- even if the approach they went with was "no curation at all". They're the ones who didn't deem it necessary to prioritize reducing harmful real-world biases in their training data. They're the ones who didn't deem it worth the tradeoff of time and money for a model that reproduces less real-world bias. And it doesn't matter of they're not particularly racist or sexist or whatever themselves. They just need to not be willing to put in a lot of time and effort combating all these biases in their models. And unfortunately that's a pretty huge swath of the industry right now.
For bias I think both things can happen. I'm not a programmer so please forgive poor technical terms. Also I'm leaving out someone who would deliberately input bias in, in some way. I think facial...
Exemplary
For bias I think both things can happen. I'm not a programmer so please forgive poor technical terms. Also I'm leaving out someone who would deliberately input bias in, in some way.
I think facial recognition software is a comparable example - the programmers trained it (I am sure there's multiple) on what turned out to be a predominantly white sample set, leading to black people consistently being misidentified. Testing should have brought that to light. Either it did, and it was a lack of care about the impact of this discrimination on Black people where these systems were used (bigger than a bug that can be worked through IMO) which is itself bias, or it didn't because the testing was not comprehensive enough.
We know that programming as a profession is mostly male and mostly white and we consistently see these biases in machine learning output- and yeah they're representative of society's biases as in this case the training material was (I'm assuming) scraped online. At some point consistently failing at this level is negligent to the point of malice. IMO this is why scraping the entire Internet for pictures and content was a bad method for AI building. We've seen chatbots become racist too so this isn't new.
What I do see here is a sort of shrug, "what can be done" or "you have to change the prompts" attitude. That's what is frustrating. Women shouldn't have to use a fundamentally different prompt than men to get non-sexualized art. The bias present in these models has real world impact - for a basic photo generation that's relatively small, but we established a long time ago that only having pictures of white men as doctors in media had ripple effects. When AI models are used for other things like sorting resumes or identifying faces, we're right back into perpetuating the same systemic biases.
I found this article one that said a lot of the things I was thinking in much more eloquent ways.
Scraping the internet indiscriminately is absolutely a bad way of gathering training data, but it happens anyway because it’s free/cheap. Nobody wants to build datasets by buying rights to known...
Scraping the internet indiscriminately is absolutely a bad way of gathering training data, but it happens anyway because it’s free/cheap. Nobody wants to build datasets by buying rights to known good data or even pay for the labor to sift through the results of scraping.
Even with multinational deep pocketed corporations like Microsoft involved, the money to train ethically hasn’t materialized, and the only reason I can imagine for that is because the potential payoff for doing it the wrong way is too great to ignore.
Which is an incredibly frustrating response - it costs too much to make sure Asian women have clothes on in their images. And well, what can you expect? It may be true. But it's frustrating.
Which is an incredibly frustrating response - it costs too much to make sure Asian women have clothes on in their images. And well, what can you expect?
Realistically, I think a reasonably well curated dataset of e.g. text or image data that's large enough to train the models we're working with today might just be bigger than humanity's biggest...
Realistically, I think a reasonably well curated dataset of e.g. text or image data that's large enough to train the models we're working with today might just be bigger than humanity's biggest singular project to date. Good data is very expensive and labor intensive, and mediocre data is practically free. It's no surprise AI is going for the "refining mediocre data" approach, particularly since that pays dividends beyond the individual dataset/problem we're working on. I'm actually somewhat hopeful that the current world beater models, if cobbled together right, and perhaps guided a bit towards the right conclusions using carefully curated data, could provide the right guidance to undo the bias, even if the training data is shitty. Imagine generating the above prompts, and asking a language/image hybrid model "here's pictures of asian women. Are there any harmful stereotypes displayed?" - and even if the images were generated from that same model, I wouldn't be surprised by an affirmative answer. Then (maybe with an intermediate step that identifies the exact stereotyping) you use that language model to provide gradients towards a less stereotyping image generation. Conceptually mostly simple, the data needs are basically nil, all it takes might be a bit more model scale. Likewise, you could ask the model whether a set of samples (e.g. images of doctors) is representative of the demographics of doctors. Maybe add more context like "this artwork is intended to represent 1930s america" and the model will say "sure, mostly white and male, that checks out" or resample to better represent "2023 globally". Feed the model with available high-quality statistical data to better inform the sampling process.
To put why this should even work simply: If all the text on the internet was 60% nazis and 40% SJWs, you wouldn't be surprised if the naive sample was nazi bullshit; it's basically a majority vote. But at the same time, you've got terabytes of text here, and a lot of it was written in an inclusive way, identifies injustices or otherwise has beneficial things to say. You just need to put your hands on the scale very lightly to make sure the model actually uses that information. The good info is in there, it just needs a little help coming out. Which is why post-training using annotated data can have outsized effects.
Is it just too early in the morning for me or is this question completely non-sequitur from the original article. The original prompt already has some degree of specificity, "Asian woman"...
Is it just too early in the morning for me or is this question completely non-sequitur from the original article. The original prompt already has some degree of specificity, "Asian woman" shouldn't need to be prompted with "Asian women WITH CLOTHES"
Like is that so hard to understand??? Minorities aren't expecting to be the default, we're literally just trying to have a level playing field.
It is a complete non-sequitor! When AI assumes doctors are white men, autistic people are sad white teenage boys and that women should be naked that's not a "you didn't get specific enough with...
It is a complete non-sequitor!
When AI assumes doctors are white men, autistic people are sad white teenage boys and that women should be naked that's not a "you didn't get specific enough with your prompts" problem it's problem with the system before the prompts.
But it doesn't do this for everyone. Does this for a specific subset of the population, demonstrating the bias baked into the AI from its training data. That is what we are saying. I'm not blaming...
If you have hundreds of different styles of clothes in the training set, no single clothing style will dominate, so it goes for the one thing that looks more or less the same each time: naked.
But it doesn't do this for everyone. Does this for a specific subset of the population, demonstrating the bias baked into the AI from its training data. That is what we are saying. I'm not blaming the AI who cannot mind read. I'm blaming the programmers, and the marketers and managers and everyone didn't consider this a larger priority. But the idea that it's ridiculous to expect literally any typical clothing but only on women and specifically on Asian women, is insulting
Your comment comes up as straight-up malicious and unwilling to learn. Did you even read any of the other several good points in this thread about how that makes people feel? Does a white man have...
Your comment comes up as straight-up malicious and unwilling to learn. Did you even read any of the other several good points in this thread about how that makes people feel?
Does a white man have to put in any extra words in their prompt to avoid a non-sexualized output? Why can't a woman have the same experience without going through extra hoops, especially an Asian woman? Can you really argue this is OK? That it's OK to perpetrate these sterotypes that are harmful?
If it's the data set that is struggling to randomize, then there is a clear biases in it and that is not an excuse! I honestly can't tell if you're trolling because you're response is so callous and ignorant to these problems.
"It's really not that hard to understand" in your own words, oh and some empathy to and actually listening and talking to women!!!!
So, in the effort of reducing bias and prejudice, we should legislate that all AI generated images of people must be naked. 1.) No AI generating pictures of people in cultural clothing as a...
So, in the effort of reducing bias and prejudice, we should legislate that all AI generated images of people must be naked.
1.) No AI generating pictures of people in cultural clothing as a default
2.) No concerns about women being naked since now everyone is.
3.) People will stop trying to AI generate themselves since they can't get that mountain background without showing their junk.
This is intended to be humorous, in light of the seriousness of the topic, but if you told people that nudity was the only legal option there would still be a variety of problems with generation of different body types.
A woman around middle age, medium brown skin, dark brown to black hair.... Oh and she's wearing the clothes, like how a man I generate with the same prompts would wear. I suspect I'd get a young,...
A woman around middle age, medium brown skin, dark brown to black hair.... Oh and she's wearing the clothes, like how a man I generate with the same prompts would wear. I suspect I'd get a young, white woman with light eyes and light brown hair, or a naked Asian woman, given the article we're commenting on. That tracks with the things I have generated or seen generated.
If your answer is that women should have to specify that they want clothes on their generated avatars when men do not have to do that, you're not having the same conversation I have been.
If all images of a person came back naked by default it'd be a weird choice but at least a consistent one.
I absolutely agree with you that they shouldn't brush this shit off like that. However, as someone who works in AI, I'd like to point out they have a pretty big incentive to make any excuse they...
If women (particularly Asian women) are being portrayed naked by default (or at a much higher frequency) that is a problem. They screwed up their training data. But the response to that being "you just have to put in different words" is unhelpful. They should absolutely be criticized and be expected to retrain their model IMO.
I absolutely agree with you that they shouldn't brush this shit off like that. However, as someone who works in AI, I'd like to point out they have a pretty big incentive to make any excuse they can to avoid retraining their models. Even ignoring the high costs of retraining itself, it's extremely difficult (and potentially not even strictly possible) to remove real-world biases from their training data. It's even harder on datasets of the massive size of this one. And "difficult" and "harder" here translate to "really really really fucking expensive". So ofc they have a pretty big reason to deflect and offer up any bandaid solution they can rather tham retraining.
Bias in machine learning models reflecting undesirable real-world bias is an old problem (well, as old as any problem can be in this field) and is super interesting even just within text-only language models (which is my field). It turns out it's really hard to remove such things from natural language data without making the models worse at their tasks -- after all, machine learning models are pattern-recognizing machines, and those biased are actual existing patterns in the data! There's a lot of interesting work out there on techniques to mitigate issues like this (at least in pure language models; I assume similar work has been done on models more like this one but I'm not up to date on it). But unfortunately it's really hard to do anything beyond just that mitigation.
Unfortunately, even mitigation requires a thoughtful approach to data collection, curation, and augmentation before training, which is something that the groups training these massive LLMs are imo not prioritizing nearly enough. To be fair to them, the sheer quantity of data needed for models like this would make a more thoughtful approach extremely expensive. On the other hand, they shouldn't get to act oblivious to what is underlyingly your classic GIGO problem.
Yeah I'm aware they have the incentive not to care but I felt quite disheartened at the response here of "just change the prompts" for example. Well yeah but that doesn't address the underlying...
Yeah I'm aware they have the incentive not to care but I felt quite disheartened at the response here of "just change the prompts" for example. Well yeah but that doesn't address the underlying problems.
I just don't think they should be let off the hook. They should start doing that mitigation yesterday, especially if they're going to keep marketing their products.
I think I'd attribute those comments mostly to two things: one, the user was in problem solving mode, when what you and perhaps the author of the OP want is empathy. That is such a common social...
I think I'd attribute those comments mostly to two things: one, the user was in problem solving mode, when what you and perhaps the author of the OP want is empathy. That is such a common social problem, books and papers are written about it. You probably know what I mean.
two, the user has limited capability of actually solving the problem. They can't retrain the model, they can't conjure up new methods that fix bias, they can't take the model offline. All they can do is prompt engineer their way around the problem. That does not mean they're not empathetic to your problem, it's just expressed differently. Nor does it mean the user is supportive of the company that hosts that model, or even finds the workaround satisfactory.
I know why it happened. I just didn't love the responses. I do keep getting told why a bunch of things that in my opinion don't actually matter - because the point to me is the discrimination. It...
I know why it happened. I just didn't love the responses. I do keep getting told why a bunch of things that in my opinion don't actually matter - because the point to me is the discrimination.
It was less the problem solving and more the dismissive "what can you do" that I disliked.
I don't think it's dismissive, at least for the most part. I'm also to a degree in "what can you do" camp (though not dismissive of the issue), because, well, the problem of bias in AI is really...
I don't think it's dismissive, at least for the most part. I'm also to a degree in "what can you do" camp (though not dismissive of the issue), because, well, the problem of bias in AI is really fucking hard. You could not publish biased models, in which case biased models will simply eat your lunch. You could work towards reducing bias, which many of the leading actors are working on, but it's either ridiculously expensive or slow-going basic research. Smarter people than most of us here are working hard on these problems. There's no quick fix, and a slow fix won't actually affect the market in any way until years down the line. Short of putting the genie back in the bottle (lol, as if), there's not much that actually can be done by any org or person that isn't already happening. (Well, orgs or persons that are generally empathetic to issues of discrimination and bias.) Sure, more people working on fairness rather than chasing another fraction of a percentage of benchmark scores would help, but like putting the genie back in the bottle, that's not a thing individual actors can meaningfully affect.
In short: Let 'em cook. They're working on it. In the meantime, there's nothing much we can do. Ignore AI until they sort it, or work around the existing issues. Oh, and to not ignore the obvious: Make sure that fairness stays near the top of the collective ToDo list of the AI community.
Talking about it is a way of ensuring that fairness stays to the forefront. The other angle is gonna end up being putting pressure on companies and institutions that start looking to AI to get...
Talking about it is a way of ensuring that fairness stays to the forefront.
The other angle is gonna end up being putting pressure on companies and institutions that start looking to AI to get them to quit paying for it if all it gives are biased outputs. And to address actual harm done - like the disordered eating organization whose chatbot gave information on calorie restriction.
Because if the response to anything we don't have direct control about is shrug let them work on it, they'll get there some day but you just gotta assume you'll get (bad outcome) til then then half the conversations on Tildes should go away.
You've had some really thoughtful responses and I understand you're not trying to be dismissive. But I want to expand on this "Let 'em cook" mindset from a female perspective. Those words scare me...
In short: Let 'em cook. They're working on it.
You've had some really thoughtful responses and I understand you're not trying to be dismissive. But I want to expand on this "Let 'em cook" mindset from a female perspective.
Those words scare me to be frank because every time we're told to wait, that it will get better, it doesn't. We saw this with Trump's election and fears of Roe being overturned we're dismissed as being too out there, just let him work. Turns out those fears had founding and our reproductive rights stay under attack. We are always told to wait and see, and it lately it hasn't been going well.
I know the cart us ahead of the horse at this point with AI, but it is so exhausting to be told repeatedly to just wait instead of never being considered from the beginning. Again, why is that the default?
Because the way AI is right now is kind of a natural outcome of the kinds of freely available data out there and the way we do AI. And I can already hear calls to "find better data" and "figure...
Again, why is that the default?
Because the way AI is right now is kind of a natural outcome of the kinds of freely available data out there and the way we do AI. And I can already hear calls to "find better data" and "figure out a better way", and both of those are happening, they're just glacially slow compared to the "move fast and break things" method. The status quo is unfair not because people like unfairness, but because the unfairness is a natural consequence of the most efficient(*) known way of building AI.
So? What do? Well, if you're not actively doing research on this, support those actors that are doing it right. Commercial and non-commercial, there's people out there doing the research, collecting data, training and releasing models that are worse(again, natural consequence), but fairer. Find them. Talk about them. If they have usable products, use their shit, even if it might suck a bit more. That's the kind of feedback that will be heard. If people ignore shitty move-fast-and-break-things startups and instead give all the spot light to those that take the time to get it right, that's I think the one signal you can reasonably send "upwards" that will actually make these people cook faster (or more realistically, more cooks with more expensive spices).
I guess what I'm saying with "let em cook" is that it's slow to make AI fair, but I can assure you that lots of people are working on it; the common opinion that AI bros are too out of touch to care is wrong to a substantial degree. So advocacy in that direction is ineffective. But because they're inherently at a disadvantage, these cooks need time to catch up, so the fact that they're not visible to a larger extent doesn't mean they don't exist. I don't mean to call for everyone to sit back down and do nothing.
Keep in mind that I think fair AI will, unless or until things change, always be a bit on the back foot. Trying to be fair is just a performance hit one way or another, at least for the time being. For some applications, we don't care about the last percentage point of benchmark scores, so we accept fairness over scores; those are the applications where fair AI has a reasonable chance. Or where for legal reasons, fairness is a requirement, e.g. anti-discrimination laws for hiring decisions. But I don't want to give off the impression that just time and a bit more research will fix AI in all domains, if the requisite awareness isn't retained, or where it doesn't exist, established.
FWIW, I don't think Trump/RoeVsWade was at all a "let him cook" situation. My take on that guy was a tad more radical and hands-on.
(*) most possible definitions of efficiency, I'd argue, lead to the same result here. I.e. if efficiency = success / cost, then counting cost as time or money, and success as money or market share or benchmark scores, doesn't really matter. Curating better datasets or building inherently less biased learning algorithms take too long and are too expensive to compete.
Exactly this. I've played around a bunch with local models like StableFusion and "online" stuff like Midjourney. There's a huge bias on hair colour and race depending on activity. You can see the...
Exactly this. I've played around a bunch with local models like StableFusion and "online" stuff like Midjourney.
There's a huge bias on hair colour and race depending on activity. You can see the Matrix behind the images when you push the limits. Every image of "woman model" is vaguely eastern European, because there is a TON of training material of women from that area.
Now try to generate a redhead for example and you'll see the style switch immediately.
And the best bit is that Midjourney's own NSFW detector thinks blondes are too sexy, but the exact same prompt but "redhead" appended is just fine ¯\_(ツ)_/¯
AI can't create things it hasn't seen. Two of the best ones I've found out is "<name of large city> without cars" and "woman lying down after workout". There just aren't photos of either so the AI doesn't know what to do. It tries, but fails: https://imgur.com/eD1HVWb
Sure, but the prompts you given it aren't example of that. Google images for "a city without cars" and you get multiple examples of cities without cars, mixing concepts like "a city without cars"...
AI can't create things it hasn't seen.
Sure, but the prompts you given it aren't example of that. Google images for "a city without cars" and you get multiple examples of cities without cars, mixing concepts like "a city without cars" and "new york city" isn't hard for the model either, this is why the "avocado chair" example astonished people when these models were first coming out. Googling "woman lying down after workout" also wields plenty of results, so I don't know why you think the models have never seen that.
You're just misunderstanding how it works. The text to image model you used doesn't use an LLM as the backend for the text portion, so it's understanding of text in more akin to word salad. the vectors for the prompt "a city without cars" and "a city with cars" are virtually the same. There's a paper explaining this phenomena in the CLIP text encoder if you want to know more.
If you want your results you need to be more creative with your prompting, try "new york with empty roads" and you'll have more luck, putting "car" in the negative prompt also helps. My GPU will be busy for the next couple days so I can't show you SDXL results, but here's what novelai can do (SDXL based anime model).
Their text interpretation models can't handle negatives like "without cars" in the prompt, you'd have put cars in the negative prompt or what describe a city like like without cars like "city with...
Their text interpretation models can't handle negatives like "without cars" in the prompt, you'd have put cars in the negative prompt or what describe a city like like without cars like "city with walkable streets, people walking, cycling, street markets and cafes" etc
If you go for the walkable streets one, it'll give you a city in Europe with buildings like that. Or one of the very few such places in the US. It can't give you a picture of New York city streets...
If you go for the walkable streets one, it'll give you a city in Europe with buildings like that. Or one of the very few such places in the US.
It can't give you a picture of New York city streets without cars for example.
Interesting trying this out with Dall-E. Only the last of these prompts actually generated two images like it was supposed to; every other prompt generated at most one image. My first attempt was...
Interesting trying this out with Dall-E. Only the last of these prompts actually generated two images like it was supposed to; every other prompt generated at most one image. My first attempt was "a woman lying down after working out." That failed. These are the ones I tried that worked:
This is a very interesting topic, and it seems to paradoxically lead to the conclusion that less dataset filtering may lead to less bias. This is not true, or rather it's phrased in a misleading...
This is a very interesting topic, and it seems to paradoxically lead to the conclusion that less dataset filtering may lead to less bias.
AI can't create things it hasn't seen. Two of the best ones I've found out is "<name of large city> without cars" and "woman lying down after workout". There just aren't photos of either so the AI doesn't know what to do. It tries, but fails: https://imgur.com/eD1HVWb
This is not true, or rather it's phrased in a misleading way. The issue is that with some current models, due to the way they are trained and strongly guided towards a certain style of photorealism (they collected user input during the training process specifically for that), certain concepts are too strongly associated together, so the problem is more accurately described as "things it has seen too much of trump everything else".
I don't know how much Midjourney suffers from this (don't use it), but it's a big issue with SDXL and Dall-E, they are strongly guided to create one particular style of semi-realistic stock-photo like style and it's quite difficult to wrangle them into doing something else, like even a cheap camera street photography style. This also guides them towards creating "true" stereotypical things more often.
SDXL cannot seem to do NYC streets without cars at all, but it can usually sort of do Prague streets without cars (I also tried "2-lane roads" instead of "streets", to push it away from just creating a pedestrian-only street).
However! If you load up Stable Diffusion 1.5, an older version which was trained on a dataset with very little filtering and without using user input for guidance, it can do NYC without cars easily, let alone Prague. It just looks less photorealistic. The 4 NYC images are 1st attempt, no selection. It also created a woman lying down after workout, but that one I had to select out of like 6 attempts because v 1.5 is bad with people in any position but standing, and anatomy in general, and to get to photorealism I'd have to use some of the existing workflows to fix the face - the raw output of older models is just bad with faces.
Dall-E 3 can kind of almost get there, not on the first try, but if you then tell it "there are still cars and taxis on this image, I want no cars at all", it spits out this on the second attempt. You could probably get to no cars if you repeated the demand, Dall-E is usually receptive to that, but it may gradually become even less photorealistic in the process.
I think a key part of generative AI that we're just figuring is how to get a model to sample how we want, when we have trained it on a lot of things we may not want. Basically, learn some aspects...
less dataset filtering may lead to less bias.
I think a key part of generative AI that we're just figuring is how to get a model to sample how we want, when we have trained it on a lot of things we may not want. Basically, learn some aspects of the data, but don't actually generate samples like it, only use it to affect other generations lightly. As an example of that, if I'm not mistaken, chatGPT was fine tuned in a relatively small dataset of high-quality helpful interactions. This fine tuning is most of what makes chatGPT helpful and keeps it from generating text that is as helpful as your average internet troll. Leverage that a bit more, and you might be able to train on completely uncurated datasets of portraits and actually generate dressed asian women.
But my examples seem to show the opposite is true with image generation, at least in the way it's done now. I believe that the kind of "airbrushed realistic" look that refuses to stray into more...
But my examples seem to show the opposite is true with image generation, at least in the way it's done now. I believe that the kind of "airbrushed realistic" look that refuses to stray into more creative territories like NYC without cars was created mainly using guidance based on users manually ranking the output, though filtering surely played a role too.
And, just like with ChatGPT, it seems to show that too much guidance for safety makes the outputs less useful. ChatGPT was most creative just after it was released, and I don't think it had any downsides for legitimate users either, it was just easier to abuse it for things that either were bad or just would look bad in media. But at least it still works well for many things.
I'm not saying that we're just going to have to deal with having strongly biased models, but these current processes that kind of work for LLMs don't really seem to work for diffusion models well. There's demand for content filtering and moderation, but I hope that some of the competitors create a better way to do it that would retain creativity.
Something else that may factor in is an abundance of training material of that type produced by risqué publications, which at least in Japan (haven’t been to other east asian countries and can’t...
Something else that may factor in is an abundance of training material of that type produced by risqué publications, which at least in Japan (haven’t been to other east asian countries and can’t speak for them) are numerous and popular, appearing in the magazine shelves of just about every convenience store in the country.
The west of course also has such publications but they don’t strike me as being nearly as prolific, ubiquitous, or socially accepted.
... So, informing people of the risks of the system along with taking steps to reduce a problem when it arises and having rules in place is... Not sexy? I dislike the tone of the article but that...
The sexualization of women is so prevalent that Lensa even addresses the issue on its FAQ page, acknowledging these societal biases, adding that LAION 5B creators have introduced filters to reduce these biases (they didn’t think to do this initially?). Lensa also states, “intentional use of the app to create explicit content is absolutely prohibited by our Terms of Use and obliges Lensa users to comply with that requirement.”
...
Consent is sexy. What Lensa is doing, and has the potential to do, is not.
So, informing people of the risks of the system along with taking steps to reduce a problem when it arises and having rules in place is... Not sexy?
I dislike the tone of the article but that doesn't necessarily invalidate any claims in it...
Both women also noted how their avatars were more images of generic Asian women—“clearly modeled on anime or video-game characters. Or most likely porn,” according to Heikkilä—than of themselves. Meanwhile, the men’s avatars were more realistic and closer likenesses.
...
But we can’t just blame the data. Keep in mind that someone is developing these models—and making the choice to depict male avatars in cool and flattering images and female avatars as half (if not fully) naked.
So, the article makes the point that people are developing these algorithms by scraping the Internet and that the people developing these systems are trying to reduce the amount of potentially unwanted sexualization but makes the claim that they're not doing anything (or enough)? I can't follow.
AI overrepresents based on stereotypes and the article correctly points out the over sexualization of women online - but also calls out the developer for not doing enough to fix it. Other than making changes to their algorithm, which they claim to have already introduced filters for that, what more can they do? I don't see a link to a 'pictures of real women™' archive that they can use to retrain their model on.
They could do more of the same: better sources of images and better curation. Maybe pay for images from higher-quality collections? I expect that machine learning will we be useful to help the...
They could do more of the same: better sources of images and better curation. Maybe pay for images from higher-quality collections?
I expect that machine learning will we be useful to help the image collectors do the curation part. This will require better understanding of what they want in their training sets, though, what's over-represented and what they're missing that customers want.
There's been a gold rush, but hopefully some of these companies will work harder on quality and it will improve.
Which is strange, because if I understand the Lensa's FAQ correctly, they're using Stable Diffusion V2, which famously did an effort to purge all NSFW data from the training dataset so that it...
Which is strange, because if I understand the Lensa's FAQ correctly, they're using Stable Diffusion V2, which famously did an effort to purge all NSFW data from the training dataset so that it couldn't be used to create any nude people, to the point that it was worse at doing anatomy in general than the previous version. That would imply that what was happening may have been caused by whatever additional training Lensa did for their app and not by Stable Diffusion itself.
It's not strange, they used Stable Diffusion for the AI model, not the dataset. They used the LAION 5B dataset, which hasn't done the nudity filtering you mention.
It's not strange, they used Stable Diffusion for the AI model, not the dataset. They used the LAION 5B dataset, which hasn't done the nudity filtering you mention.
LAION 5B was used to train Stable Diffusion v2. They certainly did not use unfiltered LAION 5B for the finetuning, because firstly that wouldn't make a lot of sense for such a specialized model,...
LAION 5B was used to train Stable Diffusion v2.
They certainly did not use unfiltered LAION 5B for the finetuning, because firstly that wouldn't make a lot of sense for such a specialized model, and secondly the price would be exorbitant. They may have filtered out useful data from it for finetuning, but to me their FAQ reads more like they're talking about SD itself and understandably keeping their internal processes secret.
I mean I'm aware that "AI" doesn't randomize well, and I think that the output is evidence of a problem with the training set. But also that this is evidence of societal bias, bias in the programmers and thus bias built into in the model.
If women (particularly Asian women) are being portrayed naked by default (or at a much higher frequency) that is a problem. They screwed up their training data. But the response to that being "you just have to put in different words" is unhelpful. They should absolutely be criticized and be expected to retrain their model IMO.
We know there's racial bias as well but if every search for a Black man shows a minstrel show character we'd be worried about that too, right? It's concerning when every search for "doctor" or "autistic" gives you white men.
I find it confusing that the response here is "well thats just how it is because of the training material." Yeah it is, but that's not ok.
No one's saying it's ok. If I'm reading the room right, the response is "it's not OK, but it's definitely not malicious, and it's also a problem we're aware of and working towards solving. In the meantime, here's a workaround". Holding software back because you've found one major bug is not usually the way to go[1], so long as it's not safety critical software or a showstopper bug. You get it into many hands, thus find many bugs fast, then solve those bugs. If you yoinked/delayed the software for every major bug, you'd see it for a week before it's yoinked, then 6 months of crickets while Dave furiously types to fix his fuckup. Then it's another week of public testing and now it's John's time to fix his fuckup for the next 6 months. Bit over the top, but still mostly accurate. Maybe even more accurate for AI, where "bugs" can be quite nebulous and might require a longer societal debate to nail down. See the OP, where a single individual might not even know they're dealing with biased treatment (aka a bug), because they only see output for themselves.
Also, I'm curious how you think programmer bias gets baked into the model? Like, is that because they approve flawed training data, or because of more active intervention of their part that biases the model?
[1] a positive, not normative statement. That's just the way the software industry does business nowadays, for better or worse. That that has consequences is another way more general can of worms.
I think you're viewing this more as a software issue rather than considering it a reflection of biaseses our society is still coming to terms it. Being provided a work around and referring to the issue as a "bug" is dehumanizing because it fails to acknowledge that these biaseses have been included from the beginning without even second thought. It's not a malicious act necessarily, but it tells me as a woman that the teams creating these algorithms are absolutely naive to these issues.
It's no hidden fact that Asian women have been dealing with being portrayed in over sexualized roles in Hollywood.
This biases is so strong in media that no doubt it influenced these algorithms. But I think what has I and others upset about this, is "why is this the default?" Like really, why did no one stop to think that this would be an issue? It took me 5 minutes to read into it and a dose of empathy to make me want to avoid it.
People are missing the fact that if those who are creating these AI image creators are ignorant to these issues, they will get baked into the code base and exemplified within them. They expose our biaseses that we continue to hold in society and how we are still unwilling to grapple them. I get the whole idea that it doesn't make sense to scrap all the work and start anew, but it also tells me that the people working on these things don't care enough to prevent them from arising. Instead the AI bubble is too big of a money making machine to care about how their portrayals of women may affect how they are viewed in real life. Instead they are inconvenience to the model and their very existence requires a "workaround" to get a more accurate reflection of who they really are. Honestly, how would that make you feel that the default to generate an image akin to your own identity in real life requires a damn "workaround" to avoid over sexualization. It disgusts me and it inherently bothers me to my core that so many others don't get it or don't care to get it.
I'm not going to engage too thoroughly as I don't think there's much I can contribute on many of the aspects you're raising, not that I even disagree to begin with. One point I want to raise more generally, and yours is just the comment that sparks that thought for me; it's not specific to your comment, so don't take it as an attack or anything:
Keep in mind that "the AI bros" are often not a homogeneous group, nor are they all organized in any way. It's entirely possible for an AI engineer at a corporation to train a specific model, but tell their manager "hey, yeah, sure... but this thing is going to be drawing asian women in the nude". The manager greenlights the model's release anyway; he doesn't have much of a choice, as fixing the issues would take years of either research or data curation, while less scrupulous companies are always around. He thus orders any mitigations to be put in place that can be achieved within a set budget. Meanwhile another AI researcher at a public university is working on making it randomize better, e.g. by explicitly deprioritizing outputs we don't want, but that's basic research and will take years. And independently of that, enthusiasts online figure out what kinda prompts make the shitty model produce reasonable outputs. (This example is only loosely inspired by your post, so if it seems like a strawman, that's why. It's not supposed to be a faithful representation of any of your arguments.)
Now we could paint this entire group with one broad stroke as "AI bros produced a shitty model and now they're telling me a fix will be ready in maybe 3 years and also, here's a workaround, that's all you're going to get because you're a minority we don't care about." But that wouldn't be fair to the engineer's awareness of what is going to happen and him raising concerns; it wouldn't be fair to the researcher who's doing his level best to actually fix the problem; nor the enthusiast who -within the scope of his abilities- is also contributing to getting the most fair results possible out of the model. The only one it's arguably doing justice is the manager who gave the green light, but that bloke also has a bottom line to pay attention to, and shelving a project for years in a very temperamental business environment probably isn't great. Whether you think some sacrifice is appropriate here is subjective, I think, but I think the impact would be limited given there's no shortage of shitty AI startups.
What I'm saying here is, it can be frustrating for those trying to actually make responsible AI to hear these broad strokes narratives that misattribute the malicious actions. A lot of the people at the core of the field are very aware of their responsibility and the impact of their actions, and also aware of the issues around bias. They're trying really damn hard, and painting the entire field as naive buffoons helps no one, except the naive buffoons that do exist in the field. Also, regarding fixes and changes that laypeople propose: xkcd never misses.
You bring up some excellent points and I completely understand. I work in a similar environment and have my own frustrations with it.
I guess more so it is the frustration that this is the default and that more effort is needed to avoid that. It sorta reminds me of that scene in Rick and Morty where Rick keeps dying and each new universe is filled with Nazis. This leads to Rick screaming "Goddamnit when did this shit become the default!" It's just all so troubling, but I appreciate your comment making me think.
Your explanation represents how things are, and it embodies the ethical problem described in Ursula K. LeGuin's short story, "The Ones Who Walk Away From Omelas" [PDF warning].
Can you knowingly abide the suffering of a minority of people in order to bring the vision of AI abundance to fruition? Can you continue to ignore the people who tell you they are being directly and preventably harmed?
Repairing AI function is not a trolley problem. There is no comparably clear utilitarian answer where we can say, "this technology helps billions, will save or improve many lives, and only a few might be harmed (even if they don't get a choice)."
We have direct evidence that AI bias is causing prolonged injuries and social harm to many people, particularly when seen across all minorities who are not the statistical average Internet example. This might ultimately be a very large number, when you count the insurance, loan, and rental denials, policing errors, rejected job applications, stereotype-driven query responses, misdiagnoses, and so on.
Continuing to offer the technology while refusing to respond to the issues raised here is engineering malpractice, in addition to the ethical and moral wrongs.
[And I say this as a more-or-less female technologist.]
Do we really? Maybe I'm emotionally stunted, but I wouldn't describe someone who wants to use an AI for face filtering and realizes they don't like the result as significantly harmed.
There are more substantial impacts, but I think those are all models that exist solely behind closed doors. Anytime you combine actually high-stakes situations like legal or financial issues. Think job applications, insurance applications, medical decision making. The case of discrimination is more difficult to make, as those models exist behind closed doors so can't be freely evaluated. It's also going on mostly independently of the current hype around generative AI, most of that is by now just plain old statistics.
Though I'm not terribly convinced we (collectively) have a good grasp of the amount of discrimination going on there. Not every error is discrimination, and not every discrimation is "erroneous" (statistically speaking), so it's difficult to measure even under transparent conditions. And with the individual case studies we're getting, I think people will absolutely turn on their confirmation bias and see or not see discrimination, depending on their general outlook.
A very concrete one is a chat bot that replaced live support for people with eating disorders. It gave them information on how to restrict calories. It's just direct harm caused by poor implementation of a product that was probably mis-sold to them.
I actually agree with that. But I think that the focus on specifically biases in image generation models is misplaced, firstly because I have doubts about the harm and secondly because the mechanisms are simply different than in LLMs and the real parallels are not that strong.
The fact that people seem to mostly focus on Stable Diffusion not because it's particularly bad but because it's the only one that's open and therefore easy and cheap to examine also isn't great.
They're referring to uses of 'AI' outside image generation
As evidenced by this part of their comment. If 'AI' can't even generate a picture that represents anything slightly deviating from the perceived norm, do you really want it helping your doctor?
True! Sorry.
I don't think those two things are similar enough to compare and I think that placing the focus on removing bias from image generation models instead of talking about those things is misguided.
That said, as someone with an uncommon chronic disease that many doctors know almost nothing about, I definitely 100% do want it helping my doctor, even though I acknowledge that many potential issues exist. I think that preliminary research actually shows that in this particular case an AI might have fewer biases than most doctors and since the doctor is still always present to veto it, the potential damage is smaller than with products related to insurance, loans, rental denials etc., where there's a higher chance that an AI would be used unsupervised.
I'm just curious though.
What, exactly, should an AI generative model do when given a prompt? Should it start running the user through a bias checklist? Admonish the user? Should it be hardcoded with a list of "acceptable" results and only respond through them? Is it the responsibility of the AI to refuse to fulfill requests that seem "adult" in nature?
Because what the AI actually does absent a specific criteria is pick randomly, weighted by what's in its dataset. Which can be changed as the user likes. Anyone who has specific needs for their AI dataset can build something that suits their needs. And people do.
Infinite diversity in infinite combination. Humanity is the very definition of that statement. A wonderful array of choices, each unique, each wonderful in his or her own way.
But details matter. If an human artist is posing a human model, the artist will get into specifics with the model. Who the model is, what to wear, how to pose, facial expressions, everything.
It's not racist, or sexist, or really any kind of -ist, for the computer to default to the most likely responses based in its dataset if it's not given details to narrow the response with. If the user has specifics in mind, tell the computer! So it can use that data to parse its response. The same as the artist would parse her prospective models and details if she had specifics in mind.
A Korean artist working in Korea will have to get very specific, and take some extra steps, to get a non-Korean model in to pose. Same for an Australian artist, an American, a French, and so on. It would be silly for those artists, if they put out a casting call, to be upset that the people who live in their city are who show up for consideration. A Korean in Seoul being angry that "I didn't mean I wanted a Korean to come pose for this picture" would be a silly complaint.
Especially since all they'd have to do is ... be specific about what and who they'd like to come pose for them. Sure maybe there might not be that many non-Koreans in Seoul at that time, but at least they'd have better odds of some of those perhaps coming in to meet with the artist for consideration.
And it occurs to me that non-Koreans, seeing a generic "I want a model to pose" call in Seoul, might assume the artist would prefer a Korean. After all, if they didn't specify, a woman who's black and six foot four could very reasonably assume she's probably not who a random Korean artist wants to pose for a painting. How should this lovely non-Korean woman be expected to somehow know that artist would very much like to meet with her, if the artist wasn't specific in the casting request?
Assumptions go both ways, see. And if details matter, details should be given.
It's not -ist to need to give the computer specifics. And anyone who doesn't like a certain model (dataset) can generate their own. On Stable Diffusion dataset websites new models and modifications of models are posted in the hundreds, every day. By people who had some very specific thing they wanted put into a dataset, and proceeded to work on.
A lot of what they post is just mindboggling to me, but they wanted it and got it (and then decided to share it). Anyone can do the same. There are even websites that will do all the computer stuff; just upload the pictures with details and it'll give you back a model after it runs the numbers.
So I see all these articles by "reporters" eager to play "hah! gotcha!" with AI and it's puzzling. If they feel they're not represented, they can build models that do represent whoever and whatever they want, however they want, whenever they want. Which then leads to the assumption they might be complaining sort of just to complain. Outrage gets clicks.
Again, why does a Asian women have to put in more inputs to a avoid a sexualized output when a white male does not? Do you really not see the unfairness and the issue here?
It is racist. It is misogynistic.
Why does an Asian womam need to put more of a quadratic equation into the prompt to avoid sexualization while a white male can avoid that with just 2+2?
The article I linked in my post refers to a Asian hate crime shooting that occurred in Atlanta a few years ago.
These type of biaseses have impacts and failing to stop their perpetuation leads to this hate. There is a clear biases in the data sets being used in these models and AI models like this are just going to further stress these problems. A fair model set would have non-sexualized outputs by just typing "Asian woman" akin to entering "White Man" without no extra work needed.
It's so frustrating as a minority to hear from the same people that "AI is the future" but also "you minorities need to go through extra hoops to use the default AI experience and have it be the same as everyone else".
This hurts to read, but it is the unfortunate reality of the dawn of AI. When I read this type of commentary I am instantly empathize that this is wrong and is hurting people. What gets me fuming is the absolute smugness of a "just deal with it" attitude that is given as a response to these concerns. It's not that I view AI itself as the boogie man, but more so its developers that are ushering it to be such. At a certain point this pure ignorance tiptoes into malicious territory when it comes to impacting something pivotal.
A while ago there was some meta commentary that Tildes can be too nice at times. I'm over it with this topic and it's time to rip the bandaid off. If you're a supporter of AI who clearly doesn't understand the issues of this and can't be bothered to read or empathize with it, you're a fucking monster who is continuing to uphold the ills we have yet to overcome as a society. Do better; we already should have at this point.
Fwiw I was the one who gave you the Exemplary tag :) I don't very often but thank you for being up and down this thread, it's very appreciated!
I read through this thread again and this is what I keep coming back to that really annoys me:
The implication that a minority literally saying that "hey maybe this is a weird thing happening" is "just outrage" is infuriating. I've actually noticed more and more even here on Tildes that there's a push back on furthering diversity and shutting out voices that maybe there's still a lot of racism out there.
I hope my posts in this thread haven’t come off as brushing these problems off. My intention was discuss why/how things wound up as they are (which I think is an important step in effecting change, e.g. identifying sources of bias to avoid in training datasets for example), but that can sometimes unintentionally read as excusing things.
Hey, sorry if I came off a bit angry in some of my comments, but I didn't mean for any to be targeting towards any one user. @pheonixrises may have similar experiences to my own on this site, but I've noticed there's not a lot of minority voices on here. Especially women.
I've experienced several threads being locked due to the majority male voice outright silencing female experiences on here. It has even led to some outright disturbing posts that enter 4Chan Territory that fortunately lead in those users being banned. But it is so dismaying to constantly have to be the lone female voice to share those experiences and deal with the continual ignorance. I'm frankly quite tired of this in 2024 for fucks sake.
Women's issues are valid! Minority issues are valid! Like why are these questioned and dismissed as if they are not realities? It makes me wonder how many of these male users actually care to be aware of these things or is it simply not worth the time or "profitable enough." I live this reality every day, just about every minority does. It really serves as a way to further dehumanize us all while our rights are being eroded away. It really exemplifies a portion of the population doesn't care and I think I'm just frustrated with that. It came out in my comment, but I am tired of being dismissed.
Sorry if I have caused any distress to you or others.
At least for me, I'm lucky enough to have a slightly more noticeable Asian presence here on Tildes, but even then it's still very clear that it's a minority. Furthermore as a model minority I feel like we have a more complex relationship with the "default" perspective.
I honestly don't know how anyone survives as a woman online. It's already really annoying as an Asian American guy, and I just avoid commenting to avoid the headache. Props to you for speaking up, I hope you know it's appreciated!
(o before e btw :) )
Very rarely have minority rights been viewed as profitable enough to be handled by the free market. I've heard libertarians who believe that a) businesses should be allowed to discriminate freely and b) no significant number of businesses would lose the money they'd lose for discriminating against Black folks, or Muslim folks, or queer folks, disabled folks, etc.
When that absolutely flies in the face of our history and our present. People don't act "rationally" but they think they do. And the number of people and institutions willing to cut their own noses off to be able to spite the face of someone they consider lesser than them is absolutely wild to me.
All this to say I appreciate your voice and am here to help amplify it when I can.
THIS. So many people working AI, particularly men, brush off reproduction of real-world biases by machine learning models as just an inevitable outcome of the code, something they could not possibly have prevented or influenced. But, like, they're the ones who ultimately curated the training data -- even if the approach they went with was "no curation at all". They're the ones who didn't deem it necessary to prioritize reducing harmful real-world biases in their training data. They're the ones who didn't deem it worth the tradeoff of time and money for a model that reproduces less real-world bias. And it doesn't matter of they're not particularly racist or sexist or whatever themselves. They just need to not be willing to put in a lot of time and effort combating all these biases in their models. And unfortunately that's a pretty huge swath of the industry right now.
For bias I think both things can happen. I'm not a programmer so please forgive poor technical terms. Also I'm leaving out someone who would deliberately input bias in, in some way.
I think facial recognition software is a comparable example - the programmers trained it (I am sure there's multiple) on what turned out to be a predominantly white sample set, leading to black people consistently being misidentified. Testing should have brought that to light. Either it did, and it was a lack of care about the impact of this discrimination on Black people where these systems were used (bigger than a bug that can be worked through IMO) which is itself bias, or it didn't because the testing was not comprehensive enough.
We know that programming as a profession is mostly male and mostly white and we consistently see these biases in machine learning output- and yeah they're representative of society's biases as in this case the training material was (I'm assuming) scraped online. At some point consistently failing at this level is negligent to the point of malice. IMO this is why scraping the entire Internet for pictures and content was a bad method for AI building. We've seen chatbots become racist too so this isn't new.
What I do see here is a sort of shrug, "what can be done" or "you have to change the prompts" attitude. That's what is frustrating. Women shouldn't have to use a fundamentally different prompt than men to get non-sexualized art. The bias present in these models has real world impact - for a basic photo generation that's relatively small, but we established a long time ago that only having pictures of white men as doctors in media had ripple effects. When AI models are used for other things like sorting resumes or identifying faces, we're right back into perpetuating the same systemic biases.
I found this article one that said a lot of the things I was thinking in much more eloquent ways.
Scraping the internet indiscriminately is absolutely a bad way of gathering training data, but it happens anyway because it’s free/cheap. Nobody wants to build datasets by buying rights to known good data or even pay for the labor to sift through the results of scraping.
Even with multinational deep pocketed corporations like Microsoft involved, the money to train ethically hasn’t materialized, and the only reason I can imagine for that is because the potential payoff for doing it the wrong way is too great to ignore.
It’s a crappy situation all around.
Which is an incredibly frustrating response - it costs too much to make sure Asian women have clothes on in their images. And well, what can you expect?
It may be true. But it's frustrating.
Realistically, I think a reasonably well curated dataset of e.g. text or image data that's large enough to train the models we're working with today might just be bigger than humanity's biggest singular project to date. Good data is very expensive and labor intensive, and mediocre data is practically free. It's no surprise AI is going for the "refining mediocre data" approach, particularly since that pays dividends beyond the individual dataset/problem we're working on. I'm actually somewhat hopeful that the current world beater models, if cobbled together right, and perhaps guided a bit towards the right conclusions using carefully curated data, could provide the right guidance to undo the bias, even if the training data is shitty. Imagine generating the above prompts, and asking a language/image hybrid model "here's pictures of asian women. Are there any harmful stereotypes displayed?" - and even if the images were generated from that same model, I wouldn't be surprised by an affirmative answer. Then (maybe with an intermediate step that identifies the exact stereotyping) you use that language model to provide gradients towards a less stereotyping image generation. Conceptually mostly simple, the data needs are basically nil, all it takes might be a bit more model scale. Likewise, you could ask the model whether a set of samples (e.g. images of doctors) is representative of the demographics of doctors. Maybe add more context like "this artwork is intended to represent 1930s america" and the model will say "sure, mostly white and male, that checks out" or resample to better represent "2023 globally". Feed the model with available high-quality statistical data to better inform the sampling process.
To put why this should even work simply: If all the text on the internet was 60% nazis and 40% SJWs, you wouldn't be surprised if the naive sample was nazi bullshit; it's basically a majority vote. But at the same time, you've got terabytes of text here, and a lot of it was written in an inclusive way, identifies injustices or otherwise has beneficial things to say. You just need to put your hands on the scale very lightly to make sure the model actually uses that information. The good info is in there, it just needs a little help coming out. Which is why post-training using annotated data can have outsized effects.
Is it just too early in the morning for me or is this question completely non-sequitur from the original article. The original prompt already has some degree of specificity, "Asian woman" shouldn't need to be prompted with "Asian women WITH CLOTHES"
Like is that so hard to understand??? Minorities aren't expecting to be the default, we're literally just trying to have a level playing field.
It is a complete non-sequitor!
When AI assumes doctors are white men, autistic people are sad white teenage boys and that women should be naked that's not a "you didn't get specific enough with your prompts" problem it's problem with the system before the prompts.
But it doesn't do this for everyone. Does this for a specific subset of the population, demonstrating the bias baked into the AI from its training data. That is what we are saying. I'm not blaming the AI who cannot mind read. I'm blaming the programmers, and the marketers and managers and everyone didn't consider this a larger priority. But the idea that it's ridiculous to expect literally any typical clothing but only on women and specifically on Asian women, is insulting
Your comment comes up as straight-up malicious and unwilling to learn. Did you even read any of the other several good points in this thread about how that makes people feel?
Does a white man have to put in any extra words in their prompt to avoid a non-sexualized output? Why can't a woman have the same experience without going through extra hoops, especially an Asian woman? Can you really argue this is OK? That it's OK to perpetrate these sterotypes that are harmful?
If it's the data set that is struggling to randomize, then there is a clear biases in it and that is not an excuse! I honestly can't tell if you're trolling because you're response is so callous and ignorant to these problems.
"It's really not that hard to understand" in your own words, oh and some empathy to and actually listening and talking to women!!!!
So, in the effort of reducing bias and prejudice, we should legislate that all AI generated images of people must be naked.
1.) No AI generating pictures of people in cultural clothing as a default
2.) No concerns about women being naked since now everyone is.
3.) People will stop trying to AI generate themselves since they can't get that mountain background without showing their junk.
This is intended to be humorous, in light of the seriousness of the topic, but if you told people that nudity was the only legal option there would still be a variety of problems with generation of different body types.
A woman around middle age, medium brown skin, dark brown to black hair.... Oh and she's wearing the clothes, like how a man I generate with the same prompts would wear. I suspect I'd get a young, white woman with light eyes and light brown hair, or a naked Asian woman, given the article we're commenting on. That tracks with the things I have generated or seen generated.
If your answer is that women should have to specify that they want clothes on their generated avatars when men do not have to do that, you're not having the same conversation I have been.
If all images of a person came back naked by default it'd be a weird choice but at least a consistent one.
I absolutely agree with you that they shouldn't brush this shit off like that. However, as someone who works in AI, I'd like to point out they have a pretty big incentive to make any excuse they can to avoid retraining their models. Even ignoring the high costs of retraining itself, it's extremely difficult (and potentially not even strictly possible) to remove real-world biases from their training data. It's even harder on datasets of the massive size of this one. And "difficult" and "harder" here translate to "really really really fucking expensive". So ofc they have a pretty big reason to deflect and offer up any bandaid solution they can rather tham retraining.
Bias in machine learning models reflecting undesirable real-world bias is an old problem (well, as old as any problem can be in this field) and is super interesting even just within text-only language models (which is my field). It turns out it's really hard to remove such things from natural language data without making the models worse at their tasks -- after all, machine learning models are pattern-recognizing machines, and those biased are actual existing patterns in the data! There's a lot of interesting work out there on techniques to mitigate issues like this (at least in pure language models; I assume similar work has been done on models more like this one but I'm not up to date on it). But unfortunately it's really hard to do anything beyond just that mitigation.
Unfortunately, even mitigation requires a thoughtful approach to data collection, curation, and augmentation before training, which is something that the groups training these massive LLMs are imo not prioritizing nearly enough. To be fair to them, the sheer quantity of data needed for models like this would make a more thoughtful approach extremely expensive. On the other hand, they shouldn't get to act oblivious to what is underlyingly your classic GIGO problem.
Yeah I'm aware they have the incentive not to care but I felt quite disheartened at the response here of "just change the prompts" for example. Well yeah but that doesn't address the underlying problems.
I just don't think they should be let off the hook. They should start doing that mitigation yesterday, especially if they're going to keep marketing their products.
I think I'd attribute those comments mostly to two things: one, the user was in problem solving mode, when what you and perhaps the author of the OP want is empathy. That is such a common social problem, books and papers are written about it. You probably know what I mean.
two, the user has limited capability of actually solving the problem. They can't retrain the model, they can't conjure up new methods that fix bias, they can't take the model offline. All they can do is prompt engineer their way around the problem. That does not mean they're not empathetic to your problem, it's just expressed differently. Nor does it mean the user is supportive of the company that hosts that model, or even finds the workaround satisfactory.
I know why it happened. I just didn't love the responses. I do keep getting told why a bunch of things that in my opinion don't actually matter - because the point to me is the discrimination.
It was less the problem solving and more the dismissive "what can you do" that I disliked.
I don't think it's dismissive, at least for the most part. I'm also to a degree in "what can you do" camp (though not dismissive of the issue), because, well, the problem of bias in AI is really fucking hard. You could not publish biased models, in which case biased models will simply eat your lunch. You could work towards reducing bias, which many of the leading actors are working on, but it's either ridiculously expensive or slow-going basic research. Smarter people than most of us here are working hard on these problems. There's no quick fix, and a slow fix won't actually affect the market in any way until years down the line. Short of putting the genie back in the bottle (lol, as if), there's not much that actually can be done by any org or person that isn't already happening. (Well, orgs or persons that are generally empathetic to issues of discrimination and bias.) Sure, more people working on fairness rather than chasing another fraction of a percentage of benchmark scores would help, but like putting the genie back in the bottle, that's not a thing individual actors can meaningfully affect.
In short: Let 'em cook. They're working on it. In the meantime, there's nothing much we can do. Ignore AI until they sort it, or work around the existing issues. Oh, and to not ignore the obvious: Make sure that fairness stays near the top of the collective ToDo list of the AI community.
Talking about it is a way of ensuring that fairness stays to the forefront.
The other angle is gonna end up being putting pressure on companies and institutions that start looking to AI to get them to quit paying for it if all it gives are biased outputs. And to address actual harm done - like the disordered eating organization whose chatbot gave information on calorie restriction.
Because if the response to anything we don't have direct control about is shrug let them work on it, they'll get there some day but you just gotta assume you'll get (bad outcome) til then then half the conversations on Tildes should go away.
You've had some really thoughtful responses and I understand you're not trying to be dismissive. But I want to expand on this "Let 'em cook" mindset from a female perspective.
Those words scare me to be frank because every time we're told to wait, that it will get better, it doesn't. We saw this with Trump's election and fears of Roe being overturned we're dismissed as being too out there, just let him work. Turns out those fears had founding and our reproductive rights stay under attack. We are always told to wait and see, and it lately it hasn't been going well.
I know the cart us ahead of the horse at this point with AI, but it is so exhausting to be told repeatedly to just wait instead of never being considered from the beginning. Again, why is that the default?
Because the way AI is right now is kind of a natural outcome of the kinds of freely available data out there and the way we do AI. And I can already hear calls to "find better data" and "figure out a better way", and both of those are happening, they're just glacially slow compared to the "move fast and break things" method. The status quo is unfair not because people like unfairness, but because the unfairness is a natural consequence of the most efficient(*) known way of building AI.
So? What do? Well, if you're not actively doing research on this, support those actors that are doing it right. Commercial and non-commercial, there's people out there doing the research, collecting data, training and releasing models that are worse(again, natural consequence), but fairer. Find them. Talk about them. If they have usable products, use their shit, even if it might suck a bit more. That's the kind of feedback that will be heard. If people ignore shitty move-fast-and-break-things startups and instead give all the spot light to those that take the time to get it right, that's I think the one signal you can reasonably send "upwards" that will actually make these people cook faster (or more realistically, more cooks with more expensive spices).
I guess what I'm saying with "let em cook" is that it's slow to make AI fair, but I can assure you that lots of people are working on it; the common opinion that AI bros are too out of touch to care is wrong to a substantial degree. So advocacy in that direction is ineffective. But because they're inherently at a disadvantage, these cooks need time to catch up, so the fact that they're not visible to a larger extent doesn't mean they don't exist. I don't mean to call for everyone to sit back down and do nothing.
Keep in mind that I think fair AI will, unless or until things change, always be a bit on the back foot. Trying to be fair is just a performance hit one way or another, at least for the time being. For some applications, we don't care about the last percentage point of benchmark scores, so we accept fairness over scores; those are the applications where fair AI has a reasonable chance. Or where for legal reasons, fairness is a requirement, e.g. anti-discrimination laws for hiring decisions. But I don't want to give off the impression that just time and a bit more research will fix AI in all domains, if the requisite awareness isn't retained, or where it doesn't exist, established.
FWIW, I don't think Trump/RoeVsWade was at all a "let him cook" situation. My take on that guy was a tad more radical and hands-on.
(*) most possible definitions of efficiency, I'd argue, lead to the same result here. I.e. if efficiency = success / cost, then counting cost as time or money, and success as money or market share or benchmark scores, doesn't really matter. Curating better datasets or building inherently less biased learning algorithms take too long and are too expensive to compete.
Oh absolutely 100% agree on that
Exactly this. I've played around a bunch with local models like StableFusion and "online" stuff like Midjourney.
There's a huge bias on hair colour and race depending on activity. You can see the Matrix behind the images when you push the limits. Every image of "woman model" is vaguely eastern European, because there is a TON of training material of women from that area.
Now try to generate a redhead for example and you'll see the style switch immediately.
And the best bit is that Midjourney's own NSFW detector thinks blondes are too sexy, but the exact same prompt but "redhead" appended is just fine ¯\_(ツ)_/¯
AI can't create things it hasn't seen. Two of the best ones I've found out is "<name of large city> without cars" and "woman lying down after workout". There just aren't photos of either so the AI doesn't know what to do. It tries, but fails: https://imgur.com/eD1HVWb
Sure, but the prompts you given it aren't example of that. Google images for "a city without cars" and you get multiple examples of cities without cars, mixing concepts like "a city without cars" and "new york city" isn't hard for the model either, this is why the "avocado chair" example astonished people when these models were first coming out. Googling "woman lying down after workout" also wields plenty of results, so I don't know why you think the models have never seen that.
You're just misunderstanding how it works. The text to image model you used doesn't use an LLM as the backend for the text portion, so it's understanding of text in more akin to word salad. the vectors for the prompt "a city without cars" and "a city with cars" are virtually the same. There's a paper explaining this phenomena in the CLIP text encoder if you want to know more.
If you want your results you need to be more creative with your prompting, try "new york with empty roads" and you'll have more luck, putting "car" in the negative prompt also helps. My GPU will be busy for the next couple days so I can't show you SDXL results, but here's what novelai can do (SDXL based anime model).
That Imgur link is actually somewhat nightmare inducing lmao. Do you have one for the "city with no cars" prompt?
Their text interpretation models can't handle negatives like "without cars" in the prompt, you'd have put cars in the negative prompt or what describe a city like like without cars like "city with walkable streets, people walking, cycling, street markets and cafes" etc
If you go for the walkable streets one, it'll give you a city in Europe with buildings like that. Or one of the very few such places in the US.
It can't give you a picture of New York city streets without cars for example.
Interesting trying this out with Dall-E. Only the last of these prompts actually generated two images like it was supposed to; every other prompt generated at most one image. My first attempt was "a woman lying down after working out." That failed. These are the ones I tried that worked:
A man lying down after working out
A woman lying down after working out ... this one seems to be a man in a feminine pose?
A woman in exercise clothes
A promotional photo for an organization that provides a space for women to lie down after exercising
My bad. Should be fixed. It's one of those pictures that just gets creepier the more you look at it.
This is a very interesting topic, and it seems to paradoxically lead to the conclusion that less dataset filtering may lead to less bias.
This is not true, or rather it's phrased in a misleading way. The issue is that with some current models, due to the way they are trained and strongly guided towards a certain style of photorealism (they collected user input during the training process specifically for that), certain concepts are too strongly associated together, so the problem is more accurately described as "things it has seen too much of trump everything else".
I don't know how much Midjourney suffers from this (don't use it), but it's a big issue with SDXL and Dall-E, they are strongly guided to create one particular style of semi-realistic stock-photo like style and it's quite difficult to wrangle them into doing something else, like even a cheap camera street photography style. This also guides them towards creating "true" stereotypical things more often.
SDXL cannot seem to do NYC streets without cars at all, but it can usually sort of do Prague streets without cars (I also tried "2-lane roads" instead of "streets", to push it away from just creating a pedestrian-only street).
However! If you load up Stable Diffusion 1.5, an older version which was trained on a dataset with very little filtering and without using user input for guidance, it can do NYC without cars easily, let alone Prague. It just looks less photorealistic. The 4 NYC images are 1st attempt, no selection. It also created a woman lying down after workout, but that one I had to select out of like 6 attempts because v 1.5 is bad with people in any position but standing, and anatomy in general, and to get to photorealism I'd have to use some of the existing workflows to fix the face - the raw output of older models is just bad with faces.
Dall-E 3 can kind of almost get there, not on the first try, but if you then tell it "there are still cars and taxis on this image, I want no cars at all", it spits out this on the second attempt. You could probably get to no cars if you repeated the demand, Dall-E is usually receptive to that, but it may gradually become even less photorealistic in the process.
I think a key part of generative AI that we're just figuring is how to get a model to sample how we want, when we have trained it on a lot of things we may not want. Basically, learn some aspects of the data, but don't actually generate samples like it, only use it to affect other generations lightly. As an example of that, if I'm not mistaken, chatGPT was fine tuned in a relatively small dataset of high-quality helpful interactions. This fine tuning is most of what makes chatGPT helpful and keeps it from generating text that is as helpful as your average internet troll. Leverage that a bit more, and you might be able to train on completely uncurated datasets of portraits and actually generate dressed asian women.
But my examples seem to show the opposite is true with image generation, at least in the way it's done now. I believe that the kind of "airbrushed realistic" look that refuses to stray into more creative territories like NYC without cars was created mainly using guidance based on users manually ranking the output, though filtering surely played a role too.
And, just like with ChatGPT, it seems to show that too much guidance for safety makes the outputs less useful. ChatGPT was most creative just after it was released, and I don't think it had any downsides for legitimate users either, it was just easier to abuse it for things that either were bad or just would look bad in media. But at least it still works well for many things.
I'm not saying that we're just going to have to deal with having strongly biased models, but these current processes that kind of work for LLMs don't really seem to work for diffusion models well. There's demand for content filtering and moderation, but I hope that some of the competitors create a better way to do it that would retain creativity.
Something else that may factor in is an abundance of training material of that type produced by risqué publications, which at least in Japan (haven’t been to other east asian countries and can’t speak for them) are numerous and popular, appearing in the magazine shelves of just about every convenience store in the country.
The west of course also has such publications but they don’t strike me as being nearly as prolific, ubiquitous, or socially accepted.
...
So, informing people of the risks of the system along with taking steps to reduce a problem when it arises and having rules in place is... Not sexy?
I dislike the tone of the article but that doesn't necessarily invalidate any claims in it...
...
So, the article makes the point that people are developing these algorithms by scraping the Internet and that the people developing these systems are trying to reduce the amount of potentially unwanted sexualization but makes the claim that they're not doing anything (or enough)? I can't follow.
AI overrepresents based on stereotypes and the article correctly points out the over sexualization of women online - but also calls out the developer for not doing enough to fix it. Other than making changes to their algorithm, which they claim to have already introduced filters for that, what more can they do? I don't see a link to a 'pictures of real women™' archive that they can use to retrain their model on.
They could do more of the same: better sources of images and better curation. Maybe pay for images from higher-quality collections?
I expect that machine learning will we be useful to help the image collectors do the curation part. This will require better understanding of what they want in their training sets, though, what's over-represented and what they're missing that customers want.
There's been a gold rush, but hopefully some of these companies will work harder on quality and it will improve.
Am I missing something, or does she not say what model she was using in the article?
It's Lensa's LAION 5B dataset using Stable Diffusion
Which is strange, because if I understand the Lensa's FAQ correctly, they're using Stable Diffusion V2, which famously did an effort to purge all NSFW data from the training dataset so that it couldn't be used to create any nude people, to the point that it was worse at doing anatomy in general than the previous version. That would imply that what was happening may have been caused by whatever additional training Lensa did for their app and not by Stable Diffusion itself.
It's not strange, they used Stable Diffusion for the AI model, not the dataset. They used the LAION 5B dataset, which hasn't done the nudity filtering you mention.
LAION 5B was used to train Stable Diffusion v2.
They certainly did not use unfiltered LAION 5B for the finetuning, because firstly that wouldn't make a lot of sense for such a specialized model, and secondly the price would be exorbitant. They may have filtered out useful data from it for finetuning, but to me their FAQ reads more like they're talking about SD itself and understandably keeping their internal processes secret.