I got it in one via DALL·E using the prompt of "Generate a picture of a Black, African doctor treating a white kid please." No racist overtones, no African tribal art, all placed in a modern...
I got it in one via DALL·E using the prompt of "Generate a picture of a Black, African doctor treating a white kid please."
No racist overtones, no African tribal art, all placed in a modern setting.
That's nice to hear. Though I think the point of this article is the inherent biases these algorithms carry because of the chosen data sets, manual constraints set by the developers and so on....
That's nice to hear. Though I think the point of this article is the inherent biases these algorithms carry because of the chosen data sets, manual constraints set by the developers and so on. Lots of folk seem to think AIs produce the objective truth when they absolutely do not.
No doubt this "bug" will be fixed but the question is when the "bug" isn't as blatant as it is in this case, will the user notice or even think to question the answers they receive?
Right, and those folks are wrong for reasons that aren't particularly limited to bias in the dataset. Racial bias in AI generation is, at this point, a known issue, a well-understood one, and...
Lots of folk seem to think AIs produce the objective truth when they absolutely do not.
Right, and those folks are wrong for reasons that aren't particularly limited to bias in the dataset.
Racial bias in AI generation is, at this point, a known issue, a well-understood one, and these articles coming out left and right "Look how I made it do another racism" are IMO ragebait.
I'd take the article more seriously if it was trying to inform people about the limitations and how to work around them / when to use vs. not to use, instead of begging for clicks.
But that's bog-standard poor science reporting. Main stream media is pretty terrible at any kind of science reporting, partly because they tend to stick to press releases rather than doing the...
But that's bog-standard poor science reporting. Main stream media is pretty terrible at any kind of science reporting, partly because they tend to stick to press releases rather than doing the hard work of reading and understanding the papers.
The actual journal article has a reasonable premise: where does the racism get introduced?
The Health Policy paper of Esmita Charani and colleagues,1
showed how stereotypical global health tropes (such as the so-called suffering subject and white saviour) can be perpetuated through the images chosen to illustrate publications on global health. We propose that generative artificial intelligence (AI), which uses real images as a basis for learning, might further serve to show how deeply embedded the existing tropes and prejudices are within global health images. This in turn can perpetuate oversimplified social categorisations and power imbalances.
[...]
In summary, we were unable to achieve our initial goal of inverting stereotypical global health images, and instead we unwittingly created hundreds of visuals representing white saviour and Black suffering tropes and gendered stereotypes; these images were created despite the AI developers' stated commitment to ensure non-abusive depictions of people, their cultures, and communities.2 This case study suggests, yet again, that global health images should be understood as political agents,1, 3 and that racism, sexism, and coloniality are embedded social processes manifesting in everyday scenarios, including AI.4
People may say "yes, but it's the prompts" -- but that's the point. Most people can't use Google, don't understand Wikipedia, and now we're asking them to perfect their AI prompts to avoid nonsense results, and blaming them when they do get nonsense results.
Another point to make:
in further probing the rendering bias, we discovered that AI couples HIV status with Blackness. Nearly all rendered patients for an HIV patient receiving care prompt (150 of 152) were rendered Black (figure 8).
Is this a reflection of poor access to HIV health care in the global south, and attempts by humanitarian orgs to get attention? HIV was, for many years, devastating to Africa.
African Region: An estimated 25.6 million [21.6–30.0 million] people were living with HIV in 2022, of which 90% [76 to >98%] knew their status, 82% [69–96%] were receiving treatment and 76% [64–89%] had suppressed viral loads. An estimated 20.9 million people were receiving antiretroviral therapy in 2022.
Region of the Americas An estimated 3.8 million [3.4–4.3 million] people were living with HIV in 2022, of which 86% [76–97%] knew their status, 71% [62–79%] were receiving treatment and 65% [57–73%] had suppressed viral loads. An estimated 2.7 million people were receiving antiretroviral therapy in 2022.
South-East Asian Region An estimated 3.9 million [3.4–4.6 million] people were living with HIV in 2022, of which 81% [70–94%] knew their status, 65% [57–76%] were receiving treatment and 61% [53–71%] had suppressed viral loads. An estimated 2.6 million people were receiving antiretroviral therapy in 2022.
European Region An estimated 3.0 million [2.6–3.3 million] people were living with HIV in 2022, of which 72% [64–80%] knew their status, 63% [55–70%] were receiving treatment and 60% [53–67%] had suppressed viral loads. An estimated 1.9 million people were receiving antiretroviral therapy in 2022.
Eastern Mediterranean Region An estimated 490 000 [420 000–600 000] people were living with HIV in 2022, of which 38% [33–47%] knew their status, 27% [23–33%] were receiving treatment and 24% [21–29%] had suppressed viral loads. An estimated 130 000 people were receiving antiretroviral therapy in 2022.
Western Pacific Region An estimated 2.2 million [1.7–2.8 million] people were living with HIV in 2022, of which 81% [60 to >98%] knew their status, 73% [55–91%] were receiving treatment and 70% [53–88%] had suppressed viral loads. An estimated 1.6 million people were receiving antiretroviral therapy in 2022.
Even at their best, journalists produce articles that they hope will be relevant to their audience. At the time this article was researched and written, ai biases were likely still an emergent...
ragebait
Even at their best, journalists produce articles that they hope will be relevant to their audience. At the time this article was researched and written, ai biases were likely still an emergent issue largely unknown to broad sectors of nor’s audience.
Additionally, the full scope of bias is probably not fully known or understood among experts.
For those of us trying now, I’d wager the “bug” has been identified already and somehow “remedied.”
A bigger issue for me is the balance between limiting exposure and utility of the tools. Already i can spot the guardrails put in place on the free tools, and how it takes away interesting useful answers. I get a lot of responses now that say things like “you should ask a doctor” or “you should consult the trade journals” without any useful information.
I'm neither black nor African so I can't speak how it feels to see this kind of racial bias highlighted from their POV but I do agree this kinda dips into the ragebait category. Could've been...
I'm neither black nor African so I can't speak how it feels to see this kind of racial bias highlighted from their POV but I do agree this kinda dips into the ragebait category. Could've been fleshed out a lot more.
Right. I'm thinking about the metric fuckton of research that's being done to address this issue. There's a lot going on there, and I think some appreciation for the hard and currently largely...
Right. I'm thinking about the metric fuckton of research that's being done to address this issue. There's a lot going on there, and I think some appreciation for the hard and currently largely thankless and underfunded work of making AI fairer is in order.
Yeah, but it does no one any good if they overstate the bias. For example, I just asked craiyon (which is ... not that great) to give me a black african doctor caring for a white child, and the...
Yeah, but it does no one any good if they overstate the bias.
For example, I just asked craiyon (which is ... not that great) to give me a black african doctor caring for a white child, and the first result was a black woman in scrubs holding a white baby. So now I'm doubting the validity of the statements in the article, and for sure everyone who repeats the experiment successfully will do the same. Luckily, I'm someone who's well aware that data biases are real, but a lot of people would see this as proof that the original article contains false claims and conclude that the rest of the article's points are equally false.
Yep, it sure would've been nice if they used the same prompts in different AI programs and compared. Would've been less rage-bait-y and still created discussion. Less clicks maybe though? I'm not...
Yep, it sure would've been nice if they used the same prompts in different AI programs and compared. Would've been less rage-bait-y and still created discussion.
Less clicks maybe though? I'm not a journalist so I don't know what kind of standards, budget and goals they have for these.
I think that might be true for text but not images. If you're asking it to generate a picture, it's never going to be a real photo, so I don't know what "objective truth" even means here. It's...
I think that might be true for text but not images. If you're asking it to generate a picture, it's never going to be a real photo, so I don't know what "objective truth" even means here. It's very, very obvious that it's not real when sometimes you get people with extra fingers or whatever.
The damage seems limited since people look at the resulting pictures and reject anything they don't like. It's common that it takes multiple tries and sometimes you can't get what you want at all. This is a frustrating waste of time, but not otherwise harmful.
The bigger picture is that users themselves have biases and they might be using (or misusing) the tool for bad purposes. And then maybe they share it and memes go viral for bad reasons? But the same is true of someone using a paint program.
It would be interesting to see a study of the images that real users decide to share. I imagine the proportion of anime women and fantasy landscapes would be rather high, and practically nobody's using it to generate pictures of doctors treating children.
It's interesting and fun to see what you can get these image generators to do, though, and researchers will do a more systematic job of it than people just trying it out casually.
If a prompt like "attractive people" gives results only of white people someone will absolutely use that as ammunition to prop their fucked up worldview. In any case I meant that this is part of...
If a prompt like "attractive people" gives results only of white people someone will absolutely use that as ammunition to prop their fucked up worldview.
In any case I meant that this is part of the bigger conversation about AIs as a whole and pictures are a good way to highlight the problems these systems have. It's a lot more boring to look at a wall of text than a giraffe in a clinic.
And so what if they did? Yes, there are racist trolls with Internet access. From a software design perspective it's not even worth thinking about them. Trolls will troll no matter what you do. But...
And so what if they did? Yes, there are racist trolls with Internet access. From a software design perspective it's not even worth thinking about them. Trolls will troll no matter what you do.
But more likely, it would be used as a gotcha to make the AI service look bad, which is embarrassing, so they'll fix it for that reason.
You asked what the misconception of AIs' "objective truth" has to do with AIs that generate images, I gave you an example. But like I tried to convey, that isn't the argument that's worth focusing...
You asked what the misconception of AIs' "objective truth" has to do with AIs that generate images, I gave you an example. But like I tried to convey, that isn't the argument that's worth focusing on here in my opinion.
This is the real issue that we need to focus on, because it's going to bring much bigger problems than some people justifying their racism, and even if it's possible to remove some biases, making...
Lots of folk seem to think AIs produce the objective truth when they absolutely do not.
This is the real issue that we need to focus on, because it's going to bring much bigger problems than some people justifying their racism, and even if it's possible to remove some biases, making an unbiased AI is impossible.
However I'm mainly replying because I think the problem mentioned in the article is not only caused by racial bias in the dataset, but largely also because the models themselves are not smart enough, they don't understand the prompts that well. If you don't understand what the user wants you to generate and seeing the word "african" makes you go "ah, tribes and giraffes!", it's not necessarily wrong, it just not really understanding the task well enough.
Dall-E 3 being better at this task confirms that - it's not necessarily better at generating images, but it's considerably better at understanding text.
The hardest thing is going to be to teach people on is “AI” is a language/image model not a knowledge model. It doesn’t know anything, yet. It’s just really really fancy autocomplete. So if you...
The hardest thing is going to be to teach people on is “AI” is a language/image model not a knowledge model. It doesn’t know anything, yet. It’s just really really fancy autocomplete. So if you ask it “whats 1+1?” Or “draw a ball” it goes through its data and answers, “The most probable answer is 2 because most of the internet has 2 following that phrase” or “heres something that looks like a ball from the internet where people said ‘hey heres a ball’ in pictures”. It does not know what 1+1 is and if you ask it something that they internet doesn’t have a lot of data to autocomplete to, it’s going to get it wrong.
The autocomplete thing isn't really true anymore. And it's evident when you compare a truly purely autoregressive token predictor to the instruction tuned models we have today. Just play around...
The autocomplete thing isn't really true anymore. And it's evident when you compare a truly purely autoregressive token predictor to the instruction tuned models we have today. Just play around with GPT-3 or GPT-2 - now that's autocomplete. For the tuned ones, after many iterations of instruction fine tuning and RLHF, it is not just the most likely next token - but the most likely next token to please humans.
After all, does it make any sense that the most likely token after a question asking anything about sex or race to be "Hi I am an AI model please stop talking about this topic" if it were merely the most likely next token in the dataset?
but that's exactly what this criticism is about: that current "AI" isn't really intelligently giving us what we ask -- that it's just regurgitating data sets we fed it. It's like yet another...
but that's exactly what this criticism is about: that current "AI" isn't really intelligently giving us what we ask -- that it's just regurgitating data sets we fed it.
It's like yet another person from the crowd point out Clever Hans isn't actually doing math, it's just responding to the cues its human is providing.
And this chant, though worn out for you and I, *needs to be rechanted again and again until everyone knows it. Not until you and I know: until middle managers, until TV informercials, until 10 year olds playing Mad Libs stop using 2023 gen AI as a buzzword.
LLMs are very exciting in their own right. But they're still not Rosie the Robot or Doraemon or Marvin. Not yet. And yet another person shouting the emperor has no clothes is appropriate until the parade stops and the emperor puts its clothes back on.
This article is from the beginning of October. Wouldn't it be possible for there to have been an improvement between now and then? Like palimpsest said, people will take being able to make the...
This article is from the beginning of October. Wouldn't it be possible for there to have been an improvement between now and then? Like palimpsest said, people will take being able to make the prompt successfully without the bias presented in the article as a fact that the article was totally infactual and the bias is made up. I'd argue that the article could be simply outdated at this time. With how fast everything has been moving, maybe some of these issues have been able to be addressed.
But also, the point of the article isn't "We couldn't make a black doctor and a white child without bullshit." it's that the data LLMs have been trained on have inherit biases, because people have biases. If you ask an image generator to make 100 attractive people, what will the outcome be? 100 men? 100 women? Will they have similar features? Will they be primarily white? Will the default of attractive be a healthy mix of people from all over the place? Data has bias and it can be reflected with image generation.
I got it in one via DALL·E using the prompt of "Generate a picture of a Black, African doctor treating a white kid please."
No racist overtones, no African tribal art, all placed in a modern setting.
Image in question: https://cdn.discordapp.com/attachments/878223980983631883/1180457805073895455/file-uw9WIxiKjwhH5WbCwGOS3ATt.png
and yes, I know they used Midjourney.
That's nice to hear. Though I think the point of this article is the inherent biases these algorithms carry because of the chosen data sets, manual constraints set by the developers and so on. Lots of folk seem to think AIs produce the objective truth when they absolutely do not.
No doubt this "bug" will be fixed but the question is when the "bug" isn't as blatant as it is in this case, will the user notice or even think to question the answers they receive?
Right, and those folks are wrong for reasons that aren't particularly limited to bias in the dataset.
Racial bias in AI generation is, at this point, a known issue, a well-understood one, and these articles coming out left and right "Look how I made it do another racism" are IMO ragebait.
I'd take the article more seriously if it was trying to inform people about the limitations and how to work around them / when to use vs. not to use, instead of begging for clicks.
But that's bog-standard poor science reporting. Main stream media is pretty terrible at any kind of science reporting, partly because they tend to stick to press releases rather than doing the hard work of reading and understanding the papers.
The actual journal article has a reasonable premise: where does the racism get introduced?
[...]
People may say "yes, but it's the prompts" -- but that's the point. Most people can't use Google, don't understand Wikipedia, and now we're asking them to perfect their AI prompts to avoid nonsense results, and blaming them when they do get nonsense results.
Another point to make:
Is this a reflection of poor access to HIV health care in the global south, and attempts by humanitarian orgs to get attention? HIV was, for many years, devastating to Africa.
Here's WHO data: https://www.who.int/teams/global-hiv-hepatitis-and-stis-programmes/hiv/strategic-information/hiv-data-and-statistics
African Region: An estimated 25.6 million [21.6–30.0 million] people were living with HIV in 2022, of which 90% [76 to >98%] knew their status, 82% [69–96%] were receiving treatment and 76% [64–89%] had suppressed viral loads. An estimated 20.9 million people were receiving antiretroviral therapy in 2022.
Region of the Americas An estimated 3.8 million [3.4–4.3 million] people were living with HIV in 2022, of which 86% [76–97%] knew their status, 71% [62–79%] were receiving treatment and 65% [57–73%] had suppressed viral loads. An estimated 2.7 million people were receiving antiretroviral therapy in 2022.
South-East Asian Region An estimated 3.9 million [3.4–4.6 million] people were living with HIV in 2022, of which 81% [70–94%] knew their status, 65% [57–76%] were receiving treatment and 61% [53–71%] had suppressed viral loads. An estimated 2.6 million people were receiving antiretroviral therapy in 2022.
European Region An estimated 3.0 million [2.6–3.3 million] people were living with HIV in 2022, of which 72% [64–80%] knew their status, 63% [55–70%] were receiving treatment and 60% [53–67%] had suppressed viral loads. An estimated 1.9 million people were receiving antiretroviral therapy in 2022.
Eastern Mediterranean Region An estimated 490 000 [420 000–600 000] people were living with HIV in 2022, of which 38% [33–47%] knew their status, 27% [23–33%] were receiving treatment and 24% [21–29%] had suppressed viral loads. An estimated 130 000 people were receiving antiretroviral therapy in 2022.
Western Pacific Region An estimated 2.2 million [1.7–2.8 million] people were living with HIV in 2022, of which 81% [60 to >98%] knew their status, 73% [55–91%] were receiving treatment and 70% [53–88%] had suppressed viral loads. An estimated 1.6 million people were receiving antiretroviral therapy in 2022.
Even at their best, journalists produce articles that they hope will be relevant to their audience. At the time this article was researched and written, ai biases were likely still an emergent issue largely unknown to broad sectors of nor’s audience.
Additionally, the full scope of bias is probably not fully known or understood among experts.
For those of us trying now, I’d wager the “bug” has been identified already and somehow “remedied.”
A bigger issue for me is the balance between limiting exposure and utility of the tools. Already i can spot the guardrails put in place on the free tools, and how it takes away interesting useful answers. I get a lot of responses now that say things like “you should ask a doctor” or “you should consult the trade journals” without any useful information.
I'm neither black nor African so I can't speak how it feels to see this kind of racial bias highlighted from their POV but I do agree this kinda dips into the ragebait category. Could've been fleshed out a lot more.
Right. I'm thinking about the metric fuckton of research that's being done to address this issue. There's a lot going on there, and I think some appreciation for the hard and currently largely thankless and underfunded work of making AI fairer is in order.
Yeah, but it does no one any good if they overstate the bias.
For example, I just asked craiyon (which is ... not that great) to give me a black african doctor caring for a white child, and the first result was a black woman in scrubs holding a white baby. So now I'm doubting the validity of the statements in the article, and for sure everyone who repeats the experiment successfully will do the same. Luckily, I'm someone who's well aware that data biases are real, but a lot of people would see this as proof that the original article contains false claims and conclude that the rest of the article's points are equally false.
Yep, it sure would've been nice if they used the same prompts in different AI programs and compared. Would've been less rage-bait-y and still created discussion.
Less clicks maybe though? I'm not a journalist so I don't know what kind of standards, budget and goals they have for these.
I think that might be true for text but not images. If you're asking it to generate a picture, it's never going to be a real photo, so I don't know what "objective truth" even means here. It's very, very obvious that it's not real when sometimes you get people with extra fingers or whatever.
The damage seems limited since people look at the resulting pictures and reject anything they don't like. It's common that it takes multiple tries and sometimes you can't get what you want at all. This is a frustrating waste of time, but not otherwise harmful.
The bigger picture is that users themselves have biases and they might be using (or misusing) the tool for bad purposes. And then maybe they share it and memes go viral for bad reasons? But the same is true of someone using a paint program.
It would be interesting to see a study of the images that real users decide to share. I imagine the proportion of anime women and fantasy landscapes would be rather high, and practically nobody's using it to generate pictures of doctors treating children.
It's interesting and fun to see what you can get these image generators to do, though, and researchers will do a more systematic job of it than people just trying it out casually.
If a prompt like "attractive people" gives results only of white people someone will absolutely use that as ammunition to prop their fucked up worldview.
In any case I meant that this is part of the bigger conversation about AIs as a whole and pictures are a good way to highlight the problems these systems have. It's a lot more boring to look at a wall of text than a giraffe in a clinic.
And so what if they did? Yes, there are racist trolls with Internet access. From a software design perspective it's not even worth thinking about them. Trolls will troll no matter what you do.
But more likely, it would be used as a gotcha to make the AI service look bad, which is embarrassing, so they'll fix it for that reason.
You asked what the misconception of AIs' "objective truth" has to do with AIs that generate images, I gave you an example. But like I tried to convey, that isn't the argument that's worth focusing on here in my opinion.
This is the real issue that we need to focus on, because it's going to bring much bigger problems than some people justifying their racism, and even if it's possible to remove some biases, making an unbiased AI is impossible.
However I'm mainly replying because I think the problem mentioned in the article is not only caused by racial bias in the dataset, but largely also because the models themselves are not smart enough, they don't understand the prompts that well. If you don't understand what the user wants you to generate and seeing the word "african" makes you go "ah, tribes and giraffes!", it's not necessarily wrong, it just not really understanding the task well enough.
Dall-E 3 being better at this task confirms that - it's not necessarily better at generating images, but it's considerably better at understanding text.
It worked in one go because you said "please". No, really. Maybe. Probably not. But maybe. https://arstechnica.com/information-technology/2023/09/telling-ai-model-to-take-a-deep-breath-causes-math-scores-to-soar-in-study/
Seriously though, I keep getting black+black with Stable Diffusion XL, 1.5, and 2.1, and with DALL-E 2. So you may have been just lucky.
All seriousness aside, that picture with the giraffe is really funny out of context. It's like he's a valuable member of the team.
I like the kid with a wire in his mouth and also the one white hand and/or croissant on the table
The hardest thing is going to be to teach people on is “AI” is a language/image model not a knowledge model. It doesn’t know anything, yet. It’s just really really fancy autocomplete. So if you ask it “whats 1+1?” Or “draw a ball” it goes through its data and answers, “The most probable answer is 2 because most of the internet has 2 following that phrase” or “heres something that looks like a ball from the internet where people said ‘hey heres a ball’ in pictures”. It does not know what 1+1 is and if you ask it something that they internet doesn’t have a lot of data to autocomplete to, it’s going to get it wrong.
The autocomplete thing isn't really true anymore. And it's evident when you compare a truly purely autoregressive token predictor to the instruction tuned models we have today. Just play around with GPT-3 or GPT-2 - now that's autocomplete. For the tuned ones, after many iterations of instruction fine tuning and RLHF, it is not just the most likely next token - but the most likely next token to please humans.
After all, does it make any sense that the most likely token after a question asking anything about sex or race to be "Hi I am an AI model please stop talking about this topic" if it were merely the most likely next token in the dataset?
but that's exactly what this criticism is about: that current "AI" isn't really intelligently giving us what we ask -- that it's just regurgitating data sets we fed it.
It's like yet another person from the crowd point out Clever Hans isn't actually doing math, it's just responding to the cues its human is providing.
And this chant, though worn out for you and I, *needs to be rechanted again and again until everyone knows it. Not until you and I know: until middle managers, until TV informercials, until 10 year olds playing Mad Libs stop using 2023 gen AI as a buzzword.
LLMs are very exciting in their own right. But they're still not Rosie the Robot or Doraemon or Marvin. Not yet. And yet another person shouting the emperor has no clothes is appropriate until the parade stops and the emperor puts its clothes back on.
This article is from the beginning of October. Wouldn't it be possible for there to have been an improvement between now and then? Like palimpsest said, people will take being able to make the prompt successfully without the bias presented in the article as a fact that the article was totally infactual and the bias is made up. I'd argue that the article could be simply outdated at this time. With how fast everything has been moving, maybe some of these issues have been able to be addressed.
But also, the point of the article isn't "We couldn't make a black doctor and a white child without bullshit." it's that the data LLMs have been trained on have inherit biases, because people have biases. If you ask an image generator to make 100 attractive people, what will the outcome be? 100 men? 100 women? Will they have similar features? Will they be primarily white? Will the default of attractive be a healthy mix of people from all over the place? Data has bias and it can be reflected with image generation.