Google stops Gemini AI chatbot from generating images of people after inaccuracies in race depiction

Gemini users this week posted screenshots on social media of historically white-dominated scenes with racially diverse characters that they say it generated, leading critics to raise questions about whether the company is overcorrecting for the risk of racial bias in its AI model.

“We’re already working to address recent issues with Gemini’s image generation feature,” Google said in a post on the social media platform X. “While we do this, we’re going to pause the image generation of people and will re-release an improved version soon.”

Previous studies have shown AI image-generators can amplify racial and gender stereotypes found in their training data, and without filters are more likely to show lighter-skinned men when asked to generate a person in various contexts.

Google said on Wednesday that it is “aware that Gemini is offering inaccuracies in some historical image generation depictions” and that it is “working to improve these kinds of depictions immediately”.

Gemini can generate a “wide range of people”, which the company said is “generally a good thing” because people around the world use the system but it is “missing the mark”.

University of Washington researcher Sourojit Ghosh, who has studied bias in AI image-generators, said he is in favour of Google pausing the generation of people’s faces but is a “little conflicted about how we got to this outcome”.

Contrary to claims of “white erasure” and the premise that Gemini refuses to generate faces of white people – ideas circulating on social media this week – Ghosh’s research has largely found the opposite.

“The rapidness of this response in the face of a lot of other literature and a lot of other research that has shown traditionally marginalised people being erased by models like this – I find a little difficult to square,” he said.

When the Associated Press asked Gemini to generate pictures of people, or even just a big crowd, it responded by saying it is “working to improve” the ability to do so. “We expect this feature to return soon and will notify you in release updates when it does,” the chatbot said.

Ghosh said it is likely that Google can find a way to filter responses to reflect the historical context of a user’s prompt, but solving the broader harms posed by image-generators built on generations of photos and artwork found on the internet requires more than a technical patch.

“You’re not going to overnight come up with a text-to-image generator that does not cause representational harm,” he said. “They are a reflection of the society in which we live.”