— Google, Meta, and others test their algorithms for bias using standardized skin tone scales. Sony says those tools ignore the yellow and red hues at work in human skin color.
Paresh Dave / Wired:
After evidence surfaced in 2018 that leading face-analysis algorithms were less accurate for people with darker skin, companies including Google and Meta adopted measures of skin tone to test the effectiveness of their AI software.
New research from Sony suggests that those tests are blind to a crucial aspect of the diversity of human skin color.
By expressing skin tone using only a sliding scale from lightest to darkest or white to black, today’s common measures ignore the contribution of yellow and red hues to the range of human skin, according to Sony researchers. They found that generative AI systems, image-cropping algorithms, and photo analysis tools all struggle with yellower skin in particular. The same weakness could apply to a variety of technologies whose accuracy is proven to be affected by skin color, such as AI software for face recognition, body tracking, and deepfake detection, or gadgets like heart rate monitors and motion detectors.
“If products are just being evaluated in this very one-dimensional way, there’s plenty of biases that will go undetected and unmitigated,” says Alice Xiang, lead research scientist and global head of AI Ethics at Sony. “Our hope is that the work that we’re doing here can help replace some of the existing skin tone scales that really just focus on light versus dark.”
But not everyone is so sure that existing options are insufficient for grading AI systems. Ellis Monk, a Harvard University sociologist, says a palette of 10 skin tones offering light to dark options that he introduced alongside Google last year isn’t unidimensional.
“I must admit being a bit puzzled by the claim that prior research in this area ignored undertones and hue,” says Monk, whose Monk Skin Tone scale Google makes available for others to use.
“Research was dedicated to deciding which undertones to prioritize along the scale and at which points.”
He chose the 10 skin tones on his scale based on his own studies of colorism and after consulting with other experts and people from underrepresented communities.
X. Eyeé, CEO of AI ethics consultancy Malo Santo and who previously founded Google’s skin tone research team, says the Monk scale was never intended as a final solution and calls Sony’s work important progress. But Eyeé also cautions that camera positioning affects the CIELAB color values in an image, one of several issues that make the standard a potentially unreliable reference point.
“Before we turn on skin hue measurement in real-world AI algorithms—like camera filters and video conferencing—more work to ensure consistent measurement is needed,” Eyeé says.
Read more at links:
https://www.wired.com/story/ai-algorithms-are-biased-against-skin-with-yellow-hues/
— Techmeme