How can the UN's new advisory body on AI help drive gender equality?

New and fast-moving technologies such as generative artificial intelligence are already changing the way we work and learn, but they are also exacerbating existing inequalities and harmful stereotypes embedded in the data behind algorithms. While the technology holds tremendous potential to accelerate progress on the Sustainable Development Goals, bias in AI also risks widening the gender divide.

Gender bias in AI is amplified in different ways, including during an algorithm’s development process, in the training of datasets, and in AI-generated decision-making, according to the International Peace Institute’s Global Observatory.

“If an AI algorithm is trained on data that reflects traditional gender norms and stereotypes, its outputs may be biased and reinforce those same norms and stereotypes. This can then lead to discriminatory practices and decisions that negatively impact women and marginalised groups,” wrote Carlien Scheele, the director of the European Institute for Gender Equality, in an email to Devex.

But the problem is difficult to address and regulate. “Unless someone goes into these datasets and does careful vetting, auditing, and correction of these models, women and women-related concepts are often represented in a very stereotypical way,” said Abeba Birhane, a senior adviser in AI accountability at the Mozilla Foundation.

More efforts are now needed to improve the governance of AI — especially its impact on gender equality. These efforts include the European Union’s recent deal on the world’s first rules for AI with its proposed Artificial Intelligence Act, and the United Nations’ new high-level advisory body on artificial intelligence, of which Birhane is a member.

Launched by U.N. Secretary-General António Guterres in October 2023, the advisory body aims to improve international governance of AI and promote a better understanding of its risks and opportunities, including for gender equality. It will release a final report later this year that includes recommendations on governance, along with how to enhance accountability and “ensure an equitable voice for all countries,” among other issues.

For now, gender and digital rights experts are watching to see how gender will be incorporated into the group’s work and the resulting recommendations.

The ‘AI divide’

One of the risks identified so far by the advisory body is the discrimination or unfair treatment of subgroups, including on the basis of gender, for example through the stereotypical depiction of skills and professions often produced by generative AI. While searches for higher-paid jobs such as judges or doctors yield mostly images of men in those roles, searches for lower-paid jobs such as nursing or caregiving almost exclusively portray women.

The European Union’s law enforcement agency predicts that as much as 90% of content on the internet could be created or edited by AI by 2026, meaning the impact of bias in AI is only set to grow.

Marwa Azelmat, a digital rights expert at RNW Media — an NGO working to advance the rights of disadvantaged youth — and a frequent speaker on tech accountability and gender, recently tried to use ChatGPT in her own work. But when she asked the AI to generate an image to illustrate an article she had written on a gender project in China, it responded with sexualized and stereotyped images of Asian women.

“It made me think about how much effort we need to put into ChatGPT to make it gender-sensitive. … We have a digital divide, and now we have an AI divide,” Azelmat told Devex.

This gender bias within AI is already having a real-life impact. For example, AI used in hiring processes can lead to discriminatory practices if biased data is used. In some cases, AI has interpreted things like employment gaps due to maternity leave on a woman’s CV as something negative, Scheele explained.

Online abuse could also drive more women out of politics, as political candidates are increasingly targeted by online threats, harassment, and even deepfakes — videos of a person in which their face or body has been digitally altered so that they appear to be someone else.

Women in the political space are not only disproportionately targeted but also subjected to different forms of harassment, including comments on their physical appearance and threats of sexual violence, according to the Carnegie Endowment for International Peace.

However, it is not only AI’s gender bias that needs to be addressed, as other intersecting identities such as race are also exacerbating inequalities. If governments want to go digital and move services such as banking and health care online, they need to think about these intersectionalities and address things such as racial profiling,  Azelmat said.

AI-driven content moderation often amplifies biases too, something that can be harmful to smaller organizations working on topics related to sexual and reproductive health which face restrictions from tech platforms. If a post is flagged by the algorithm, it is often removed by the platform without any human oversight — even if it doesn’t break any guidelines.

“That’s costly for grassroots movements that depend on their social media presence. Sometimes you get penalized, and even if your content is back then you might get less visibility, or damage the trust with your audience,” Azelmat said.

Addressing the risks of AI

Policymakers are in agreement that AI requires more governance and regulation, “not merely to address the challenges and risks but to ensure we harness its potential in ways that leave no one behind,” read the U.N. advisory body’s interim report released in October.

The report also highlights the need for AI “to be governed inclusively, by and for the benefit of all” and for more women to be included in the development, deployment, use, and governance of technology.

However, Azelmat would like to see one of the advisory body’s working groups be focused exclusively on gender, not just as a cross-cutting theme.

“We’ve seen the establishment of all these advisory bodies, but gender slips between the lines,” she said.

She added that it’s not just about gender, but rather calls for an intersectional approach, making the case for the inclusion of more local perspectives, including more youth and feminist representation.

“When I saw the selection [of the members], it was very high level and exclusive, oriented towards high-caliber experts [rather] than looking for grassroot movement leaders or people outside of the AI field,” she said. Seeking thematic input from specific grassroots organizations could be one way to ensure more diverse voices are included to help address some of the underlying issues and biases in AI, she added.

Pressure to ensure AI promotes rather than harms gender equality will also have to come from a movement that can bring people together, she explained.

“We need a Greta [Thunberg] kind of movement,” Azelmat said. “The turning point would be when feminist movements start stepping up their games in terms of understanding the implication of technology on their work, but also on the issues that they advocate for.”

Others call for the establishment of stronger accountability frameworks. “Be it through the UN, the EU or at Member State level, I would first like to see the implementation of measures that focus on the detection and mitigation of bias in AI algorithms, as well as on the establishment of guidelines and standards,” Scheele wrote. We also need harmonized measures and strong accountability frameworks that apply to both companies and users.”

Beyond governance, another essential measure for fostering more equitable AI is the promotion and inclusion of more women in AI development teams, she added. The underrepresentation of women working in the fields of STEM — science, technology, engineering, and mathematics — has been known for a long time and recent research from UNESCO shows that only 12% of AI researchers and 6% of software developers are women.

Ultimately, Birhane hopes her research and the new advisory body will be a first step in helping to push for more corporate responsibility and for ways to balance power inequalities in AI.

“At the moment, there’s so much power in the hands of a few, big corporations and individual actors who draw most of the benefits from the development and deployment of AI,” she said, adding that “It’s the people at the margins of society that pay the highest price when AI systems go wrong. And AI systems always go wrong.”

Visit Ctrl Shift Equality — a series produced by Devex in partnership with UN Women and the Generation Equality Action Coalitions on Technology and Innovation for Gender Equality, and Gender-Based Violence.

To learn more about the multi-stakeholder commitments made to the Generation Equality Action Coalitions, click here.