From Bias to Harm: AI’s Impact on Women and Girls
AI’s representation of women is an ongoing feminist issue. The UN found in a study of 133 AI generative platforms 44% of them showed gender bias and roughly 25% of them showed gender and racial bias. When AI was prompted to write a story about a doctor and a nurse the AI made the doctor a male and the nurse a female giving increasingly stereotypical gendered roles to each according to writer Beyza Doğuç (UN, 2024). Ana Carmo (UN, 2026) writes that AI Large Language Models (LLMs) associate women with the home and childcare whilst men are linked with business and success and in some cases they even present women as sexual objects and as subordinate to men. When asked a question that began with gendered statements LLMs gave misogynistic or sexist answers. UN women state that AI is trained on unequal gendered representations and it is not just a design flaw as it pulls from pre-existing data written by people who existed in a world where women were filed under home and men were filed under business and career.
The economic impact of AI is also an issue in women’s representation as women are less likely to be involved in the creation of AI models with less than a third of the AI workforce being made up of women. Experts from ILO (International Labour organisation) and AI Workforce state that if the industry stays male dominated AI will never overcome its gender bias. The economic impact of AI on gender equality also affects women unequally as nearly twice the number of women as men working outside of the AI industry could have their jobs replaced by automation. The gender bias that AI demonstrates is also compounded by race, disability, geography and income.
According to Glitch (2022) online abuse isn’t just hate speech or harassment but a wide variety of online behaviours that particularly affect black women. As AI algorithms are pushed by those that made them (dis)information can come from anywhere including white supremacists. AI technologies also do not work the same for black women including facial recognition which is most accurate on white men and least accurate on black women as technology is predominantly created by white men and tested on white male data. Platforms such as Tiktok have been known to host harmful sexual racialised AI videos of black women with Onlyfans links and captions such as “why I need a white guy in my life”. Tiktoker Riya Ulan had her videos altered by an account who posted racialised videos to promote AI Onlyfans models. She stated that she didn’t know if she was angrier that her videos had been stolen to promote explicit content or that people keep falling for AI videos with several of these accounts stating that they are not AI.
The United Nations (2024) write that AI discrimination against women and girls is a democratic threat targeting social cohesion and human rights. AI is not creating misogyny but amplifying it by giving abusers the mechanism to create abuse faster and cheaper. 24% of surveyed woman human rights defendants reported experiencing AI assisted online abuse with a further 12% of them experiencing nonconsensual sharing of their images. AI has rapidly changed how women and girls exercise their human rights with the use of AI in both the public and private sector being found to show patterns of discrimination. This discrimination is felt by all women across the public and private sector particularly those who are at the intersection of various forms of discrimination such as disability and race.
Feminist, gender responsive and human rights based approaches recognise the gender unequal nature of societies and how AI reproduces these inequalities based on biases in the data. There is concern about how AI intensifies existing harms for women and girls through its integration into infrastructure and the technologies of daily life. These harms are compounded by the existing structural barriers such as the male dominated nature of the professional field, financial violations by the private sector, inadequate regulation and discriminatory markets. As stated, misogyny is not a design flaw it is a feature of the systems designed to work in patriarchal world.
This article is the second part of a series that aims to educate on feminist concerns and responses to AI with the overall hope to better inform feminists what they can do to combat AI harms. The series will culminate in a lunch time seminar online on the 7th September at 1pm, you can register your interest here.
By Orlaith Hegarty Moore
This article is the second in a series on feminist responses to AI aiming to explore what feminists should be concerned about and what responses feminists can take. You can read the first article here and watch a seminar on the Feminist Responses to AI here.
Reference list
Carmo, A. (2026). AI is getting women wrong as gender bias persists, data reveals. [online] UN News. Available at: https://news.un.org/en/story/2026/06/1167776 [Accessed 26 Aug. 2026].
Glitch (2022). Glitch. [online] Glitch. Available at: https://glitchcharity.co.uk/blog/online-abuse-bill-k65am [Accessed 27 Aug. 2026].
Sharihan Al-Akhras (2026). AI videos of sexualised black women removed from TikTok after BBC investigation. BBC News. [online] 22 Mar. Available at: https://www.bbc.co.uk/news/articles/c070e283k8vo [Accessed 27 Aug. 2026].
UN Women (2024). Artificial intelligence and gender equality | UN women – headquarters. [online] UN Women – Headquarters. Available at: https://www.unwomen.org/en/articles/explainer/artificial-intelligence-and-gender-equality [Accessed 26 Aug. 2026].
United Nations (2026). Women’s and girls’ rights and artificial intelligence and related digital technologies - report of the working group on discrimination against women and girls. [online] OHCHR. Available at: https://www.ohchr.org/en/documents/thematic-reports/ahrc6248-womens-and-girls-rights-and-artificial-intelligence-and-related [Accessed 26 Aug. 2026].