The project has explored, with the direct contribution of social scientists, and with the support of other EU-funded projects under the same call [“Tackling gender, race and other biases in AI – HORIZON-CL4-2021-HUMAN-01-24 (RIA)], additional domains where AI bias may be rooted. The present inventory of protected grounds in domains beyond the project’s three use cases (finance, face verification, and scientific publications) aims to guide the integration of further potential implementation dimensions by third parties into the MAI-BIAS toolkit, as well as for the broader adoption of the project’s solutions in additional thematic areas.

The prohibition of discrimination in the EU legal context is laid down in Article 2 of the Treaty on European Union (TEU), and Articles 10 and 19 of the Treaty on the Functioning of the European Union (TFEU) by establishing a list of the following grounds:

• Sex;
• Racial or ethnic origin;
• Religion or belief;
• Disability;
• Age;
• Sexual orientation.

In addition, the EU Charter of Fundamental Rights in Article 21 enacts a non-exhaustive list of grounds for non-discrimination as follows:

• Sex;
• Race;
• Colour;
• Ethnic or social origin;
• Genetic features;
• Language;
• Religion or belief;
• Political or any other opinion;
• Membership of a national minority;
• Property;
• Birth;
• Disability;
• Age;
• Sexual orientation.

Moreover, also the Directive 2000/43/EC prohibits discrimination on grounds of race and ethnic origin.

All AI systems that utilise one or more of the above characteristics, or proxies of these characteristics, should be checked for relevant biases. However, the literature on AI bias also identifies a list of characteristics that are not protected under the law, but which have been shown to lead to bias in AI systems.

There is extensive literature on forms of bias in AI systems. Here, we present a list of protected and non-protected characteristics across six domains not covered by the use cases of the MAMMOth project: health; education; justice; personal security; living standards; work and participation. The list has been published on the MAMMOth website and is aimed at providing some general guidance to AI developers working in the listed domains. For each domain, we also provide some example publications on the topic.

Health

AI is used for many different purposes in the health domain, like in disease detection, healthcare needs estimation, healthcare appointments management, and public health surveillance. Numerous attributes have been shown to be potentially connected to bias in AI systems used for the healthcare sector (Hutchinson et al., 2020; Straw, Callison-Burch, 2020; Obermeyer et al., 2019; Adamson & Smith, 2018):

• Age;
• Disability;
• Gender;
• Nationality;
• Native language and dialect;
• Race/ethnicity;
• Religion;
• Sexuality;
• Socio-economic status.

The website of the EU-funded project AEQUITAS (https://www.aequitas-project.eu/) provide more details on how AI systems used in the healthcare domain can reproduce forms of biases.

Education

In the field of education, AI systems are used for a number of different aims, like dropout prediction, automated essay scoring, graduate admissions, and knowledge inference. Here, we present a list of attributes that have been identified as potential sources of bias in AI systems used in education, as outlined in a review of the literature by Baker and Hawn (2022):

• Disability;
• Gender;
• Military-connected status;
• Nationality;
• Native language and dialect;
• Parental education background;
• Race/ethnicity;
• Socioeconomic status;
• Urbanicity.

The website of the EU-funded project AEQUITAS (https://www.aequitas-project.eu/) provide more details on how AI systems used in education can reproduce forms of biases.

Justice

AI systems are used in the justice system to assess the potential risk of recidivism. A well-known example of an AI system reproducing bias in this field is the COMPAS case (Angwin et al., 2016). Such systems need to be checked for biases related to any kind of personal information collected about the offender, like (Malek, 2022):

• Age;
• Criminal history;
• Employment and work performance;
• Gender;
• Geography;
• Living situation;
• Marital and family relationships;
• Racial and Ethnic Background;
• School grades.

 

Personal security

In this domain, AI can be used in areas such as national security criminal investigations, counterterrorism activities, border security, or surveillance programs. AI systems for face verification are commonly used in this domain and they can reproduce a number of biases related to (Cook et al., 2025; 2023; Buolamwini et al., 2025):

• Age;
• Gender;
• Glasses wearing;
• Head covering;
• Height;
• Race;
• Skin lightness.

 

Living standards

The Directive 2004/113/EC calls for equal treatment of men and women in the access to and supply of goods and services. In the area of living standards, AI can be used with many different scopes, like tailored social services or urban planning. For example, in the case of urban planning, biases can be related to (Kontokosta, Hong, 2021):

• Employment situation;
• Geography/neighborhood;
• Level of education;
• Race;
• Socioeconomic status.

 

Work and participation

In addition to the general laws on discrimination mentioned at the beginning of the section, the Directive 2000/78/EC prohibits discrimination at work on grounds of religion or belief, disability, age or sexual orientation. Moreover, the Directive 2006/54/EC calls for equal treatment of men and women in the field of employment and occupation.

As demonstrated by Vinod Bhatia et al. (2024), although sensitive information about applicants (e.g. ethnicity, sexual orientation and religion) is rarely included in CVs, they still contain references to personal, professional and educational details that can reveal a candidate’s protected characteristics, such as:

• Age (based on date of birth or date of graduation);
• Gender identity (by looking at work gaps due to maternity leave, or title of honors and awards);
• Nationality and ethnicity (through information like: address and postcode; name; spoken languages; name of the educational institution and its location or of early employment; presence of grammatical errors);
• Political affiliations (political campaigning experience; publications; links to personal websites and social media);
• Religion (e.g., wearing a headscarf in the picture);
• Socioeconomic status (through level of education or expensive hobbies reported in the CV).

The websites of the EU-funded projects FINDHR (https://findhr.eu/), BIAS (https://www.biasproject.eu/) and AEQUITAS (https://www.aequitas-project.eu/) provide more details on how AI systems used for hiring can reproduce forms of biases.

References

1. Adamson, A. S., & Smith, A. (2018). Machine learning and health care disparities in dermatology. JAMA dermatology, 154(11), 1247-1248.
2. Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016). Machine Bias. There’s software used across the country to predict future criminals. And it’s biased against blacks. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing.
3. Baker, R. S., & Hawn, A. (2022). Algorithmic bias in education. International Journal of Artificial Intelligence in Education, 32(4), 1052-1092.
4. Buolamwini, J., Raman, S., & Dean, A. (2025). Comply to fly? How Airport Travelers Experience TSA’s Facial Recognition Experiment. https://afp.oxford-aiethics.ox.ac.uk/sitefiles/complytoflyreport-ajl.pdf.
5. Cook, C. M., Cuffney, L., Howard, J. J., Sirotin, Y. B., Tipton, J. L., & Vemury, A. R. (2025). Performance Differentials in Deployed Biometric Systems Caused by Open-Source Face Detectors. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (pp. 2630-2639).
6. Cook, C. M., Howard, J. J., Sirotin, Y. B., Tipton, J. L., & Vemury, A. R. (2023). Demographic effects across 158 facial recognition systems. Technical report, DHS.
7. Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020). Social biases in NLP models as barriers for persons with disabilities. arXiv preprint arXiv:2005.00813.
8. Kontokosta, C. E., & Hong, B. (2021). Bias in smart city governance: How socio-spatial disparities in 311 complaint behavior impact the fairness of data-driven decisions. Sustainable Cities and Society, 64, 102503.
9. Malek, M. A. (2022). Criminal courts’ artificial intelligence: the way it reinforces bias and discrimination. AI and Ethics, 2(1), 233-245.
10. Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447-453.
11. Straw, I., & Callison-Burch, C. (2020). Artificial Intelligence in mental health and the biases of language based models. PloS one, 15(12), e0240376.
12. Vinod Bhatia, K., Capasso, M., Arora, P., Castillo, C., & Saldivar, J. (2024). Proxy Discrimination Risks in Hiring: A Qualitative Analysis of a Set of Real CVs. Available at SSRN: https://ssrn.com/abstract=5048771 or http://dx.doi.org/10.2139/ssrn.5048771.