AI Fairness & Inclusion: Key Insights from Recent Workshops

During the months of March and April 2025, we conducted four co-creation workshops dedicated to the ethical implications and potential biases in AI systems. These workshops brought together diverse participants belonging to groups that, according to research, are at risk of discrimination by AI systems. In the workshops, we focused on three AI applications.

The first is related to the use of AI in finance as a tool that can help banks to assess loan requests; these AI applications evaluate the probability of loan repayment and indicate whether it is safe for the bank to approve a loan. The second is used to facilitate access to online services that require identity verification, such as opening a bank account; AI can automate this process by comparing a selfie to the photo on the ID to verify the identity of the person requesting the service. The third is related to search and ranking systems used in the academic sector – like Google Scholar; when searching for a specific topic, AI systems automatically recommend and rank articles based on how frequently other authors have cited them.

The workshops were organized by Associacio Forum Dona Activa, IASIS NGO, Diversity Development Group, Complexity Science Hub Vienna, and the University of Bologna. The four workshops took place on March 20th 2025, and March 26th 2025, in Barcelona, and on April 3rd 2025, in Thessaloniki and Vilnius.  

Pictures from the co-creation workshop on 26/04/2025 in Barcelona, Spain

Participants in the workshops emphasized the importance of identifying and addressing the biases inherent in AI systems. These biases can perpetuate many different forms of inequality related to gender, ethnicity, age, and socioeconomic status, among others. A central concern was the “black box” nature of many AI algorithms; participants emphasized the need for transparent and understandable AI models to ensure fairness and build trust. Moreover, participants stressed the importance of maintaining human oversight of AI systems; human judgment was considered to be crucial for interpreting complex scenarios, handling exceptions, and ensuring ethical alignment.

Figure 2. Pictures from the co-creation workshop on 03/04/2025 in Thessaloniki, Greece

Regarding the use case on finance, participants thought that AI can improve speed and efficiency in loan evaluations but must be carefully monitored to avoid perpetuating biases in loan approval processes. Participants discussed how personal data should and should not be used when assessing loan eligibility. In this sense, a key discussion regarded how we can evaluate individual’s effort in determining if a loan request should be approved. Personal effort was perceived as something very subjective and not the main factor that should be used to determine loan eligibility. Illness, inability to work, or personal difficulties might be something out of individual’s control, but that can affect individual’s effort and appear unfavorable to an AI system.

Moreover, it was added that people in these situations might need loans more than employed individuals, and that they should be given more support in the loan application process. This demonstrates the close relationship between ethical considerations and the use of AI systems. In the MAMMOth project, Bundeswehr University Munich and EXUS are working to make AI systems in finance more equitable. 

It was common opinion that AI can streamline identity verification, but biases can lead to unfair or inaccurate outcomes, particularly for minorities and older individuals – e.g., people denied access to a service because inaccurately identified as impostors. This is often because AI systems are trained using data that lacks diversity in terms of gender and ethnicity, being then less accurate with females and non-white people. In the workshops, the identity verification scenario was discussed as less severe, in terms of potential negative consequences, compared to the finance use case that may have severe consequences on a person’s life and opportunities.

However, it was common agreement that it is important to make sure that imposters – that is, people who try to access an online service under a fake identity – are correctly denied, because this has potential consequences on security standards. Our partners Centre for Research and Technology and IDNow are working in this direction. 

According to the participants, several factors contribute to biases within academic citations and collaborations. Researchers are often judged based on citation counts, with a noticeable bias favoring quantitative fields and methodologies over qualitative approaches. This further disadvantages early-career researchers and individuals from less prestigious universities due to structural factors, such as researchers’ working conditions and long-term funding opportunities, which are often contingent upon mobility and resilience, especially in precarious positions. This affects their possibility of appearing in the top positions in search engine results.

Moreover, search engines often display in the first positions people who are part of the same network – e.g., those who work in the same research group. Implicit biases can also arise from ethnicity or regional identity, as inferred through names. Our partner Complexity Science Hub Vienna is working on the mitigation of these aspects by promoting a search engine that can be adjusted by gender, nationality, and network, to increase the diversity of search results. 

These workshops underscore the critical importance of ethical considerations in the development and deployment of AI. By addressing biases, promoting transparency, and maintaining human oversight, we can harness the power of AI to create a more equitable and inclusive future.