As part of the second COMFORT progress meeting, held on May 15–16, 2025, in Thessaloniki, Greece, the consortium hosted a dedicated workshop on gender and sex minorities and related bias in AI-driven healthcare solutions. The session was chaired by Dr. Lili Jiang from Umeå University and sparked important conversations on equity, inclusivity, and responsibility in medical AI.

While women make up 49.6% of the global population, they remain underrepresented in terms of power and privilege – especially in healthcare research and decision-making. A clear example of this systemic gender bias in medicine: studies show that women are, on average, diagnosed with heart disease 7–10 years later than men due to long-standing gaps in medical protocols and research that have historically focused on male physiology.

Why is this relevant to COMFORT and AI in healthcare?

A hands-on demonstration during the workshop illustrated this challenge. When prompted with the questions, “What job is suitable for my 18-year-old daughter?” versus “What job is suitable for my 18-year-old son?”, ChatGPT produced noticeably different responses—highlighting a persistent gender bias in AI-generated content.

AI models trained on biased or incomplete datasets risk reinforcing existing healthcare disparities. Diagnostic tools powered by AI have sometimes shown reduced accuracy for women or people of colour. This underscores the critical need for AI systems that are lawful, ethical, and robust.

From a technical perspective, bias can be mitigated at three different stages of the AI system. During pre-processing, data can be modified to be inclusive and representative before training. The algorithm can be modified, during in-processing, and the predictions of the model can be modified, in feedback loops during post-processing.

How can COMFORT address and reduce bias in its AI models?

The first important step in mitigating bias is recognising its potential presence. For instance, kidney cancer is statistically less prevalent in women than in men. Consequently, if a model is trained on randomly sampled clinical data, it will naturally include fewer cases involving female patients—leading to skewed representation.

At the same time, COMFORT must operate within the framework of European data protection laws, which classify gender and other data as sensitive, meaning it cannot be recorded. As a result, the datasets used in COMFORT’s model development do not include gender-specific metadata.

However, there are ways to compensate. One key approach is thorough clinical validation. COMFORT’s planned prospective study, scheduled for the final year of the project, will play a crucial role in testing whether the model performs with equal accuracy across gender groups—ensuring that no patient is left behind.