About us

Consortium

Team

Meet the Alumni

Flanders AI

Contact

<h5>Description:</h5>
<p>Various metrics have been proposed to measure how social biases affect predictions of large language models. However, these metrics rely on different interpretations of 'fairness' and different world-views, thus they measure different aspects. We wrote a survey paper about fairness evaluation methods for language models and we implemented the metrics in one Python package.</p>

<h5>More Information:</h5>
<ul>
<li><strong>Grand Challenge/Workpackage(s)</strong>: <a href="/en/research/research-challenges-2019-2023/ai-driven-data-science"[entry:airesearch:58:/research/research-challenges/ai-driven-data-science]data-entry-id="58">GC1</a> - WP5</li>
<li><strong>Responsible Research Lead</strong>: <a href="mailto:Bettina.berendt@kuleuven.be" target="_blank" rel="noopener">Bettina Berendt</a>&nbsp;</li>
<li><strong>Authors</strong>: <a href="mailto:pieter.delobelle@kuleuven.be" target="_blank" rel="noopener">Pieter Delobelle</a>, <a href="mailto:ewoenamkwaku.tokpo@uantwerpen.be" target="_blank" rel="noopener">Ewoenam Tokpo</a>, <a href="mailto:toon.calders@uantwerpen.be" target="_blank" rel="noopener">Toon Calders</a>,&nbsp; <a href="mailto:Bettina.berendt@kuleuven.be" target="_blank" rel="noopener">Bettina Berendt&nbsp;</a></li>
<li><strong>Programming Language</strong>: Python</li>
<li><a href="https://pieter.ai/measuring-fairness-with-biased-rulers" target="_blank" rel="noopener">Link to the website</a></li>
<li><a href="https://github.com/iPieter/biased-rulers" target="_blank" rel="noopener">Link to the repository</a></li>
<li><a href="https://youtu.be/8_81GsyJGnA" target="_blank" rel="noopener">Link to a video tutorial</a></li>
<li><strong>Categories</strong>: Text, Natural language processing</li>
</ul>

<h5>Publication:</h5>
<ul>
<li>it can be found <a href="https://aclanthology.org/2022.naacl-main.122/" target="_blank" rel="noopener">here</a></li>
</ul>