Biased-rulers
A Python package for fairness metrics in language models
Description:
Various metrics have been proposed to measure how social biases affect predictions of large language models. However, these metrics rely on different interpretations of 'fairness' and different world-views, thus they measure different aspects. We wrote a survey paper about fairness evaluation methods for language models and we implemented the metrics in one Python package.
More Information:
- Grand Challenge/Workpackage(s): GC1 - WP5
- Responsible Research Lead: Bettina Berendt
- Authors: Pieter Delobelle, Ewoenam Tokpo, Toon Calders, Bettina Berendt
- Programming Language: Python
- Link to the website
- Link to the repository
- Link to a video tutorial
- Categories: Text, Natural language processing
Publication:
- it can be found here
