Toolboxes & Code Repositories
Biased-rulers
A Python package for fairness metrics in language models
Description:
Various metrics have been proposed to measure how social biases affect predictions of large language models. However, these metrics rely on different interpretations of 'fairness' and different world-views, thus they measure different aspects. We wrote a survey paper about fairness evaluation methods for language models and we implemented the metrics in one Python package.
More Information:
Originates from the AI-Driven Data Science research challenge (FAIR 2019–2023).
Responsible Research Lead: Bettina Berendt
Authors: Pieter Delobelle, Ewoenam Tokpo, Toon Calders, Bettina Berendt
Programming Language: Python
Categories: Text, Natural language processing
Publication:
it can be found here