Bias and Fairness in Large Language Models: A Survey

Reviews, theory, and governance2024MIT PressApproved editorial review

Authors: Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, Nesreen K. Ahmed

Keywords: Large Language Models, Bias evaluation, Fairness, Social bias, Natural language processing, AI ethics

Source: Open primary source (opens in a new tab)

9
Authors
14
Findings
31
Limitations
14
Evidence

Editorial summary

English

This article is a narrative, taxonomic review, not an experiment measuring bias in a particular model. Its main contribution is to separate three components that are often conflated: the social harm being studied, the metric used to quantify it, and the dataset supplying evaluation cases. It defines social bias as disparate treatment or outcomes between groups arising from historical and structural power asymmetries. It distinguishes representational harms, derogatory language, disparate performance, erasure, exclusionary norms, misrepresentation, stereotyping, and toxicity, from direct or indirect allocational harms that distribute resources or opportunities unequally.

The first taxonomy organizes metrics by what they obtain from a model: embeddings, probabilities, or generated text. Embedding metrics compare distances and associations; probability metrics use masked tokens, pseudo-likelihoods, perplexity, or sentence choices; generated-text metrics compare distributions, auxiliary classifiers, or lexicons. The paper stresses that an association in representation space is not equivalent to downstream harm, and that embedding- and probability-based measures relate weakly or inconsistently to application outcomes. Prompts, templates, temperature, length, decoding, and auxiliary models can change or even reverse conclusions. It therefore recommends evaluating the use-case task directly, publishing the full configuration, and tying every metric to an explicit harm and sociolinguistic theory.

The second taxonomy lists 23 datasets. Fifteen use counterfactual inputs: eight masked-token datasets and seven complete-sentence datasets. Eight use prompts: five sentence-completion and three question-answering resources. The table links each resource to target harms and social groups, but the authors emphasize construct, content, and ecological validity problems. Many instances are synthetic, binary, US-centric, or focused on gender and stereotypes, and substituting an identity term can alter a sentence's meaning. There is no universal fairness benchmark; shared benchmarks improve comparability but may erase context and affected communities' perspectives.

The third taxonomy classifies mitigations by intervention point. Pre-processing covers augmentation, filtering or reweighting, data generation, instruction tuning, and projection; in-training methods modify architecture, losses, parameter subsets, or filter parameters; intra-processing changes decoding, redistributes weights, or adds modular networks; post-processing rewrites outputs. The survey does not empirically compare effectiveness or select a winner. It identifies common risks: erasing identities, treating groups as interchangeable, reducing linguistic diversity, censoring minoritized dialects, inheriting bias from auxiliary classifiers, and improving one metric while harm reappears elsewhere.

The final section places technical limits within power relations. It recommends centering marginalized communities, participatory research, explicit values and assumptions, broader languages and data governance, disaggregated and intersectional groups, reporting standards, validity and reliability studies, hybrid mitigations, and analysis of guarantees and performance–fairness trade-offs. The authors acknowledge that organizing the field by technical mechanisms adopts a technical-solutionist perspective, omits parts of the lifecycle, and is limited to English-language papers.

The editorial audit confirms broad coverage and useful formalization, but it is not a systematic review: it reports no searched databases, search strings, cutoff dates, selection flow, screened-study count, or quality appraisal. Its LLM definition includes encoder models such as BERT and even WEAT for static embeddings, making the scope broader than contemporary generative-LLM usage. The associated repository, frozen at a September 2023 commit, has 21 folders versus 23 datasets in the final table; GAP-Subjective and Bias-STS-B are absent, and RealToxicityPrompts and Equity Evaluation Corpus contain documentation or links rather than data. Five folders lack a local license and so does the repository root. The defensible contribution is a conceptual and technical map of the field through 2024, not evidence that any method eliminates bias or an audit of synthetic personality.

Español

Este artículo es una revisión narrativa y taxonómica, no un experimento que mida el sesgo de un modelo concreto. Su principal aportación consiste en separar tres componentes que a menudo se confunden: qué daño social se quiere estudiar, con qué métrica se cuantifica y qué conjunto de datos proporciona los casos de evaluación. Define el sesgo social como trato o resultados dispares entre grupos surgidos de asimetrías históricas y estructurales de poder. Distingue daños representacionales, lenguaje derogatorio, rendimiento dispar, borrado, normas excluyentes, representación errónea, estereotipos y toxicidad, de daños de asignación, directos o indirectos, que distribuyen de forma desigual recursos u oportunidades.

La primera taxonomía organiza las métricas por aquello que obtienen del modelo: embeddings, probabilidades o texto generado. Las métricas de embeddings comparan distancias y asociaciones; las probabilísticas usan tokens enmascarados, pseudo-verosimilitud, perplejidad o elecciones entre frases; las de texto generado comparan distribuciones, clasificadores auxiliares o léxicos. El artículo insiste en que una asociación en el espacio de representación no equivale a daño en una aplicación y que embeddings y probabilidades se correlacionan de forma débil o inestable con resultados posteriores. También advierte que prompts, plantillas, temperatura, longitud, decodificación y modelos auxiliares pueden cambiar e incluso invertir las conclusiones. Por eso recomienda evaluar directamente la tarea de uso, publicar toda la configuración y vincular cada métrica con un daño y una teoría sociolingüística explícitos.

La segunda taxonomía enumera 23 datasets. Quince usan entradas contrafactuales: ocho con tokens enmascarados y siete con oraciones completas. Ocho usan prompts: cinco de continuación y tres de pregunta-respuesta. La tabla relaciona cada recurso con el daño y los grupos sociales que pretende cubrir, pero los autores recalcan problemas de validez de constructo, contenido y ecología: muchos ejemplos son artificiales, binarios, centrados en Estados Unidos o en género y estereotipos, y la sustitución de una identidad puede cambiar el significado de la oración. No existe un benchmark universal de equidad; la comparabilidad que aporta un conjunto común puede ocultar contexto y perspectivas de los grupos afectados.

La tercera taxonomía clasifica mitigaciones por el punto de intervención. El preprocesamiento incluye aumento, filtrado o reponderación, generación de datos, instruction tuning y proyecciones; el entrenamiento modifica arquitectura, función de pérdida, subconjuntos de parámetros o filtra parámetros; el intraprocesamiento cambia la decodificación, redistribuye pesos o añade redes modulares; el posprocesamiento reescribe la salida. El artículo no compara empíricamente su eficacia ni elige una técnica ganadora. Señala riesgos transversales: borrar identidades, tratar grupos como intercambiables, reducir diversidad lingüística, censurar dialectos minoritarios, trasladar sesgos desde clasificadores auxiliares y mejorar una métrica mientras el daño reaparece en otra etapa.

La sección final sitúa los límites técnicos dentro de relaciones de poder. Recomienda centrar a comunidades marginadas, adoptar investigación participativa, hacer explícitos valores y supuestos, ampliar idiomas y gobernanza de datos, desagregar grupos e intersecciones, crear estándares de reporte, estudiar validez y fiabilidad, combinar etapas de mitigación y analizar garantías y compromisos entre rendimiento y equidad. Los propios autores reconocen que organizar el campo por mecanismos técnicos adopta una perspectiva de solucionismo técnico, que el alcance omite partes del ciclo de vida y que se limita a trabajos en inglés.

La auditoría editorial confirma una cobertura amplia y una formalización útil, pero no permite llamarla revisión sistemática: no se publican bases consultadas, cadenas de búsqueda, fechas de corte, flujo de selección, número de estudios cribados ni evaluación de calidad. Su definición de LLM incluye modelos encoder como BERT y hasta WEAT para embeddings estáticos, de modo que el alcance es más amplio que el uso contemporáneo de «LLM generativo». El repositorio asociado, congelado en un commit de septiembre de 2023, contiene 21 carpetas frente a 23 datasets en la tabla final; no incluye GAP-Subjective ni Bias-STS-B, y RealToxicityPrompts y Equity Evaluation Corpus solo aportan documentación o enlaces. Cinco carpetas no contienen licencia local y el repositorio raíz tampoco. La contribución defendible es un mapa conceptual y técnico del campo hasta 2024, no evidencia de que un método elimine el sesgo ni una auditoría de personalidad sintética.

Research question

How can social bias harms and equity in LLMs be precisely defined, and how can the literature on metrics, datasets, and mitigation techniques be organized according to the type of model access and the point of intervention?

Method

Narrative review with mathematical formalization and taxonomic synthesis. Includes works in English that propose closed metrics, datasets, or mitigation techniques applicable to social bias in pretrained models broadly defined as LLMs. Definitions of harms and five equity desiderata are constructed; metrics are classified by embeddings, probabilities, or generated text, datasets by counterfactual inputs or prompts, and mitigations by preprocessing, training, in-processing, or post-processing. No systematic protocol for search, screening, or quality evaluation is described. The editorial audit read and rendered the 79 pages, cross-checked the dataset table, and reviewed the associated repository at its current commit.

Sample: There is no experimental sample or meta-analysis. The unit of analysis is publications selected narratively, with no total number of included or screened studies. Table 4 summarizes 23 datasets: 15 of counterfactual inputs and 8 of prompts. The associated repository contains 21 resource folders and 291 versioned files; two folders link to external data without including it.

Findings

  • The article separates nine harm mechanisms: seven representational and two allocation, which are not mutually exclusive.
  • It proposes five desiderata: group unawareness, invariance, equal group associations, equal neutral associations, and replication of a reference distribution.
  • It classifies metrics by embeddings, probabilities, and generated text, clarifying that metric and dataset are distinct components.
  • The reviewed literature shows weak or inconsistent relationships between bias in embeddings and harm in downstream applications.
  • Probabilistic metrics can also diverge from final behavior and depend on templates, normalization, and linguistic assumptions.
  • Metrics on generated text change with prompts and decoding parameters; classifiers and lexicons introduce their own biases and context losses.
  • The evaluation table gathers 23 datasets: 15 counterfactual and 8 prompt-based.
  • The authors point out recurrent problems of validity, ambiguity, limited social coverage, and dependence on United States contexts.
  • Mitigations are ordered into preprocessing, training, in-processing, and post-processing, with fourteen concrete mechanisms.
  • No empirical comparison is presented that would allow ordering mitigations by efficacy or safety.
  • Recommendations prioritize direct evaluation of the use case, configuration reporting, explicit harm, and construct and ecological validity.
  • The work insists that neutralizing identities or equalizing all associations can erase relevant differences and minority voices.
  • Open problems include community participation, data governance, intersectionality, standards, living benchmarks, hybrid mitigation, and theoretical guarantees.
  • The repository audit confirms a useful but incomplete collection relative to the final table and with heterogeneous reuse permissions.

Limitations

  • It is not a systematic review: databases consulted, queries, dates, deduplication, detailed criteria, and a selection diagram are missing.
  • It does not report how many works were located, excluded, or included, nor does it evaluate their quality or risk of bias.
  • The selection is restricted to proposals of closed metrics, datasets, or mitigations and leaves out much of qualitative auditing, governance, and deployment studies.
  • The scope is limited to works in English, except for additional languages within some datasets and tasks.
  • The definition of LLM includes encoder and encoder-decoder pretrained models, broader than the current usage centered on generative models.
  • It includes WEAT for static embeddings for historical continuity, even though it is not strictly an LLM metric.
  • It is a snapshot up to July 2024 and does not cover subsequent models, regulations, benchmarks, or mitigations.
  • The categories are an interpretive synthesis by the authors; no inter-coder agreement of the taxonomy is evaluated.
  • Organizing by technical details can obscure the context of use, who suffers the harm, and what power relation produces it.
  • The desiderata of equality and invariance are themselves normative and may be inadequate when distinct groups require distinct treatments.
  • Fairness through unawareness does not avoid proxies or indirect discrimination and can prevent measuring disparities.
  • A reference distribution is not intrinsically fair; the outcome depends on which population and values are chosen.
  • Embedding and probability metrics should not be interpreted as equivalent to downstream harm.
  • Prompts, seeds, temperature, length, and decoding can produce contradictory conclusions without a consolidated reporting standard.
  • Auxiliary toxicity or sentiment classifiers can penalize dialects and marginalized identities.
  • Lexicons ignore relations between words, context, irony, use-mention, and harms expressed without individually offensive terms.
  • Counterfactual inputs may cease to be semantically equivalent when a social group is substituted.
  • Many datasets use binary categories, interchangeable groups, artificial stereotypes, and United States or Western context.
  • Static datasets age and can create complacency by turning a contextual question into a universal score.
  • Benchmarks do not exhaustively cover harms, intersections, languages, cultures, or deployment contexts.
  • Mitigating a representation does not guarantee mitigating the application; bias can reappear in later stages.
  • Augmentation, filtering, and rewriting can erase identities, history, dialects, and linguistic diversity.
  • Mitigations that use lists, annotation, or human feedback do not scale easily to all groups and harms.
  • Objective functions embed values and trade-offs, but many works do not make explicit which notion of equity they optimize.
  • Broad comparisons across stages, hybrid techniques, theoretical guarantees, and robust performance-equity analyses are missing.
  • The survey summarizes others' results and does not reproduce metrics, datasets, or mitigations, so it does not validate their efficacy.
  • The associated repository remained at a September 2023 commit and is not synchronized with the entire table of the final version.
  • The repository contains 21 folders versus 23 datasets; GAP-Subjective and Bias-STS-B do not appear.
  • RealToxicityPrompts and Equity Evaluation Corpus do not include local data, only license/README or link.
  • Five folders do not contain a local license and there is no root license, so not all permissions are communicated uniformly.
  • The work does not specifically address synthetic personality, psychometrics, or trait inference; its relevance is cross-cutting given bias risks.

What the study does not establish

  • It does not demonstrate that a specific LLM is fair or unfair.
  • It does not estimate the prevalence or magnitude of bias in current models.
  • It does not demonstrate that a metric measures the social harm it claims to measure.
  • It does not establish a universal metric or benchmark of equity.
  • It does not prove that embeddings or probabilities predict the harm of an application.
  • It does not rank datasets by quality or mitigations by efficacy.
  • It does not demonstrate that any technique eliminates bias completely, stably, or in a generalizable manner.
  • It does not offer theoretical guarantees of equity.
  • It does not resolve which values, groups, or reference distribution should be prioritized.
  • It does not replace audits of real context, community participation, or governance.
  • It does not validate uses in profiling, selection, health, education, credit, or justice.
  • It does not provide evidence on human or synthetic personality, Big Five traits, or psychometric validity.
  • It does not comprehensively cover research published after July 2024.

Traceability

Scope: Full text

Version: arXiv:2309.00770v3 (12 Jul 2024); accepted at Computational Linguistics, Volume 50, Number 3; publisher DOI 10.1162/coli_a_00524

Consulted source: https://arxiv.org/pdf/2309.00770v3

Review: Codex full-text, visual, taxonomy, dataset and repository audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • No experimental model sample; literature includes autoregressive, autoencoding and encoder-decoder pretrained language models
  • Examples discussed include GPT, GPT-2, GPT-3, GPT-4, BERT, RoBERTa, XLM-R, BART and T5
  • Auxiliary toxicity, sentiment, regard and style classifiers used by surveyed metrics

Instruments and metrics

  • Taxonomy of seven representational harms and two allocational harms
  • Five fairness desiderata for LLMs
  • Embedding-based bias metrics
  • Probability-based bias metrics
  • Generated-text distribution, classifier and lexicon metrics
  • Counterfactual-input and prompt dataset taxonomy
  • Four-stage mitigation taxonomy

Data used

  • Twenty-three bias-evaluation datasets listed in Table 4
  • Fifteen counterfactual-input datasets: eight masked-token and seven unmasked-sentence resources
  • Eight prompt datasets: five sentence-completion and three question-answering resources
  • Fair-LLM-Benchmark repository at commit b21f3912c4722f1b067e646d1cb100771b65095f

Evidence and location

  • Scope, contributions, and general inclusion criteria: arXiv v3, sections 1–2, pp. 1–13
  • Taxonomy of representational and allocation harms: arXiv v3, Table 1 and section 2.2, pp. 6–10
  • Five equity desiderata: arXiv v3, Definitions 8–12, pp. 10–11
  • Taxonomy and limitations of metrics: arXiv v3, section 3 and Table 3, pp. 13–27
  • Recommendations for evaluation: arXiv v3, section 3.6, pp. 26–27
  • Twenty-three datasets, structures, harms, and groups: arXiv v3, section 4 and Table 4, pp. 27–34
  • Limitations and recommendations of datasets: arXiv v3, sections 4.1.3, 4.2.3 and 4.3, pp. 31–34
  • Taxonomy of mitigations: arXiv v3, section 5, Tables 5–6 and Figures 6–10, pp. 34–55
  • Mitigation recommendations: arXiv v3, section 5.5, pp. 54–55
  • Power, participation, groups, standards, and guarantees: arXiv v3, section 6, pp. 56–60
  • Limitations acknowledged by the authors: arXiv v3, section 7, pp. 60–61
  • Version and acceptance: arXiv:2309.00770v3, revised 12 Jul 2024; accepted at Computational Linguistics 50(3)
  • Content and lag of the collection: Official Fair-LLM-Benchmark repository commit b21f3912c4722f1b067e646d1cb100771b65095f; audited 15 Jul 2026
  • Licenses and folders without local data: Official repository commit b21f3912, root README and dataset folders; audited 15 Jul 2026