This article is a narrative, taxonomic review, not an experiment measuring bias in a particular model. Its main contribution is to separate three components that are often conflated: the social harm being studied, the metric used to quantify it, and the dataset supplying evaluation cases. It defines social bias as disparate treatment or outcomes between groups arising from historical and structural power asymmetries. It distinguishes representational harms, derogatory language, disparate performance, erasure, exclusionary norms, misrepresentation, stereotyping, and toxicity, from direct or indirect allocational harms that distribute resources or opportunities unequally.
The first taxonomy organizes metrics by what they obtain from a model: embeddings, probabilities, or generated text. Embedding metrics compare distances and associations; probability metrics use masked tokens, pseudo-likelihoods, perplexity, or sentence choices; generated-text metrics compare distributions, auxiliary classifiers, or lexicons. The paper stresses that an association in representation space is not equivalent to downstream harm, and that embedding- and probability-based measures relate weakly or inconsistently to application outcomes. Prompts, templates, temperature, length, decoding, and auxiliary models can change or even reverse conclusions. It therefore recommends evaluating the use-case task directly, publishing the full configuration, and tying every metric to an explicit harm and sociolinguistic theory.
The second taxonomy lists 23 datasets. Fifteen use counterfactual inputs: eight masked-token datasets and seven complete-sentence datasets. Eight use prompts: five sentence-completion and three question-answering resources. The table links each resource to target harms and social groups, but the authors emphasize construct, content, and ecological validity problems. Many instances are synthetic, binary, US-centric, or focused on gender and stereotypes, and substituting an identity term can alter a sentence's meaning. There is no universal fairness benchmark; shared benchmarks improve comparability but may erase context and affected communities' perspectives.
The third taxonomy classifies mitigations by intervention point. Pre-processing covers augmentation, filtering or reweighting, data generation, instruction tuning, and projection; in-training methods modify architecture, losses, parameter subsets, or filter parameters; intra-processing changes decoding, redistributes weights, or adds modular networks; post-processing rewrites outputs. The survey does not empirically compare effectiveness or select a winner. It identifies common risks: erasing identities, treating groups as interchangeable, reducing linguistic diversity, censoring minoritized dialects, inheriting bias from auxiliary classifiers, and improving one metric while harm reappears elsewhere.
The final section places technical limits within power relations. It recommends centering marginalized communities, participatory research, explicit values and assumptions, broader languages and data governance, disaggregated and intersectional groups, reporting standards, validity and reliability studies, hybrid mitigations, and analysis of guarantees and performance–fairness trade-offs. The authors acknowledge that organizing the field by technical mechanisms adopts a technical-solutionist perspective, omits parts of the lifecycle, and is limited to English-language papers.
The editorial audit confirms broad coverage and useful formalization, but it is not a systematic review: it reports no searched databases, search strings, cutoff dates, selection flow, screened-study count, or quality appraisal. Its LLM definition includes encoder models such as BERT and even WEAT for static embeddings, making the scope broader than contemporary generative-LLM usage. The associated repository, frozen at a September 2023 commit, has 21 folders versus 23 datasets in the final table; GAP-Subjective and Bias-STS-B are absent, and RealToxicityPrompts and Equity Evaluation Corpus contain documentation or links rather than data. Five folders lack a local license and so does the repository root. The defensible contribution is a conceptual and technical map of the field through 2024, not evidence that any method eliminates bias or an audit of synthetic personality.