Synthetic Personality Review

Measuring gender and racial biases in large language models: Intersectional evidence from automated resume evaluation

Applications, bias, and safety2025PNAS NexusApproved editorial review

Original title: Measuring Gender and Racial Biases in Large Language Models: Intersectional Evidence from Automatic Resume Evaluation

Authors: Jiafu An, Difang Huang, Chen Lin, Mingzhu Tai

Keywords: Large Language Models, Gender bias, Racial bias, Intersectionality, Resume evaluation, Hiring discrimination

Source: Open primary source (opens in a new tab)

4
Authors
10
Findings
33
Limitations
12
Evidence

Editorial summary

English

The study audits whether five language models assign different scores to comparably qualified resumes when the social identity implied by the applicant's name changes. The authors started from 149 job advertisements and selected 20 entry-level positions. They extracted work history, education, and skills from 25,000 Indeed resumes and recombined those characteristics into approximately 361,000 fictitious resumes across five US states. Each resume received a randomized identity, balanced among Black women, White women, Black men, and White men. GPT-3.5 Turbo, GPT-4o, Gemini 1.5 Flash, Claude 3.5 Sonnet, and Llama 3-70b rated job suitability from 0 to 100 at temperature zero, except Llama at 0.01. The regressions control for observed resume characteristics and absorb position, state, and latest-job-title fixed effects, with standard errors clustered across 100 position-state cells. For GPT-3.5 Turbo, the estimated average effect of being female is +0.452 score points and that of being Black is −0.074. Relative to White men, the differences are +0.379 for Black women, +0.223 for White women, and −0.303 for Black men. The other four models also give women higher scores, with overall female coefficients from +0.646 to +1.110 points. For Black men relative to White men, the difference is −0.591 in GPT-4o, −0.252 in Gemini, and −0.611 in Claude; Llama's −0.104 estimate is not statistically distinguishable from zero. The aggregate race coefficient changes direction across models because it combines opposing intersectional patterns. Although the large sample yields narrow intervals, the score differences are small relative to the 0–100 scale and observed score dispersion. Their apparent impact grows when a cutoff is imposed: at GPT-3.5's 80-point threshold, the paper estimates roughly +1.7 percentage points for Black women, +1.4 for White women, and −1.4 for Black men. These are not observed hiring outcomes; they are hypothetical probabilities obtained by converting highly rounded model scores into a binary decision. Longer job-description prompts, subsamples, and stratified analyses broadly preserve the direction of results, but only GPT-3.5 receives the alternative-prompt test. An extension based on another 360,000 simulations reports large disadvantages for Asian- and Hispanic-associated names in GPT-3.5 and different patterns in newer models, showing that there is no single stable axis of racial bias. The evidence establishes systematic sensitivity of model scores to names associated with gender and racial or ethnic origin under this protocol. It does not measure real employment decisions or show whether AI improves or worsens human discrimination. Reproducibility is partial: OSF provides five derived datasets and Stata code for the five main figures, but not the resumes, name lists, generation code, API calls and raw outputs, additional-ethnicity data, or exact model snapshot identifiers. Two figures also rely on manually stored estimates, and the repeated subsampling does not set a random seed.

Español

El estudio audita si cinco modelos de lenguaje asignan puntuaciones distintas a currículos de cualificación comparable cuando cambia la identidad social sugerida por el nombre. Los autores partieron de 149 anuncios de empleo y eligieron 20 puestos de entrada. Extrajeron experiencia, educación y habilidades de 25.000 currículos de Indeed y recombinaron esas características para generar aproximadamente 361.000 currículos ficticios en cinco estados de Estados Unidos. Cada currículo recibió una identidad aleatoria y equilibrada entre mujer negra, mujer blanca, hombre negro y hombre blanco. GPT-3.5 Turbo, GPT-4o, Gemini 1.5 Flash, Claude 3.5 Sonnet y Llama 3-70b puntuaron la adecuación al puesto entre 0 y 100 con temperatura cero, salvo Llama con 0,01. Las regresiones controlan características observables del currículo e incluyen efectos fijos de puesto, estado y último cargo, con errores agrupados en 100 combinaciones puesto-estado. En GPT-3.5 Turbo, el efecto medio estimado de ser mujer es +0,452 puntos y el de ser una persona negra es −0,074; frente a hombres blancos, las diferencias son +0,379 para mujeres negras, +0,223 para mujeres blancas y −0,303 para hombres negros. Los otros cuatro modelos también puntúan más alto a las mujeres: los efectos globales oscilan entre +0,646 y +1,110 puntos. Para hombres negros, la diferencia frente a hombres blancos es −0,591 en GPT-4o, −0,252 en Gemini y −0,611 en Claude; en Llama es −0,104 y no resulta estadísticamente distinguible de cero. La dirección del efecto racial agregado cambia según modelo porque combina patrones interseccionales opuestos. Aunque la muestra produce intervalos estrechos, las diferencias son pequeñas respecto a la escala y a la dispersión de las puntuaciones. Su posible impacto aumenta al imponer umbrales: con GPT-3.5 y corte en 80, el artículo estima aproximadamente +1,7 puntos porcentuales para mujeres negras, +1,4 para mujeres blancas y −1,4 para hombres negros. Esas no son contrataciones observadas, sino probabilidades hipotéticas derivadas de convertir puntuaciones muy redondeadas en una decisión binaria. Las pruebas con descripciones de puesto más largas, submuestras y estratos conservan la dirección general, pero solo GPT-3.5 recibe la prueba alternativa de prompt. Una extensión con otras 360.000 simulaciones encuentra desventajas grandes para nombres asiáticos e hispanos en GPT-3.5 y patrones distintos en modelos posteriores, reforzando que no existe un único eje estable de “sesgo racial”. La evidencia demuestra sensibilidad sistemática de las puntuaciones a nombres asociados con género y origen racial o étnico bajo este protocolo; no mide decisiones laborales reales ni demuestra que la IA mejore o empeore la discriminación humana. La reproducibilidad es parcial: OSF publica cinco datasets derivados y código Stata para las cinco figuras principales, pero no los currículos, nombres, generador, llamadas y salidas crudas, datos de etnias adicionales ni identificadores exactos de los modelos. Dos figuras dependen además de resultados almacenados manualmente y el muestreo repetido no fija semilla.

Research question

Do the scores that five LLMs assign to entry-level applications change when the gender and racial or ethnic identity suggested by the name varies randomly, how do both dimensions interact, and how do those differences translate under hypothetical selection thresholds?

Method

Audit experiment with synthetic resumes. Starting from 25,000 real resumes for 20 jobs, the authors sample with replacement experience, education, and skills within job-state cells, construct approximately 361,000 profiles, and randomly assign names associated with four intersections of gender and race. Five LLMs score each profile from 0 to 100. Linear regressions are estimated with controls, fixed effects for job, state, and last position, and errors clustered by 100 job-state cells; quantile regressions are added, along with interactions with a derived measure of quality, 500 random subsamples of 20 %, alternative prompts for GPT-3.5, and linear probability models for six thresholds. An extension incorporates Asian and Hispanic names.

Sample: The framework starts from 20 entry-level jobs, five states, and 25,000 real resumes. The main experiment generates 361,000 resumes balanced at 25 % among Black woman, white woman, Black man, and white man. After outliers, invalid scores, and absorbed observations, the regression Ns are 332,886 for GPT-3.5, 319,478 for GPT-4o, 322,819 for Gemini, 318,145 for Claude, and 330,438 for Llama. The extension combines another 360,000 resumes with Asian and Hispanic identities and presents Ns between 622,235 and 659,900 depending on the model. There are no applicants, recruiters, or real hiring decisions in the experiment.

Findings

  • In GPT-3.5 Turbo, being female is associated with +0.4521 points and being a Black person with −0.0736 points, conditional on controls and fixed effects; the second result only reaches the 10 % threshold.
  • Relative to white men, GPT-3.5 scores +0.3786 for Black women, +0.2227 for white women, and −0.3032 for Black men.
  • The four additional models show positive global coefficients for women: +0.6455 in GPT-4o, +1.0174 in Gemini, +1.1098 in Claude, and +0.7488 in Llama.
  • The difference for Black men relative to white men is −0.5907 in GPT-4o, −0.2517 in Gemini, and −0.6114 in Claude; in Llama it is −0.1043 and not significant.
  • The aggregate racial coefficient is negative in GPT-4o and Claude, not significant in Gemini, and positive in Llama, so concealing the intersection between gender and race substantially changes the reading.
  • Scores concentrate on multiples of five: 99.4 % of those from GPT-3.5 and 100 % of those from Gemini in the published data are multiples of five. This means that small score differences can produce jumps when applying a cutoff.
  • With GPT-3.5 and a hypothetical cutoff of 80, the article estimates approximately +1.7 percentage points for Black women, +1.4 for white women, and −1.4 for Black men relative to white men.
  • The general direction persists in strata by occupational composition, assigned state, 500 subsamples of 20 %, and, for GPT-3.5, prompts with a real job description with or without an equal opportunity clause.
  • In the extension, GPT-3.5 strongly penalizes Asian and Hispanic names in both genders, whereas the four subsequent models show different signs and magnitudes; there is no single racial pattern across models.
  • The OSF audit confirms five derived datasets and 1,045 lines of Stata for the main figures. The published Ns and means are consistent with those files, but the raw generation and scoring cannot be reconstructed.

Limitations

  • The observed result is a score produced by an LLM, not an interview, offer, hire, salary, or real career trajectory.
  • There is no comparison with human recruiters or with a validated criterion of job quality or performance, so it cannot be determined whether the models reduce or aggravate human bias.
  • The effects are small relative to the scale and dispersion of scores; their practical relevance depends on unobserved decision rules.
  • Hiring probabilities are hypothetical constructs based on six arbitrary cutoffs, not probabilities calibrated with real decisions.
  • The strong concentration on multiples of five makes threshold results discontinuous and sensitive to choosing 60, 65, 70, 75, 80, or 85.
  • Extrapolation to hundreds of thousands of jobs assumes universal adoption, equal threshold, applicability to all jobs, and a direct relationship between score and vacancy; the experiment does not validate those assumptions.
  • Identity is inferred only through names; there is no check on how the models perceive each name, and names may convey social signals in addition to gender and race.
  • The main design reduces gender to female/male and race to Black/white; the extension adds Asian and Hispanic, but does not represent non-binary, multiracial, or intragroup identities.
  • Each identity is assigned to a different resume rather than scoring matched versions of the same resume with several names; the large sample balances covariates on average, but a within-profile contrast is lost.
  • The fictitious resumes recombine experiences, education, and skills with replacement and assume trajectories without gaps; it is not validated with people whether the resulting profiles are coherent or realistic.
  • The 25,000 Indeed resumes were scraped from the web and provide real job text, but the article does not present a detailed discussion of consent, ethical review, governance, or terms of use.
  • Only 20 entry-level jobs and five U.S. states in English are studied; validity for specialized jobs, management, other countries, languages, or cultures is not established.
  • The main prompt only indicates the job title and deliberately simulates an untrained recruiter; it does not represent a configured, evaluated, and supervised professional selection system.
  • The test with full descriptions and an equality clause is limited to GPT-3.5 and does not allow asserting prompt robustness for the other four models.
  • Exact snapshot identifiers, API dates for each model, provider or Llama checkpoint, and full generation parameters are not reported.
  • The supplement states that all items are delivered in a single session but also that resumes are sent in several passes due to token limits; it does not document batch size, order, restarts, or context effects.
  • Each model appears to score each resume only once; there are no repetitions per resume to measure temporal, stochastic, or between-session stability.
  • The models discard invalid scores without a flow diagram or formal selection analysis. In the four-model dataset, approximately 3.8 % of GPT-4o, 2.8 % of Gemini, 4.2 % of Claude, and 0.5 % of Llama are missing, with slightly different rates by identity.
  • The alternative measure of resume quality is derived from the same model score and in the same sample, with no external criterion or out-of-sample validation, making its interpretation circular.
  • The quantile regressions rank the model's own score and present it as resume quality; with discrete and rounded results, the extreme coefficients do not cleanly identify objective quality.
  • Comparing R² between groups does not prove that the model uses information equally or that it eliminates attention discrimination, because R² also depends on the variance of the outcome and the error.
  • Many comparisons are made across groups, models, jobs, states, quantiles, and thresholds without explicit correction for multiplicity.
  • Errors are clustered in 100 job-state cells; stratified analyses have fewer clusters and are not complemented with randomization inference.
  • The comparative table of models includes sizes, training data, and estimated or speculative mitigation strategies without sources; it contains internal inconsistencies, such as attributing to Gemini a window of about 7,000 tokens and also of one million.
  • Differences between models are not a controlled experiment: they change architecture, training, alignment, provider, date, and scoring behavior, so they do not identify the cause of the differences.
  • The explanation attributing the female advantage to post-training or debiasing is a conjecture; the study does not compare base and adjusted models in a matched way, nor does it observe the alignment data or processes.
  • OSF publishes derived variables, not the resumes, names, identifiers, generator, batch prompts, raw responses, API errors, or cleaning scripts; therefore the experiment cannot be reproduced end to end.
  • The Asian and Hispanic extension data and the alternative prompt experiments are not in the five deposited datasets, so those tables cannot be recalculated from the public package.
  • The files do not include a resume identifier. In the GPT-3.5 dataset there is one missing score and 186 additional rows exactly duplicated across all published variables; without text or ID it cannot be determined whether they are true duplicates or distinct profiles that collapse to the same fields.
  • The code for figures 3 and 4 uses manually saved estimates; repeated sampling does not set a seed, and the README does not list all dependencies, such as the eststo command, so a clean run is not fully deterministic.
  • The OSF project is not a preregistration and was created a few days before publication; it does not allow distinguishing confirmatory decisions from explorations conducted during the analysis.
  • There are internal drafting errors: the significance statement erroneously says that Black men are favored, while the central result shows a penalty, and the discussion mentions more than 36,000 instead of approximately 361,000 resumes.
  • The experiment is a snapshot of 2023–2024. Proprietary services can change without notice, so the coefficients should not be treated as current and permanent properties of the evaluated brands.

What the study does not establish

  • It does not demonstrate discrimination in real hiring; it demonstrates differences in simulated scores under a specific prompt.
  • It does not prove that replacing human recruiters with LLMs reduces or increases labor inequality, because there is no human comparison or observed labor outcome.
  • It does not demonstrate that Black or white women would obtain hundreds of thousands of additional jobs, nor that Black men would lose that number of jobs.
  • It does not establish that a statistically significant difference is materially important without specifying a real selection policy and its calibration.
  • It does not identify the causal origin of the patterns in pretraining data, RLHF, filters, annotators, instructions, or bias mitigation.
  • It does not demonstrate that the models have corrected or “overcorrected” prior discrimination; those terms are interpretations, not longitudinal contrasts.
  • It does not establish that favoring one group on average makes the system fair, because equity does not equate to inverting the sign of a mean difference.
  • It does not allow inferring the hiring probability of a specific person or the job quality of candidates.
  • It does not generalize to other languages, countries, periods, highly qualified jobs, interviews, cover letters, or multistage processes.
  • It does not prove that names exclusively measure perceived race and gender, nor that all people would interpret the identities in the same way.
  • It does not demonstrate that Llama is more transparent or fair for being open, nor that proprietary models share a common bias mechanism.
  • It does not establish that current models with those commercial names would reproduce the coefficients, because exact snapshots are missing and services change.
  • It does not prove that the models use resume information equally or that they have eliminated attention discrimination based on R² similarities.

Traceability

Scope: Full text

Version: PNAS Nexus 4(3), pgaf089; advance access 12 March 2025; PMC11937954.1; CC BY-NC 4.0

Consulted source: https://pmc-oa-opendata.s3.amazonaws.com/PMC11937954.1/PMC11937954.1.pdf

Review: Codex full-text, visual, supplementary-material, OSF, code and dataset audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-3.5 Turbo (snapshot exacto no informado)
  • GPT-4o (snapshot exacto no informado)
  • Gemini 1.5 Flash (snapshot exacto no informado)
  • Claude 3.5 Sonnet (snapshot exacto no informado)
  • Llama 3-70b (checkpoint y proveedor de API no informados)

Instruments and metrics

  • Experimento de auditoría con identidades asignadas mediante nombres
  • Puntuación LLM de adecuación al puesto de 0 a 100
  • Regresión lineal con efectos fijos de alta dimensión
  • Errores estándar agrupados por combinación puesto-estado
  • Regresión cuantílica
  • Modelos lineales de probabilidad bajo umbrales de 60 a 85
  • 500 submuestras aleatorias del 20 %

Data used

  • 149 anuncios filtrados de Indeed y Snagajob
  • 25.000 currículos de Indeed usados para construir distribuciones, no publicados
  • Aproximadamente 361.000 currículos ficticios del experimento principal
  • Aproximadamente 360.000 currículos adicionales para identidades asiáticas e hispanas
  • OSF 4dahv: data_gpt35_baseline.dta
  • OSF 4dahv: data_gpt35_heter.dta
  • OSF 4dahv: data_score_4models.dta
  • OSF 4dahv: data_bootstrap_gpt35.dta
  • OSF 4dahv: data_magnitude_gpt35.dta

Evidence and location

  • Question, models, general result, and license: Article, p. 1, Abstract, Significance Statement, and license notice
  • Selection of 20 jobs and origin of postings: Supplement, pp. 3–6, Supplementary Note 1 and Material S1
  • Construction of 361,000 resumes and name assignment: Article, pp. 10–11, Resume creation; supplement, pp. 7–13, Supplementary Note 2
  • Prompt, temperature, models, and batch processing: Article, p. 11, LLM scoring; supplement, pp. 13–15, Supplementary Note 3
  • Statistical specification and error clustering: Article, pp. 11–12, Regression model; supplement, Tables S6–S19
  • GPT-3.5 coefficients and intersections: Article, pp. 2–3, Figure 1; supplement, pp. 22–23, Tables S5–S6
  • Results for GPT-4o, Gemini, Claude, and Llama: Article, pp. 6–8, Figure 5; supplement, pp. 31–32, Tables S12–S13
  • Robustness, quality, and repeated subsamples: Article, pp. 3–6, Figures 2–3; supplement, Tables S7–S17 and Figures S3–S7
  • Conversion to hypothetical hiring probabilities: Article, pp. 6–7, Figure 4; supplement, pp. 42–46, Table S18
  • Extension to Asian and Hispanic identities: Article, p. 7, Assessing more ethnicity groups; supplement, pp. 47–49 and 65–67, Table S19 and Figure S9
  • Stated limitations and post-training conjecture: Article, pp. 9–10, Discussion and limitations
  • Reproducibility availability and audit: OSF 4dahv, ReadMe.txt, Code.do, and five Stata files; project created 7 March 2025 and modified 8 March 2025