Invisible Influences: Investigating Implicit Intersectional Biases through Persona Engineering in Large Language Models

Research question

Do associations interpreted as intersectional bias change when five LLMs respond to compound identity classes and adopt six persona frames, and can BADx summarize direction, sensitivity, stability, and influential words better than static measures?

Method

Task 1 selects six classes from a curated corpus and scores responses with ad hoc variants named CEAT, I-WEAT, and I-SEAT. Task 2 crosses six personas and one neutral control with five questions and five model labels; BAD averages signed persona-control differences, PSI averages BAD across five prompts, and the volatility table calculates its deviation across prompts. The article claims five generations per cell and LIME over outputs. The repository audit shows a single response per cell, a synthetic CSV constructed with manual parameters, and a dummy LIME predictor not connected to the responses.

Sample: The repository contains 261 sentences in 21 labels including Neutral: 160 intersectional sentences and 101 neutral sentences. Task 1 retains 150 raw responses for 6 classes x 5 prompts x 5 models, but its CSV has 140 rows and omits two prompts for the five models. Task 2 retains one response per 7 persona/control conditions x 5 prompts x 5 models and a CSV of 175 synthetic scores; the five generations per cell claimed by the article are absent.

Findings

• The article reports distinct profiles: GPT-4o sensitive and volatile, DeepSeek-R1 suppressor but unstable, LLaMA-4 stable, Claude balanced, and Gemma low volatility.
• The BAD table assigns a negative sign to all A-C personas labeled marginalized and positive to all D-F labeled privileged, across all models and prompts.
• PSI is the arithmetic mean of the five BAD values published per persona and model; the table volatility is its deviation across prompts, not across generations.
• The public responses qualitatively show that the persona frame changes the content and emphasis of an answer.
• The repository provides corpus, responses, notebooks, CSV, and supplements, allowing discrepancies to be detected that the article does not make visible.
• The Task 2 notebook generates synthetic scores from manual parameters and does not analyze the published responses.
• The BADx table in the article does not reproduce with the CSV: GPT-4o/Persona A/Prompt 1 publishes -0.086, while its formula applied to the CSV gives +0.472.
• The saved LIME lists are labeled as simulated and repeat the same list per model for all personas and questions.

Limitations

• The central quantitative tables have no reproducible path from the published responses and code.
• Task 2 uses bias profiles, persona multipliers, and prompt adjustments fixed by hand; its results are synthetic.
• The Task 2 LIME is a dummy predictor of length plus noise and does not explain model scores or responses.
• The Task 1 LIME uses word rules over the prompt, not over outputs or over embedding measures.
• The article and artifacts disagree on Claude and Gemma; the Task 1 notebook also names LLaMA-3.
• The article claims five generations and seed 42, but there is only one response and one score per cell.
• The published volatility is calculated across five different questions, confusing prompt heterogeneity with stochastic instability.
• BAD is described as an absolute difference but the formula retains the sign; PSI allows cancellations despite being called magnitude.
• No formula exists that combines BAD, PSI, volatility, and LIME into a single BADx score.
• The implementations do not reproduce standard CEAT, WEAT, or SEAT procedures and mix incompatible directions.
• The I-WEAT example from the corpus cancels algebraically to zero for any sentence.
• The corpus notebook shows outputs incompatible with its own code and does not regenerate the 261 published scores.
• The Task 2 seed uses Python hash(), variable across processes unless PYTHONHASHSEED is fixed.
• The intersectional sentences contain explicit stereotypes and exclusion, while many controls contain success and inclusion; they are not lexically equivalent pairs.
• The six classes are chosen for their high scores in the same corpus, without out-of-sample validation.
• Identities are grouped without single-axis conditions or factorial design, so intersectional interactions are not identified.
• The classification of D-F as privileged is assumed despite combining unemployment, low income, or immigration.
• There is no human validation of perceived bias, annotator agreement, behavioral criterion, or external benchmark.
• No inferential tests, intervals, or uncertainty are reported to support the word significantly.
• The 0.2 threshold is attributed to Cohen although these ad hoc scores are not Cohen's d.
• The real corpus has 261 rows and 101 controls, not 260 and 100; the article also oscillates between six, twenty, and twenty-one classes.
• API snapshots, endpoints, execution dates, request IDs, retries, and traceability between outputs and tables are missing.
• The repository does not include requirements.txt despite the README, nor a lockfile, tests, CI, releases, or a detectable license.

What the study does not establish

• It does not validate BADx as a unified, psychometric, or fairness measure.
• It does not demonstrate that BADx outperforms standard CEAT, WEAT, or SEAT.
• It does not establish statistically significant modulation by persona.
• It does not support a reliable classification of which model is more biased, stable, or suitable for deployment.
• It does not identify causal effects of marginalized or privileged identity nor properly intersectional interactions.
• It does not demonstrate volatility across five generations because those generations are not published.
• It does not validate the LIME influential words as explanations of LLM behavior.
• It does not distinguish implicit bias from stereotypical content explicitly inserted into sentences and personas.
• It does not generalize beyond five questions, one wording per persona, English, and ambiguous model versions.
• It does not allow exact reproduction of results without a connected pipeline, versioned dependencies, immutable models, and stable seeds.

Traceability

Scope: Full text

Version: arXiv:2604.06213v1; ACM DOI 10.1145/3795766.3799772

Consulted source: https://arxiv.org/abs/2604.06213

Review: Codex 11-page article plus 27-page supplement visual full-text, repository, metric-validity, synthetic-score, model-identity, LIME, statistical and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

• GPT-4o
• DeepSeek-R1
• LLaMA-4
• Claude 4.0 Sonnet (artículo; artefactos usan etiquetas Claude-3.5-Sonnet y Claude-4o Sonnet)
• Gemma-3n E4B (artículo; artefactos usan etiquetas Gemma-2o-8B y Gemma-3n 4B)
• RoBERTa-base vía SentenceTransformer como encoder de puntuación

Instruments and metrics

• Variantes ad hoc de CEAT, I-WEAT e I-SEAT
• BAD/BADx
• Persona Sensitivity Index (PSI)
• Volatilidad por desviación estándar
• LIME
• Implicit Intersectional Bias Score (IIBS)

Data used

• Corpus público del estudio: 261 frases, 160 interseccionales y 101 neutrales
• Respuestas Markdown de Task 1 y Task 2
• outputs/task1_bias_scores_with_synthetic.csv
• outputs/task2_bias_scores.csv

Evidence and location

• Design, formulas, tables, discussion, limitations, ethics, and references: arXiv:2604.06213v1, 11/11 PDF pages rendered and individually inspected
• ACM publication, DOI, pages, authors, dates, and CC BY 4.0 license: Official arXiv record and Crossref DOI metadata inspected 2026-07-17
• Corpus, responses, notebooks, CSV, history, absence of release and license, and model discrepancies: bias-in-LLMs/Bias-in-LLMs commit b31b0a938cd161759d13bd58397491c68e8d8593 inspected 2026-07-17
• Synthetic configuration, counts, and absence of error bars: All 27/27 pages of Appendix.pdf and five Supplementary PDFs rendered and individually inspected
• Task 2 synthetic generation, dummy LIME, disconnected pipeline, I-WEAT bug, hash seed, and non-reproduction of BADx: reports/verification/article-391-badx-intersectional-persona-synthetic-scores-metric-validity-model-identity-lime-statistics-and-reproducibility-audit.json