Personality Shapes Gender Bias in Persona-Conditioned LLM Narratives Across English and Hindi: An Empirical Investigation

Applications, bias, and safety2026arXivApproved editorial review

Authors: Tanay Kumar, Shreya Gautam, Aman Chadha, Vinija Jain, Francesco Pierri

Keywords: Persona-conditioned generation, Gender stereotypes, English-Hindi evaluation, HEXACO, Dark Triad, Occupational narratives, IndicSBERT, Centroid bias metric, Human annotation, Corpus integrity, Fairness, Reproducibility

Source: Open primary source (opens in a new tab)

5
Authors
7
Findings
14
Limitations
3
Evidence

Editorial summary

English

This preprint asks whether explicit personality descriptions change gender-stereotypical language in English and Hindi occupational artifacts. It crosses six LLMs, 50 India-grounded occupations, two languages, persona gender, and 18 trait conditions, high and low descriptions for six HEXACO and three Dark Triad dimensions, plus no-personality baselines. Its metric embeds each sentence with IndicSBERT and subtracts similarity to female from male stereotype centroids; each story receives the signed score of its maximum-absolute-bias sentence. The published results associate Machiavellianism and Psychopathy with positive male-stereotypical shifts, Narcissism with a smaller shift, and HEXACO with a heterogeneous pattern: Honesty-Humility, Agreeableness, and Conscientiousness also increase male alignment, while Openness and Emotionality reduce it and Extraversion is not significant. Hindi has a more male-leaning baseline in five of six models, whereas personality modulation is larger in English. Three annotators judge the conditioned story more stereotyped in 66% of 50 English pairs and 72% of 50 Hindi pairs. The audit confirms exactly 23,400 public rows but not 23,400 valid distinct stories: 17 stories are empty and 126 excess duplicates in 45 groups reuse text across conditions; one story appears 19 times, and some female-labeled rows retain male pronouns. The anonymous code endpoint returns 401, and the data omit bias scores, embeddings, regressions, and annotation rows. The metric depends on translated stereotype lexicons, one encoder, and an extreme sentence; mean aggregation reverses direction relative to max-abs for 12.8% of stories. Regression reporting omits a complete coefficient table, errors, exact p-values, R-squared, and diagnostics, while Table 18 runs 216 uncorrected t-tests. The faithful conclusion is that trait descriptions are associated with shifts in one stereotype-alignment metric under this factorial prompt design; the study does not establish that Dark Triad is universally more biasing than HEXACO, that prosocial traits generally attenuate bias, or that language, culture, grammar, or model scale causes the differences.

Español

Este preprint estudia si descripciones explícitas de personalidad cambian el lenguaje estereotípico de género en artefactos profesionales en inglés e hindi. Cruza seis LLM, 50 ocupaciones de India, dos idiomas, género de persona y 18 condiciones de rasgo, niveles alto y bajo de seis dimensiones HEXACO y tres Dark Triad, más baselines sin personalidad. La medida incrusta cada frase con IndicSBERT y resta su similitud a centroides estereotípicos femenino y masculino; cada historia recibe la puntuación firmada de su frase con máximo sesgo absoluto. Los resultados publicados asocian Maquiavelismo y Psicopatía con desplazamientos masculinos positivos, Narcisismo con uno menor, y HEXACO con un patrón heterogéneo: Honestidad-Humildad, Amabilidad y Responsabilidad también aumentan la alineación masculina, mientras Apertura y Emocionalidad la reducen y Extraversión no muestra efecto significativo. El hindi presenta un baseline más masculino en cinco de seis modelos, pero la modulación por personalidad es mayor en inglés. Tres anotadores juzgan como más estereotipada la historia condicionada en el 66% de 50 pares ingleses y el 72% de 50 pares hindi. La auditoría confirma exactamente 23.400 filas públicas, pero no 23.400 historias válidas y distintas: hay 17 textos vacíos y 126 duplicados excedentes en 45 grupos que reutilizan relatos entre condiciones; un relato aparece 19 veces y algunas filas femeninas conservan pronombres masculinos. El código anónimo devuelve 401 y los datos no incluyen puntuaciones, embeddings, regresiones ni anotaciones. La métrica depende de léxicos estereotípicos traducidos, un único encoder y la frase extrema; la media cambia de dirección frente al máximo en el 12,8% de historias. La regresión no publica tabla completa, errores, p-valores exactos, R² ni diagnóstico, y la Tabla 18 ejecuta 216 t-tests sin corrección. Por tanto, la conclusión fiel es que las descripciones de rasgo se asocian con cambios en una medida concreta de estereotipo bajo este diseño factorial; no que Dark Triad sea universalmente más sesgada que HEXACO, que los rasgos prosociales atenúen en general, ni que idioma, cultura, gramática o escala del modelo causen las diferencias.

Research question

How are HEXACO and Dark Triad descriptions associated with the direction and magnitude of a gender stereotype measure in professional artifacts generated by six LLMs in English and Hindi, and how do they interact with gender, occupation, language, and model?

Method

A factorial design of 23,400 rows is constructed: per model, 300 baselines (three genders by 50 occupations by two languages) and 3,600 conditioned cases (two genders by 18 trait-level descriptions by 50 occupations by two languages). Each prompt requests a first-person artifact of 6-8 sentences. Sentences are segmented, embedded with indic-sentence-similarity-sbert, and scored by cosine difference against male and female centroids formed with curated lexicons; the story inherits the sentence with maximum absolute value. OLS is fitted with gender, personality condition, language, occupation, and model as categories, plus stratified models. Three annotators compare 100 conditioned-baseline pairs. The audit inspected the 34 pages, the TeX, the 24 CSVs from the Hugging Face commit, and checked counts, empties, duplicates, metric, aggregation, statistics, annotation, and code availability.

Sample: Six models contribute 3,900 rows each: 300 baselines and 3,600 personality conditions, for 23,400 rows. The audited download contains 23,383 non-empty stories and 23,257 unique texts; 17 rows are empty and 126 are surplus exact duplicates across conditions. Human validation uses 100 pairs, 50 per language, evaluated by three annotators. Stability uses only 180 repeated stories for 36 configurations in one English occupation.

Findings

  • Machiavellianism and Psychopathy show positive shifts toward the male centroid; Narcissism has a smaller positive effect in the aggregate analysis.
  • HEXACO is not uniformly protective: Honesty-Humility, Agreeableness, and Conscientiousness also show comparable positive shifts, Openness and Emotionality negative, and Extraversion non-significant.
  • Five of six models have a more masculine Hindi baseline than English; the modulation attributed to personality is greater in English.
  • Majority annotation identifies the conditioned story as more stereotyped in 66% of English pairs and 72% of Hindi pairs; Fleiss kappa is 0.660 and 0.688.
  • The 24 CSVs sum to exactly 23,400 rows, but include 17 empty stories and 126 surplus duplicates reused across distinct conditions.
  • The greatest contamination is in Mixtral: 16 of 17 empties and 98 of 126 surplus duplicates appear in its personality files.
  • The absolute maximum and the mean coincide in direction in 87.2%, which also implies sign inversion in 12.8% of stories.

Limitations

  • The code linked at 4open.science returned HTTP 401 and could not be audited or executed.
  • The data do not include scores, segmentation, embeddings, centroids, OLS matrices, statistical summaries, annotations, or figure/table inputs.
  • The Hugging Face card is a minimal YAML fragment, with no license, risk documentation, citation, model versions, or field contract.
  • The 17 empty stories and duplicates across conditions contradict the claim of regenerating every defective output until a valid artifact is obtained.
  • The measure operationalizes alignment with predefined stereotypical lexicons, not harm, discrimination, professional competence, or general severity of bias.
  • The 50 occupations are selected and separated in advance as stereotypically masculine or feminine; they do not represent the Indian labor market.
  • The Hindi lexicon derives from English and the entire geometry depends on a single IndicSBERT; cross-language invariance is not validated.
  • Choosing the most extreme sentence discards the rest of the story and may depend on length and segmentation.
  • Human annotation covers 100 pairs, is forced and relative to the baseline; it does not validate continuous magnitude, direction, or absolute severity.
  • SD3 only checks Dark Triad prompt compliance in GPT-5 nano; HEXACO and the other five models do not receive equivalent validation.
  • The regression treats stories as independent, does not document robust/clustered covariance, and does not publish complete diagnostic results.
  • The 216 tests in Table 18 use p<0.05 without correction for multiple comparisons.
  • One main story per cell does not estimate generative variability; the five-sample repetition is limited to one occupation in English.
  • The experiment does not reproduce real interaction, user distribution, or deployment consequences.

What the study does not establish

  • It does not demonstrate that Dark Triad is always more biased than HEXACO or that prosocial traits attenuate bias.
  • It does not demonstrate that personality has a greater effect than gender in general or outside this metric and prompts.
  • It does not identify grammar, culture, or language as the cause of the English-Hindi differences.
  • It does not identify model size or architecture as the cause of effects across unmatched families.
  • It does not psychometrically validate the generated personalities in the six models.
  • It does not test real harm or discrimination toward women, men, or workers in the listed occupations.
  • It does not allow reproducing regressions, tables, and figures from the public artifacts available at audit time.
  • It does not generalize to multi-turn dialogue, other languages, non-stereotyped occupations, or intersectional categories.

Traceability

Scope: Full text

Version: arXiv:2604.23600v2

Consulted source: https://arxiv.org/abs/2604.23600

Review: Codex 34-page visual full-text, TeX, Hugging Face 24-CSV corpus, metric, statistical, annotation and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • GPT-5 nano
  • Llama-3.3-70B-Instruct
  • Gemma-3-1b-it
  • DeepSeek-R1
  • Mixtral-8x7B-Instruct
  • Falcon-Mamba-7B-Instruct
  • indic-sentence-similarity-sbert

Instruments and metrics

  • Six HEXACO high/low descriptions
  • Three Dark Triad high/low descriptions
  • Short Dark Triad SD3 prompt-compliance check
  • English and Hindi stereotype lexicons
  • IndicSBERT stereotype centroids
  • Sentence cosine-difference bias score
  • Maximum-absolute story aggregation
  • Treatment-coded OLS
  • Forced pairwise human annotation
  • Fleiss kappa and pairwise Cohen kappa

Data used

  • Personality-Gendered-Artifacts revision da4b1fcc6ae6136cd7a281e75847e27d8828fbb7
  • 24 English/Hindi model CSV files with 23,400 rows
  • 50 India-grounded occupation-artifact-scenario mappings
  • English and Hindi male/female stereotype word lists
  • Unreleased human annotation rows
  • Unreleased sentence scores, embeddings and regression outputs

Evidence and location

  • Design, prompts, models, metric, regression, results, limitations, ethics, and complete appendices: arXiv:2604.23600v2, 34 pages inspected; §§3-5 and Appendix A
  • Public corpus, schema, per-file count, empty stories, duplicates, and documentation: Hugging Face unknown-submission/Personality-Gendered-Artifacts revision da4b1fcc6ae6136cd7a281e75847e27d8828fbb7
  • Integrity audit, metric, statistics, artifacts, and limits of interpretation: reports/verification/article-359-gender-bias-persona-corpus-metric-statistics-data-and-claim-audit.json