Synthetic Personality Review

The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models

Personas, identity, and agents2026arXivApproved editorial review

Authors: Yunze Xiao, Vivienne J. Zhang, Chenghao Yang, Ningshan Ma, Weihao Xuan, Jen-tse Huang

Keywords: Persona collapse, Persona conditioning, Human simulation, Population-level evaluation, BFI-44, Moral reasoning, Self-introduction, Coverage, Hopkins statistic, Local intrinsic dimensionality, Demographic variance, Reproducibility

Source: Open primary source (opens in a new tab)

6
Authors
8
Findings
12
Limitations
3
Evidence

Editorial summary

English

This preprint proposes a population-level account of persona collapse that separates human-space Coverage, spatial Uniformity, and local Complexity. It evaluates ten LLMs on 1,144 synthetic profiles retained from 2,000, using 44 BFI items, 131 moral dilemmas, and three self-introductions per persona. The published results are strongly task-dependent: Qwen3-4B reaches the highest BFI Coverage (0.80) but local intrinsic dimensionality of 7.3 versus 14.4 for the human reference; CoSER-Llama-8B compresses BFI responses (EffL 1.36; Coverage 0.16) while varying widely on moral judgments; MiniMax-M2-Her has high BFI complexity (22.3) but almost no human-space Coverage (0.06). The framework is useful because it distinguishes surface variety from structural diversity. Its stronger claims require substantial qualification. A human reference exists only for BFI, so moral and self-introduction findings are model-to-model comparisons. The public artifact omits all ten-model outputs and the self-introduction embedding code required for Table 8. The 856 exclusions are produced by six deterministic rules rather than a documented manual review. Incremental R-squared uses ordinal integer encodings instead of the claimed one-hot predictors; gender keyword matching has demonstrable substring false positives; and template normalization lowercases text before searching for capitalized names, so names are not replaced. Fidelity and Cohen's d are also not independent evidence of caricature because both respond to the same assigned High/Low OCEAN targets, while dominant-share claims divide very small demographic R-squared totals. The faithful conclusion is that these model snapshots exhibit task-contingent homogenization and off-manifold behavior under this persona population, prompt set, and metric implementation; the study does not establish a universal collapse law, a causal training effect, or human population realism.

Español

Este preprint propone diagnosticar el colapso de persona a nivel de población separando cobertura del espacio humano, uniformidad espacial y complejidad local. Evalúa diez LLM con 1.144 perfiles sintéticos retenidos de 2.000, 44 ítems BFI, 131 dilemas morales y tres auto-presentaciones por persona. Los resultados publicados muestran perfiles dependientes de la tarea: Qwen3-4B alcanza la mayor cobertura BFI (0,80) pero una dimensionalidad local de 7,3 frente a 14,4 en la referencia humana; CoSER-Llama-8B concentra su BFI en pocas respuestas (EffL 1,36; cobertura 0,16) pero es muy variable en moralidad; MiniMax-M2-Her exhibe alta complejidad BFI (22,3) fuera del soporte humano (cobertura 0,06). Es una aportación conceptual útil para no confundir variedad superficial con diversidad estructural. Sin embargo, la referencia humana existe solo para BFI; moralidad y auto-presentación admiten únicamente comparaciones entre modelos. La auditoría del artefacto limita además la fuerza de las conclusiones: no se publican las respuestas de los diez modelos ni el código de geometría de embeddings de la Tabla 8; las 856 exclusiones proceden de seis reglas deterministas, no de una revisión manual documentada; el R² incremental usa códigos ordinales de categorías en vez del one-hot descrito; el detector de género genera falsos positivos por subcadenas y el normalizador de plantillas nunca sustituye nombres después de convertirlos a minúsculas. La asociación entre fidelidad y Cohen d tampoco es evidencia independiente de caricaturización: ambas métricas responden a las mismas etiquetas OCEAN High/Low, y los porcentajes dominantes se calculan sobre R² demográficos diminutos. La conclusión fiel es que estos snapshots muestran homogeneización y desalineación dependientes de tarea bajo esta población, prompts y métricas; no que exista una ley universal de colapso, que el entrenamiento la cause o que las simulaciones reproduzcan poblaciones humanas.

Research question

How to distinguish whether a population of agents conditioned by diverse profiles covers human behavioral space, is distributed without degenerate clustering, and preserves multidimensional complexity, and which person attributes survive in BFI, moral judgments, and self-presentations?

Method

Person-by-item matrices are constructed for BFI-44 and 131 moral dilemmas, plus three self-presentation texts per person. On BFI, 1,144 model outputs are compared with 2,058 human responses via coverage and k-NN density; in both domains, effective Likert range, PCA participation, LID, Hopkins, clustering, V-Measure, eta squared, and incremental R² are computed. BFI adds Spearman correlation with assigned OCEAN labels and Cohen d between High and Low groups. The texts are described by keyword mentions, lexical traits, templates, ICC, and, according to the article, embedding geometry. The audit reproduced the person filtering and the human row, inspected the 27 pages, the TeX, and the public commit, and ran directed tests on R², mention detectors, and templates.

Sample: 2,000 initial synthetic combinations of 26 attributes; six public rules exclude 856 identifiers and leave 1,144 persons per model. Each model contributes 44 BFI responses, 131 moral judgments, and three self-presentations per person. The only human reference is 2,058 complete BFI rows from Twin-2K-500.

Findings

  • The published results separate distinct failures: high coverage with low complexity, strong clustering, impoverished Likert vocabulary, and complexity outside human support.
  • Qwen3-4B obtains the highest BFI coverage (0.80) but LID 7.3; Claude-Haiku-4.5 obtains coverage 0.71 and LID 5.4, versus human LID 14.4.
  • CoSER-Llama-8B presents the most compressed BFI (EffL 1.36, coverage 0.16, Hopkins 0.91) but the highest moral EffL (4.27) and moral LID 45.3.
  • MiniMax-M2-Her combines BFI LID 22.3 and Hopkins 0.50 with human coverage 0.06 and fidelity 0.41, compatible with complex but misaligned variation.
  • The human row of the public artifact reproduces exactly: n 2,058, EffL 3.69, PR 11.2, LID 14.4, Hopkins 0.568, separation 5.22, coverage 1, and density 0.474.
  • The public filtering exactly reproduces 856 exclusions and 1,144 retained profiles, but via deterministic rules and not via the described manual review.
  • The code audit detects that categorical R², gender mentions, and template normalization do not correctly implement the narrated method.
  • Table 2 and the text disagree on mention means: 0.89/0.86/0.60/0.33/0.25 in the table versus 0.91/0.90/0.62/0.36/0.27 in the prose.

Limitations

  • The outputs of the ten models, the analyzed matrices, and the intermediate results are not published; the main tables are not regenerable from the snapshot.
  • The embedding and self-presentation geometry code needed to reproduce Table 8 is missing.
  • The description of manual review of 856 profiles does not match the six released deterministic rules; 42.8% of the sample is removed without sensitivity analysis.
  • The incremental R² applies LabelEncoder and retains numeric columns, thus introducing arbitrary ordinality instead of the declared one-hot.
  • The gender detector uses substrings: 'his ' appears in 'this ', 'man' in 'human' or 'woman', and 'her ' in 'rather ', potentially inflating mentions.
  • The template analysis converts openings to lowercase before searching for capitalized names, so it does not normalize names as claimed.
  • Human LID is computed on standardized coordinates and model LID on raw Likert values; neither intervals nor sensitivity to k are applied.
  • Coverage uses a single subsample of 500 points per population; human coverage 1 is a comparison of the subset with itself.
  • The code fixes temperature 0.7 for BFI and morality on non-OpenAI providers, contradicting the claim of leaving default parameters.
  • There are no tests, CI, lockfile, version tags, or license, and exact reviews and execution manifests of the ten models are missing.
  • The demographic R² and eta squared are very small; Dom% may appear large because its denominator is also tiny.
  • Morality and self-presentation lack a human reference, so they do not allow measuring realism or collapse relative to humans.

What the study does not establish

  • It does not demonstrate that all LLMs or all persona schemes suffer universal collapse.
  • It does not demonstrate that SFT or RL cause the observed changes between distinct checkpoints.
  • It does not demonstrate that high fidelity causes stereotypes: fidelity and Cohen d share the same target OCEAN labels.
  • It does not demonstrate large or socially predictive demographic effects from R² between 0.0005 and 0.0075 per attribute.
  • It does not validate that self-presentations or moral judgments reproduce real human persons or populations.
  • It provides no evidence on the traits, decisions, or typical language of the sensitive groups included in the prompts.
  • It does not allow reproducing the full results without the missing outputs, corrected code, and exact inference configurations.

Traceability

Scope: Full text

Version: arXiv:2604.24698v1

Consulted source: https://arxiv.org/abs/2604.24698

Review: Codex 27-page visual full-text, TeX, repository, metric implementation, detector smoke-test, construct and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • Llama-3.1-8B-Instruct
  • Qwen3-4B
  • Qwen3-30B-A3B
  • Qwen3-32B
  • Claude-Haiku-4.5
  • MiniMax-M2
  • CoSER-Llama-8B
  • CoSER-Qwen-32B
  • HER-32B
  • MiniMax-M2-Her

Instruments and metrics

  • Big Five Inventory BFI-44
  • 131 moral dilemmas
  • Three self-introductions per persona
  • Density and Coverage k-NN
  • Hopkins statistic
  • Local Intrinsic Dimensionality
  • Participation ratio
  • Spearman persona fidelity
  • Cohen's d
  • Eta-squared and incremental R-squared
  • Keyword mention rates
  • Intraclass correlation

Data used

  • Public 2,000-profile synthetic persona input
  • Twin-2K-500 BFI reference
  • Public 131-scenario moral input
  • Unreleased ten-model BFI response matrices
  • Unreleased ten-model moral response matrices
  • Unreleased self-introduction outputs

Evidence and location

  • Framework, design, results, ethics, metrics, prompts, tables, and limitations: arXiv:2604.24698v1, 27 pages inspected; §§2-4 and appendices A-J
  • Code, profiles, exclusion rules, scenarios, human reference, and public pipeline: GitHub Algoroxyolo/PersonaCollapse commit b7a52694ab7ad2dd413e16143a93c2c14e9caf5d; tree 95646bb79b985984b332764459891c28047d481d
  • Reproduction of filtering and the human row; audit of R², mentions, templates, artifacts, and limits of interpretation: reports/verification/article-357-persona-collapse-metric-code-data-and-claim-audit.json