Stories of Your Life as Others: A Round-Trip Evaluation of LLM-Generated Life Stories Conditioned on Rich Psychometric Profiles

Personas, identity, and agents2026arXivApproved editorial review

Authors: Ben Wigler, Maria Tsfasman, Tiffany Matej Hrkalovic

Keywords: Personality, Persona conditioning, Psychometrics, Human simulation, Safety and bias

Source: Open primary source (opens in a new tab)

3
Authors
8
Findings
13
Limitations
8
Evidence

Editorial summary

English

This paper proposes a round-trip evaluation: turn human psychometric profiles into synthetic life stories and test whether other LLMs recover the original scores from the stories alone. It uses PARSEL, a corpus of conversations and cooperative tasks with 297 participants; 290 have complete profiles and 248 have at least three conversations. Claude Opus 4.6 receives all 60 HEXACO items and six domains, nine additional subscales, trust, psychopathic traits, and social interaction anxiety, biographical facts extracted from conversations, and an appearance description derived from webcam images. It writes an approximately 1,000-word immersive profile. Ten generators produce 24-turn, roughly 8,000-word McAdams interviews; Sonnet 4.6, GPT-5.4, and Gemini 3 Flash score them blindly. In the primary GPT-4.1→Sonnet path, mean source-to-recovered correlation is r=.750, bootstrap 95% CI [.730, .768], ranging from .682 for Conscientiousness to .825 for Extraversion. The nine non-HEXACO subscales reach r=.314-.645. The three primary generators scored by Sonnet are close: GPT-4.1 .750, Gemini 3 Flash .744, and Grok 4.1 Fast .740. In five-way matching, Haiku and Grok select the correct personality-only profile on 79% of trials and Gemini on 95.2%, although the four distractors are random and Gemini also prepared the stripped profiles. The central result is useful but should be read as designed information transfer: the input contains exact scores and Claude explicitly translates them into facet language and examples before story generation. r=.750 shows that this signal survives a score→profile→story→score channel; it does not establish autonomous, natural, or stable personality. The paper calls .887 a human ceiling and describes .750 as 85% of it, but .887 comes from another sample and a 13-day HEXACO-100 retest, whereas this study uses HEXACO-60 and synthetic text. It is contextual reference data, not a comparable held-out ceiling. The real-conversation validation contains a material statistical error: the text, abstract, and Table 7 caption say that nine of ten story–conversation correlations survive Bonferroni at alpha .005. The reported p values show only five do: vulnerability, agency, emotional valence, dominance, and emotional intensity. Communion p=.008, warmth .010, emotional complexity .013, and disclosure depth .039 are significant only without that correction; humor is not. The real-conversation emotional-variability correlation, r=.170 and p=.007, also misses .005 and uses the same participants and coders, making it within-sample convergence rather than independent replication. Two further confounds matter. Biographical facts supplied to story generation come from the same conversations later used as criterion, creating a conversation→prompt→story→comparison path without a psychometrics-only ablation. Story and conversation features may also correlate because both depend on the same profile; the study does not control the 15 supplied constructs to establish incremental individual fidelity. The same LLM rubric coders label both contexts, while conversation reliability is modest, three-coder ICC .483. Methods say three annotators are averaged, yet result tables use only the Gemini–Haiku pair after calling it the best pair, with no prespecified rule for dropping GPT-5.4 Mini. The Jaccard check finds no near-verbatim item copying, but cannot detect semantic paraphrase, and the paper itself counts 7.9 thematically overlapping self-descriptions per story. The main text also says nine generator–scorer pairs; after self-scoring is excluded, Table 4 contains eight nonempty primary pairs with the same r=.719-.750 range. Models are compared at different N, 290 or 154; prompt and scoring choices are selected on pilots reused in evaluation; and one diffusion model cannot causally locate the effect in training data. For privacy, full profiles, dark traits, anxiety, conversations, and appearance pass through several providers. The original PARSEL paper reports approval VCWE-2021.168, but this preprint anonymizes the identifier and does not clarify whether consent and approval cover this reuse, API transfers, or impersonating story generation. No code, prompts, outputs, or analyses are public; release is promised upon acceptance. The evidence supports robust transfer of supplied scores into long text, not individual human-behavior simulation or a validated replacement for psychometric assessment.

Español

Este trabajo propone una evaluación de ida y vuelta: convertir perfiles psicométricos humanos en relatos de vida sintéticos y comprobar si otros LLM recuperan los puntajes originales leyendo solo esos relatos. Usa PARSEL, un corpus de conversaciones y tareas cooperativas con 297 participantes; 290 tienen perfiles completos y 248, al menos tres conversaciones. Claude Opus 4.6 recibe los 60 ítems y seis dominios HEXACO, nueve subescalas adicionales, confianza, rasgos psicopáticos y ansiedad de interacción, hechos biográficos extraídos de conversaciones y una descripción de apariencia obtenida de webcam. Con ello redacta un perfil inmersivo de unas 1.000 palabras. Diez generadores producen entrevistas McAdams de 24 turnos y unas 8.000 palabras; Sonnet 4.6, GPT-5.4 y Gemini 3 Flash las puntúan a ciegas. En la ruta principal GPT-4.1→Sonnet, la correlación media entre puntaje original y recuperado es r=0,750, IC bootstrap 95% [0,730, 0,768], desde 0,682 en Conscientiousness hasta 0,825 en Extraversion. Las nueve subescalas no HEXACO alcanzan r=0,314–0,645. Los tres generadores principales, puntuados por Sonnet, quedan muy próximos: GPT-4.1 0,750, Gemini 3 Flash 0,744 y Grok 4.1 Fast 0,740. En una tarea de emparejamiento cinco-alternativas, Haiku y Grok identifican el perfil correcto en 79% de los casos y Gemini en 95,2%, aunque los cuatro distractores son aleatorios y Gemini también preparó los perfiles anonimizados. El resultado central es valioso pero debe interpretarse como transmisión diseñada de información: el input contiene los puntajes exactos y Claude los traduce explícitamente a lenguaje de facetas y ejemplos antes de que se genere la historia. r=0,750 demuestra que esa señal sobrevive al canal puntaje→perfil→relato→puntaje; no demuestra una personalidad autónoma, natural o estable. El artículo llama a 0,887 «techo humano» y presenta 0,750 como 85% de él, pero 0,887 procede de otra muestra, un retest de 13 días con HEXACO-100, mientras aquí se usa HEXACO-60 y texto sintético. Es una referencia contextual, no un techo held-out comparable. La validación con conversación real presenta un error estadístico material: texto, abstract y Tabla 7 afirman que 9 de 10 correlaciones relato–conversación sobreviven Bonferroni con α=0,005. Según los p publicados, solo cinco cumplen: vulnerabilidad, agencia, valencia emocional, dominancia e intensidad emocional. Comunión p=0,008, calidez 0,010, complejidad 0,013 y profundidad de revelación 0,039 solo son significativas sin esa corrección; humor no lo es. La correlación de variabilidad emocional en conversaciones, r=0,170 y p=0,007, tampoco cruza 0,005 y procede de la misma muestra y los mismos codificadores, por lo que es convergencia interna, no réplica independiente. Hay dos confusores adicionales. Los hechos biográficos que alimentan el relato se extraen de las mismas conversaciones usadas después como criterio, creando una ruta directa conversación→prompt→relato→comparación sin ablación psicometría-only. Además, relato y conversación pueden correlacionar porque ambos dependen del mismo perfil; no se controla el perfil para demostrar fidelidad individual adicional. Las mismas etiquetas LLM se aplican en ambos contextos, con fiabilidad modesta en conversación, ICC de tres codificadores 0,483. El método dice promediar tres anotadores, pero las tablas usan solo el par Gemini–Haiku después de identificarlo como el mejor, sin regla preespecificada para excluir GPT-5.4 Mini. La búsqueda Jaccard no encuentra copia casi literal de ítems, pero no detecta paráfrasis semánticas y el propio trabajo cuenta 7,9 autodescripciones temáticamente solapadas por relato. El texto también habla de nueve pares generador–scorer; al excluir self-scoring, su Tabla 4 contiene ocho pares primarios no vacíos y el mismo rango r=0,719–0,750. Los modelos se comparan con N distintos, 290 o 154; las decisiones de prompt y scoring se seleccionan en pilotos reutilizados en la evaluación; y un solo modelo de difusión no permite atribuir causalmente el patrón a datos de entrenamiento. En privacidad, el estudio procesa perfiles completos, rasgos oscuros, ansiedad, conversaciones y apariencia a través de varios proveedores. El PARSEL original informa aprobación VCWE-2021.168, pero el preprint anonimiza el identificador y no aclara si consentimiento y aprobación cubren esta reutilización, transferencias API o generación de impersonaciones. No hay código, prompts, outputs ni análisis públicos: se prometen tras aceptación. La evidencia apoya una transferencia robusta de puntajes a texto largo, pero no una simulación individual del comportamiento humano ni una alternativa ya validada a la evaluación psicométrica.

Research question

Does human psychometric information survive being transformed into narrative profiles and life stories generated by LLMs, can other LLMs recover it, and are the characteristics of those stories related to real conversations of the same people?

Method

Claude Opus 4.6 transforms 15 psychometric constructs, biographical facts, and appearance of 290 participants into prose profiles. Ten LLMs generate 24-turn life interviews; three scorers reconstruct HEXACO and other scales. Correlations, matching with four distractors, Jaccard search, bias decomposition, LLM coding of ten content traits, ICC, comparisons with 577 conversations, and BERTopic are calculated.

Sample: PARSEL gathers 297 participants; 290 complete profiles are used. Conversation analyses include 248 people with at least three conversations, 577 conversations. GPT-4.1, Gemini 3 Flash, Grok 4.1 Fast, and Mercury 2 run with N=290; most additional generators, with a random subset N=154. Pilots use 15 or 50 participants with no subsequent exclusion from the final analysis.

Findings

  • The GPT-4.1 to Sonnet route recovers HEXACO with mean r 0.750, 95% CI [0.730, 0.768], and the six domains fall between 0.682 and 0.825.
  • The nine additional subscales are significant with Bonferroni and span r=0.314 to 0.645; interpersonal/affective ones are recovered better.
  • Three main generators scored by Sonnet land at 0.750, 0.744, and 0.740; the extended table mixes N=290 and N=154 and does not establish a stable ranking.
  • Matching with random distractors achieves 79% with Haiku/Grok and 95.2% with Gemini versus 20% chance; it shows coarse discrimination, not fine identity recovery.
  • Five of ten, not nine, story-conversation correlations meet the declared Bonferroni alpha=0.005; their magnitudes are small, up to r=0.268.
  • Valence variability correlates with Emotionality in stories, r=0.303, and in conversations, r=0.170/p=0.007; the second is convergent evidence but does not exceed alpha=0.005 nor is it an independent replication.
  • In stories, 55 of 204 content-HEXACO associations survive Bonferroni, with vulnerability-Emotionality r=0.744 as a strong example.
  • Unconditioned models show a shared extreme profile; its causal attribution to alignment remains unidentified.

Limitations

  • The round trip starts from exact scores translated into prose, so it measures transmission of designed signal and not emergence of a natural personality.
  • The assumed human ceiling uses another sample, HEXACO-100, and a 13-day retest; it is not held-out nor directly comparable with synthetic HEXACO-60.
  • The 9/10 claim with Bonferroni is incorrect according to Table 7 itself; only five p meet 0.005.
  • The conversations used as criterion contribute biographical facts to the prompt beforehand, and there is no ablation without those facts to rule out leakage.
  • The shared psychometric profile is not controlled; cross-associations may be mediated by the same traits rather than reflecting additional individual fidelity.
  • The same LLMs code story and conversation, conversational reliability is low/modest, and the tables use the best pair although the method announces three annotators.
  • Jaccard >0.7 only detects near-literal copying, not paraphrase; 7.9 self-descriptions per story are thematically close to the items.
  • Matching uses random distractors and Gemini participates in both stripping and matching, which facilitates the task and confounds its superior result.
  • There are eight primary pairs visible after excluding self-scoring, not the nine declared; furthermore models with different sample sizes are compared.
  • The selection of prompt-generator and scoring format reuses participants from the pilots; there is no fully held-out selection.
  • Human-authored LSI, human content/personality judges, quality validation, behavioral results, or cultural/linguistic generalization are missing.
  • Data, code, prompts, exact versions, outputs, annotations, and scripts are not public and depend on mutable proprietary APIs.
  • It is not clarified whether the original consent covers sending psychometry, conversations, and appearance to multiple providers nor how non-identification of stories is audited.

What the study does not establish

  • It does not demonstrate that LLMs possess an internal, autonomous, or stable personality; it demonstrates recovery of supplied information.
  • It does not equate r=0.750 with 85% human reliability on the same test, population, or context.
  • It does not demonstrate nine corrected behavioral bridges; only five pass the specified correction.
  • It does not establish an independent replication of the reactivity pattern nor that p=0.007 exceeds alpha=0.005.
  • It does not test individual simulation beyond the profile, because it does not control for shared traits, conversation leakage, or encoder bias.
  • It does not demonstrate that the signal causally originates in pretraining or that it is independent of autoregressive generation.
  • It does not validate synthetic LSI stories as a naturalistic substitute for questionnaires, human interviews, or clinical/occupational assessment.
  • It does not guarantee future reproducibility with proprietary models without snapshots, restricted data, and artifacts not yet published.
  • It does not establish the absence of sensitive information or impersonation risk by not finding direct identifiers.

Traceability

Scope: Full text

Version: arXiv:2604.06071v1 preprint; under review at COLM

Consulted source: https://arxiv.org/pdf/2604.06071v1

Review: Codex 19-page visual full-text, TeX/appendix, statistical recalculation, psychometric construct, conversation-leakage, annotator-selection, human-data privacy, artifact and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • Claude Opus 4.6 (immersive-profile generator)
  • GPT-4.1
  • Gemini 3 Flash
  • Grok 4.1 Fast
  • GPT-4.1 Mini
  • GPT-5.4
  • Mercury 2
  • Claude Haiku 4.5
  • Gemini 3 Flash Lite
  • Claude Sonnet 4.6
  • Hermes 4 405B
  • GPT-5.4 Mini (content coder)
  • OLMo contamination-control scorer
  • Gemma 3, Hermes 4 70B and Aion 2.0 in prompt-generator pilots

Instruments and metrics

  • HEXACO-60 and six domain means
  • Four trust subscales
  • Four Psychopathic Personality Traits subscales
  • Social Interaction Anxiety Scale
  • Adapted 24-turn McAdams Life Story Interview
  • Pearson correlation and Fisher-z/bootstrap confidence intervals
  • Five-way masked profile matching
  • Sentence-level Jaccard questionnaire-overlap check
  • Ten-feature LLM content rubric
  • ICC(2,1) inter-coder reliability
  • BERTopic with UMAP and HDBSCAN

Data used

  • PARSEL multimodal partner-selection corpus
  • 290 complete PARSEL psychometric profiles
  • 577 PARSEL conversations from 248 participants
  • Public OSF HEXACO-PI-R test-retest data as an external reference
  • LLM-generated immersive profiles and life-story narratives, not publicly released

Evidence and location

  • Questions, contributions, and scope of the round trip: arXiv v1, pp. 1-3, Abstract, Introduction and Related Work
  • PARSEL, 15 constructs, three stages, and use of biography/appearance: arXiv v1, pp. 4-5, sections 3.1-3.4
  • Recovery HEXACO, external test-retest, subscales, and models: arXiv v1, pp. 5-7 and 16-17, sections 4.1-4.2 and Tables 4-5
  • Error 9/10 Bonferroni and reactivity p=0.007: arXiv v1, pp. 7 and 17-18, section 4.3 and Tables 7-9; recalculation from reported r and N=248
  • Bias, declared limits, reproduction, and ethics: arXiv v1, pp. 7-10, Discussion, Limitations, Reproducibility Statement and Ethics Statement
  • Selection of coders, pilots, controls, and exact table values: arXiv v1, pp. 14-19, Appendices A-L
  • Provenance of PARSEL and test-retest reference: TU Delft PARSEL publication portal DOI 10.1109/TAFFC.2025.3600687 and OSF node wz3du
  • Integrated audit of transfer, leakage, statistics, privacy, and reproducibility: reports/verification/article-376-life-story-roundtrip-score-transfer-conversation-leakage-bonferroni-annotator-selection-privacy-artifact-and-reproducibility-audit.json