Synthetic Personality Review

The Pragmatic Persona: Discovering LLM Persona through Bridging Inference

Personas, identity, and agents2026arXivApproved editorial review

Authors: Jisoo Yang, Jongwon Ryu, Minuk Ma, Trung X. Pham, Junyeong Kim

Keywords: Injected persona recovery, Bridging inference, Discourse graphs, Persona conditioning, Adaptive interviewing, Big Five labels, Cosine similarity, Prompt leakage, Construct validity, Reproducibility

Source: Open primary source (opens in a new tab)

5
Authors
7
Findings
11
Limitations
3
Evidence

Editorial summary

English

This preprint presents PD-Agent, a pipeline that interviews a target LLM for 3–5 turns, asks another LLM to extract seven predefined bridging-inference types, builds a graph, and predicts four attributes. The experiment does not discover an inherent persona: it first injects a social role, one binary Big Five trait, one background value, and one interest into the target, then tries to recover that configuration. Table 3 reports PD-Agent cell similarities of 0.87–0.99 and method averages of 0.90–0.98, above Vanilla and Frequency-Aware; o1-mini has the highest printed average. The defensible contribution is a structured discourse-reasoning proposal for recovering induced personas. The public evidence does not support the strongest claims. The predictor is told which Big Five trait and which background and interest categories to complete; the similarity scorer serializes those same labels in both truth and prediction, which can reward shared text even when the predicted value is wrong. Every released experiment script fixes GPT-4 as the agent and does not instantiate the six backbones or six targets in the table; only Qwen3-1.7B matches the reported panel. The repository publishes no dialogue corpus, sampled-persona manifest, run-level results, table data, aggregation, or statistical tests. Its two ablation scripts are identical, default to Qwen3-1.7B, run once, and overwrite fixed outputs. The paper claims significance and a standard deviation below 0.03 across five runs but gives no per-cell n, test, p-value, interval, or dispersion. The faithful conclusion is that the printed table assigns higher recovery scores to a multi-stage LLM pipeline under an artificial, partially disclosed schema; it does not establish a latent identity, structural encoding of persona traits, or a causal benefit from the graph.

Español

Este preprint presenta PD-Agent, una tubería que entrevista durante 3–5 turnos a un LLM objetivo, extrae con otro LLM siete tipos predefinidos de inferencia puente, construye un grafo y predice cuatro atributos. El experimento no descubre una personalidad inherente: primero inyecta al objetivo un rol social, un único rasgo Big Five binario, un valor de contexto y un interés, y después intenta recuperar esa configuración. La Tabla 3 publica similitudes de 0,87–0,99 para PD-Agent y promedios de 0,90–0,98, superiores a Vanilla y Frequency-Aware; o1-mini obtiene el promedio más alto. La aportación defendible es una propuesta de razonamiento discursivo estructurado para recuperar personas inducidas. La evidencia pública no permite sostener las conclusiones más fuertes. El predictor conoce de antemano qué rasgo Big Five y qué categorías de contexto e interés debe completar; la similitud serializa esas mismas etiquetas en verdad y predicción, lo que puede premiar coincidencias aunque el valor sea erróneo. El repositorio fija GPT-4 como agente en todos los scripts y no implementa los seis backbones ni los seis objetivos de la tabla; solo Qwen3-1.7B coincide con el panel reportado. No publica diálogos, personas muestreadas, resultados por ejecución, datos de la tabla, agregación ni tests estadísticos. Los dos scripts de ablación son idénticos, usan Qwen3-1.7B, ejecutan una sola repetición y sobrescriben salidas. El texto afirma significación y desviación estándar menor de 0,03 en cinco corridas, pero no da n por celda, pruebas, p-valores, intervalos ni dispersiones por celda. La conclusión fiel es que la tabla impresa atribuye mejores puntuaciones de recuperación a una tubería LLM multietapa bajo un esquema artificial y parcialmente revelado; no demuestra una identidad latente, que los rasgos estén codificados en la estructura discursiva ni que el grafo cause la mejora.

Research question

Can an interview pipeline, extraction of seven bridge inference relations, graph construction, and LLM reasoning recover four previously injected person attributes in another LLM better than two baselines?

Method

A person is sampled without a retained seed with four fields: social role, one of five Big Five traits with a yes/no value, a context category and value, and an interest category and value. That configuration is inserted into the system prompt of the target LLM. A reasoning agent conducts an adaptive interview of 3-5 turns, extracts anchor-anaphora pairs classified into seven relations, constructs a graph, and uses the conversation and derived representations to predict the person. Vanilla, Frequency-Aware, and PD-Agent are compared via cosine similarity across four dimensions. The audit visually inspected the 15 pages, the TeX, the commit and its history, compiled the Python sources, and contrasted model configuration, sampling, revealed template, metric, ablations, artifacts, and statistical claims.

Sample: The article crosses six reasoning backbones with three small and three large targets, but does not report how many persons are sampled per cell nor publish the sample manifest. It says it repeats five times and obtains a standard deviation below 0.03. Each case uses a synthetic person of four fields and an interview of 3-5 turns. The current repository only offers loose scripts with GPT-4 as the agent and default targets Qwen3-1.7B, Llama-3.2-1B, and deepseek-llm-7b-base; only the first matches the table.

Findings

  • Table 3 prints for PD-Agent similarities from 0.87 to 0.99 and averages per backbone from 0.90 to 0.98; o1-mini obtains the highest printed average.
  • In the table, PD-Agent outperforms Vanilla and Frequency-Aware in all shown combinations; the maximum reported improvement over frequency is +0.15.
  • The described qualitative errors concentrate on context, where clues such as relocations may confuse location with family situation.
  • The real objective is to recover values of a person explicitly inserted into the prompt, not to discover an inherent identity of the model.
  • The prediction template reveals the trait name and the context and interest categories, leaving to infer mainly their values and the role.
  • The public code does not configure the experimental panel of the table and lacks data, aggregation, and statistical analysis to reproduce it.
  • The public metric shares textual labels between truth and prediction and uses hidden states from a causal LLM, without validation that they measure semantic correctness of attributes.

Limitations

  • The number of persons per cell, the complete sampling scheme, and a reproducible seed are not reported.
  • There is no Big Five inventory: personality is reduced to a single chosen trait and a yes/no label.
  • The predictor knows in advance the personality trait and the context and interest categories.
  • Similarity may be inflated by shared text and has no demonstrated calibration between embedding spaces of distinct models.
  • Significance claims lack a test, p-values, intervals, and per-cell deviations.
  • The scripts fix GPT-4 and do not reproduce the six reported backbones; two of three default targets also do not belong to the table.
  • Only the main Qwen script computes the actual similarity; Llama and DeepSeek do not share a uniform evaluation or leak redaction path.
  • No persons, dialogues, graphs, predictions, per-run results, Table 3 data, or aggregation code are published.
  • The ablation scripts are identical, NUM_RUNS=1 and overwrite files, so they do not support the stability of five runs.
  • There is no human annotation of bridge relations, psychometric validation, human person criterion, or human participants.
  • Unfixed dependencies, absence of CI, unit tests, lockfile, and license reduce software reproducibility.

What the study does not establish

  • It does not discover an inherent or spontaneous personality of the target LLM.
  • It does not demonstrate that person traits are encoded in the structure of the discourse.
  • It does not isolate that the graph, its centrality, or the bridge relations cause the improvement.
  • It does not prove that o1-mini is better due to its reasoning or that target size causes greater recoverability.
  • It does not establish statistical significance with the published information.
  • It does not validate the recovered traits against persons, psychometrics, or human annotators.
  • It does not allow reproducing Table 3 from the public snapshot.
  • It does not support generalization to free conversation, multidimensional person, other languages, or modalities.

Traceability

Scope: Full text

Version: arXiv:2604.24079v2

Consulted source: https://arxiv.org/abs/2604.24079

Review: Codex 15-page visual full-text, TeX, repository-history, metric, construct, statistical and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o
  • Claude 3.5 Sonnet
  • Gemini 1.5 Pro
  • o1-mini
  • DeepSeek-V3
  • Llama-3.1-70B
  • Qwen3-1.7B
  • Llama-3.1-8B
  • Gemini-2.5-Flash
  • Qwen3-30B
  • Qwen3-80B

Instruments and metrics

  • Four-field injected persona schema
  • Adaptive 3–5-turn interview
  • Seven bridging-relation taxonomy
  • Bridging-inference graph
  • Graph centrality
  • Embedding cosine similarity
  • Vanilla baseline
  • Frequency-Aware baseline

Data used

  • Randomly generated injected personas, not released
  • Generated interview dialogues, not released
  • Generated bridging relations and graphs, not released
  • Run-level predictions and Table 3 data, not released

Evidence and location

  • Framework, injected person, interview, prompts, models, full table, results, limitations, and credits: arXiv:2604.24079v2, 15 pages inspected; §§3-6 and Table 3
  • Implementation, schemas, configurations, metric, ablations, demo, and deleted results history: GitHub JiSoo-Yang/Persona_Bridging commit b6d4d2b7e76a8f3ea32f71b5903c5901d5ca0b0f; tree 1d53b85c5916ab35c5e99f8c1bce7a96ce7d3c72
  • Construct audit, template leaks, metric validity, statistics, artifacts, and reproducibility: reports/verification/article-358-pragmatic-persona-injected-ground-truth-metric-code-and-reproducibility-audit.json