Synthetic Personality Review

Do LLMs Experience an Internal Polylogue? Investigating Reasoning through the Lens of Personas

Personas, identity, and agents2026arXivApproved editorial review

Authors: Nils A. Herrmann, Leander Girrbach, Kirill Bykov, Zeynep Akata

Keywords: Reasoning personas, Persona vectors, Activation trajectories, MMLU-Pro, Semantic faithfulness, Functional faithfulness, Paragraph-conditioned steering, Sparse logistic regression, Mean Reciprocal Rank, Reasoning-time control

Source: Open primary source (opens in a new tab)

4
Authors
7
Findings
16
Limitations
3
Evidence

Editorial summary

English

The paper defines a “polylogue” as the time series of projections between hidden activations and eight synthetic directions labelled Interpreter, Analyst, Planner, Solver, Explorer, Verifier, Monitor and Arbiter. On MMLU-Pro these signals carry correctness information, but semantic correspondence with paragraph labels assigned by another LLM is modest: polylogue beats the empirical-frequency baseline in only two of four models. Correctness prediction is unevenly competitive with eight fixed random directions and a PCA last-activation baseline. Paragraph steering improves three models by 0.8, 2.2 and 4.4 points but harms Phi-4 by 14.5 points; because every model uses 504 questions, the mean change is -1.775 points. Control and treatment also use different generation engines, and the released code injects vectors one transformer stage after their extraction locus, so polylogue-specific causality is not isolated. “Polylogue” is a descriptive metaphor: the study does not demonstrate experience, consciousness, inner agents or a literal internal dialogue. The reported signals therefore remain model-, layer-, generator-, and intervention-specific, and require matched-engine controls before they can support a causal interpretation.

Español

El trabajo propone “polylogue” como la serie temporal de proyecciones entre activaciones internas y ocho direcciones sintéticas etiquetadas como Interpreter, Analyst, Planner, Solver, Explorer, Verifier, Monitor y Arbiter. En MMLU-Pro, estas señales contienen información sobre la corrección, pero su correspondencia semántica con etiquetas de párrafo asignadas por otro LLM es modesta: superan el baseline de frecuencia sólo en dos de cuatro modelos. La predicción de corrección resulta competitiva de forma desigual con ocho direcciones aleatorias y con un baseline PCA de la activación final. Un steering por párrafos mejora 0,8, 2,2 y 4,4 puntos en tres modelos, pero reduce 14,5 puntos Phi-4; al tener todos 504 preguntas, el cambio medio es -1,775 puntos. Además, control y tratamiento usan motores de generación distintos y el código aplica el vector una capa después de su locus de extracción, por lo que no queda aislada una causalidad específica del polylogue. “Polylogue” es una metáfora descriptiva: el estudio no demuestra experiencia, consciencia, agentes internos ni un diálogo literal.

Research question

Can temporal trajectories of alignment with eight synthetic directions of "reasoning person" reflect the functional content of a reasoning chain, predict its correctness, and serve as causal steering targets during generation?

Method

The authors generate contrastive vectors for eight functional labels, project token-by-token activations from four open models onto them, and aggregate the signals by paragraphs. They evaluate semantic fidelity using Llama-3.3-70B labels and MRR; functional fidelity with nested L1 logistic regression over 186 features and random/PCA baselines; and causal steering by transferring five positional coefficients at paragraph intervals on a non-overlapping MMLU-Pro subset. This review checked the 10 pages, the complete TeX, the tables, the official repository, and its public submodule.

Sample: MMLU-Pro contains 12,032 questions across 14 domains. Functional fidelity uses 190 questions per domain (2,660) and causal steering 36 per domain (504), on stratified and non-overlapping subsets. Each of the four models is evaluated on those samples. The exact quantities of contrastive pairs retained to construct each vector are not published in the article.

Findings

  • Semantic MRR Polylogue/Frequency: Qwen 0.35/0.59; DeepSeek 0.51/0.50; Phi 0.54/0.49; Llama 0.47/0.51. Only DeepSeek and Phi surpass the frequency baseline.
  • AUC Polylogue/Random/Activation: Qwen 0.59/0.59/0.51; DeepSeek 0.81/0.78/0.84; Phi 0.87/0.84/0.85; Llama 0.76/0.70/0.76.
  • The random baseline is a single fixed extraction of eight directions; the semantic advantages are 0 in Qwen, 0.03 in DeepSeek/Phi, and 0.06 in Llama.
  • Steering: Qwen 63.3→64.1%; DeepSeek 62.3→64.5%; Phi 53.0→38.5%; Llama 44.4→48.8%.
  • With 504 questions per model, the mean accuracy change is -1.775 points, despite improving the count in three of four.
  • The causal set does not reuse the questions from functional training, a positive methodological decision.
  • The polylogue metaphor describes projected signals and does not evidence experience or discrete agents.

Limitations

  • Eight synthetic labels forced to be mutually exclusive, without human validation, inter-annotator reliability, or independent taxonomy.
  • The same Llama-3.3-70B participates in configuration selection and semantic labeling.
  • Whitening and frequency baseline fitted on the entire evaluated corpus.
  • A single extraction of random directions; without a distribution of repeated nulls.
  • Coefficients from a full refit and temporally correlated inputs interpreted as "which persona matters".
  • Twenty bins duplicate paragraphs in short answers and depend on the double line break format.
  • Without repetition of splits, calibration, analysis by domain, or correction for multiplicity.
  • Control generated with vLLM and treatment with Transformers, without a Transformers no-op control.
  • An offset of one layer between output_hidden_states used to extract vectors and model.layers used for steering.
  • The LogitsProcessor updates the mask after the forward pass, not before as the appendix claims.
  • The DeepSeek strategy asks for positive and negative Interpreter in different intervals, but the code dictionary retains a single sign per feature.
  • Without random steering, permuted schedule, no-op hook, intervals, paired tests, or repeated seeds.
  • The explanation of the Phi drop by "high non-linearity" is speculative.
  • Environment without lock, empty dependencies in pyproject, hardcoded local paths, without CI/tests/releases or exact outputs from the paper.
  • The public submodule uses an SSH URL that blocks recursive clone without a GitHub key.
  • The audited public snapshot is four days later than arXiv v1 and is not frozen as an experimental release.

What the study does not establish

  • That an LLM experiences, is conscious, or maintains a literal internal dialogue.
  • That eight discrete agents, modules, or persons exist within the model.
  • Psychometric personality or equivalence with human cognitive episodes.
  • That the eight labels are the natural or universal ontology of LLM reasoning.
  • That the visible chain of thought is a faithful causal explanation.
  • General semantic advantage over random directions or over final activation.
  • Mean, robust, or safe improvement of reasoning across models.
  • Person-, paragraph-, or polylogue-specific causality under comparable engines and layers.
  • Transfer to other tasks, families, domains, or black-box APIs.
  • Exact independent reproduction of the tables and results.
  • Acceptance at ICML 2026.

Traceability

Scope: Full text

Version: arXiv:2605.09159v1; official code commit 0bac577390415a47c6094b8f68ceb58030083fb5

Consulted source: https://arxiv.org/abs/2605.09159v1

Review: Codex 10-page visual full-text, complete TeX, construct, semantic, frequency-baseline, prediction, steering-code and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • Qwen2.5-14B-Instruct
  • DeepSeek-R1-Distill-Qwen-14B
  • Phi-4-reasoning
  • Llama-3.1-Nemotron-Nano-8B-v1
  • Llama-3.3-70B-Instruct (annotation and persona-configuration judge)

Instruments and metrics

  • Contrastive response-mean persona vectors
  • Token-level activation projection trajectories
  • LLM single-label paragraph annotation
  • Mahalanobis whitening
  • Mean Reciprocal Rank
  • 186-feature paragraph trajectory representation
  • Nested L1 logistic regression
  • 128-component PCA activation baseline
  • Paragraph-conditioned activation steering

Data used

  • MMLU-Pro

Evidence and location

  • Text, figures, tables, formulas, and methodological appendix: arXiv:2605.09159v1; PDF sha256 9ec3c4edd7dae2b8f6e3825faaff9161a570a334796d6537f0368c22be10eefc; TeX sha256 820c357d87af18eedfa7aa6c46f4b48ddb9b39618d6b0910578ee9b88a6d8593
  • Official implementation of monitoring, prediction, steering, and vector submodule: https://github.com/nils-herrmann/polylogue commit 0bac577390415a47c6094b8f68ceb58030083fb5; persona_vectors commit 5089205503b25f3246c88373c3a0ca6b53c4db90
  • Recalculations of MRR, AUC, aggregated steering, layer conventions, engines, signs, and reproducibility: reports/verification/article-344-polylogue-reasoning-persona-construct-semantic-frequency-baseline-prediction-steering-code-and-reproducibility-audit.json