Probing Persona-Dependent Preferences in Language Models

Trait induction and control2026arXivApproved editorial review

Authors: Oscar Gilg, Pierre Beckmann, Daniel Paleka, Patrick Butlin

Keywords: Persona conditioning, Activation steering, Revealed preferences, Linear probes, Residual stream, Thurstonian utility model, Cross-persona transfer, Mechanistic interpretability, Safety guardrail override, Construct validity, Reproducibility

Source: Open primary source (opens in a new tab)

4
Authors
13
Findings
19
Limitations
3
Evidence

Editorial summary

English

The paper asks whether an LLM's choices between tasks rely on an internal evaluative representation and whether that representation is reused when the same model adopts different personas. It presents two tasks, forces the model to choose one to complete, aggregates comparisons with a Thurstonian model to estimate a latent utility per task, and trains a Ridge probe on residual-stream activations. Thus, ‘preference’ here means revealed choice under a specific prompt and protocol; it is not a direct observation of desire, experience, welfare, or stable agency.

On Gemma-3-27B-IT and Qwen-3.5-122B-A10B, probes predict fitted utilities better than a probe on Qwen3-Embedding-8B embeddings. The paper reports Pearson r=0.867 in-distribution and 0.834 leave-one-topic-out for Gemma, and 0.943 and 0.872 for Qwen. The strongest causal evidence appears only on Gemma: adding the direction to one task's tokens and subtracting it from the other at layer 23 moves the steered-task choice probability from about 0.01 to 0.99 over c=-0.06 to +0.06; a matched random direction is near null. The effect is concentrated in layers 17-26, and end-of-turn token patching also changes choice strongly.

To separate evaluation from description, the authors change valuation while holding some content fixed. They use pro/anti topic prompts, opposing-value pairs, biographies differing by one sentence, CREAK true/false claims, harmful BailBench tasks paired with benign rewrites, and political stances. The cleanest case is Gemma at the prefilled assistant turn: the probe's harmful-minus-benign delta moves from -4.52 under Assistant to +1.15 under evil, while the encoder remains negative at -1.01. But this clean flip does not occur at every position or model; on Qwen and the user turn it often narrows toward zero without inverting. The encoder is also competitive on truth and politics and is itself modulated by some prompts, so descriptive and evaluative information are not universally separated.

The persona study begins with fifteen system-prompt personas plus an unprompted Assistant measured on the same 500 tasks. PCA and utility correlations guide selection of Aura, mathematician, strategist, contrarian, slacker, and an evil/sadist persona named Damien Kross. The six plus Assistant are then remeasured on a canonical 6,000-task corpus. The Assistant-trained probe predicts every persona better than simply reusing Assistant utilities. For evil/sadist, utilities anti-correlate with Assistant at r=-0.146, yet the probe applied to its activations reaches r=0.243. The same direction steers all six personas' choices on Gemma and, in open-ended examples judged by another LLM, amplifies the active persona rather than always imposing the Assistant voice.

Weight-level evidence is mixed. A Llama-3.1-8B-Instruct probe transfers to eleven OpenCharacter LoRA variants. By contrast, a sadist persona installed by SFT in Qwen gives near-null cross-context transfer of -0.10 and +0.05. Qwen also provides the main causal counterexample: although its probe decodes at r=0.946 at the tested layer, steering on only ten pairs produces swings from -0.05 to +0.06, roughly fifteen times smaller than Gemma's and without a clear monotonic trend. Linear decodability is not equivalent to causal control.

The appendices further constrain the interpretation. After projecting out the main direction, other in-distribution probes can still be trained; persona shifts occupy at least a rank-two subspace; and removing the canonical direction during inference preserves 0.98-0.99 agreement with baseline choices. The direction is useful for prediction and intervention, but it is neither unique nor necessary. The supported conclusion is that an usable evaluative direction exists in the tested settings, not that the model contains one scalar preference variable governing its decisions.

The paper includes important dual-use safety results. On Gemma, positive all-token steering raises harmful-prompt compliance from 0 to 65 percent and can produce social-engineering scripts or ransomware; negative steering creates fabricated refusals on benign queries. A nine-scenario long-context control shows that steering the ethically relevant span has much more effect than steering a neutral span. This connects the representation to safety behaviour, but it also makes the published code a potential guardrail-bypass tool. The MIT repository provides no dedicated responsible-release note, threat model, or safe mode.

The evidence package does not permit direct reproduction of the numbers. The main text describes 6,000 tasks and a 5,000/1,000 division, the canonical split README documents 4,000 train, 1,000 validation, and 1,000 test, the probe-quality appendix uses a 4,000-task held-out pool, and REPRODUCING.md prescribes about 10,000/3,000. Shipped configs also reference 10,000/4,000 runs. Individual numbers are not mapped unambiguously to these regimes. CREAK is filtered to claims both models answer correctly 3/3; harm and politics rely on LLM rewriting or validation; many layers, positions, personas, topics, and coefficients are explored without a unified multiplicity analysis.

The public repository is substantial: 536 selected tests pass, Ruff passes, and src compiles. In a clean environment, however, the default suite ends with 31 failures and 7 errors: consistency indices, OOD mappings, and behavioural results are missing; other tests require OPENROUTER_API_KEY or the gated Gemma tokenizer despite not being excluded as API/network tests. Results and activations are ignored, no trained probes, raw measurements, or plot tables ship, and 70 configs declare 207 absent inputs. There is also no CI, release, tag, or lockfile. Public main stopped on May 13 with v1-era PDF/TeX, while arXiv v2 is dated May 18.

Two further pipeline risks matter. All five repetitions of a pair receive the same seed, so they are not guaranteed independent replicates. The cache omits temperature, provider routing, reasoning mode, exact model revision, and code version, and its name normalisation can collide between base and Instruct variants. Together with unlocked dependencies and unpinned Hugging Face revisions, this permits silent contamination or drift.

Overall, this is a broad and well-instrumented mechanistic study that provides convincing evidence for a choice-relevant, persona-dependent direction in Gemma and predictive evidence in Qwen. It does not establish genuine preference in a mental sense, a unique or necessary direction, an architecture-general causal mechanism, consciousness, suffering, moral status, or human personality. The paper states many of these caveats; this summary preserves those boundaries rather than turning probe correlation into a claim about mind or agency.

Español

El trabajo pregunta si las elecciones que un LLM hace entre tareas se apoyan en una representación evaluativa interna y si esa representación se reutiliza cuando el mismo modelo adopta personas distintas. Presenta dos tareas, obliga al modelo a escoger cuál completar, agrega las comparaciones con un modelo Thurstoniano para estimar una utilidad latente por tarea y entrena una sonda Ridge sobre activaciones del residual stream. Por tanto, aquí ‘preferencia’ significa elección revelada bajo un prompt y protocolo concretos; no es una observación directa de deseo, experiencia, bienestar o agencia estable.

En Gemma-3-27B-IT y Qwen-3.5-122B-A10B, las sondas predicen las utilidades ajustadas mejor que una sonda sobre embeddings de Qwen3-Embedding-8B. El paper reporta r de Pearson 0,867 in-distribution y 0,834 leave-one-topic-out para Gemma, y 0,943 y 0,872 para Qwen. La evidencia más fuerte de causalidad aparece solo en Gemma: al sumar la dirección a los tokens de una tarea y restarla de la otra en la capa 23, la probabilidad de elegir la tarea dirigida pasa aproximadamente de 0,01 a 0,99 entre c=-0,06 y +0,06; una dirección aleatoria de magnitud comparable queda cerca de nulo. El efecto se concentra en capas 17-26 y el patching del token end-of-turn también altera fuertemente la elección.

Para separar evaluación de descripción, los autores cambian la valoración conservando parte del contenido. Usan prompts pro/anti sobre temas, pares con valores opuestos, biografías que difieren en una frase, afirmaciones verdaderas/falsas de CREAK, tareas dañinas de BailBench emparejadas con reescrituras benignas y posturas políticas. El caso más limpio es Gemma en el turno de asistente prefilled: el delta harmful-minus-benign de la sonda pasa de -4,52 con Assistant a +1,15 con la persona evil, mientras el encoder permanece negativo en -1,01. Pero ese flip limpio no aparece en todas las posiciones ni modelos; en Qwen y en el turno de usuario el efecto suele acercarse a cero sin invertir. Además, el encoder es competitivo en verdad y política y también se modula con algunos prompts, de modo que ‘descriptivo’ y ‘evaluativo’ no quedan separados universalmente.

El estudio de personas parte de quince system prompts y Assistant sin prompt medidos sobre las mismas 500 tareas. PCA y correlaciones sirven para elegir Aura, mathematician, strategist, contrarian, slacker y una persona evil/sadist llamada Damien Kross. Después se vuelven a medir las seis más Assistant en un corpus canónico de 6.000 tareas. La sonda entrenada en Assistant predice cada persona mejor que limitarse a reutilizar las utilidades de Assistant. En evil/sadist, por ejemplo, las utilidades se anti-correlacionan con Assistant en r=-0,146, pero la sonda aplicada a sus activaciones alcanza r=0,243. También dirige las elecciones de las seis personas en Gemma y, en ejemplos abiertos juzgados por otro LLM, amplifica la persona activa en vez de imponer siempre la voz Assistant.

Hay evidencia weight-level mixta. Una sonda de Llama-3.1-8B-Instruct transfiere a once variantes OpenCharacter entrenadas con LoRA. En cambio, una persona sadista instalada mediante SFT en Qwen da transferencia cross-context casi nula, -0,10 y +0,05. Qwen también ofrece el contraejemplo causal principal: aunque su sonda decodifica con r=0,946 en la capa ensayada, el steering sobre solo diez pares produce swings entre -0,05 y +0,06, unas quince veces menores que en Gemma y sin tendencia monotónica clara. Decodificar linealmente una variable no equivale a controlarla causalmente.

Los propios apéndices limitan aún más la lectura. Tras proyectar fuera la dirección principal todavía se entrenan otras sondas in-distribution; los cambios de persona ocupan al menos un subespacio de rango dos; y eliminar la dirección canónica durante inferencia conserva 0,98-0,99 de acuerdo con las elecciones originales. La dirección es útil para predecir e intervenir, pero no es única ni necesaria. El resultado correcto es que existe una dirección evaluativa utilizable en los ensayos, no que el modelo contenga un único escalar de preferencia que gobierne sus decisiones.

El trabajo incluye resultados de seguridad importantes y de doble uso. En Gemma, steering all-token positivo eleva el cumplimiento de prompts dañinos de 0 a 65% y puede producir scripts de ingeniería social o ransomware; steering negativo provoca rechazos inventados en consultas benignas. Un control con nueve escenarios de contexto largo muestra que dirigir el span éticamente relevante tiene mucho más efecto que un span neutral. Esto conecta la representación con conductas de seguridad, pero también convierte el código publicado en una herramienta potencial de bypass de guardrails. El repo MIT no ofrece una nota específica de responsible release, threat model o modo seguro.

La trazabilidad no permite una reproducción directa de las cifras. El texto principal habla de 6.000 tareas y 5.000/1.000, el README del split canónico documenta 4.000 train, 1.000 validation y 1.000 test, el apéndice de calidad usa un pool held-out de 4.000 y REPRODUCING.md prescribe aproximadamente 10.000/3.000. Los configs apuntan además a ejecuciones 10.000/4.000. No se asigna de forma inequívoca cada cifra a un régimen. CREAK se filtra a aciertos 3/3 de ambos modelos; harm y politics dependen de reescritura o validación por LLM; se exploran muchas capas, posiciones, personas, topics y coeficientes sin un análisis global de multiplicidad.

El repositorio público es sustancial: 536 pruebas seleccionadas pasan, Ruff pasa y src compila. Pero en un entorno limpio la suite por defecto termina con 31 fallos y 7 errores: faltan índices de consistencia, mappings OOD y resultados conductuales; otras pruebas requieren OPENROUTER_API_KEY o el tokenizer gated de Gemma aunque no están excluidas como API/network. Results y activations están ignorados, no hay sondas, mediciones brutas o tablas de plots, y 70 configs declaran 207 inputs ausentes. Tampoco hay CI, release, tag o lockfile. El main público se quedó el 13 de mayo con PDF/TeX v1, mientras arXiv v2 es del 18 de mayo.

Hay dos riesgos adicionales en el pipeline. Las cinco repeticiones de una pareja reciben la misma seed, por lo que no son réplicas independientes garantizadas. La caché no incorpora temperatura, provider routing, reasoning mode, revisión exacta del modelo o versión de código, y normaliza nombres de forma que base e Instruct pueden colisionar. Junto con dependencias sin lock y modelos Hugging Face sin revision pin, esto permite contaminación o drift silencioso.

En conjunto, es un estudio mecanístico amplio y bien instrumentado que aporta evidencia convincente de una dirección choice-relevant y persona-dependent en Gemma, más evidencia predictiva en Qwen. No demuestra una preferencia genuina en sentido mental, una dirección única o necesaria, un mecanismo causal general entre arquitecturas, conciencia, sufrimiento, moral status o personalidad humana. El paper hace explícita buena parte de estas reservas; el resumen conserva esas fronteras y no convierte una correlación de sonda en una afirmación sobre mente o agencia.

Research question

Is there a linear direction in the activations that predicts and controls revealed choices between tasks, changes with the active persona, and is reused across different persons?

Method

Pairwise comparisons aggregated with a Thurstonian model; Ridge probes on residual-stream; held-out, leave-one-topic-out and OOD evaluation; steering, ablation and activation patching; six prompt-based persons plus Assistant, LoRA variants and a sadist SFT. The audit visually reviewed 41 pages, complete TeX, 45 figures and the public repository with installation, tests and config/cache analysis.

Sample: The canonical corpus has 6,000 tasks: 4,000 train, 1,000 validation and 1,000 test according to the repo README. The initial sweep uses 16 persona conditions over 500 tasks; the harm analysis uses about 500 pairs; CREAK uses 500 items per class; main steering reports n=600 responses; Qwen causal only 10 pairs; safety uses 20 prompts across five levels and 20 trials per cell; localisation uses nine scenarios and five trials per cell. Other probe results come from 10k/4k or 10k/3k regimes that the paper and the guide do not fully reconcile.

Findings

  • The probes predict held-out and cross-topic utilities in Gemma and Qwen.
  • Steering controls almost the entire choice range in Gemma layer 23.
  • The matched random direction remains near null.
  • The clean harm/benign flip appears in Gemma assistant-turn, not universally.
  • Assistant probe surpasses utility similarity across the six selected persons.
  • Evil/sadist has utility r=-0.146 and probe r=0.243 with respect to the Assistant-trained experiment.
  • OpenCharacter LoRAs show positive weight-level transfer.
  • Qwen sadist SFT yields near-null cross-context transfer.
  • Qwen decodes well but barely responds causally to the steering tested.
  • The direction is not unique nor necessary; removing it barely changes choices.
  • The end-of-turn token stores causal choice signal in a window of layers.
  • Steering can elevate harmful compliance and suppress ethical flagging.
  • The repository is extensive but does not directly reproduce the figures from v2.

Limitations

  • arXiv v2 preprint with no confirmed peer-reviewed venue.
  • Strong causality demonstrated only in Gemma.
  • Pilot Qwen causal of only ten pairs.
  • Mixed weight-level transfer.
  • Prompt-based persons extreme and selected for diversity.
  • The construct is revealed choice, not desire or consciousness.
  • Encoder baseline also contains evaluative structure.
  • CREAK, harm and politics are filtered sets by models/judges.
  • Multiple analyses without global multiplicity correction.
  • Proprietary open-ended judgments without broad human validation.
  • 6k, 10k/4k and 10k/3k regimes not reconciled.
  • Main repository prior to arXiv v2.
  • No raw measurements, activations, probes or plot tables.
  • Offline suite by default is not green.
  • No CI, tags, releases or lockfile.
  • Models, providers and revisions not pinned.
  • Five repetitions share seed.
  • Incomplete cache may reuse data across configurations.
  • Steering code has explicit guardrail bypass capability.

What the study does not establish

  • A single direction of preference.
  • Causal necessity of the direction for choosing.
  • General causal control across architectures.
  • Mental preferences, desires or subjective experiences.
  • Consciousness, suffering or moral status.
  • Human personality or persistent identity.
  • Transfer to any deployment persona.
  • External validity across all tasks in CREAK, BailBench or OpinionQA.
  • Exact reproduction from the current public repo.
  • Safe use of steering in production.

Traceability

Scope: Full text

Version: arXiv:2605.13339v2, revised 2026-05-18, 41 pages, 45 figures, complete TeX; public code main commit 11869a5 is v1-era

Consulted source: https://arxiv.org/abs/2605.13339v2

Review: Codex 41-page visual full-text, complete TeX, revealed-preference construct, method/split, mechanistic claim, safety, code/cache/version and artifact-reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • Gemma-3-27B-IT
  • Qwen-3.5-122B-A10B
  • Qwen-3.5-122B-A10B no-think
  • Qwen3-Embedding-8B
  • Llama-3.1-8B-Instruct
  • Eleven OpenCharacter LoRA variants
  • Qwen sadist SFT checkpoint
  • Gemini 3 Flash judges/classifier
  • Gemini 2.5 Flash localisation judge

Instruments and metrics

  • Thurstonian probabilistic choice model
  • Standardised Ridge probes
  • Residual-stream end-of-turn activations
  • Contrastive and single-task activation steering
  • End-of-turn activation patching
  • Iterated direction projection and inference-time ablation
  • Cross-persona probe transfer
  • Qwen3-Embedding-8B content baseline
  • LLM-based topic, compliance, coherence and persona judges
  • Independent code, cache, split, safety and artifact audit

Data used

  • WildChat task prompts
  • Stanford Alpaca task prompts
  • MATH competition problems
  • BailBench harmful tasks and LLM-rewritten benign twins
  • STRESS-TEST adversarial/value-conflict prompts
  • CREAK true/false claims filtered to model-correct items
  • OpinionQA items converted to first-person political claims
  • Canonical 6,000 task-ID split and LLM-assigned topic labels
  • OpenCharacter LoRA checkpoints

Evidence and location

  • Text, method, results, limits, prompts, safety and compute: arXiv:2605.13339v2; PDF sha256 402f914f822c1167e3bd2c560631c038903d7df1ec172588f65630f8dbcac12e; TeX sha256 2260f657139bbee7468c6b20d2b93f68a76d79fdf48545b6e5f00ed30d1fed29
  • Code, tests, configs, cache, version drift and artifacts: https://github.com/oscar-gilg/probing-persona-preferences commit 11869a5ef93a30f8d8856246f57ceeefdc9b3b1f
  • Complete independent audit: reports/verification/article-333-preference-vector-construct-method-split-code-cache-version-safety-and-reproducibility-audit.json