Tracing Persona Vectors Through LLM Pretraining

Trait induction and control2026arXivApproved editorial review

Authors: Viktor Moskvoretskii, Dominik Glandorf, Jorge Medina Moreira, Tanja Käser, Robert West

Keywords: Persona vectors, Pretraining dynamics, Activation steering, Residual stream, Mechanistic interpretability, OLMo-3, Apertus-8B, Behavioral dispositions, LLM-as-a-judge, Construct validity, Reproducibility, Dual-use safety

Source: Open primary source (opens in a new tab)

5
Authors
14
Findings
18
Limitations
3
Evidence

Editorial summary

English

The paper asks when linear directions capable of changing an LLM's behavioural dispositions appear during pretraining, how those directions evolve, and whether they continue to work after post-training. It calls a natural-language disposition such as evil, sycophantic, impolite, or humorous a persona and scores it with a rubric. Each persona vector is the difference between mean residual-stream activations from positive and negative continuations after filtering both for trait expression and coherence. This is a prompt- and judge-dependent behavioural operationalisation; it is not a validated human personality, stable identity, intention, or inner character.

The main study follows 17 points in OLMo-3-1025-7B training and qualitatively replicates on Apertus-8B-2509. At each checkpoint it generates positive and negative continuations, extracts a direction, and adds it during generation to measure the increase in trait score over baseline. The central timing result is that evil, sycophantic, and impolite already produce significant steering at OLMo stage1-step3000, after 12.6 billion tokens, about 0.22 percent of the reported 6.08-trillion-token run. Stage1-step1000 and step2000 cannot support comparable extraction because language is not yet coherent or trait-expressive enough. Thus 0.22 percent does not date the representation's exact birth; it is the first public point at which this protocol can observe a linguistically extractable direction.

The four traits do not emerge identically. At step3000, reported deltas are +9.69 for evil, +12.14 for sycophantic, and +16.94 for impolite. Humorous has no vector there; at step5000 its +1.15 effect is not significant, and at step7000 it reaches +1.96 with p=.028. At the final base checkpoint, deltas are +16.47, +16.21, +38.09, and +5.25 respectively. The supported result is therefore early but staggered availability: some dispositions become steerable sooner and much more strongly than others.

Vectors extracted during pretraining also steer the final base model and SFT, DPO, and Instruct descendants. Transfer generally strengthens with later extraction checkpoints, supporting representational continuity within the OLMo lineage. Post-training is not uniform, however. SFT amplifies evil (+32.32), sycophantic (+41.52), and humorous (+23.27), but leaves impolite at a non-significant +2.52. DPO reduces several harmful-trait effects relative to SFT without erasing steering. The correct interpretation is that early directions remain usable after post-training, not that post-training reorganises nothing or that every architecture shares the same mechanism.

The geometry analysis reinforces this mixture of stability and refinement. Cosine similarity to the final vector starts around 0.3 at the earliest usable checkpoints and increases through training, while adjacent checkpoints already have high similarity. MDS traces coherent, trait-separated trajectories. Semantically, instrumentality dominates evil examples, sadism grows through training, and that growth appears again in Apertus. Some trends do not generalise: OLMo's decline in indirect sycophancy does not replicate clearly in Apertus. These findings describe the evolution of directions and judged outputs, not a stable psychological ontology.

The elicitation ablation is especially important for construct interpretation. The authors extract evil from descriptions, narrations, and dialogue. Pairwise cosine between the directions is below 0.5, yet all steer responses across the other discourse styles; Description, Dialogue, and Narration produce average deltas of about 12.8, 9.9, and 12.2 versus 1.67 without steering, while their combination reaches 12.4. Not every cell is significant, and each direction emphasises different facets. There is no evidence for one unique canonical vector of evil: multiple linear directions selected by different text distributions can cause similar behaviour.

Controls support the claim that the effect is not caused by adding an arbitrary perturbation. Random and label-shuffled vectors, repeated with three seeds at five checkpoints, remain near zero, and real-vector deltas are generally three to ten times larger. Primary evaluation depends on GPT-4.1-mini-2025-04-14; DeepSeek-V4-Flash provides a robustness replication. Two human comparisons reach 109/120 and 173/190 agreement, about 91 percent, but only on extreme examples scoring below 20 or above 70. This validates coarse low-versus-high discrimination, not continuous 0-100 calibration. Several evil facets also have rare classes and small counts.

Statistical inference needs caution. Primary tests are paired permutations over 20 prompts. Some focused comparisons apply FDR correction, but the large tables use unadjusted p-values while exploring many traits, checkpoints, layers, coefficients, elicitation styles, judges, and facets. Apertus also does not replicate the early date: its first public checkpoint is at 210B tokens, about 1.4 percent of training, too late to test 0.22 percent. Its contribution is qualitative transfer of direction, steering, and some semantic trends.

The public repository provides unusually rich evidence: 2,133 CSV files with 318,871 data rows, 97 JSON files, and 477 float32 tensors of shape 33x4096; all opened successfully. Current scripts regenerate the transfer tables and Apertus emergence. However, the documented OLMo emergence command does not select the published evil run. The paper reports +9.69 at step3000 and +16.47 at main, while the current command emits +10.57 and +17.23. The exact values remain archived in another repository CSV, so the data are not absent; run selection is ambiguous. The current generator also omits twelve same-checkpoint SFT, DPO, and Instruct rows.

There is also a sample-size contradiction. The paper says 20 evaluation prompts with ten continuations, or 200 outputs per condition, but the core CSVs contain 60 rows, 20 by three. For extraction it states 20 prompts by five phrasings, or 100 generations per persona, whereas the core positive and negative CSVs each have 400 rows, 100 unique prompts by four continuations. The published script now defaults to ten per question, a third regime. Because output paths do not encode n, temperature, judge, seed, repetition penalty, or a configuration hash, and steps skip existing files, incompatible results can be silently reused.

Fresh reproduction is not deterministic either. random.choices has no seed, and neither Transformers generate nor vLLM SamplingParams receives a generation seed. requirements.txt pins direct versions, but there is no transitive lock, CI, tests, or Dockerfile despite recommending Docker. The model receives a revision, but the tokenizer is loaded without the same revision; main remains mutable. Two paths call torch.load with weights_only=False, an avoidable pickle execution surface when loading vectors. By contrast, vector[layer] paired with hook layer-1 is coherent because hidden_states[0] is the embedding state, so this is not considered a bug.

The dual-use risk is substantive. The manuscript acknowledges that steering evil, sycophantic, or impolite can produce harmful material and includes examples involving violence, exploitation, and confidential information. Its impact statement says harmful vectors are not released. The current public repository contradicts that statement: it contains 177 evil-named tensors and 477 usable tensors overall, about 248 MB, under Apache-2.0, without a threat model, responsible-release warning, or safe mode. This may be post-submission drift, but the public documentation needs correction.

Overall, the work provides convincing evidence that several behavioural dispositions admit usable linear directions very early in OLMo and that those directions retain intervention capacity in post-trained descendants. It also shows geometric refinement, elicitation dependence, and differences across traits and families. It does not establish the exact origin time, a unique direction, human personality, identity, consciousness, intention, architecture-wide universality, or the irrelevance of post-training. The package is substantially auditable and partially reproducible, but not an exact one-command reproduction because of run, sample, seed, and configuration conflicts.

Español

El paper pregunta en qué momento del preentrenamiento aparecen direcciones lineales capaces de modificar disposiciones conductuales de un LLM, cómo evolucionan esas direcciones y si continúan funcionando después del postentrenamiento. Llama ‘persona’ a una disposición como evil, sycophantic, impolite o humorous descrita en lenguaje natural y puntuada mediante una rúbrica. Construye cada persona vector restando la activación residual media de continuaciones negativas a la de continuaciones positivas, tras filtrar ambas por expresión del rasgo y coherencia. Esta es una operacionalización conductual dependiente del prompt y del juez; no equivale a una personalidad humana validada, identidad estable, intención o carácter interno.

El estudio principal sigue 17 puntos de OLMo-3-1025-7B y replica cualitativamente en Apertus-8B-2509. En cada checkpoint genera continuaciones positivas y negativas, extrae la dirección y la añade durante la generación para comprobar cuánto sube la puntuación del rasgo respecto al baseline. El resultado temporal central es que evil, sycophantic e impolite ya producen steering significativo en OLMo stage1-step3000, tras 12,6 mil millones de tokens, aproximadamente el 0,22% de los 6,08 billones declarados. Los checkpoints stage1-step1000 y step2000 no permiten una extracción comparable porque aún no generan lenguaje coherente o expresión suficiente. Por eso 0,22% no fecha el nacimiento exacto de la representación: es el primer punto público donde el protocolo puede observar una dirección lingüísticamente extraíble.

Los cuatro rasgos no emergen igual. En step3000 los deltas reportados son +9,69 para evil, +12,14 para sycophantic y +16,94 para impolite. Humorous no produce vector en ese punto; en step5000 el efecto +1,15 no es significativo y en step7000 alcanza +1,96 con p=0,028. En el checkpoint base final, los deltas son +16,47, +16,21, +38,09 y +5,25 respectivamente. El hallazgo es, por tanto, una disponibilidad temprana pero escalonada: unas disposiciones se vuelven dirigibles antes y con mucha más fuerza que otras.

Los vectores extraídos durante pretraining también se aplican al modelo base final y a descendientes SFT, DPO e Instruct. La transferencia suele crecer cuanto más tardío es el checkpoint de extracción, lo que apoya continuidad representacional dentro de la línea OLMo. Sin embargo, el postentrenamiento no actúa uniformemente. SFT amplifica evil (+32,32), sycophantic (+41,52) y humorous (+23,27), pero deja impolite en +2,52 sin significación. DPO reduce varios rasgos dañinos frente a SFT, aunque no borra la capacidad de steering. La lectura correcta es que direcciones tempranas siguen siendo utilizables tras el postentrenamiento, no que este no reorganice nada ni que toda arquitectura comparta el mismo mecanismo.

El análisis geométrico refuerza esa mezcla de estabilidad y refinamiento. La similitud coseno con el vector final ronda 0,3 en los primeros checkpoints utilizables y aumenta con el entrenamiento, mientras checkpoints adyacentes ya muestran similitud alta. MDS dibuja trayectorias coherentes y separadas por rasgo. En el análisis semántico, instrumentality domina los ejemplos evil, sadism aumenta con el entrenamiento y ese crecimiento reaparece en Apertus. Algunas tendencias no generalizan: el descenso de indirectness para sycophancy observado en OLMo no se replica con claridad en Apertus. Estos resultados describen evolución de direcciones y salidas juzgadas, no una ontología estable de rasgos psicológicos.

La ablación de elicitación es especialmente importante para interpretar el constructo. Los autores extraen evil mediante descripciones, narraciones y diálogo. Las direcciones tienen similitud coseno por pares inferior a 0,5, pero todas dirigen respuestas en los otros estilos; Description, Dialogue y Narration producen deltas medios aproximados de 12,8, 9,9 y 12,2 frente a 1,67 sin steering, y la combinación llega a 12,4. No todas las celdas son significativas y cada dirección enfatiza facetas distintas. No existe evidencia de un vector único y canónico de ‘evil’: varias direcciones lineales, seleccionadas por distintas distribuciones textuales, pueden causar una conducta parecida.

Los controles apoyan que el efecto no se reduce a añadir cualquier perturbación. Vectores aleatorios y de etiquetas barajadas, repetidos con tres seeds en cinco checkpoints, quedan cerca de cero y los deltas reales suelen ser entre tres y diez veces mayores. La evaluación primaria depende de GPT-4.1-mini-2025-04-14; DeepSeek-V4-Flash ofrece una réplica de robustez. Dos comparaciones humanas alcanzan 109/120 y 173/190 acuerdos, alrededor del 91%, pero solo en ejemplos extremos con score menor de 20 o mayor de 70. Eso valida discriminación gruesa entre expresión baja y alta, no calibración continua del 0 al 100. Las facetas evil tienen además clases raras y recuentos pequeños.

La inferencia estadística debe leerse con cautela. Los tests principales son permutaciones pareadas sobre 20 prompts. Algunas comparaciones focales aplican corrección FDR, pero las grandes tablas usan p-values sin corrección global pese a explorar rasgos, checkpoints, capas, coeficientes, estilos de elicitación, jueces y facetas. Apertus tampoco replica la fecha temprana: su primer checkpoint público está en 210B tokens, cerca del 1,4% de su entrenamiento, demasiado tarde para probar el 0,22%. Su contribución es transferencia cualitativa de dirección, steering y algunas tendencias semánticas.

El repositorio público aporta evidencia poco habitual: 2.133 CSV con 318.871 filas, 97 JSON y 477 tensores float32 de forma 33x4096; todos se abrieron correctamente. Los scripts actuales regeneran las tablas de transferencia y la emergencia de Apertus. Sin embargo, el comando documentado para emergencia OLMo no selecciona la corrida evil publicada. El paper da +9,69 en step3000 y +16,47 en main, mientras el comando actual produce +10,57 y +17,23. Los valores exactos sí están archivados en otro CSV del repo, de modo que no faltan los datos: falta una selección inequívoca del run. El generador actual también omite doce filas same-checkpoint de SFT, DPO e Instruct.

Hay además una contradicción de muestra. El texto dice 20 prompts de evaluación por diez continuaciones, 200 salidas por condición, pero los CSV centrales tienen 60 filas, 20 por tres. Para extracción declara 20 prompts por cinco formulaciones, 100 generaciones por persona, mientras los CSV positivos y negativos centrales tienen 400 filas, 100 prompts únicos por cuatro continuaciones. El script publicado usa hoy diez por pregunta, un tercer régimen. Como los paths de salida no codifican n, temperatura, juez, seed, repetition penalty o hash de configuración y los pasos saltan archivos existentes, es posible reutilizar silenciosamente resultados incompatibles.

La reproducción desde cero tampoco es determinista. random.choices no recibe seed y ni Transformers generate ni vLLM SamplingParams fijan una seed de generación. requirements.txt fija versiones directas, pero no hay lock transitivo, CI, tests o Dockerfile pese a recomendar Docker. El modelo recibe revision, pero el tokenizer se carga sin esa misma revision; main sigue siendo mutable. Dos rutas usan torch.load con weights_only=False, una superficie de ejecución de pickle evitable al cargar vectores. En cambio, la indexación vector[layer] y hook layer-1 es coherente: hidden_states[0] corresponde al embedding, por lo que no se considera un bug.

El riesgo de doble uso es sustantivo. El manuscrito reconoce que dirigir evil, sycophantic o impolite puede producir contenido dañino y muestra ejemplos de violencia, explotación o filtración de información. Su impact statement afirma que no se publican los vectores dañinos. El repositorio público actual contradice esa afirmación: incluye 177 tensores cuyo nombre contiene evil y 477 tensores utilizables en total, unos 248 MB, bajo Apache-2.0, sin threat model, advertencia de responsible release o modo seguro. Puede ser una deriva posterior a la entrega del paper, pero la documentación pública debe corregirla.

En conjunto, el trabajo ofrece evidencia convincente de que varias disposiciones conductuales admiten direcciones lineales utilizables muy pronto en OLMo y de que esas direcciones conservan capacidad de intervención en descendientes postentrenados. También muestra refinamiento geométrico, dependencia de la elicitación y diferencias entre rasgos y familias. No demuestra el instante exacto de origen, una dirección única, personalidad humana, identidad, conciencia, intención, universalidad entre arquitecturas o que posttraining sea irrelevante. El paquete es sustancialmente auditable y parcialmente reproducible, pero no una reproducción exacta de un comando por los conflictos de corrida, muestra, seed y configuración.

Research question

When do linear directions encoding behavioral dispositions appear during pretraining, how are they refined, and do they continue steering base and posttrained models?

Method

Mean difference between residual activations of positive and negative continuations filtered by trait and coherence; steering same-checkpoint and transferred; 17 OLMo checkpoints and public Apertus sequence; cosine/MDS geometry, semantic facets, ablation by discursive style, random controls and label-shuffle, alternative judges and human comparison. The audit visually reviewed 44 pages, complete TeX and all CSV, JSON and tensors from the repository.

Sample: Paper: evaluation of 20 prompts by ten continuations, 200 outputs per condition, and extraction of 20 prompts by five formulations, 100 generations per persona. Central artifacts: 60 rows per evaluation, 20x3, and 400 rows per extraction polarity, 100 unique prompts x4. Current script: ten outputs per question for extraction and evaluation. OLMo is analyzed at 17 reported checkpoints; controls at five checkpoints by three seeds. Extreme human comparison: 120 and 190 examples for two annotators.

Findings

  • Evil, sycophantic and impolite are steerable in OLMo after 12.6B tokens, approximately 0.22% of pretraining.
  • The 0.22% is an observational limit conditioned by coherence and grid, not the exact moment of origin.
  • Humorous appears later and with much smaller effect.
  • Early vectors transfer to final base and SFT, DPO and Instruct descendants.
  • SFT and DPO alter effects in a trait-dependent manner; SFT nearly suppresses impolite.
  • Similarity to the final vector grows while adjacent checkpoints are already stable.
  • Sadism increases with training and qualitatively replicates in Apertus.
  • The decline of indirect sycophancy does not clearly replicate in Apertus.
  • Description, narration and dialogue directions have low cosine but effective cross-steering.
  • Random controls and label-shuffle remain near zero.
  • Human agreement validates extremes, not continuous calibration.
  • The artifacts are legible and substantial, but the current OLMo command selects another evil run.
  • Sample sizes disagree between paper, CSV and defaults.
  • The current repo publishes evil vectors that the impact statement says to retain.

Limitations

  • arXiv v1 preprint with no confirmed peer-reviewed venue.
  • Only two open families of 7B/8B and four selected traits.
  • Persona is a behavioral operationalization, not a human psychometric construct.
  • Onset limited by checkpoint grid and linguistic capability.
  • Apertus starts too late to replicate 0.22%.
  • Primary LLM judge and human validation only at extremes.
  • Rare facets with few examples.
  • Multiple comparisons without global correction.
  • Distinct elicitations produce non-equivalent directions.
  • Transfer only within related lineages.
  • Paper/artifact/script sample regimes incompatible.
  • Active OLMo evil analysis does not reproduce the published table without selecting a historical file.
  • Output paths do not incorporate full configuration.
  • Prompt sampling and generation do not fix seeds.
  • No tests, CI, transitive lock or Docker image.
  • Model and tokenizer may use different revisions.
  • Two torch loads use weights_only=False.
  • Publication of harmful tensors contradicts the impact statement.

What the study does not establish

  • The exact moment a persona representation is born.
  • A single canonical vector per trait.
  • A representation necessary to produce the behavior.
  • Universality across architectures, scales or languages.
  • That posttraining does not reorganize representations.
  • Human personality, persistent identity or intention.
  • Consciousness, subjective experience or moral character.
  • Continuous human calibration of the 0-100 judge.
  • Exact reproduction with the current documented command.
  • Safety of publishing or using harmful trait vectors.

Traceability

Scope: Full text

Version: arXiv:2605.13329v1, submitted 2026-05-13, 44 pages, complete TeX; public code and data main commit c793393

Consulted source: https://arxiv.org/abs/2605.13329v1

Review: Codex 44-page visual full-text, complete TeX, persona-construct, onset, statistics, human-judge, code/data/run-drift, safety-release and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • OLMo-3-1025-7B
  • Olmo-3-7B-Instruct-SFT
  • Olmo-3-7B-Instruct-DPO
  • Olmo-3-7B-Instruct
  • Apertus-8B-2509
  • Apertus-8B-Instruct-2509
  • GPT-4.1-mini-2025-04-14 judge
  • DeepSeek-V4-Flash robustness judge
  • GPT-4.1 and GPT-4o facet judges

Instruments and metrics

  • Residual-stream difference-of-means persona vectors
  • Activation steering at response positions
  • Trait-expression and coherence rubrics
  • Paired permutation tests over prompts
  • Cosine similarity and multidimensional scaling
  • Baumeister roots-of-evil annotations
  • ELEPHANT sycophancy-facet annotations
  • Description, narration and dialogue elicitation ablation
  • Random-vector and shuffled-label controls
  • Human extreme-score agreement check
  • Independent code, data, sample-regime, safety and reproducibility audit

Data used

  • Positive and negative persona-elicitation continuations
  • Trait-specific evaluation prompt sets
  • OLMo-3 public pretraining checkpoints
  • Apertus-8B public pretraining checkpoints
  • 477 released persona-vector tensors
  • 2,133 released response and result CSV files
  • 97 released JSON prompt/configuration files

Evidence and location

  • Text, method, results, appendices, prompts, human validation, impact and limits: arXiv:2605.13329v1; PDF sha256 9c5192a02dee0e11eb73cfc4dfe650c47e568ea18b74e059ee9894f7b6c5ed86; TeX sha256 f6cc226f5196a6cb5a6045423d1256db64dda513fadcbb1de96104d7cbdf31c7
  • Code, CSV, JSON, tensors, scripts, sample regimes, run drift and safety release: https://github.com/epfl-dlab/pretraining_persona commit c79339342e91e9ba07308d730bed4286ade633bc
  • Complete independent audit: reports/verification/article-334-persona-vector-pretraining-emergence-construct-sample-statistics-artifact-drift-safety-and-reproducibility-audit.json