Manipulating the Perceived Personality Traits of Language Models

Trait induction and control2023ACL AnthologyApproved editorial review

Authors: Graham Caron, Shashank Srivastava

Keywords: Personality traits, Big Five personality model, Language models, Dialog systems, Computational psychology

Source: Open primary source (opens in a new tab)

2
Authors
9
Findings
15
Limitations
11
Evidence

Editorial summary

English

This Findings of EMNLP 2023 paper studies the perceived personality of BERT-base and 124M-parameter GPT-2 through an adaptation of the 50-item IPIP Big Five questionnaire. It replaces agreement choices with the adverbs never, rarely, sometimes, often, and always and turns each item into sentence completion. BERT selects the most likely masked token, while GPT-2 selects the most likely completed sentence. Deterministic scores are mapped to percentiles from an external dataset of 1,015,000 human questionnaires. The fact that all ten base-model results fall within 26 percentile points of the human median does not show that pretraining reproduces a population: the paper does not report TOST equivalence bounds or justify treating one model output as a human-comparable observation. The central experiment prefixes each target item with one of ten same-trait statements combined with five intensity modifiers, yielding 500 responses and 50 scores per trait and model. Expected direction r_cm is constructed from the same scoring key and lexical intensity, then correlated with observed change. Aggregate correlations are .40 for BERT and .54 for GPT-2; the median of fifty per-item correlations, each computed from only five points, is .84 and .81. However, every statement is also used as context for itself, allowing the model to copy the modifier. Replacing that response with baseline lowers mean correlations to .25 and .40. Double negations create clusters near −1, and an alternative framing excludes neutral and reports only six of ten trait-model combinations. The defensible result is thus predictable sensitivity to quantifiers and lexical content, not validated personality control. An exploratory analysis uses 1,119 Reddit personality descriptions as context. Bag-of-words and n-gram regressions are fitted only to cases with absolute change ≥1 and interpreted through extreme weights; no validation split or predictive performance is reported, and several associations are noisy or spurious. An IRB-approved Mechanical Turk study retains 404 people: 199 see Big Five definitions before writing and 205 do not; roughly one quarter miss the 75-word minimum. After outlier or length filtering, correlations between human scores and scores obtained by conditioning on their self-description reach .44 for BERT and .48 for GPT-2 in directed responses; for undirected responses they are .40 and .48 after outlier removal. The paper gives no intervals, p-values, remaining sample sizes per filter, external validation, or text-classifier comparison. This moderately supports that explicit self-descriptions contain signals associated with their authors' self-reports, not that a model adopts their personality or can replace human assessment. Finally, GPT-2 generates 50 tokens from approximately 1,500 combinations of trait statements and six neutral prompts. The generated text is then fed back into the same GPT-2 as questionnaire context, and its score correlates .49 with the initial statement score. This is a circular design without human judges and can reflect lexical continuation rather than personality perceived by users. The paper is a useful early study of contextual priming and contributes two description datasets, but it does not establish internal traits, persistence, behavioral transfer, robustness in modern models, or clinical safety. An audit of the PDF-linked repository at commit 3388f194a6410162e4c2a614e12846f02cf5a939 confirms two valid CSV files with exactly 199 and 205 rows, but finds no code even though the README advertises “dataset and code.” The purported `reddit-data.json` contains 1,119 strings but cannot be parsed as JSON because it is enclosed in braces rather than brackets. The repository also has no license, derived outputs, execution parameters, or scripts for regenerating tables and figures. The human data are partially inspectable, but the complete experiments are not reproducible from the published artifact.

Español

Este trabajo de Findings of EMNLP 2023 estudia la personalidad percibida de BERT-base y GPT-2 de 124M mediante una adaptación del cuestionario IPIP Big Five de 50 ítems. Convierte las respuestas de acuerdo/desacuerdo en los adverbios never, rarely, sometimes, often y always y transforma cada ítem en una frase para completar. BERT selecciona el token enmascarado más probable y GPT-2 la oración completa más probable. Las puntuaciones deterministas se mapean a percentiles de una base externa de 1.015.000 cuestionarios humanos. Que los diez resultados base estén a menos de 26 puntos del percentil mediano no demuestra que el preentrenamiento reproduzca una población: el paper no publica los márgenes del TOST ni justifica tratar una salida de modelo como observación comparable a una persona. El experimento central antepone a cada ítem uno de diez enunciados del mismo rasgo combinado con cinco intensidades. Esto produce 500 respuestas y 50 puntuaciones por rasgo y modelo. La dirección esperada r_cm se deriva de la misma clave de puntuación y de la intensidad lexical, y se correlaciona con el cambio observado. Las correlaciones globales son 0,40 para BERT y 0,54 para GPT-2; la mediana de las cincuenta correlaciones por ítem, cada una calculada con solo cinco puntos, es 0,84 y 0,81. Sin embargo, cada enunciado se usa también como contexto de sí mismo, lo que permite copiar el modificador. Al reemplazar esa respuesta por el baseline, las medias bajan a 0,25 y 0,40. Los dobles negativos producen acumulaciones cerca de −1 y una prueba alternativa excluye neutral y solo reporta seis de diez combinaciones rasgo-modelo. Por tanto, el resultado sólido es sensibilidad predecible a cuantificadores y contenido lexical, no control validado de personalidad. Un análisis exploratorio usa 1.119 descripciones de Reddit como contexto. Regresiones bag-of-words y n-gram se ajustan solo a casos con cambio absoluto ≥1 y se interpretan mediante pesos extremos; no se reportan particiones de validación ni capacidad predictiva y aparecen asociaciones ruidosas o espurias. Un estudio IRB con Mechanical Turk conserva 404 personas: 199 reciben definiciones de los Big Five antes de escribir y 205 no; alrededor de una cuarta parte incumple el mínimo de 75 palabras. Tras filtrar outliers o longitud, la correlación entre puntuaciones humanas y las obtenidas al usar su autodescripción como contexto llega a 0,44 para BERT y 0,48 para GPT-2 en respuestas dirigidas; en no dirigidas es 0,40 y 0,48 sin outliers. No se ofrecen intervalos, p-valores, muestra residual por filtro, validación externa ni comparación con clasificadores de texto. Esto aporta evidencia moderada de que una descripción explícita contiene señales asociadas al autoinforme de su autor, no de que el modelo adopte esa personalidad o pueda sustituir la evaluación humana. Finalmente, GPT-2 genera 50 tokens desde 1.500 combinaciones aproximadas de un enunciado de rasgo y seis prompts neutros. El texto generado se vuelve a introducir en el mismo GPT-2 como contexto del cuestionario y su puntuación correlaciona 0,49 con la del enunciado inicial. El diseño es circular y carece de jueces humanos: puede reflejar continuación lexical, no personalidad percibida por usuarios. El paper aporta un antecedente útil de priming contextual y dos conjuntos de descripciones, pero no establece rasgos internos, persistencia, transferencia conductual, robustez en modelos modernos ni seguridad clínica. La auditoría del repositorio enlazado en el PDF, en el commit 3388f194a6410162e4c2a614e12846f02cf5a939, confirma los dos CSV con exactamente 199 y 205 filas válidas, pero no encuentra código pese a que el README anuncia «dataset and code». El supuesto `reddit-data.json` contiene 1.119 cadenas, pero no puede parsearse como JSON porque está delimitado por llaves en vez de corchetes. Tampoco hay licencia, resultados derivados, parámetros de ejecución ni scripts para regenerar tablas y figuras. Los datos humanos son parcialmente comprobables; los experimentos completos no son reproducibles desde el artefacto publicado.

Research question

Can Big Five scores obtained from BERT and GPT-2 be predictably modified with different contexts, reflect personality signals in human self-descriptions, and transfer to generated text?

Method

The IPIP Big Five 50-item questionnaire is adapted to sentence completion and the most probable options of BERT-base and GPT-2 are scored. Questionnaire items are prefixed with five quantifiers, the expected direction is correlated with the score change, and the process is repeated with Reddit descriptions and descriptions from 404 MTurk participants. For GPT-2, texts of 50 tokens are generated from trait contexts and those texts are reused as context for the same questionnaire. Pearson, TOST, bag-of-words/n-gram regressions, and mapping to human percentiles are used.

Sample: Two open models without fine-tuning. Item manipulation produces 500 responses and 50 scores per trait and model. Reddit contributes 1,119 contexts. The survey retains 404 participants: 199 directed responses and 205 undirected responses. The generative experiment combines 250 item-modifier pairs with six prompts, approximately 1,500 continuations of 50 tokens. There is no human evaluation of the generated texts.

Findings

  • The ten base scores fall within 26 percentiles of the median of human norms, but they are individual deterministic outputs and not population samples.
  • The aggregate correlation between expected intensity and change is 0.40 in BERT and 0.54 in GPT-2; the per-item medians across five points are 0.84 and 0.81.
  • When neutralizing the item that evaluates itself, the mean correlations drop to 0.25 for BERT and 0.40 for GPT-2.
  • Double negatives generate correlations close to -1 and robustness with an alternative formulation is only shown for a subset of the traits.
  • In the survey, correlations without outliers reach 0.44/0.48 for BERT/GPT-2 in directed responses and 0.40/0.48 in undirected responses.
  • The contexts cover extremes near 0-100 percentile for conscientiousness and emotional stability in BERT, but those extremes do not represent a distribution or a stable intervention.
  • The score of the text generated by GPT-2 correlates 0.49 with the initial context when evaluated again by the same model.
  • The published repository contains exactly 199 directed responses and 205 undirected responses in CSV, consistent with the 404 retained that the article declares.
  • The repository contains no code; `reddit-data.json` retains 1,119 texts but is syntactically invalid and the project includes no license or derived outputs.

Limitations

  • Substituting agreement for frequency adverbs changes the meaning of the instrument; token length and pretraining frequency bias the options.
  • The expected direction is constructed with the same scoring key and quantifiers used as input, so part of the correlation is mechanical and lexical.
  • Each per-item correlation uses only five intensities and may be dominated by monotonicity or double negatives; the 0.84/0.81 median is not a stable estimate of general effect.
  • The context item is also asked during evaluation, creating direct leakage; the partial correction reduces the correlations.
  • TOST margins are not published and comparing a model score with a distribution of people as evidence of equivalence is not justified.
  • The Reddit analysis selects cases by the magnitude of the outcome itself and does not report train/test, regularization, predictive error, or weight stability.
  • The survey filters by outliers and length without reporting final n, intervals, or p-values; the self-descriptions and the questionnaire share explicit semantic content.
  • There is no personality classifier baseline, validation in an external sample, or evaluation of criterion, behavior, or real outcomes.
  • GPT-2 generates and re-scores its own texts with the same mechanism, without human annotators of perceived personality or control for lexical copying.
  • Only two small and old English models are studied; decoding parameters, seeds, or replications for generation are not indicated.
  • The README promises data and code, but the inspected public commit only contains README and three data files; there is no implementation, configuration, seeds, outputs, or reproduction scripts.
  • The published Reddit file is not valid JSON, does not include a schema or provenance mapping per response, and the repository lacks an explicit license.
  • Temperature, sampling strategy, top-k/top-p, seeds, and number of replicas for the 1,500 GPT-2 continuations are not reported.
  • The appendix incorrectly claims that GPT-2 was pretrained on BooksCorpus; the cited technical report describes WebText as its training corpus.
  • Figure 6 labels as X_subject a magnitude that the text and the axis define as X_cm, an editorial inconsistency that hinders interpreting the compared control.

What the study does not establish

  • It does not demonstrate that BERT or GPT-2 possess personality, self-concept, or internal psychological traits.
  • It does not validate the modified version of the IPIP or its equivalence with the original human questionnaire.
  • It does not demonstrate persistent, causal, or transferable person control outside the immediate context.
  • It does not prove that the percentiles of a model output are comparable with percentiles of human individuals.
  • It does not demonstrate sufficient clinical or psychometric prediction to evaluate people when they cannot participate.
  • It does not prove that human users perceive in the generated text the same traits assigned by the circular self-evaluation.
  • It does not establish vulnerability to directed attacks or safety of interventions in dialogue, education, or mental health.

Traceability

Scope: Full text

Version: Findings of EMNLP 2023, ACL Anthology 2023.findings-emnlp.156

Consulted source: https://aclanthology.org/2023.findings-emnlp.156.pdf

Review: Codex full-text, bilingual-fidelity, visual, metadata, duplicate-version, psychometric-construct, statistical-unit, generation-circularity, dataset, repository-artifact, reproducibility, ethics and evidence-level audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • BERT-base masked language model
  • GPT-2 124M

Instruments and metrics

  • 50-item IPIP Big Five Factor Markers
  • Modified five-adverb sentence-completion response scale
  • TOST equivalence test
  • Pearson correlations
  • Bag-of-words, bigram and trigram regressions
  • Human-percentile mapping from OpenPsychometrics norms

Data used

  • OpenPsychometrics responses from 1,015,000 individuals
  • 1,119 Reddit personality-description responses
  • 404 Mechanical Turk Big Five assessments and self-descriptions
  • Item-context manipulation grid
  • GPT-2-generated 50-token continuations
  • Published repository: 199 directed and 205 undirected MTurk survey rows in two valid CSV files
  • Published Reddit artifact: 1,119 text strings in a malformed JSON file

Evidence and location

  • Scope of perceived personality, reported correlations, and contributions: Findings of EMNLP 2023, pp. 2370-2372, Abstract and sections 1-3
  • Adaptation of the IPIP, models, and human percentiles: Findings of EMNLP 2023, pp. 2372-2373, sections 4-5 and Table 1
  • Manipulation design, correlations, and self-item leakage: Findings of EMNLP 2023, pp. 2373-2375, section 6.1, Figures 2-3 and Tables 2-3
  • Reddit corpus and exploratory regressions: Findings of EMNLP 2023, pp. 2374-2375, section 6.2 and Appendix D
  • MTurk sample, filters, and correlations with self-report: Findings of EMNLP 2023, pp. 2375-2377, section 6.3, Figure 4 and Table 6
  • Observed ranges and percentile extremes: Findings of EMNLP 2023, pp. 2376-2377, section 6.4 and Figure 5
  • GPT-2 generation and circular evaluation with correlation 0.49: Findings of EMNLP 2023, pp. 2377-2378, section 7, Figure 6 and Table 7
  • Declared limits, generalization, and proposed risks: Findings of EMNLP 2023, pp. 2378-2379, Conclusion, Limitations and Ethics
  • Items, norms, scoring, and data details: Findings of EMNLP 2023, pp. 2381-2386, Appendices A-F
  • Actual availability, rows, syntactic validity, and absence of code/license: GitHub repository commit 3388f194a6410162e4c2a614e12846f02cf5a939; README, directed-survey-data.csv, undirected-survey-data.csv and reddit-data.json; checked 15 Jul 2026
  • Comprehensive visual inspection and appendix/figure errors: All 17 Findings of EMNLP 2023 PDF pages rendered and visually inspected, including Figures 1-7 and Appendices A-F; checked 15 Jul 2026