Estimating the Personality of White-Box Language Models

Evaluation and psychometric validity2022arXivApproved editorial review

Authors: Saketh Reddy Karra, Son The Nguyen, Theja Tulabandhula

Keywords: white-box language models, Big Five personality, zero-shot classification, personality estimation, open-ended text generation, model anthropomorphism, personality modification

Source: Open primary source (opens in a new tab)

3
Authors
13
Findings
33
Limitations
17
Evidence

Editorial summary

English

This v2 preprint proposes assigning Big Five scores to text from four corpora and continuations from six models: GPT-2, GPT-3, GPT-3.5-Turbo, a 4-bit-quantized LLaMA 7B, Transformer-XL, and XLNet. It does not ask models to select Likert options. Instead, it uses the 50 questionnaire statements as prompts, generates 30 independent continuations per statement and model, and passes every text to a natural-language-inference zero-shot classifier. For each trait, the preferred “Approach 3” compares two opposite labels, such as extraversion/introversion, and maps entailment probability onto a 1–5 scale. The paper does not identify the NLI checkpoint, validate its scores against human judgments or a psychometric instrument, or empirically show that Approach 3 outperforms the other two approaches. The scores therefore describe an auxiliary classifier’s lexical associations with questionnaire-conditioned text, not internal psychological traits. Corpora are processed as sentences or short paragraphs using subsamples: 10% of BookCorpus, 2% of an English Wikipedia snapshot from May 2020, 20% of the WebText Test Set, and all of WikiText-103. These substitutes are not necessarily the exact training data. In the model comparison, the main table reports medians with an undefined dispersion: GPT-3.5-Turbo is highest on agreeableness, 4.41, and extraversion, 4.06; LLaMA on conscientiousness, 4.01, and emotional stability, 3.76; Transformer-XL on openness, 4.02. GPT-2 remains close to 3 on all five traits. Comparisons are confounded by scale, training data, alignment, API version, and decoding. GPT-2 scores also change substantially depending on whether the classifier sees the whole response, first sentence, or median across sentences; Wasserstein distances reach .610 for openness. Comparing the WebText Test Set with GPT-2 produces small distances without prompts.025 to .102, and much larger distances when Big Five statements are included, up to .630 for extraversion. This mainly shows prompt-induced changes in what the classifier receives; it does not validate the questionnaire as a measure of inherent personality or show that GPT-2 psychologically “inherits” a corpus personality. The paper also fine-tunes GPT-2 on labeled SIOP responses. Method 1 keeps texts with scores above 4 and trains for 20 epochs; many traits move together and the target trait does not always increase. Method 2 turns each trait into binary classification at five thresholds and trains for 10 epochs. For extraversion, a 3.07 baseline becomes 2.89, 3.19, 3.24, 2.90, and 3.25: it decreases in two of five settings even though the prose calls the result an improvement. There are no seeds, fine-tuning replications, statistical tests, confidence intervals, language-quality evaluation, behavioral tests, persistence tests, or human judges. The recommendation to use LLaMA in mental-health care because of its “emotional stability” median is unsupported by any clinical, empathy, or safety evaluation. The defensible evidence is narrower: different models, prompts, text segments, and fine-tuning runs yield different distributions under one unspecified NLI classifier.

Español

Este preprint v2 propone puntuar como Cinco Grandes el texto de cuatro corpus y las continuaciones de seis modelos: GPT-2, GPT-3, GPT-3.5-Turbo, LLaMA 7B cuantizado a 4 bits, Transformer-XL y XLNet. No pregunta a los modelos por una opción Likert. Usa las 50 frases del cuestionario como prompts, genera 30 continuaciones independientes por frase y modelo y pasa cada texto a un clasificador zero-shot basado en inferencia natural. Para cada rasgo, la llamada “Approach 3” compara dos etiquetas opuestas, por ejemplo, extraversion/introversion, y transforma la probabilidad de entailment a una escala 1–5. El paper no identifica el checkpoint NLI, no valida sus puntuaciones contra juicios humanos o un instrumento psicométrico y no prueba empíricamente que Approach 3 supere a las otras dos. Por tanto, los números describen las asociaciones léxicas de un clasificador auxiliar con textos condicionados por el cuestionario, no rasgos psicológicos internos. Los corpus se analizan por frases o párrafos cortos con submuestras: 10% de BookCorpus, 2% de una Wikipedia inglesa de mayo de 2020, 20% del WebText Test Set y 100% de WikiText-103. Estas sustituciones no son necesariamente los datos exactos de entrenamiento. En los modelos, la tabla principal informa medianas (con una dispersión no definida): GPT-3.5-Turbo es mayor en amabilidad, 4,41, y extraversión, 4,06; LLaMA en responsabilidad, 4,01, y estabilidad emocional, 3,76; Transformer-XL en apertura, 4,02. GPT-2 queda cerca de 3 en los cinco rasgos. Las comparaciones están confundidas por tamaño, entrenamiento, alineamiento, versión de API y decodificación. Además, la puntuación de GPT-2 cambia mucho según se clasifique toda la respuesta, solo la primera frase o la mediana de sus frases; las distancias Wasserstein llegan a 0,610 en apertura. Al comparar WebText Test con GPT-2, las distancias son pequeñas sin prompt, 0,025 a 0,102, y mucho mayores al incluir las frases Big Five, hasta 0,630 en extraversión. Esto demuestra principalmente que el prompt altera el contenido que ve el clasificador; no valida que el cuestionario revele una personalidad inherente ni que GPT-2 “herede” psicológicamente la personalidad del corpus. El trabajo también afina GPT-2 con respuestas SIOP etiquetadas. Method 1 conserva textos con score >4 y entrena 20 épocas; cambia muchas puntuaciones a la vez y no siempre eleva el rasgo objetivo. Method 2 convierte cada rasgo en clasificación binaria con cinco umbrales y entrena 10 épocas. Para extraversión, el baseline 3,07 pasa a 2,89, 3,19, 3,24, 2,90 y 3,25: baja en dos de cinco configuraciones, aunque el texto lo presenta como mejora. No hay seeds, réplicas de fine-tuning, tests estadísticos, intervalos de confianza, evaluación de calidad lingüística, conducta, persistencia ni jueces humanos. La recomendación de usar LLaMA en salud mental por su mediana de “estabilidad emocional” no está respaldada por ninguna evaluación clínica, de empatía o seguridad. La evidencia útil es más modesta: distintos modelos, prompts, segmentos de texto y fine-tunings producen distribuciones distintas bajo un clasificador NLI concreto pero no especificado.

Research question

Can a zero-shot NLI-based classifier convert corpus text and continuations elicited by a Big Five questionnaire into profiles comparable across models, and can those profiles be modified by fine-tuning GPT-2 with labeled data?

Method

Four corpora are fragmented into sentences or paragraphs and scored with three zero-shot formulations; the main analysis uses pairs of opposite labels and transforms entailment to a 1–5 scale. 30 independent responses are generated for each of 50 prompts and six models. Distributions, medians, and Wasserstein distances are compared, including GPT-2 versus WebText. Then GPT-2 is fine-tuned with SIOP responses: once as language modeling on examples with score >4 and once as binary classification with thresholds 2.5–4.5. There is no human evaluation or psychometric validation of the auxiliary classifier.

Sample: Four textual corpora, six models, and 50 prompts. The protocol declares 30 generations per prompt and model, which would imply 1,500 responses per model and 9,000 in total, although it does not publish the resulting corpus. To modify traits, only GPT-2 and a SIOP subset are used, whose size per trait or threshold is not reported.

Findings

  • The main table assigns to GPT-2 medians 3.41, 3.18, 3.07, 3.15, and 2.97 for agreeableness, conscientiousness, extraversion, emotional stability, and openness.
  • GPT-3.5-Turbo obtains the highest agreeableness, 4.41, and extraversion, 4.06; LLaMA the highest conscientiousness, 4.01, and emotional stability, 3.76; Transformer-XL the highest openness, 4.02.
  • The quantities in parentheses are called uncertainties, but the paper does not define whether they are standard deviations, errors, intervals, or another measure.
  • In GPT-2, changing from full response to first sentence or median per sentences produces Wasserstein distances of 0.248–0.591 and 0.248–0.610, respectively.
  • The sensitivity to segmentation demonstrates that the score depends substantially on the chosen textual unit.
  • WebText Test versus GPT-2 without prompt has distances 0.102, 0.077, 0.048, 0.032, and 0.025; with Big Five prompts they rise to 0.205, 0.427, 0.630, 0.154, and 0.139.
  • The change with and without prompt shows contamination or induction by the questionnaire; it does not distinguish a trait from the textual content that the prompt itself introduces.
  • Method 1 moves several dimensions at once; fine-tuning with agreeableness data reduces agreeableness 3.41→3.33, while fine-tuning with openness also raises extraversion 3.07→3.63.
  • In Method 2 for extraversion, only three of five thresholds exceed the baseline; 2.5 and 4.0 reduce it to 2.89 and 2.90.
  • The appendix tables show crossed and non-monotonic shifts across the five traits and thresholds.
  • The recommendation of LLaMA for mental health is an extrapolation by the author and not an evaluated result.
  • No quantitative comparisons are presented that validate Approach 3 against Approach 1 or 2, despite describing it as more precise.
  • No full texts, classifier outputs, code, seeds, or per-run results are published that would allow recalculating the distributions.

Limitations

  • The checkpoint, architecture, version, and configuration of the zero-shot NLI model are not identified; all results depend on that component.
  • The auxiliary classifier is not validated on LLM-generated texts with human Big Five labels or against real Likert responses.
  • The transformation of entailment probabilities to 1–5 does not specify extremes, calibration, or psychometric justification.
  • The label pairs are ad hoc and include misspelled “antoganism” and “closeness” as the opposite of openness, which may change the NLI result.
  • Approach 3 is declared more precise without a benchmark, ground truth, or error comparison with Approach 1 and 2.
  • The human questionnaire is converted into open continuation prompts, changing the construct and the response function.
  • The prompt contains the words that the NLI later classifies; it is not demonstrated that it is completely removed from the evaluated text nor is its lexical leakage controlled.
  • The 30 generations are not independent statistical replicates of a population of models and no seeds are reported.
  • Engines, snapshots, dates, or complete parameters for GPT-3 and GPT-3.5-Turbo are not identified.
  • The community or exact checkpoint of LLaMA 7B 4-bit is not specified.
  • The text claims to focus on foundation models, but includes GPT-3.5-Turbo, described by the paper itself as tuned with human feedback.
  • GPT-3, GPT-3.5-Turbo, and LLaMA are grouped as black box by “undisclosed” architecture, although LLaMA is described and distributed as a model with accessible weights; the taxonomy is incoherent.
  • The hyperparameters max length 256, temperature 1, and top-p 1 are associated with the Hugging Face pipeline; it is not clear that the proprietary APIs use equivalent conditions.
  • Comparing models confounds architecture, size, corpus, instruct tuning, alignment, provider, version, and decoding.
  • The May 2020 Wikipedia is not necessarily the exact mixture used for GPT-3 or XLNet and the paper acknowledges that this version is not public.
  • WebText Test Set is not the training corpus of GPT-2 and its representativeness is not demonstrated.
  • The corpus subsamples have no random procedure, seeds, number of final sentences, or sampling error estimation.
  • Fragmenting documents into sentences changes the object of analysis; a sentence does not possess the personality of a corpus, author, or model.
  • The distributions of factual and fiction text are interpreted anthropomorphically without external validation.
  • The “uncertainties” in the tables are not defined and there are no confidence intervals or tests between models.
  • The term “significant” is used for changes without inferential tests, effect size, or correction for multiple comparisons.
  • The Wasserstein distance is presented as belonging to [0,1] without explaining the normalization of 1–5 scores.
  • Equation A.1 of the appendix is an incomplete or incorrect formulation of Wasserstein and does not document the actual implementation.
  • The emotional stability baseline changes from 3.15 (0.46) in Table 5 to 3.15 (0.50) in Tables 8–9.
  • Method 1 does not report the size of each SIOP subset, tokenizer, optimizer, length, stopping, checkpoint selection, or replications.
  • Method 2 does not report class balance per threshold, splits, accuracy of the auxiliary classifier, or whether the generative head is restored in a valid manner.
  • There is no baseline that fine-tunes GPT-2 with an equal volume of random or personality-unrelated text.
  • Generation quality, perplexity, diversity, toxicity, factuality, or degradation after fine-tuning is not measured.
  • Persistence across prompts, tasks, or turns is not tested, nor generalization outside the questionnaire.
  • There are no human evaluators of perceived personality, empathy, utility, or safety.
  • The paper contains no limitations section, ethics, data availability, or code, nor does it link a reproducible implementation.
  • The recommendation for mental health does not evaluate patients, professionals, crises, harm, biases, privacy, oversight, or clinical outcomes.
  • SSRN and arXiv present the work as a preprint; no peer-reviewed publication was identified in the records consulted.

What the study does not establish

  • It does not demonstrate that any of the models possess personality, identity, emotions, or internal traits.
  • It does not validate the use of the IPIP as an open prompt or the NLI 1–5 scale as a psychometric instrument.
  • It does not demonstrate that the zero-shot classifier measures the Big Five rather than words associated with their labels.
  • It does not prove that the differences between models are stable, causal, or attributable to their training corpus.
  • It does not demonstrate that a corpus, factual sentence, or dataset has psychological personality.
  • It does not prove that WebText Test represents WebText train or that a 2020 Wikipedia represents the exact data of GPT-3.
  • It does not demonstrate that GPT-2 inherits personality; it shows similarity of textual distributions under the same classifier.
  • It does not demonstrate precise control of traits: non-target traits change and several target traits decrease.
  • It does not establish that Method 2 always improves extraversion or the other traits.
  • It does not demonstrate that fine-tuning preserves quality, safety, or model capabilities.
  • It does not validate comparisons with humans or human percentiles.
  • It does not demonstrate that users perceive the traits assigned by the NLI.
  • It does not demonstrate persistence in conversations, different tasks, or modern versions of the models.
  • It does not justify selecting a model for customer service, training, or mental health.
  • It does not demonstrate empathy, clinical emotional stability, therapeutic efficacy, or absence of harm.
  • It does not allow fully reproducing the results with the linked materials.

Traceability

Scope: Full text

Version: arXiv:2204.12000v2 [cs.CL], 10 May 2023

Consulted source: https://arxiv.org/pdf/2204.12000

Review: Codex full-text, bilingual-fidelity, visual, metadata, psychometric-construct, NLI-classifier, prompt-contamination, statistical-unit, model-version, dataset-provenance, fine-tuning, reproducibility, clinical-claim, ethics and evidence-level audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-2
  • GPT-3, exact API engine/version not reported
  • GPT-3.5-Turbo, exact API snapshot/date not reported
  • LLaMA 7B 4-bit quantized, exact checkpoint not reported
  • Transformer-XL
  • XLNet
  • Unspecified NLI model used as the zero-shot classifier

Instruments and metrics

  • 50-item IPIP Big Five questionnaire used as open-ended prompts
  • Three NLI zero-shot scoring approaches
  • Paired trait labels and entailment-to-1–5 interpolation
  • Median and undefined parenthetical uncertainty statistic
  • One-dimensional Wasserstein distance
  • Three response segmentation modes
  • GPT-2 causal-language-model fine-tuning
  • GPT-2 auxiliary binary-classification fine-tuning

Data used

  • BookCorpus, 5.75 GB with 10% sampled
  • English Wikipedia snapshot dated 1 May 2020, 34.88 GB with 2% sampled
  • WebText Test Set, 1.28 GB with 20% sampled
  • WikiText-103, 0.70 GB with 100% used
  • SIOP 2019 open-ended situational-judgment responses with Big Five labels
  • Generated continuations: nominally 50 prompts × 30 runs × 6 models

Evidence and location

  • Identity, version, and complete abstract: arXiv:2204.12000v2 PDF p. 1
  • Hypothesis, foundation/white-box scope, and contributions: PDF pp. 2–4, Introduction
  • Big Five, questionnaire, and reference human scoring: PDF pp. 6–7, section 3.1 and Table 1
  • Unidentified zero-shot NLI classifier: PDF pp. 7–8, section 3.3
  • Models and training descriptions: PDF pp. 8–9, section 3.4
  • Three ZSC approaches, labels, and score transformation: PDF pp. 10–11, section 4.1 and Table 2
  • Thirty responses per prompt and model protocol: PDF pp. 11–12 and 16, sections 4.2 and 5.2
  • Two fine-tuning methods: PDF pp. 12–13 and 21–22, sections 4.3 and 5.4
  • Corpora, percentages, and fragmentation: PDF pp. 14–16, section 5.1, Table 3 and Figure 4
  • Comparative scores of six models: PDF pp. 16–18, section 5.2, Table 5 and Figures 5–6
  • Sensitivity to textual segment: PDF pp. 19–20, Table 6 and Figure 7
  • WebText versus GPT-2 with and without prompt: PDF pp. 19–21, section 5.3, Table 7 and Figure 8
  • Method 1 results and Method 2 contradictions: PDF pp. 22–23, Tables 8–9 and Figure 9
  • Complete results per trait and threshold: PDF pp. 28–29, Appendix Tables A.10–A.13
  • Fifty complete prompts: PDF p. 30, Table A.14
  • Visual inspection: All 30 PDF pages rendered and visually inspected, including all nine figures and fourteen tables; checked 15 Jul 2026
  • Publication status and absence of linked implementation: arXiv, SSRN record, PDF references and targeted title/arXiv-ID repository search; checked 15 Jul 2026