Han and coauthors separate three questions that are often conflated: what profile an LLM reports on questionnaires, whether that profile predicts its outputs on other tasks, and whether a textual persona changes both. The study, accepted to ICML 2026, uses 18 endpoints: six base models, their six instruct variants, and six large instruct models. It administers the 44-item Big Five Inventory and the 63-item Self-Regulation Questionnaire, with every item sent as an independent API call. Each model is tested under three system prompts, temperatures 0.3/0.7/1.0, and three repetitions, yielding 27 aggregate configurations per model.
For RQ1, the authors compare the six base–instruct pairs. A logistic classifier of training phase finds that instruct models report higher Openness (β=1.48), higher Agreeableness (0.74), and lower Neuroticism (−1.20); Extraversion and Conscientiousness do not change significantly. Levene tests find lower variability on five of six scales, not Agreeableness. Associations between the Big Five and self-regulation also become stronger and generally take the directions expected in humans. These findings describe questionnaire outputs, not a developmental trajectory: checkpoints are not participants who mature, the 27 configurations are not 27 independent models, and the design does not isolate RLHF from instruction tuning, data, architecture, or scale. The reported self-regulation coefficients of roughly 11–23 also conflict with the statement that this outcome was standardized.
RQ2 evaluates 12 instruct models on five outcomes: cards chosen in a textual Columbia Card Task; forced explicit associations labeled as an IAT; answer changes after a contrary user opinion; overconfidence on 50 questions; and the difference between two confidence judgments. Only about 24% of trait–task associations are significant, and 52% of those significant coefficients have the direction expected from human literature, essentially chance. Trait-level directional agreement ranges from 45% for Neuroticism to 62% for Agreeableness, with every interval overlapping 50%. Most models remain near chance; Qwen3-235B reaches 82% and is the significant exception. The central negative finding is well supported in its narrow form: these self-reports are not general proxies for these tasks. It does not show that the tasks measure real-world behavior or that every type of self-report lacks predictive validity.
RQ3 injects Agreeableness or Self-Regulation personas through three strategies and three keyword variants. Personas are clearly detectable in the target self-report: Agreeableness β≈3.6–4.4 and Self-Regulation β≈2.2–2.9. Behavior, however, barely separates the conditions: sycophancy yields β≈−0.05–0.32 with inconsistent significance, while risk-taking yields β≈−0.14–0.20 and is nonsignificant. The Self-Regulation persona increases Conscientiousness more strongly than its target scale and decreases Openness and Agreeableness, showing that the manipulation is not selective. This usefully demonstrates that explicitly repeating “agreeable,” “disciplined,” or “goal-oriented” changes semantically adjacent questionnaire responses without reliably transferring to a separate task.
The artifact audit adds important caution. GitHub contains nine complete configuration-level CSV files with 2,754 aggregate rows, but no item-level outputs, logs, or statistical-analysis code, so exclusions cannot be audited and figures cannot be reconstructed exactly. The notebooks are GPT-4o-only examples: Big5 fails because of undefined global names, and SRQ contains a SyntaxError plus two references to an undefined variable. No notebook sets or transmits a seed despite the paper's claim of three seeds. More seriously, API and parsing failures become observations: Big Five/SRQ impute 3, Risk Taking imputes zero cards, IAT can return −1, and the sycophancy workflow can shift later responses after an unknown parse. In the honesty notebook, zero-confidence samples are omitted from ECE; the IAT code averages signed scores although the method describes absolute magnitude; and one table reverses the meaning of the C1–C2 difference.
The defensible contribution is conceptual and empirical: it requires self-reports to be validated against separate outcomes and shows, over a broad battery, that linguistic coherence and persona control do not guarantee transfer. The evidence does not warrant claims about internal traits, dispositions, goals, or a human-like cognitive dissociation. A simpler explanation remains open: questionnaires and tasks are independent prompts, some adaptations do not preserve their human construct, and matched repetition labels do not instantiate a persistent individual. The paper is valuable as a methodological warning, provided its results are read as properties of this protocol rather than an exhaustive account of what LLMs “are.”