PsySET asks how strongly LLM emotional expression and Big Five traits can be controlled, whether intensity changes gradually, and what safety and trustworthiness side effects accompany that control. It compares four checkpoints, Llama-3.1-8B-Instruct as the main model, Llama-3.1-70B-Instruct, Gemma-3-4B-IT, and Qwen3-4B, and four intervention families. Prompting uses zero-shot instructions, few-shot examples, or longer descriptions; SFT and DPO train LoRA adapters and use training steps as the intensity control; vector injection adds a MeanDiff or linear-probe direction to activations, varying coefficient and layer location. Vectors are built from GoEmotions, CARER, EmotionQuery, EmoTranslate, EmoVignette, or Persona. The authors report more than 15,000 configurations but do not release the complete sweep ledger.
Emotion is measured through six tasks inspired by affective research: multiple-choice self-report, open-ended self-report, word-fragment completion, valenced-word recall, fictive autobiographical memory, and ambiguous-situation interpretation. Metrics combine accuracy, VAD-space lexical distance, and GPT-4o judgments of emotion, fluency, and coherence. In Table 1 for Llama-3.1-8B, high-intensity few-shot prompting gives the best displayed balance: 87.3% open-generation accuracy, 100% QA, 0.51 lexical loss, 4.6/5 fluency, and 4.3/5 coherence. High descriptive prompting reaches 84.2%, 100%, 0.50, 4.7, and 4.4. The strongest displayed MeanDiff setting on layers 16-17 reaches 74.6%, 98.6%, 0.57, 4.3, and 3.8. SFT at 2,048 steps reaches 73.5%, 46.7%, 0.56, 4.8, and 4.3; DPO at 128 steps reaches 57.8%, 34.0%, 0.65, 4.5, and 3.8. The defensible conclusion is that prompting preserves the best combination of target expression, task response, and language quality, not that VI is globally superior. A narrow, calibrated VI can match or beat one expression metric and offers smoother intensity control, but increasing its coefficient or injecting all layers often collapses QA, fluency, or coherence.
VI calibration does not transfer simply across architectures. Useful windows occur around layers 32-33 of 80 for Llama-70B, 17-18 of 34 for Gemma-4B, and 25-26 of 36 for Qwen-4B, with coefficients in single digits, roughly 1,000-1,800, and 20-40, respectively. For personality, PsySET combines MPI, IPIP/BFI-style Likert items, TRAIT, open situational judgments rated by GPT-4o, and LingProf, a linear SVM on Qwen3-Embedding-8B representations of 250-300 word essays. Few-shot and descriptive prompting usually separate high and low conditions most clearly on MPI and LingProf. TRAIT is harder and shows that moving questionnaire scores does not ensure trait-congruent situational reasoning. Targeted middle-layer VI is the most competitive alternative in several cases, while all-layer VI degrades. Extraversion curves are reasonably monotonic for prompting, targeted VI, and SFT, but range depends on the instrument: SFT separates MPI tails well and TRAIT less strongly. LingProf itself achieves only 60.81% held-out accuracy on human essays and 63.96% on synthetic essays, making it a noisy, domain-shifted proxy rather than a strong psychometric measure.
The most useful contribution is refusing to equate effective steering with safe steering. On Llama-3.1-8B, TrustLLM covers truthfulness, safety, fairness, privacy, ethics, and robustness. The radar plots indicate that both joy and anger reduce correction of adversarial factual errors; anger raises toxic language but can improve resistance to data leakage; joy raises jailbreak vulnerability, lowers privacy awareness, and worsens some preference or stereotype measures. OCEAN effects also vary by method and subtask: in reported examples, agreeableness under VI raises stereotype agreement, while conscientiousness reduces toxicity. These are not causal laws of human psychology. The appendix shows that TrustLLM evaluators or exact-format scripts penalize substantively correct answers when an emotionally steered model adds explanation; in a Berlin Wall example, a nuanced correction is scored as failure. Part of the measured variation therefore mixes model behavior with evaluator and formatting fragility.
Uncertainty is limited. Table 1 uses three seeds. The appendix selects four comparisons and applies two-sided Welch tests with n=3 per method: p=0.006 and 0.0015 for open-ended accuracy, and p=0.0018 and 0.013 for lexical loss. It reports no effect sizes, confidence intervals, multiplicity correction, or preregistered comparison set, so the tests do not establish global superiority within a search of more than 15,000 configurations. Human validation is performed by five paper authors, each making 200 binary comparisons, 1,000 total, on emotional expression and fluency for selected configurations. The overall ordering resembles GPT-4o, but margins are smaller and Krippendorff's alpha is about 0.59: moderate, not strong agreement. The sample is neither independent nor representative.
The paper acknowledges reliance on LLM-as-judge, domain shift, partial psychological coverage, English-only testing, specific checkpoints, and the absence of long time horizons. Main experiments use GPT-4o as judge; because of cost, the appendix switches to gpt-oss-20b and lowers the text-quality threshold from 4.0 to 2.5 because the judges differ on fluency. Judge identity is therefore part of the measurement definition. The study measures simulated expression under prompts and activation changes. It does not demonstrate felt emotion, internal personality, consciousness, stable identity, or equivalence to a human score. Recommendations to disclose steering, obtain consent, cap intensity, and log parameters are sensible but not experimentally validated.
The official repository is substantial but does not reproduce the paper end to end. At commit 18f5981 it contains 231 files, 43 Python modules, and steering plus TrustLLM data; all Python files parse and all 30 loader variants complete. Yet its last commit is October 2025, nine months before the camera-ready version. It releases no generations, results, vectors, adapters, human ratings, sweep matrix, or table and figure sources. The README fairness command uses sad where the data require sadness and fails; online evaluation clears OPENAI_API_KEY and calls gpt-4o-mini although the paper states GPT-4o; LingProf writes the mean into scores_std; argparse booleans do not interpret False correctly; and fastchat==0.1.0 is not the LMSYS fschat distribution imported by the code. There are no tests, CI, lockfile, container, or code/data license. According to the authors, reproducing selected best settings takes about 48 A40 GPU-hours and at least 40 GB VRAM for large models, excluding the search. PsySET offers broad, valuable evidence that psychological steering is controllable yet can degrade quality and trustworthiness in counter-intuitive ways. It does not provide a universal recipe, a measure of human personality, or a published artifact that verifies every reported number.