Synthetic Personality Review

PerMix-RLVR: Preserving Persona Expressivity under Verifiable-Reward Alignment

Trait induction and control2026arXivApproved editorial review

Authors: Jihwan Oh, Soowon Oh, Murad Aghazada, Minchan Jeong, MyeongSeok Kang, Sungnyun Kim, Se-Young Yun

Keywords: Persona conditioning, Role-playing agents

Source: Open primary source (opens in a new tab)

7
Authors
7
Findings
8
Limitations
10
Evidence

Editorial summary

English

The paper asks whether reinforcement learning with verifiable rewards (RLVR) makes task performance less dependent on a prompted persona while also weakening the model's ability to stay in character. It proposes PerMix-RLVR: during GSM8K training, one of 25 personas is sampled uniformly and prepended as a system message while retaining the same GRPO objective and binary verifier as standard RLVR. The controlled comparison starts from Llama-3.1-8B-Instruct and uses QLoRA; it covers SFT, persona-mixed SFT, sequence distillation from three teachers, RLVR, and PerMix-RLVR. Sixteen persona prompts are evaluated with five runs per condition on GSM8K, MATH500, AIME2024, and LiveCodeBench v6. Persona fidelity is measured with PersonaGym, using questions generated by GPT-5-mini and judgments from GPT-5-mini and DeepSeek-v3.2. On MATH500, PerMix-RLVR raises the worst-persona accuracy from 34.0% to 41.0%, lowers the best from 49.6% to 48.6%, and barely changes the mean from 46.8% to 47.1%. PSS, defined as minimum divided by maximum accuracy, rises from 0.675 to 0.818: the advertised +21.2% is a relative improvement in that ratio, not an accuracy gain. On GSM8K, mean accuracy changes from 86.1% to 86.4% and PSS from 0.959 to 0.975. On PersonaGym, Persona Consistency rises from 3.06 to 3.41, producing the abstract's +11.4%; the five-dimension mean only rises from 2.79 to 2.87, and PerMix worsens Action Justification and Toxicity Control. The advantage is not uniform: compared with RLVR, PSS falls from 0.400 to 0.355 on LiveCodeBench Medium; worst, best, mean, and PSS all worsen on AIME24; and every PSS is zero on LiveCodeBench Hard. The method is a simple intervention with a promising lower-tail MATH500 result, but PSS depends on two extremes and can improve by lowering the maximum or even when performance is uniformly poor. Prompts combine identity, style, and problem-solving strategy; although train and test labels are distinct, their semantics overlap substantially. The paper provides no inferential tests for significance claims, human fidelity validation, code, checkpoints, outputs, seeds, or judge logs. Its theory describes an ideal optimum with a style-independent verifier and does not guarantee finite-GRPO or PersonaGym behavior. This is an arXiv v1 preprint, not a confirmed accepted publication.

Español

El trabajo estudia si el aprendizaje por refuerzo con recompensas verificables (RLVR) hace que el rendimiento de un modelo dependa menos de la persona indicada en el prompt y, a la vez, reduzca su capacidad de mantenerse en personaje. Propone PerMix-RLVR: durante el entrenamiento con GSM8K se elige uniformemente una de 25 personas, se añade como mensaje de sistema y se conserva el mismo objetivo GRPO y el mismo verificador binario que en RLVR. La comparación controlada parte de Llama-3.1-8B-Instruct y usa QLoRA; incluye SFT, SFT con personas, destilación secuencial con tres profesores, RLVR y PerMix-RLVR. Se evalúan 16 prompts de persona, cinco ejecuciones por condición, en GSM8K, MATH500, AIME2024 y LiveCodeBench v6. La fidelidad se mide con PersonaGym, cuyas preguntas genera GPT-5-mini y cuyas respuestas juzgan GPT-5-mini y DeepSeek-v3.2. En MATH500, PerMix-RLVR eleva la precisión de la peor persona de 34,0 % a 41,0 %, reduce la mejor de 49,6 % a 48,6 % y apenas cambia la media, de 46,8 % a 47,1 %. El PSS, mínimo dividido por máximo, pasa de 0,675 a 0,818: el +21,2 % anunciado es una mejora relativa de ese cociente, no de precisión. En GSM8K, la media pasa de 86,1 % a 86,4 % y el PSS de 0,959 a 0,975. En PersonaGym, Persona Consistency sube de 3,06 a 3,41, que corresponde al +11,4 % del resumen; el promedio de las cinco dimensiones solo pasa de 2,79 a 2,87 y PerMix empeora Action Justification y Toxicity Control. La ventaja tampoco es uniforme: frente a RLVR, su PSS baja de 0,400 a 0,355 en LiveCodeBench Medium; en AIME24 empeoran peor caso, mejor caso, media y PSS; y en LiveCodeBench Hard todos los PSS son cero. La propuesta aporta una intervención sencilla y un resultado prometedor en la cola inferior de MATH500, pero el PSS depende de dos extremos y puede subir al reducir el máximo o aun con rendimiento uniformemente bajo. Los prompts mezclan identidad, estilo y estrategia de resolución; aunque sus etiquetas de entrenamiento y prueba son distintas, hay solapamientos semánticos claros. No se aportan pruebas inferenciales para las afirmaciones de significación, evaluación humana de la fidelidad ni código, checkpoints, salidas, semillas o registros de jueces. La teoría describe un óptimo idealizado con un verificador independiente del estilo y no garantiza el comportamiento del GRPO finito ni de PersonaGym. Es una prepublicación arXiv v1, no una publicación aceptada confirmada.

Research question

Does RLVR reduce the sensitivity of performance to different persona prompts at the cost of expressiveness, and can RLVR training with a mixture of personas simultaneously improve stability on verifiable tasks and fidelity to the persona?

Method

Initial observational study across model families and controlled experiment. The initial analysis compares PSS across Qwen3, Llama 3.1/3.2, and Gemma 3 variants. The main experiment starts from Llama-3.1-8B-Instruct and trains, with QLoRA, SFT, PerMix-SFT, SeqKD with Qwen3-32B, Llama-3.1-70B, or Gemma-3-27B, RLVR, and PerMix-RLVR variants. The latter uniformly samples one training persona per GSM8K example and applies GRPO with a binary math-verify reward. Sixteen evaluation personas are tested five times on four benchmarks; two LLMs judge PersonaGym. Pass@1 accuracy, worst and best condition, mean, PSS, and five fidelity dimensions are reported.

Sample: The controlled training uses GSM8K and a Llama-3.1-8B-Instruct base. There are 25 training personas and 16 evaluation personas, distributed across specialty, educational level, traits, roles, and other styles. Each persona condition is run five times. The article does not report the exact number of PersonaGym questions, judge comparisons, or released outputs; it does not recruit human evaluators.

Findings

  • On MATH500, PerMix-RLVR raises the worst case from 34.0% to 41.0% over RLVR, reduces the best from 49.6% to 48.6%, and leaves the mean nearly unchanged, 47.1% versus 46.8%; PSS rises from 0.675 to 0.818.
  • The +21.2% on MATH500 is the relative change in PSS: the absolute change is +0.143 and the mean accuracy gain is +0.3 percentage points.
  • On GSM8K, PerMix-RLVR obtains 86.4% mean and PSS 0.975 versus 86.1% and 0.959 with RLVR; the difference is small.
  • Persona Consistency improves from 3.06 to 3.41, a relative +11.4%, while the full PersonaGym mean only rises from 2.79 to 2.87 and two of five dimensions worsen.
  • On LiveCodeBench Easy, PerMix maintains the RLVR mean and raises PSS; on Medium it slightly raises the mean but reduces PSS from 0.400 to 0.355, contrary to the legend's claim that both improve across all splits.
  • On AIME24, PerMix-RLVR falls below RLVR in worst case, best case, mean, and PSS; on LiveCodeBench Hard both have PSS 0 and PerMix's mean is lower.
  • The cross-family comparison associates Qwen3 with greater stability, but it confounds family, scale, data, and post-training recipe and does not causally identify RLVR.

Limitations

  • PSS uses only the minimum and maximum across 16 conditions, with no confidence interval; it is sensitive to noisy extremes and does not distinguish high stability with good accuracy from high stability with poor accuracy.
  • The expressions 'significantly' are not supported by hypothesis tests, intervals, bootstrap, or multiple-comparison adjustments; the reported deviations describe dispersion across personas.
  • The training and evaluation persona labels do not match literally, but their contents do overlap: elementary student/kindergartener, emeritus professor/PhD, and technical specialists/STEM, among others.
  • The prompts change identity, knowledge, tone, length, and resolution strategy at the same time. Therefore, the variation cannot be attributed to identity or personality separately and some personas reproduce stereotypes.
  • PersonaGym is evaluated only with LLMs, uses judges different from the original work, and provides no human calibration; the kindergartener case compares models of different family and size.
  • The theory presupposes a binary verifier independent of style, but the implementation extracts answers from stylized text, so formatting may affect the observed reward.
  • No code, checkpoints, rewritten data, outputs, PersonaGym questions, judge decisions, seeds, or full decoding configurations are released.
  • The article is a v1 preprint. The PDF includes MyeongSeok Kang, omitted from the arXiv metadata, and no formal publication or official artifact was identified.

What the study does not establish

  • It does not demonstrate that PerMix-RLVR improves stability or accuracy across any domain, difficulty, model, or set of personas; the difficult results and LiveCodeBench Medium contain counterexamples.
  • It does not prove a 21.2% improvement in accuracy or an 11.4% improvement in the total PersonaGym average; both percentages correspond to specific, relative metrics.
  • It does not isolate RLVR as the cause of the lower fidelity observed between Qwen3-32B and Llama-3.1-8B, because they differ in family, size, data, and post-training.
  • It does not validate the personas as authentic human representations or demonstrate safety against harmful or adversarial personas; 'harmful variation' here means loss of performance.
  • It does not demonstrate the theoretical model for the implemented finite algorithm or extend its guarantee to role-playing tasks, which the article itself places outside the formal scope.
  • It does not allow independent reproduction of the figures or auditing of the fidelity judgments with the available public artifacts.

Traceability

Scope: Full text

Version: arXiv:2604.08986v1 preprint

Consulted source: https://arxiv.org/pdf/2604.08986v1

Review: Codex 31-page visual full-text, TeX/source, metric arithmetic, prompt-pool, theory-scope, LLM-judge, artifact-search and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • meta-llama/Llama-3.1-8B-Instruct
  • Qwen3-32B teacher
  • Llama-3.1-70B teacher
  • Gemma-3-27B teacher
  • Qwen3 0.6B, 1.7B, 4B, 8B and 32B observational variants
  • Llama 3.1/3.2 1B, 8B and 70B observational variants
  • Gemma 3 1B, 4B, 12B and 27B observational variants
  • GPT-5-mini generator and judge
  • DeepSeek-v3.2 judge

Instruments and metrics

  • Persona Stability Score (minimum persona accuracy / maximum persona accuracy)
  • Pass@1 accuracy
  • PersonaGym: Expected Action, Action Justification, Linguistic Habits, Persona Consistency and Toxicity Control
  • DeepSeek-v3.2 pairwise persona-fidelity judgment

Data used

  • GSM8K
  • MATH500
  • AIME2024
  • LiveCodeBench v6
  • PersonaGym
  • GPT-5-mini persona-rewritten GSM8K targets
  • Sequence-level teacher generations

Evidence and location

  • Question, cross-family observation, PSS definition, and kindergartener comparison: Paper, pp. 1-5, abstract and sections 1-3.1; Figure 1 and Table 1
  • PerMix-RLVR design, baselines, and controlled results on GSM8K and MATH500: Paper, pp. 5-8, section 4; Table 2
  • LiveCodeBench results and generalization limits: Paper, pp. 8-9 and 23-27; Table 4 and appendix hard-task tables
  • QLoRA configuration, hyperparameters, verifier, and algorithm: Paper, pp. 13-15, Appendix A; Tables 5-8 and Algorithm 1
  • Full training and evaluation personas and semantic overlap: Paper, pp. 17-19, Appendix C; Tables 9-11
  • Judges, five dimensions, and full PersonaGym results: Paper, pp. 22-23, Appendix B.2; Table 15
  • Assumptions, ideal guarantee, and exclusion of role-playing from theoretical scope: Paper, pp. 4-6 and 20, sections 3.2-3.3 and Appendix D; Proposition 1
  • v1 status, license, and authorship divergence: arXiv abstract/Atom metadata and paper title page; PDF SHA-256 1919c9f5ff9bae5aeafdf83f8f327e00827e964019d2c59c4621c486ec876d4f
  • Absence of official code, models, and datasets: GitHub repository API and Hugging Face model/dataset API exact-name searches performed 2026-07-17
  • Comprehensive audit of metrics, prompts, theory, judges, artifacts, and reproducibility: reports/verification/article-372-permix-rlvr-pss-extreme-ratio-persona-pool-semantic-overlap-verifier-style-assumption-llm-judge-statistics-artifact-and-reproducibility-audit.json