Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment

Trait induction and control2024ACL AnthologyApproved editorial review

Authors: Keming Lu, Bowen Yu, Chang Zhou, Jingren Zhou

Keywords: Computation and Language, Machine Learning

Source: Open primary source (opens in a new tab)

4
Authors
9
Findings
20
Limitations
6
Evidence

Editorial summary

English

Ditto asks an LLM to generate its own role-play supervision. For each character it collects Wikidata attributes and a Wikipedia profile, pairs characters to produce relevant and incompatible questions, gives the full profile to the same model for response simulation, and finally fine-tunes it with only a short introduction rather than the detailed profile. The paper applies this process to Qwen-Chat at 1.8B, 7B, 14B and 72B. WikiRole is reported as 3,902 roles, 7,086 English/Chinese sessions and 36,164 turns; the test holds out 100 characters, 53 English and 47 Chinese, not used for training, with 498 turns generated by GPT-4. GPT-4-1106-preview judges three dimensions: identifying the character among four candidates, scoring evidence-supported knowledge from 1 to 10, and classifying whether the model rejects a question outside the character's era or scope. Ditto improves all three reported figures at every size. Qwen-72B moves from 0.54/4.92/0.67 to 0.90/6.64/0.82 for identity/knowledge/rejection and from 8.13 to 8.43 on the MT-Bench role-play subset. Qwen-Max nevertheless scores 0.92/8.33/0.91, and GPT-4 exceeds Ditto-72B on knowledge and rejection. The 4 x 4 cross-supervision experiment suggests that role-play format or style can be learned even from a smaller supervisor, while knowledge depends more on the seed model. This is plausible but descriptive: there are no intervals, tests or seed replications. Artifact auditing reveals important limits. Identity is judged one turn at a time and the judge also sees the user question; an answer-blind lexical baseline selects the correct candidate for 249/498 turns (50%, versus 25% chance), so the metric does not isolate style or multi-turn consistency. Only 97/498 questions require rejection, meaning never rejecting already yields 80.52% accuracy. Ditto-72B does achieve 92.78% recall on those unknown questions, but falsely rejects 85 of 401 answerable questions. The repository publishes the test, one Ditto-72B response set and 25 score files, but not the training corpus, simulation/fine-tuning code or checkpoints. Several outputs omit or duplicate roles because evaluator failures can disappear silently. Overall, the paper provides useful evidence of improved Qwen role-play and an interesting benchmark, but it does not show that LLMs are literally superpositions of characters, that identity is stable, or that Ditto achieves arbitrary and safe role-play.

Español

Ditto propone que un LLM genere su propio entrenamiento de role-play. Para cada personaje recoge atributos de Wikidata y un perfil de Wikipedia, empareja personajes para producir preguntas pertinentes y preguntas incompatibles, entrega el perfil completo al mismo modelo para simular respuestas y finalmente lo ajusta con una introducción breve, sin el perfil detallado. El paper aplica el proceso a Qwen-Chat de 1,8B, 7B, 14B y 72B. WikiRole se describe como 3.902 roles, 7.086 sesiones en inglés/chino y 36.164 turnos; el test separa 100 personajes, 53 ingleses y 47 chinos, no presentes en entrenamiento, con 498 turnos generados por GPT-4. GPT-4-1106-preview juzga tres dimensiones: identificar al personaje entre cuatro candidatos, puntuar de 1 a 10 el conocimiento apoyado por evidencia y clasificar si rechaza una pregunta fuera de su época o ámbito. Ditto mejora las tres cifras en los cuatro tamaños. Qwen-72B pasa de 0,54/4,92/0,67 a 0,90/6,64/0,82 en identidad/conocimiento/rechazo y de 8,13 a 8,43 en el subset role-play de MT-Bench. Aun así, Qwen-Max obtiene 0,92/8,33/0,91 y GPT-4 supera a Ditto-72B en conocimiento y rechazo. El experimento cruzado 4x4 sugiere que el formato o estilo de role-play puede aprenderse incluso de un supervisor menor, mientras el conocimiento depende más del modelo base. Esa interpretación es plausible, pero descriptiva: no hay intervalos, tests ni repeticiones por semilla. La auditoría del artefacto detecta límites esenciales. La métrica de identidad evalúa cada turno por separado y muestra al juez también la pregunta; un baseline léxico que ignora la respuesta acierta 249/498 candidatos (50%, azar 25%), por lo que no aísla estilo ni consistencia multi-turno. Solo 97/498 preguntas requieren rechazo: no rechazar nunca ya daría 80,52% de accuracy. Ditto-72B sí logra 92,78% de recall en esas preguntas, pero rechaza erróneamente 85 de 401 preguntas válidas. El repositorio publica el test, una respuesta de Ditto-72B y 25 archivos de puntuación, pero no el corpus de entrenamiento, código de simulación/ajuste ni checkpoints. Varios resultados omiten o duplican personajes porque el evaluador pierde silenciosamente futuros fallidos. En suma, el trabajo aporta evidencia útil de mejora de role-play en Qwen y un benchmark interesante, pero no demuestra que los LLM sean literalmente una superposición de personajes, que la identidad sea estable o que Ditto logre role-play arbitrario y seguro.

Research question

Can an LLM improve its role-play by generating conversations from profiles it already knows how to read and then fine-tuning with them, and what does cross-supervision reveal about style transfer versus the limits of the base model's knowledge?

Method

Ditto extracts profiles from Wikidata/Wikipedia, pairs characters, generates three specific or contrastive questions, simulates answers with the complete profile, and fine-tunes the same Qwen with a brief introduction. It evaluates four sizes on 100 non-overlapping roles and 498 turns. GPT-4-1106-preview performs three judgments per turn on identity, knowledge, and rejection; MT-Bench role-play and a 4x4 grid of supervisor by seed model are added.

Sample: The declared training set contains 3,902 roles and 7,086 sessions (3,902 English and 3,184 Chinese), with 36,164 turns. The test set has 100 non-overlapping roles, 53 in English and 47 in Chinese, and 498 turns: 401 answerable questions and 97 that require rejection. Each conversation contributes between one and six turns.

Findings

  • Ditto improves identity, knowledge, and rejection across the four evaluated Qwen sizes.
  • Qwen-1.8B goes from 0.60/3.13/0.65 to 0.78/3.81/0.73; MT-Bench role-play, from 5.85 to 6.34.
  • Qwen-7B goes from 0.52/3.87/0.70 to 0.82/4.97/0.76; MT-Bench, from 6.73 to 6.90.
  • Qwen-14B goes from 0.52/4.15/0.68 to 0.90/6.03/0.80; MT-Bench, from 7.10 to 7.65.
  • Qwen-72B goes from 0.54/4.92/0.67 to 0.90/6.64/0.82; MT-Bench, from 8.13 to 8.43.
  • Qwen-Max retains the best result from Table 2, 0.92/8.33/0.91; Ditto-72B does not dominate the proprietary systems across all metrics.
  • Adding complete knowledge when simulating data for Qwen-1.8B especially improves knowledge, 3.77 to 4.40, and rejection, 0.73 to 0.79.
  • The cross-grid shows that identifiable identity can improve with weaker supervision; knowledge and rejection remain more tied to the seed model's capacity.
  • The audit reproduces the main Qwen aggregates and several proprietary ones from the published files, although some use incomplete denominators.

Limitations

  • 'Superposition of all characters' is a motivating metaphor, not a demonstrated mechanism.
  • The 100 roles are public Wikipedia entities predictably present in pretraining; they do not prove arbitrary role-play.
  • The three results depend on the same GPT-4-1106-preview as judge and are not independent measures of evaluator bias.
  • Identity is judged per turn, not by contradictions or drift over the course of a conversation.
  • The identity judge sees the question, which often reveals the character; a no-answer baseline achieves 50% versus 25% at random.
  • Knowledge measures coherence with evidence from the same generating profile, not independently verified historical truth.
  • The 97 rejection questions are only 19.48% of the test; raw accuracy hides recall, specificity, and over-rejection.
  • Ditto-72B achieves 92.78% recall on unknown questions, but incorrectly rejects 85 of 401 answerable questions.
  • No intervals, significance, multiple correction, or variation by seed are reported.
  • Human annotation does not report the number of annotators, agreement, adjudication, or per-item quality labels.
  • The WikiRole corpus of 36,164 turns, exact role selection, simulation/training code, and checkpoints are missing.
  • The public pool has 7,511 metadata entries and does not allow reconstruction of the trained subset of 3,902 roles.
  • Several outputs lose roles; Xingchen duplicates Edward III and omits Freddy Krueger.
  • The code ignores failed futures and never reaches the writing of the failure file after re-raising the exception.
  • The knowledge metric is also computed on 97 deliberately unanswerable questions and without evidence.
  • The main text and the appendix disagree between learning rate 2e-7 and initial 2e-6/minimum 2e-7.
  • The degraded Qwen checkpoints used as the base are not public, and the proprietary APIs are not fully controlled against each other.
  • Safety, biases, privacy, copyright, impersonation, or harm from representing real people are not evaluated.
  • The paper itself warns that models may generate toxic or harmful content under prompting.
  • Only English, Chinese, Qwen, and characters with Wiki profiles are studied.

What the study does not establish

  • It does not demonstrate that LLMs are literally a superposition of all characters.
  • It does not demonstrate arbitrary role-play for any person, character, language, or model family.
  • It does not turn 0.90 identity into evidence of stable multi-turn consistency.
  • It does not demonstrate that WikiRoleEval is objective or free from the LLM judge's bias.
  • It does not demonstrate that Ditto-72B matches or surpasses GPT-4, GPT-4-Turbo, or Qwen-Max globally.
  • It does not validate knowledge as independent factuality or absence of hallucinations.
  • It does not prove as a general law that style is easy to transfer and knowledge is limited by capacity.
  • It does not demonstrate that raw rejection accuracy adequately measures cognitive limits.
  • It does not allow reproduction of the training or all end-to-end results.
  • It does not establish safety for deployment or absence of risks from impersonating real people.
  • It does not generalize beyond Wiki roles in English/Chinese and the evaluated Qwen family.

Traceability

Scope: Full text

Version: ACL 2024 Volume 1, pages 7828-7840; arXiv:2401.12474v1 and official GitHub/LFS artifacts audited separately

Consulted source: https://aclanthology.org/2024.acl-long.423/

Review: Codex 13-page visual, official-ACL, arXiv-v1, GitHub-LFS, public-fallback-archive, all-25-judge-output, denominator, query-leakage, rejection-imbalance, silent-failure, construct, statistics, safety and claim-boundary audit, 2026-07-16

Approval: Codex fidelity pass, 2026-07-16

English translation: approved, 2026-07-18

Models evaluated

  • Qwen-1.8B-Chat
  • Qwen-7B-Chat
  • Qwen-14B-Chat
  • Qwen-72B-Chat
  • GPT-4-1106-preview evaluator
  • GPT-4 test-data generator
  • OpenChat-3.5-1210
  • Mistral-7B-Instruct-v0.2
  • Mixtral-7x8B-Instruct-v0.1
  • Claude 2.1
  • Wenxin 4.0
  • GPT-3.5-Turbo
  • GPT-4
  • GPT-4-Turbo
  • Qwen-Max
  • CharacterGLM
  • Xingchen

Instruments and metrics

  • WikiRoleEval consistent role identity
  • WikiRoleEval evidence-grounded role knowledge
  • WikiRoleEval unknown-question rejection
  • Four-candidate GPT-4-Turbo identity judgment
  • One-to-ten GPT-4-Turbo knowledge judgment
  • Human rejection-required labels
  • MT-Bench role-play subset
  • Four-by-four cross-supervision matrix
  • Four-hundred-query human quality audit

Data used

  • WikiRole train: reported 3,902 roles, 7,086 sessions and 36,164 turns
  • WikiRoleEval test: 100 held-out roles and 498 turns
  • Public role metadata pool: 7,511 rows and 7,493 unique labels
  • Public Ditto-Qwen-72B response set: 100 roles and 498 turns
  • Twenty-five public collapsed GPT-4-1106-preview judge-output files
  • MT-Bench role-play subset

Evidence and location

  • Metadata, DOI, license, and exact abstract: ACL Anthology 2024.acl-long.423
  • Method, metrics, configuration, tables, limitations, ethics, and appendices: ACL 2024 PDF, 13 pages, sha256 669ddfd495ebe9dcd123a9abab5ec8e67edfd161a2a7b1ba1bb44bfb2330aca7
  • Version, history, and official subjects: arXiv:2401.12474v1
  • Code, benchmark, answers, judge outputs, failures, and reproducibility: github.com/OFA-Sys/Ditto commit 7d4cb56ba5f058865ab5b1890b5b3916d2b042a6
  • Integrity and form of the public fallback: ditto_data.zip, sha256 c68bfaf20893a1cf966e988bddbbc1273d926bf636bab962d934a48a7f15d2f4
  • Audit of leakage, imbalance, denominators, data, code, judge, and claim limits: reports/verification/article-245-acl-ditto-wikirole-judge-leakage-missing-training-data-and-claim-audit.json