From Persona to Personalization: A Survey on Role-Playing Language Agents

Reviews, theory, and governance2024arXivApproved editorial review

Authors: Jiangjie Chen, Xintao Wang, Rui Xu, Siyu Yuan, Yikai Zhang, Wei Shi, Jian Xie, Shuang Li, Ruihan Yang, Tinghui Zhu, Aili Chen, Nianqi Li, Lida Chen, Caiyu Hu, Siye Wu, Scott Ren, Ziquan Fu, Yanghua Xiao

Keywords: role-playing agents, persona modeling, personalization, character simulation, interactive systems, LLM applications, human-likeness

Source: Open primary source (opens in a new tab)

18
Authors
24
Findings
35
Limitations
23
Evidence

Editorial summary

English

This 50-page narrative survey, accepted in TMLR in 2024, organizes role-playing language agents (RPLAs) around a useful but conceptual progression: demographic persona, character persona, and individualized persona. The first represents groups, professions, identities, interests, or personality types, and draws on statistical patterns and stereotypes learned by an LLM; the second represents public, historical, or fictional figures with established attributes and narratives; the third builds a dynamic profile of a particular person from their data, preferences, and interactions. The categories can coexist: a Socrates tutor may combine the demographic archetype of an ancient Greek philosopher, the Socrates character, and personalization learned from the user. The paper neither proposes nor tests a new model. It synthesizes literature and products to classify how data are sourced, how agents are constructed, how they are evaluated, and what risks they pose. For construction, it separates parametric training, pretraining, supervised fine-tuning, and reinforcement learning, from nonparametric prompting, including profiles, demonstrations, retrieval, and memory. For evaluation, it distinguishes general role-playing capability, engagement, conversation quality, and anthropomorphic abilities, from fidelity to a particular persona, linguistic style, knowledge, personality, and thought process, and summarizes four evaluation families: metrics or judges with ground truth, judges without ground truth, multiple-choice questions, and human evaluation. The text itself notes that LLM judges may lack character knowledge, that ground truth is often synthetic, and that human evaluations are costly and hard to reproduce. The demographic section reviews human questionnaire use to characterize inherent traits and prompt-based steering, while recognizing that assigned roles can amplify toxicity and bias. The character section separates descriptions from demonstrations and compares experience extraction, dialogue synthesis, and human annotation; Table 2 lists 18 dataset rows ranging from PDP and Character-LLM to PersonaHub. The individualized section organizes data as profiles, interactions, and domain knowledge; Table 3 lists 14 datasets across English, Chinese, Japanese, and Korean, and contrasts offline learning with online adaptation through memory, retrieval, fine-tuning, or feedback. It also groups personalized applications into conversation, recommendation, and task solving. The paper's most responsible contribution is that it does not reduce the topic to “personality imitation”: dedicated sections cover toxicity, bias, character hallucination, privacy, lack of social intelligence, long context, knowledge gaps, and anthropomorphism. It notes that fidelity to harmful characters conflicts with safety, personalization requires storing sensitive data, and emotional dependence or undisclosed impersonation may foster isolation or manipulation. Its four future directions concern causal analysis of decisions, improved decision making, comprehensive personal assistants, and autonomous social simulation. The appendix adds a snapshot of the 2024 market: 49 product rows, 13 persona-oriented and 36 task-oriented, but only 48 unique names because Squirrel AI is listed twice. This inventory is illustrative rather than a product evaluation: it supplies no cutoff date, search protocol, feature testing, row-level sources, or inclusion criteria, and several descriptions reproduce commercial claims not verified in the paper. The same qualification applies to the literature review. Although the paper calls the review systematic and received TMLR survey certification, it reports no search query, databases, time window, screening procedure, inclusion or exclusion criteria, selection flow, duplicate extraction, or quality and bias appraisal. The work should therefore be treated as a broad and useful taxonomy for orientation through October 2024, not as a reproducible systematic review or empirical evidence that LLMs possess personality, social intelligence, consciousness, or a faithful replica of an individual. Its language about “human likeness,” “digital life,” and human-RPLA coexistence is field framing, not an ontological finding. For this project it can serve as a conceptual backbone and historical catalog of methods, datasets, evaluations, and risks, provided its temporal scope and selection limitations remain explicit.

Español

Esta revisión narrativa de 50 páginas, aceptada en TMLR en 2024, organiza el campo de los agentes de lenguaje de juego de rol (RPLA) alrededor de una progresión útil pero conceptual: persona demográfica, persona de personaje y persona individualizada. La primera representa grupos, profesiones, identidades, intereses o tipos de personalidad, y explota patrones y estereotipos estadísticos aprendidos por el LLM; la segunda representa figuras públicas, históricas o ficticias con atributos y narrativas establecidas; la tercera construye un perfil dinámico de una persona concreta a partir de sus datos, preferencias e interacciones. Las categorías pueden coexistir: un tutor que interpreta a Sócrates puede reunir el arquetipo de filósofo griego, el personaje Sócrates y una personalización aprendida del usuario. El artículo no propone ni prueba un modelo nuevo. Sintetiza literatura y productos para clasificar cómo se obtienen los datos, cómo se construyen los agentes, cómo se evalúan y qué riesgos plantean. Para la construcción distingue entrenamiento paramétrico, preentrenamiento, ajuste supervisado y refuerzo, de prompting no paramétrico, incluidos perfiles, demostraciones, recuperación y memoria. Para la evaluación separa capacidad general de role-play, participación, calidad conversacional y habilidades antropomórficas, de fidelidad a una persona concreta, estilo lingüístico, conocimiento, personalidad y proceso de pensamiento, y resume cuatro familias de evaluación: métricas o jueces con ground truth, jueces sin ground truth, preguntas de opción múltiple y evaluación humana. El propio texto advierte que los jueces LLM pueden desconocer personajes, que el ground truth suele ser sintético y que las evaluaciones humanas son costosas y difíciles de reproducir. La sección demográfica revisa la medición de rasgos inherentes mediante cuestionarios humanos y el steering por prompt, pero reconoce que asignar roles puede amplificar toxicidad y sesgo. La sección de personajes diferencia descripciones y demostraciones, y compara extracción de experiencias, síntesis de diálogos y anotación humana; la Tabla 2 reúne 18 filas de datasets, desde PDP y Character-LLM hasta PersonaHub. La sección individualizada organiza los datos en perfil, interacciones y conocimiento de dominio; la Tabla 3 reúne 14 datasets en inglés, chino, japonés y coreano, y contrasta aprendizaje offline con adaptación online mediante memoria, recuperación, fine-tuning o feedback. También ordena las aplicaciones personalizadas en conversación, recomendación y resolución de tareas. La contribución más responsable del artículo es que no reduce el tema a “imitar personalidad”: dedica secciones a toxicidad, sesgo, alucinación de personaje, privacidad, falta de inteligencia social, contexto largo, lagunas de conocimiento y antropomorfización. Señala que la fidelidad a personajes dañinos entra en tensión con la seguridad, que personalizar requiere almacenar datos sensibles y que la dependencia emocional o la suplantación no declarada puede fomentar aislamiento o manipulación. Sus cuatro direcciones futuras son análisis causal de decisiones, mejor toma de decisiones, asistentes personales integrales y simulación social autónoma. El apéndice añade una fotografía del mercado de 2024: 49 filas de productos, 13 orientadas a persona y 36 a tareas, pero solo 48 nombres únicos porque Squirrel AI aparece dos veces. Ese inventario es ilustrativo, no una evaluación de producto: no aporta fecha de corte, protocolo de búsqueda, pruebas de funcionalidades, fuentes por fila ni criterios de inclusión, y varias descripciones son afirmaciones comerciales no verificadas en el paper. Esta misma limitación afecta a la literatura. Aunque se autodenomina revisión sistemática y obtuvo la certificación de survey de TMLR, el texto no publica consulta de búsqueda, bases consultadas, periodo, cribado, criterios de inclusión o exclusión, diagrama de selección, extracción duplicada ni evaluación de calidad o sesgo. Por tanto, el trabajo es una taxonomía amplia y muy útil para orientarse hasta octubre de 2024, no una revisión sistemática reproducible ni una medición empírica de que los LLM posean personalidad, inteligencia social, consciencia o una réplica fiel de individuos. Su lenguaje de “apariencia humana”, “vida digital” y convivencia debe leerse como framing del campo, no como evidencia ontológica. Para este proyecto sirve como columna vertebral conceptual y como catálogo histórico de métodos, datasets, evaluaciones y riesgos, siempre etiquetando su alcance temporal y sus límites de selección.

Research question

How can research and the market for role-playing language agents be organized according to the type of persona represented, data sources, construction and evaluation methods, and associated risks?

Method

Narrative and taxonomic literature review on LLMs and agents, structured into three progressive types of persona: demographic, character, and individualized. The article synthesizes construction methods, datasets, evaluation dimensions and techniques, risks, and commercial products through three research tables, a graphical taxonomy, and a market appendix. It does not document a reproducible search, screening, extraction, or quality assessment strategy, so "systematic" describes organizational breadth, not a verifiable systematic review protocol.

Sample: The paper does not define a reproducible bibliographic sample. Its visible corpus is materialized in references distributed across a taxonomy, 18 rows of character datasets, 14 rows of individualized datasets, and 49 rows of market products, with a duplication of Squirrel AI leaving 48 unique names. Coverage extends to cited works from 2024 and the final version is from October 2024.

Findings

  • It proposes a conceptual progression from demographic persona to character and individualized.
  • The three categories are not mutually exclusive and can coexist in the same agent.
  • The demographic persona explicitly relies on statistical patterns and stereotypes from pretraining.
  • The character persona requires representing attributes, history, relationships, knowledge, style, and decisions of an established figure.
  • The individualized persona is updated with data and preferences of a specific person.
  • Construction is divided into parametric training and non-parametric prompting, which can be combined.
  • Prompting includes descriptions, demonstrations, retrieval, and long-term memory.
  • Character data is distinguished between descriptions and demonstrations.
  • Demonstrations can come from experience extraction, LLM synthesis, or human annotation.
  • Synthesis without a reference source is described as limited by the teacher LLM and in need of filtering.
  • Evaluation separates general role-play capability from fidelity to a specific persona.
  • Fidelity includes linguistic style, knowledge, personality, and thought process.
  • The article acknowledges that LLM judges may poorly evaluate characters they are unfamiliar with.
  • The article acknowledges that part of the evaluation ground truth is synthesized with advanced models.
  • Individualized data is organized into profile, interactions, and domain knowledge.
  • Personalization is organized by conversation, recommendation, and task resolution.
  • Individualized learning is divided into offline and online, with a central role of memory and retrieval.
  • The review devotes explicit sections to toxicity, bias, character hallucination, privacy, and deployment.
  • Anthropomorphization is linked with dependence, social isolation, and public opinion manipulation.
  • Proposed future directions include causality, decisions, personal assistants, and social simulation.
  • Table 2 presents 18 rows of character datasets and Table 3 presents 14 individualized rows.
  • The commercial appendix presents 49 rows but 48 unique names due to duplicating Squirrel AI.
  • The audited version is arXiv v2 from October 2024 and the work was accepted at TMLR 2024.
  • No official repository of code or dataset specific to the survey is identified, and the work does not depend on one for its taxonomic contribution.

Limitations

  • It is a narrative review; it does not publish a reproducible search query.
  • It does not indicate which bibliographic databases were consulted.
  • It does not declare an explicit cutoff date for the literature search.
  • It does not define inclusion and exclusion criteria.
  • It does not show a screening process or a selection diagram.
  • It does not report extraction by independent reviewers or disagreement resolution.
  • It does not assess quality, risk of bias, or strength of evidence of the included studies.
  • It does not explain how the representative works appearing in the taxonomy were chosen.
  • It does not allow determining the exhaustiveness or coverage bias of the corpus.
  • Coverage ends in 2024 and does not represent the current state of 2026.
  • Grouping demographics, characters, and individuals into a progression may obscure deep ethical and epistemological differences.
  • The demographic category mixes professions, social groups, interests, and heterogeneous psychological typologies.
  • Using MBTI as an example does not imply it is a validated scientific measure for LLMs.
  • Applying human questionnaires to LLM text does not demonstrate that the psychometric construct measures the same thing.
  • The review cites results from studies but rarely compares designs, sizes, uncertainty, or replications.
  • It does not perform meta-analysis or quantitative synthesis.
  • The text lists evaluation methods as "three-fold" and then lists four categories.
  • Metrics with synthetic ground truth may inherit biases and errors from the generator model.
  • LLM judges without ground truth often lack sufficient knowledge about lesser-known characters.
  • Human evaluation is characterized as costly and poorly reproducible, but no validated alternative standard is offered.
  • The term character hallucination redefines hallucination as role-breaking or out-of-date knowledge, not as general factual falsehood.
  • The tension between fidelity to a harmful character and safety is discussed, but not resolved with operational criteria.
  • The privacy discussion is general and does not compare threats, guarantees, or regulatory requirements by technique.
  • The anthropomorphism section formulates plausible risks without providing a specific review of clinical or longitudinal evidence.
  • The product inventory does not publish search method, selection criteria, or sources per product.
  • The functionalities, models, or data practices attributed to commercial products are not empirically verified.
  • Table 4 does not declare a cutoff date and was quickly exposed to obsolescence.
  • Squirrel AI appears duplicated in Table 4 with two different descriptions.
  • Some references are incomplete and API-Bank is cited twice in the bibliography.
  • The "systematic review" label may be misleading because it is not accompanied by a reproducible systematic protocol.
  • The TMLR survey certification accredits the type of contribution, it does not guarantee methodological exhaustiveness.
  • Expressions such as human likeness, digital life, and harmonious coexistence are aspirational framing and favor anthropomorphization.
  • The review does not consistently distinguish evidence about observable behavior from claims about cognition or internal states.
  • There is no structured artifact that allows automatically updating the taxonomy, tables, or product inventory.
  • No official repository is provided with the bibliographic selection, extraction sheets, or change history.

What the study does not establish

  • It does not demonstrate that LLMs have human personality.
  • It does not demonstrate that RPLAs possess consciousness, identity, emotions, or subjective experience.
  • It does not demonstrate that a score on a human questionnaire has the same psychometric validity in an LLM.
  • It does not demonstrate that an agent that imitates a style is faithful to the personality or thinking of a person.
  • It does not demonstrate that the three types of persona are a unique or exhaustive ontology.
  • It does not causally demonstrate that assigning personas improves task resolution in general.
  • It does not demonstrate that greater role-play engagement implies greater fidelity.
  • It does not demonstrate that LLM judges are valid or impartial evaluators of personality.
  • It does not demonstrate that synthetic ground truth represents human or canonical responses.
  • It does not demonstrate that interaction memory produces a digital replica of a person.
  • It does not demonstrate that RPLAs are safe for emotional support, health, education, or personal decisions.
  • It does not demonstrate that anonymization, filters, or general protocols eliminate privacy and abuse risks.
  • It does not demonstrate that multi-agent simulation reproduces human societies.
  • It does not demonstrate that the listed products actually implement all the attributed capabilities.
  • It does not establish the state of the market or research after October 2024.
  • It does not constitute a reproducible systematic review even though it uses that term.
  • It does not offer its own empirical evidence to compare models, methods, or products.
  • It does not validate a psychological theory of synthetic personality.

Traceability

Scope: Full text

Version: arXiv:2404.18231v2, revised 9 Oct 2024; accepted and published in Transactions on Machine Learning Research, October 2024; OpenReview forum xrO70E8UIZ

Consulted source: https://arxiv.org/pdf/2404.18231

Review: Codex full-text, bilingual-fidelity, visual, bibliographic, taxonomy, review-method, persona-construction, evaluation, psychometrics, dataset, risk, anthropomorphism, privacy, product-inventory and temporal-scope audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • Large language models as a general model class
  • GPT-4 and ChatGPT as examples and automated evaluators in surveyed work
  • Claude, PaLM 2 and other contemporary foundation models as background examples
  • Parametrically trained role-playing language agents surveyed across prior work
  • Prompted and retrieval-augmented role-playing language agents surveyed across prior work

Instruments and metrics

  • Three-level RPLA persona taxonomy
  • Construction taxonomy: pretraining, supervised fine-tuning, reinforcement learning and in-context prompting
  • Role-playing capability versus persona fidelity distinction
  • Automatic evaluation with ground truth
  • Automatic evaluation without ground truth and LLM-as-judge
  • Multiple-choice evaluation
  • Human evaluation
  • Psychological questionnaires cited in surveyed studies, including Big Five instruments
  • Application-tier taxonomy: conversation, recommendation and task solving
  • Risk taxonomy: toxicity, bias, character hallucination, privacy, deployment challenges and anthropomorphism

Data used

  • Table 2: 18 rows of character-persona datasets
  • Table 3: 14 rows of individualized-persona datasets
  • PDP, Character-LLM, ChatHaruhi, RoleLLM, HPD and CharacterGLM
  • PIPPA, RoleEval, CharacterEval, DITTO, RolePersonality and MORTISE
  • CroSS-MR, SocialBench, TimeChara, LifeChoice, InCharacter and PersonaHub
  • PERSONA-CHAT, ConvAI, Qianyan, P-Ubuntu, P-Weibo and FoCus
  • MPCHAT, OpinionQA, Synthetic-Persona-Chat, COMSET, RealPersonaChat, LiveChat, KBP and Cho et al. 2023
  • Table 4: 49 product rows representing 48 unique product names

Evidence and location

  • Official record and temporal scope: arXiv:2404.18231v2, submitted 28 Apr 2024, revised 9 Oct 2024; accepted to TMLR 2024; OpenReview forum xrO70E8UIZ
  • Complete audited source: .cache/editorial-sources/article-104/source.pdf; 50 pages; sha256 dc0680f09336e4be4c49ab278d619dba1a52c3d7f6e5bf45a79a586c6bfb64d7
  • Objective and three-persona taxonomy: Full text pp. 1-3, Abstract, Introduction and Figure 1
  • Background on LLMs and agents: Full text pp. 3-5, sections 2.1-2.2
  • Global research taxonomy: Full text p. 6, Figure 2
  • Construction methods: Full text pp. 7-8, Table 1 and sections 3.2-3.3
  • Demographic persona: Full text pp. 9-11, section 4
  • Datasets and character data: Full text pp. 11-13, section 5.2 and Table 2
  • Character construction: Full text pp. 13-14, section 5.3
  • Evaluation dimensions and methods: Full text pp. 14-16, section 5.4
  • Three-fold error versus four categories: Full text pp. 8 and 15, sections 3.3 and 5.4
  • Individualized persona and datasets: Full text pp. 16-18, sections 6.1-6.3 and Table 3
  • Personalization evaluation: Full text pp. 18-19, section 6.4
  • Toxicity and bias: Full text pp. 19-21, sections 7.1-7.2
  • Character hallucination and privacy: Full text pp. 21-22, sections 7.3-7.4
  • Deployment and anthropomorphization: Full text pp. 22-23, sections 7.5-7.6
  • Conclusions and future directions: Full text pp. 23-24, section 8
  • Absence of reproducible systematic protocol: Full text pp. 1-24: no search query, databases, cutoff, inclusion/exclusion criteria, screening flow, duplicate extraction or study-quality appraisal reported
  • Product inventory: Full text pp. 43-50, Appendix A and Table 4
  • Inventory count and duplication: Table 4 audit: 13 persona-oriented rows plus 36 task-oriented rows, 49 total rows, 48 unique names; Squirrel AI listed twice
  • Bibliographic issues: Full text References: API-Bank appears in two adjacent entries; several web/preprint references lack complete publication metadata
  • Absence of survey artifact: Official arXiv record exposes PDF, HTML and TeX source but no author-provided code/data repository; checked 15 Jul 2026
  • Integral visual verification: All 50 PDF pages rendered and visually inspected: 24 main-text pages, 19 reference pages and 7 product-appendix pages; checked 15 Jul 2026