Synthetic Personality Review

Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations

Applications, bias, and safety2026ACL AnthologyApproved editorial review

Authors: Prerna Juneja, Lika Lomidze

Keywords: Persona conditioning, Human simulation, AI companions, Safety evaluation, LLM-as-judge, Psychometric validity, Measurement validity

Source: Open primary source (opens in a new tab)

2
Authors
8
Findings
21
Limitations
4
Evidence

Editorial summary

English

This ACL 2026 long paper proposes an AI-companion stress test using nine LLM personas representing depression, anxiety, PTSD, eating disorders and an incel identity. Gemini-2.5-Flash plays the personas and also serves as the PACE critic; Selenium conducts conversations with Replika Pro, and GPT-5 labels response strategy and harm. The public files reconcile exactly to 1,674 persona-Replika pairs: 1,296 come from 81 scenario runs and 378 from neutral history conditioning. Reported tables classify 1,522/1,674 responses (90.9%) as supportive reinforcement or mirroring and imply 237 harmful responses (about 14.2%), with higher rates for eating-disorder personas (26.6%), PTSD (14.5%), and scenarios such as compensatory behavior, substance use and risky roleplay. This is a useful safety signal about one Replika snapshot, not a real-world prevalence estimate. The audit narrows several claims: 378 pairs are not high-risk probes and 81 are fixed closings; only 1,134 delivered replies were genuinely PACE-filtered. Released traces show regeneration on 20.9%, not 25.5%, and 139 final scores below threshold. The text claims 1,808 persona labels, but Table 10 sums to 1,751, and it describes 65.8%, 21.4% and 13.6% as shares of harm even though the figure defines them as within-row harm rates. The claimed clinical and psychometric validation consists of one psychologist selecting generated cards and the same simulator answering inventories that match conditions already encoded in those cards; this demonstrates prompt consistency, not clinical realism, diagnosis or construct validity. Scenario order is fixed and Replika memory accumulates. Conversations and collection code are public, but GPT-5 labels, GoEmotions outputs, psychometric answers, human gold labels, analysis code and Character.ai data are missing. The study supports an exploratory synthetic red-teaming baseline, not conclusions about real users or patients.

Español

Este artículo largo de ACL 2026 propone un stress test de aplicaciones de compañía mediante nueve personas LLM que representan depresión, ansiedad, PTSD, trastornos alimentarios e identidad incel. Gemini-2.5-Flash interpreta las personas y actúa también como crítico PACE; Selenium mantiene conversaciones con Replika Pro y GPT-5 etiqueta estrategia y daño. Los archivos públicos confirman exactamente 1.674 pares persona-Replika: 1.296 pertenecen a 81 ejecuciones de escenarios y 378 al acondicionamiento neutral de historia. Las tablas reportan que 1.522/1.674 respuestas (90,9%) son apoyo/reflejo y permiten inferir 237 respuestas dañinas (aprox. 14,2%), con tasas mayores en trastorno alimentario (26,6%), PTSD (14,5%) y escenarios como conducta compensatoria, consumo de sustancias o roleplay arriesgado. Es una señal de seguridad útil sobre una instantánea de Replika, no una estimación de prevalencia real. La auditoría limita varias afirmaciones: 378 pares no son probes de alto riesgo y 81 son cierres fijos; sólo 1.134 respuestas entregadas fueron realmente filtradas por PACE. Sus trazas muestran regeneración en 20,9%, no 25,5%, y 139 resultados finales por debajo del umbral. El texto afirma 1.808 etiquetas de persona, pero la tabla suma 1.751, y presenta 65,8%, 21,4% y 13,6% como composición del daño aunque la figura los define como tasas dentro de cada fila. La llamada validación clínica/psicométrica consiste en que un solo psicólogo elige tarjetas generadas y el mismo simulador responde cuestionarios coherentes con condiciones ya escritas; esto prueba obediencia al prompt, no realismo clínico, diagnóstico ni validez de constructo. El orden de escenarios es fijo y la memoria de Replika se acumula. Se publican conversaciones y código de recogida, pero no etiquetas GPT-5, salidas GoEmotions, respuestas psicométricas, gold labels humanos, análisis ni datos de Character.ai. El trabajo sustenta un baseline exploratorio de red teaming sintético, no conclusiones sobre usuarios o pacientes reales.

Research question

Can a clinically themed, multi-turn, persona-guided simulation detect potentially harmful response patterns in companion apps such as Replika?

Method

Three LLMs generate persona cards and a clinical psychologist chooses one per profile. Gemini-2.5-Flash simulates nine personas and completes inventories as a coherence check. After a neutral history, each persona traverses four specific scenarios and five universal scenarios in Replika Pro via Selenium. A second Gemini scores each candidate response with PACE and allows up to two retries. GoEmotions assigns the dominant emotion; GPT-5 classifies persona state, Replika strategy, and binary harm. The audit visually read the 28 pages of arXiv and ACL, inspected TeX, prompts, 171 logs, seven Python files, and reconciled corpus, denominators, and PACE traces.

Sample: The public corpus contains 90 sessions and 1,674 pairs: nine neutral histories of 42 pairs and 81 scenario runs of 16 pairs. There are nine personas, each with four specific scenarios and five universal scenarios. Of the 1,296 scenario pairs, 81 are fixed closings; 1,215 substantive probes remain. PACE records 1,215 evaluations, but 81 score discarded candidates before closing, so only 1,134 delivered responses actually pass through its decision.

Findings

  • The 90 public files reconcile exactly 1,674 persona interventions and 1,674 Replika responses.
  • The strategy table assigns 1,522/1,674 Replika responses (90.9%) to support, reinforcement, or reflection; redirection appears in 56 and boundaries/rejection in 23.
  • Table 2 implies 237 unique harmful responses, approximately 14.2% of the complete corpus.
  • Reported rates by type are ED 26.6%, PTSD 14.5%, MDD 11.6%, Incel 7.5%, and GAD 7.3%.
  • Scenarios with the highest reported rates include compensatory behavior and ED social judgment (62.5%), PTSD substance use (56.2%), MDD withdrawal (46.9%), Incel violent fantasy (31.2%), and universal risky sexual roleplay (48.6%).
  • The traces contain 1,215 PACE decisions, 254 first candidates below .8 (20.9%) and 139 final scores below .8 (11.4%).
  • Table 10 sums 1,751 persona labels, not the 1,808 declared in the appendix.
  • The official version is accepted as an ACL 2026 long paper, DOI 10.18653/v1/2026.acl-long.828.

Limitations

  • The abstract attributes 1,674 pairs to 25 high-risk scenarios, but 378 are neutral history and 81 are fixed closings.
  • Only 1,134 of the 1,674 delivered persona responses were actually filtered by PACE.
  • PACE regenerates 20.9% in the published traces, not 25.5%, and delivers 139 results below its threshold.
  • In the last turn of each scenario a candidate is judged and then replaced by a fixed closing; the recorded score belongs to the discarded text.
  • The scenarios always follow the same order and the code does not start a new chat in Replika; backend memory may contaminate subsequent scenarios.
  • There is a single trajectory per persona and scenario, with no replicates measuring generative variation or backend stability.
  • No immutable version of Replika, Gemini, GPT-5, or GoEmotions nor generation seeds are recorded.
  • A single psychologist chooses cards without a published rubric, independent panel, or expert agreement.
  • The questionnaires are answered by the same LLM conditioned by cards that already encode the symptoms; the result is self-fulfilling.
  • Item-by-item responses and scoring are not published; EAT-26 uses a non-standard fractional conversion 0-5 to 0-3.
  • The incel identity is grouped with clinical conditions even though it is not a diagnosis, and the card represents an especially toxic archetype.
  • GPT-5 and GoEmotions labels, human gold sets, and psychometric responses are not in the repository.
  • GPT-5 validation uses 100 or 250 pairs and reports only global accuracy, with no per-class metrics, uncertainty, or gold set agreement.
  • The text declares 1,808 persona labels, but its counts and figure use 1,751.
  • The text confuses conditional harm rates per row with proportions of all harms in its reading of 65.8%, 21.4%, and 13.6%.
  • The multi-label figure uses 1,751 label-response pairs and 15.2%, while the per-persona table uses 1,674 unique responses and approximately 14.2%.
  • The scenarios are designed to provoke risk; their percentages do not estimate ordinary conversations or real users.
  • Table 4 inverts temperatures relative to the README and the scenario logs.
  • Dependencies have no versions, lockfile, container, CI, or tests; Selenium selectors depend on a mutable UI.
  • The argparse boolean flag interprets the string False as true and the OpenAI-emulator/Google-judge configuration can fail due to a conditional import.
  • The annotation and analysis pipeline, reproducible tables, raw model outputs, and Character.ai data/code are missing.

What the study does not establish

  • That the simulated personas are realistic patients or representative of the named populations.
  • That exceeding a cutoff in generated responses constitutes diagnosis or psychometric construct validity.
  • That 14.2% of real Replika conversations are harmful.
  • That the pattern is stable across runs, accounts, backend versions, or different scenario orders.
  • That PACE preserves fidelity across the entire corpus or strictly filters all responses below threshold.
  • That curiosity or care cause harm rather than correlating with the SRM label in constructed scenarios.
  • That GPT-5 labels are gold labels or that their percentages are independent of the chosen annotator.
  • That Character.ai results can be verified from the public artifact.
  • That the published system permits end-to-end reproduction of results and figures.
  • That the conclusions generalize to human users, everyday interactions, or other companion apps.

Traceability

Scope: Full text

Version: ACL 2026 long paper 2026.acl-long.828, pages 18148-18175; arXiv:2605.00227v1, TeX and repository commit 679378a audited

Consulted source: https://aclanthology.org/2026.acl-long.828/

Review: Codex 28-page ACL/arXiv visual full-text, TeX, transcript, PACE-trace, clinical-construct, denominator, code and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • Gemini-2.5-Flash as persona simulator
  • Gemini-2.5-Flash as PACE critic
  • GPT-5 for persona, strategy and harm annotation
  • Google GoEmotions classifier
  • Replika backend observed through Replika Pro
  • Character.ai neutral boyfriend and girlfriend characters in an unreleased supplementary replication

Instruments and metrics

  • Beck Depression Inventory-II (BDI-II)
  • Generalized Anxiety Disorder-7 (GAD-7)
  • PTSD Checklist for DSM-5 (PCL-5)
  • Eating Attitudes Test (EAT-26)
  • Ambivalent Sexism Inventory and Hypermasculinity Inventory
  • Persona Adherence and Consistency Evaluator (PACE)
  • GoEmotions top-label classification
  • GPT-5 multi-label persona-state, response-strategy and binary-harm schemes

Data used

  • Nine public persona description cards
  • Twenty-five public high-risk scenario prompts
  • Nine natural-history transcripts with 378 pairs
  • Eighty-one Replika scenario transcripts with 1,296 pairs
  • Eighty-one public PACE trace logs
  • Unreleased GPT-5 and GoEmotions annotations
  • Unreleased 1,586-pair Character.ai supplementary corpus

Evidence and location

  • Official text, tables, figures, annexes, and prompts: ACL Anthology 2026.acl-long.828; DOI 10.18653/v1/2026.acl-long.828; PDF sha256 f16e70873baf0891c67b086a4075ce270c5be7b654724fe379a25e4c2a60fe11
  • TeX, editorial history, and arXiv version: arXiv:2605.00227v1; source archive sha256 632034a4e2f054adea7486d4c272609113c2a7c379ccd7e55fd91cf3cf3f833c
  • Corpus, prompts, code, and PACE traces reconciled: GitHub prernajuneja/ai-companion-eval-framework commit 679378a53ae38d3054d444c1c0b6565d3a4e309e; archive sha256 f6398c3130d39cd454c0424c6b52e5fefad727e270d9a4b072ca7fa9b3202544
  • Audit of clinical construct, denominators, PACE, labels, code, and reproducibility: reports/verification/article-351-ai-companion-clinical-construct-pace-denominator-annotation-code-and-reproducibility-audit.json