Persona-E²: A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events

Evaluation and psychometric validity2026arXivApproved editorial review

Original title: Persona-E$^2$: A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events

Authors: Yuqin Yang, Haowu Zhou, Haoran Tu, Zhiwen Hui, Shiqi Yan, HaoYang Li, Dong She, Xianrong Yao, Yang Gao, Zhanpeng Jin

Keywords: Personality, Persona conditioning, Role-playing agents, Human simulation

Source: Open primary source (opens in a new tab)

10
Authors
6
Findings
10
Limitations
8
Evidence

Editorial summary

English

Yang and colleagues present Persona-E², a dataset for studying how different people say they feel about the same text. Its main value is not raw size but density: the same 36 people label the same 3,111 events with one primary emotion, disgust, fear, anger, sadness, surprise, joy, or neutral, and confidence from 1 to 5. Each participant also supplies MBTI and five Big Five scores. This supports analysis of disagreement within a fixed panel without confounding it with different annotators seeing different items.

Events come from news, social media, and everyday narratives. A pipeline reduces 76,773 texts through two NSFW classifiers, Qwen3-Max differential-potential scoring, and review by five experts. Some Chinese sources are translated into English with Qwen3-Max. The annotators are all Chinese, university educated, aged 18–25, and proficient in English; over several weeks they answer how they would feel when reading each event. The paper reports training, latency and repetition monitoring, reminders, and reannotation, but not thresholds, numbers of corrected labels, total duration, or attrition.

For personality analysis, the authors cluster the 36 Big Five vectors into six K-means groups of 3, 4, 7, 11, 6, and 5 people. Personality Agreement Gap compares within- and out-group agreement. All six clusters have positive reported gaps from 8.27 to 25.96 points, and six MBTI types with at least three members are also positive. This is an interesting descriptive association, not causal identification: groups are built and evaluated on the same 36 people, equalized out-group sampling is unspecified, and there is no person-clustered uncertainty, independent replication, preregistration, or multiplicity control.

The paper's interpretive labels, an Emotional Black Hole for social media, a Psychological Immune System for life narratives, Anxious Empathy, and Negative Passivation, go beyond the measurements. General Writer is not an author's reported emotion but an external classifier output compared with human majorities. The Openness-neutralization correlation uses only six cluster means: it reproduces at r=.862 and p=.027 with an approximate 95% interval [.17,.985], loses significance when any of five clusters is omitted, and becomes p=.135 under a minimal five-trait Bonferroni adjustment. It is an exploratory hypothesis, not an established psychological mechanism.

In RQ2, GPT-5.1, Llama-3-8B, Qwen3-8B, Gemma-3-12B, and Ministral-3-8B predict two emotions under general, Big Five, and Big Five-CoT prompts. The study focuses on a 413-event Subjective Divergence Subset said to be consistent within clusters and divergent across them. That subset cannot be reconstructed: released consensus scores are on 0–100 while the threshold is written as .3, significant difference is not operationalized, and neither IDs nor a seed are released. Literal application yields 2,635 cases; treating the threshold as 30 yields 1,275, not 413.

Prediction results do not support a general significant benefit from personality. On the SDS, some models or domains improve and others worsen. On 100 random events, GPT-5.1 top-1 accuracy is 35% without personality, 36% with Big Five, and 35% with CoT; top-2 falls from 56% to 54% and 52%. The prompts are not controlled equivalents: Big Five adds five numbers, forces a polarity decision, and even lists Surprise as both positive and negative. Information, format, and personality content are confounded.

RQ3 does not test whether a model discovers an emotion or reproduces a cognitive process. Every prompt is given the human emotion and intensity and produces a post-hoc justification. Five reviewers make forced best-of-three choices on persona consistency, plausibility, and specificity. Big Five dominates, but it receives much richer information and explicit instructions to cite OCEAN dimensions, while baseline receives no profile. Without human rationales, concurrent reports, reviewer agreement, intervals, or verifiable blinding, the result shows preference for better-informed explanations under this protocol, not cognitive soundness or removal of personality illusion.

The data release is useful but has concrete integrity defects. The main CSV has 3,113 rows: two The Paper events have no annotations and are absent from the cluster file. One of the remaining 3,111 events lacks E34's emotion and confidence, leaving 111,995 observable labels rather than 111,996; another confidence is 28 on a 1–5 scale. There are duplicate events, two '[translation failed]' texts, mixed category vocabularies, and 38 English-text differences across files. General Writer also publishes 27/28 GoEmotions-like labels rather than the seven classes claimed in the paper, with no released mapping.

The current GitHub repository is the project's Vue website, not scientific code: it has no filtering, clustering, analysis, executable prompts, model responses, human judgments, tests, or reproducible environment, and it declares no repository license. Hugging Face and Kaggle distribute the three CSVs under CC BY-NC-SA 4.0, although the card adds use restrictions not identical to the standard license and does not document redistribution rights for each source. The faithful conclusion is that Persona-E² offers a dense, open panel for exploring emotional disagreement among 36 young Chinese participants; it does not establish population ground truth, trait causality, human cognition in LLMs, or fully reproducible findings.

Español

Yang y colaboradores presentan Persona-E², un dataset para estudiar cómo distintas personas dicen sentirse ante el mismo texto. Su rasgo más valioso no es el tamaño bruto, sino la densidad: las mismas 36 personas etiquetan los mismos 3.111 eventos con una emoción principal, asco, miedo, ira, tristeza, sorpresa, alegría o neutral, y una confianza de 1 a 5. Cada participante aporta además MBTI y cinco puntuaciones Big Five. Esto permite observar desacuerdos dentro de un panel fijo sin confundirlos con que cada ítem haya sido visto por personas distintas.

Los eventos proceden de noticias, redes sociales y relatos cotidianos. Un pipeline reduce 76.773 textos mediante dos clasificadores NSFW, puntuación de “potencial diferencial” con Qwen3-Max y revisión de cinco expertos. Algunas fuentes chinas se traducen al inglés con Qwen3-Max. Los anotadores, todos de China, universitarios, con 18–25 años y dominio avanzado de inglés, responden durante varias semanas a “¿cómo te sentirías al leer este evento?”. El paper declara entrenamiento, monitorización de latencia y repetición, recordatorios y reanotación, pero no informa umbrales, cuántas etiquetas fueron corregidas, duración total ni abandono.

Para analizar personalidad, los autores agrupan los 36 vectores Big Five con K-means en seis clusters de 3, 4, 7, 11, 6 y 5 personas. Comparan acuerdo dentro y fuera del grupo mediante Personality Agreement Gap. Los seis clusters dan gaps positivos de 8,27 a 25,96 puntos; seis tipos MBTI con al menos tres miembros también son positivos. Es una asociación descriptiva interesante, no una identificación causal: los grupos se construyen y evalúan sobre las mismas 36 personas, el muestreo externo “igualado” no está especificado, no hay incertidumbre agrupada por persona, réplica independiente, preregistro ni control de múltiples análisis.

Las etiquetas interpretativas del artículo, “Emotional Black Hole” para redes, “Psychological Immune System” para relatos vitales, “Anxious Empathy” o “Negative Passivation”, exceden los datos. “General Writer” no es la emoción del autor, sino la salida de un clasificador externo, comparada con mayorías humanas. La correlación entre Openness y neutralización usa solo seis medias de cluster: se reproduce como r=.862 y p=.027, con IC95% aproximado [.17,.985]; deja de ser significativa al retirar cualquiera de cinco clusters y un ajuste elemental por cinco rasgos da p=.135. Debe tratarse como hipótesis exploratoria, no mecanismo psicológico demostrado.

En RQ2, GPT-5.1, Llama-3-8B, Qwen3-8B, Gemma-3-12B y Ministral-3-8B predicen dos emociones con prompts general, Big Five y Big Five-CoT. La atención se concentra en una Subjective Divergence Subset de 413 eventos supuestamente consistentes dentro de cada cluster y divergentes entre clusters. Esa selección no se puede reconstruir: los scores liberados están en 0–100 aunque el umbral se escribe .3, “diferencia significativa” no se operacionaliza y no se publican IDs ni semilla. Aplicar literalmente el criterio produce 2.635 casos; interpretarlo como >30 produce 1.275, no 413.

Los resultados de predicción tampoco sostienen una mejora general “significativa” por personalidad. En la SDS, algunos modelos o dominios mejoran y otros empeoran. En una muestra aleatoria de 100 eventos, GPT-5.1 obtiene top-1 35% sin personalidad, 36% con Big Five y 35% con CoT; top-2 baja de 56% a 54% y 52%. Los prompts no son controles equivalentes: Big Five añade cinco números, obliga a decidir polaridad y hasta incluye Surprise tanto en candidatos positivos como negativos. Efecto de información, formato y personalidad quedan mezclados.

RQ3 no evalúa si el modelo descubre una emoción o reproduce un proceso cognitivo. Cada prompt recibe ya la emoción humana y su intensidad y genera una justificación posterior. Cinco revisores eligen forzosamente la mejor de tres explicaciones en consistencia de persona, plausibilidad y especificidad. Big Five domina, pero recibe mucha más información y la instrucción explícita de nombrar dimensiones OCEAN; el baseline no recibe perfil. Sin racionales humanos, informes concurrentes, acuerdo entre revisores, intervalos o cegamiento verificable, el resultado demuestra preferencia por explicaciones más informadas bajo ese protocolo, no “cognitive soundness” ni eliminación de la ilusión de personalidad.

La publicación de datos es útil pero presenta defectos concretos. El CSV principal tiene 3.113 filas: dos eventos de The Paper carecen de las 36 anotaciones y no aparecen en el archivo de clusters. Entre los 3.111 restantes falta la emoción/confianza de E34 en un evento, por lo que hay 111.995 etiquetas observables, no 111.996; otra confianza vale 28 en una escala 1–5. Hay duplicados, dos textos “[translation failed]”, categorías mezcladas y 38 textos ingleses distintos entre archivos. Además, el campo “General Writer” publica 27/28 etiquetas tipo GoEmotions, no las siete clases que afirma el paper, sin liberar el mapeo usado.

El repositorio GitHub actual es la web Vue del proyecto, no el código científico: no contiene filtrado, clustering, análisis, prompts ejecutables, respuestas de modelos, juicios humanos, tests ni entorno reproducible, y no declara licencia propia. Hugging Face y Kaggle sí distribuyen los tres CSV bajo CC BY-NC-SA 4.0, aunque la tarjeta añade restricciones de uso que no coinciden exactamente con la licencia estándar y no documenta derechos de redistribución por cada fuente. La conclusión fiel es que Persona-E² aporta un panel denso y abierto para explorar desacuerdo emocional en 36 jóvenes chinos; no establece ground truth poblacional, causalidad de rasgos, cognición humana en LLMs ni resultados plenamente reproducibles.

Research question

How do the emotions that different people report toward the same events vary, what association do those differences have with MBTI and Big Five, and to what extent can several LLMs predict them or retrospectively justify responses conditioned by profiles?

Method

Intra-subject dataset of 3,111 bilingual events labeled by the same 36 participants; MBTI-93 and IPIP-NEO-120 profiles; Big Five clustering, Personality Agreement Gap and polarity transitions; top-1/top-2 prediction with five LLMs on an SDS of 413 events and a random sample of 100; forced human evaluation of baseline, MBTI and Big Five rationales.

Sample: 36 participants from China, university students, aged 18-25 with advanced English; all label 3,111 events, target 111,996 units. The release contains 111,995 observable labels, two extra rows without annotations and one out-of-range confidence value.

Findings

  • The six Big Five clusters have positive descriptive PAG from +8.27 to +25.96 points.
  • The correlation of six clusters between Openness and neutralization is r=.862, p=.027, but it is fragile, ecological and does not survive a minimal correction for five traits.
  • On the SDS of 413 events, personality or CoT improves some cells and worsens others; global top-1 remains approximately between 20% and 31%.
  • On 100 random events, GPT-5.1 obtains top-1 35/36/35 and top-2 56/54/52 for general/persona/CoT.
  • Big Five rationales receive the highest proportion of forced choices, under prompts with unequal information and formats.
  • The data release allows validating profiles and cluster majorities, but not reconstructing SDS, external PAG, transitions of seven classes or LLM experiments.

Limitations

  • Narrow panel of 36 young Chinese university students; density does not equate to representativeness.
  • Small clusters derived and evaluated on the same sample, with no person holdout or hierarchical uncertainty.
  • Observational personality confounded with other differences; PAG does not identify causality.
  • Post-hoc psychological mechanisms and ecological correlation of only six points without multiple correction.
  • SDS not reproducible due to ambiguous threshold/units, absent divergence rule, IDs and seed not published.
  • Unequal RQ2 and RQ3 prompts; target emotion leaks into rationales and Big Five receives more information.
  • No agreement, uncertainty, assignment or sufficient blinding of the five reviewers is reported.
  • Integrity defects: two extra empty rows, one missing annotation, confidence 28, duplicates, failed translations and drift between files.
  • Released General Writer uses 27/28 classes although the paper claims a common space of seven.
  • Repository without scientific code, outputs, seeds, environment or license; source text rights not traced by record.

What the study does not establish

  • It does not establish a true emotion or universal ground truth for each event.
  • It does not demonstrate that Big Five or MBTI cause the observed differences.
  • It does not generalize to other ages, countries, educational levels, cultures or clinical populations.
  • It does not validate the post-hoc names as real psychological mechanisms.
  • It does not demonstrate a general and significant improvement in prediction from adding personality.
  • It does not demonstrate that an LLM rationale reproduces human cognition or appraisal.
  • It does not prove that Big Five eliminates the personality illusion.
  • It does not allow reproducing the SDS, the LLM runs, the human evaluation or all the statistics.
  • It does not justify using the dataset for individual psychological inference, mental health or user profiling.

Traceability

Scope: Full text

Version: ACL 2026 main paper 2026.acl-long.1350; DOI 10.18653/v1/2026.acl-long.1350; arXiv:2604.09162v2 and public data artifacts also audited

Consulted source: https://arxiv.org/abs/2604.09162

Review: Codex 26-page official ACL visual full-text, two-page checklist, TeX/source, Hugging Face Dataset Viewer and three-CSV data-quality, Kaggle, GitHub, clustering, ecological-correlation, SDS-selection, prompt-confounding, human-evaluation, licensing and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • GPT-5.1
  • meta-llama/Meta-Llama-3-8B-Instruct
  • Qwen/Qwen3-8B
  • google/gemma-3-12b-it
  • mistralai/Ministral-3-8B-Instruct-2512
  • Qwen3-Max
  • Distilbert-NSFW
  • Roberta-large-NSFW
  • External writer-emotion classifier reported as j-hartmann/emotion-english-distilroberta-base

Instruments and metrics

  • MBTI-93 Questionnaire
  • IPIP-NEO-120 Big Five questionnaire
  • Seven-class reader emotion label
  • Self-reported emotion confidence 1-5
  • Personality Agreement Gap
  • Normalized-entropy Group Consensus Score
  • Top-1 and Top-2 accuracy
  • Best-of-three reviewer preference

Data used

  • Persona-E² all-annotators CSV
  • Persona-E² Big Five group-consensus CSV
  • Persona-E² anonymized annotator-profile CSV
  • Subjective Divergence Subset (413 events; IDs not released)
  • Social Chemistry 101 and twelve news/social/life sources

Evidence and location

  • Definitive publication, design, results, prompts, ethics, limitations and appendices: ACL Anthology 2026.acl-long.1350, DOI 10.18653/v1/2026.acl-long.1350; all 26 pages rendered and inspected
  • Consent, compensation, institutional review, privacy and declared use of AI assistants: Official Responsible NLP Checklist for 2026.acl-long.1350; both pages rendered and inspected
  • Editable source, SDS criteria, prompts, tables and internal mention of 3,113: arXiv:2604.09162v2 source package SHA-256 25d6c27d972b9e9660d1fc1cd18ffa41464b4aab04418f58fe21c50d761bc01e; main TeX SHA-256 476c601b4114e72d19818f2e596d1c8544ff178e3d3df9cbd6bb78dedb070742
  • Status and defects of the public dataset: Hugging Face CRIS-Yang/Persona-E2-Dataset commit 1ddbbc304bf894feb98381867fde92c9ff02bc22; all three CSVs independently parsed
  • Alternative replica of the release and license metadata: Kaggle crisyang777/peronsa-e-personality-shaped-emotion-dataset version 4, updated 2026-04-22
  • Absence of scientific code and actual scope of the repository: https://github.com/SCUT-HAI/Persona-E2 commit b7260dfa29ec54ecf8eec5000df74cf315505835
  • Recomputation of rows, annotations, clusters, consensus, SDS and correlation: Independent audit of released CSVs SHA-256 ac47965fbf7b5b1cb4c0b3aa947ddd4cfd83e4e38ac9f8a22727644a19e2ee22, f083a381e8327c9a938554740ddecbba0577e6351f6e8af53bfe176661c36b73 and 0bc2e72eab25ecb9d0a9b0e7db333b8113682420c74280394575704af205869c
  • Comprehensive audit of population, measurement, clustering, SDS, evaluation, data, license and reproducibility: reports/verification/article-371-acl-personae2-annotator-population-personality-clustering-ecological-correlation-sds-selection-rationale-evaluation-data-license-and-reproducibility-audit.json