PersonaAgent with GraphRAG: Community-Aware Knowledge Graphs for Personalized LLM

Personas, identity, and agents2025arXivApproved editorial review

Authors: Siqi Liang, Yudi Zhang, Yingying Guo

Keywords: Large Language Models, Personality Control, Prompting, Persona, Steering

Source: Open primary source (opens in a new tab)

3: Authors
5: Findings
6: Limitations
5: Evidence

Editorial summary

English

PersonaAgent with GraphRAG is a personalization framework that combines an individual user's history with patterns retrieved from other users. It represents interactions, concepts, and categories in a heterogeneous graph. For each query, TF-IDF similarity retrieves both user-specific and global interactions; category preferences and related concepts are then linearized into a prompt for the language model. The evaluation covers three LaMP tasks: personalized news categorization, movie tagging, and product-rating prediction. Relative to the earlier PersonaAgent baseline, the method reaches 0.804 accuracy and 0.591 F1 on LaMP-2N, 0.653 and 0.662 on LaMP-2M, and lowers MAE to 0.216 and RMSE to 0.484 on LaMP-3. The headline improvements, 11.1% higher news F1, 56.1% higher movie F1, and 10.4% lower product MAE, are relative changes over PersonaAgent. A qualitative example shows global neighbor articles correcting a Llama 3 8B classification that had been pulled toward an overrepresented topic in one user's history. The study therefore supports the narrower claim that community context can improve these personalization metrics on the tested datasets and setup. It does not report uncertainty estimates, significance tests, or an ablation separating the effects of graph structure, personal retrieval, global retrieval, and prompt formatting. Nor does it evaluate the claimed explainability with users, longitudinal adaptation, privacy, cross-user leakage, or homogenization risks. In this paper, “persona” denotes a profile inferred from interaction history rather than a validated psychological trait.

Español

PersonaAgent con GraphRAG es una propuesta de personalización que combina el historial individual de un usuario con patrones recuperados de otros usuarios. El sistema representa interacciones, conceptos y categorías en un grafo heterogéneo. Para cada consulta, selecciona mediante similitud TF-IDF tanto interacciones del propio usuario como interacciones globales, añade preferencias por categoría y conceptos relacionados, y linealiza ese contexto en un prompt para el LLM. La evaluación usa tres tareas de LaMP: categorización personalizada de noticias, etiquetado de películas y predicción de valoraciones de productos. Frente a la versión previa PersonaAgent, el método obtiene en LaMP-2N 0,804 de accuracy y 0,591 de F1; en LaMP-2M, 0,653 y 0,662; y en LaMP-3 reduce MAE a 0,216 y RMSE a 0,484. Los porcentajes destacados por los autores, +11,1 % de F1 en noticias, +56,1 % en películas y −10,4 % de MAE en productos, son mejoras relativas sobre PersonaAgent. Un ejemplo cualitativo muestra que añadir noticias globalmente similares corrige una clasificación de Llama 3 8B sesgada por el historial temático del usuario. El trabajo evidencia que, en estos tres conjuntos y con esta configuración, incorporar contexto comunitario puede mejorar las métricas de personalización. Sin embargo, no incluye intervalos, pruebas de significación ni una ablación que separe la contribución del grafo, la recuperación personal, la recuperación global y el formato del prompt. Tampoco evalúa con usuarios la explicabilidad alegada, la adaptación longitudinal ni los riesgos de privacidad, homogeneización o filtración entre perfiles. Aquí «persona» significa un perfil inferido del historial de interacción, no un rasgo psicológico validado.

Research question

Does personalization improve in LaMP tasks when constructing an agent's prompts with a graph that combines individual history, category preferences, concepts, and interaction patterns retrieved from the community?

Method

The system constructs a heterogeneous graph with interaction, concept, and category nodes, and interaction-category, interaction-concept, and concept-concept edges. It retrieves Top-K from the personal history and from the global set through cosine similarity over TF-IDF, adds preference distributions and conceptual clusters, and forms a personalized prompt. It is evaluated against Non-Personalized, ReAct, MemBank, and PersonaAgent on LaMP-2N, LaMP-2M, and LaMP-3 using accuracy, F1, MAE, and RMSE. The test sets include the 100 users with the most extensive histories from each task, based on a temporal partition.

Sample: For testing, the 100 users with the largest history in each task are selected. The training data used to construct the graph contain 274 users in news, 829 in movies, and 1,000 in product reviews.

Findings

In LaMP-2N, PersonaAgent with GraphRAG obtains 0.804 accuracy and 0.591 F1, compared to 0.796 and 0.532 for PersonaAgent.
In LaMP-2M it reaches 0.653 accuracy and 0.662 F1, compared to 0.513 and 0.424; the reported relative improvement of F1 is 56.1%.
In LaMP-3 it reduces MAE from 0.241 to 0.216 and RMSE from 0.509 to 0.484 compared to PersonaAgent.
In the model comparison for LaMP-2N, Claude 3.5 Sonnet offers the best described F1 and recall, while Claude 4 falls below; the article attributes this latter result to overcomplicated responses, without a causal analysis.
The presented case shows that similar interactions from other users can correct a prediction dominated by a thematically narrow personal history.

Limitations

No deviations, confidence intervals, or significance tests are presented for the differences between methods.
An ablation isolating how much each component contributes is missing: graph structure, personal retrieval, global retrieval, concepts, categories, or prompt construction.
The evaluation is limited to three LaMP tasks and to the 100 most active users, which may not represent users with sparse histories.
The qualitative explanation is based on one case; there is no human evaluation of explainability, utility, or profile coherence.
Privacy, information leakage between users, community biases, and the risk of reducing minority preferences toward global patterns are not analyzed.
Although the motivation mentions changing preferences, the experiment does not measure longitudinal adaptation or consistency over time.

What the study does not establish

It does not demonstrate that GraphRAG is superior in any domain, model, or user population.
It does not prove that the agent possesses a psychological personality; the persona is an operational representation of observed preferences.
It does not validate that the system is explainable to users or that its recommendations are more fair, safe, or private.
It does not allow attributing the improvement specifically to community knowledge without the corresponding ablations.

Traceability

Scope: Full text

Version: arXiv:2511.17467v2

Consulted source: https://arxiv.org/pdf/2511.17467

Review: Codex editorial review, 2026-07-14

Approval: Codex fidelity pass, 2026-07-14

English translation: approved, 2026-07-18

Models evaluated

Mistral Small
LLaMA2-7B
LLaMA3-8B
Claude 3.5 Sonnet
Claude 4

Instruments and metrics

TF-IDF cosine similarity retrieval
Heterogeneous knowledge graph
Accuracy
F1
Mean absolute error (MAE)
Root mean squared error (RMSE)

Data used

LaMP-2N personalized news categorization
LaMP-2M personalized movie tagging
LaMP-3 personalized product rating

Evidence and location

Graph architecture, node types, and dual retrieval: Paper, pp. 2-3, Methodology, Knowledge Graph Construction and GraphRAG Retrieval Mechanism
Personalized prompt construction: Paper, p. 3, Personalized Prompt Generation and Algorithm 1
Datasets, user selection, and training sizes: Paper, p. 3, Results, Data Description
Quantitative results against baselines: Paper, pp. 3-4, Metrics Comparison and Table 1
Qualitative example and scope of conclusions: Paper, p. 4, Case Study and Conclusion