Synthetic Personality Review

A-MEM: Agentic Memory for LLM Agents

Personas, identity, and agents2025arXivApproved editorial review

Authors: Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, Yongfeng Zhang

Keywords: Computation and Language, Human-Computer Interaction

Source: Open primary source (opens in a new tab)

6
Authors
7
Findings
18
Limitations
11
Evidence

Editorial summary

English

A-MEM is a Zettelkasten-inspired memory system for LLM agents; it is not a synthetic-personality study. Each conversational turn is transformed by an LLM into a note with content, context, keywords, tags, and timestamps. Embedding retrieval identifies candidate memories, another LLM step decides whether to create links or update neighboring memories' context and tags, and final answers use the top-k retrieved memories. The paper evaluates long-conversation question answering on LoCoMo and DialSim. On LoCoMo it compares A-MEM with full-context prompting, ReadAgent, MemoryBank, and MemGPT across GPT-4o-mini, GPT-4o, Qwen2.5-1.5B/3B, and Llama-3.2-1B/3B; the appendix adds DeepSeek-R1-32B and Claude 3 Haiku/3.5 Haiku. It reports F1, BLEU-1, ROUGE-2/L, METEOR, and SBERT similarity. A-MEM improves especially on Multi-Hop and Temporal questions, and on DialSim reaches F1 3.45 versus 2.55 for full context and 1.18 for MemGPT. It does not, however, lead every LoCoMo cell: full context or MemGPT score better in several Single-Hop, Open-Domain, or Adversarial model/category combinations. The broad claim of superiority over all baselines therefore exceeds the paper's own tables. A GPT-4o-mini ablation supports contributions from Link Generation and Memory Evolution, but there are no repeated runs, intervals, significance tests, or human evaluation; the checklist explicitly confirms that statistical significance was omitted because repeated API calls would be costly. Retrieval k is tuned by model and category up to 50 without a documented held-out validation set, creating benchmark-selection risk. The t-SNE figures provide no seed, configuration, or quantitative structure metric and do not establish superior semantic organization. Efficiency results do not show absolute leadership either: A-MEM uses about 1,200–2,500 tokens versus roughly 16,900 for full context/MemGPT, but ReadAgent and MemoryBank use fewer tokens in Table 1; at one million entries A-MEM reports 3.70 µs retrieval versus MemoryBank's 1.91 µs. The claimed sub-$0.0003 operation cost lacks a dated pricing model and reproducible calculation. The official evaluation repository audited at commit 0c8039f28fdcc08189a23c07a3437d9d2482f9c2 postdates the paper and contains no published result files, DialSim implementation, lockfile, or CI. It includes only ten LoCoMo conversations with 1,986 QA items while the paper describes 7,512 pairs, randomizes adversarial option order without a seed, uses nonzero decoding temperatures, and provides full-run scripts with author-specific absolute server paths. The system repository audited at f303dfc71e07bdc787f4bc135d4cea328ae30e99 adds an MIT license, packaging, and tests, but relies on external models/APIs, does not lock versions, and does not reproduce the paper's tables. Its initialization resets a ChromaDB collection named “memories,” and several backends turn LLM failures into empty JSON, both consequential production choices. A-MEM is contextually relevant to the continuity that could support an agent persona, but the paper does not induce, measure, validate, or demonstrate personality traits, persistent identity, or behavioral consistency.

Español

A-MEM es un sistema de memoria para agentes LLM inspirado en Zettelkasten; no es un estudio de personalidad sintética. Cada intervención conversacional se transforma mediante un LLM en una nota con contenido, contexto, palabras clave, etiquetas y marcas temporales. Un embedding recupera recuerdos candidatos, otro paso LLM decide si crear enlaces o actualizar el contexto y las etiquetas de recuerdos vecinos, y la respuesta final usa los k recuerdos recuperados. El artículo evalúa preguntas sobre conversaciones largas de LoCoMo y DialSim. En LoCoMo compara A-MEM con introducir todo el contexto, ReadAgent, MemoryBank y MemGPT sobre GPT-4o-mini, GPT-4o, Qwen2.5-1.5B/3B y Llama-3.2-1B/3B; el apéndice añade DeepSeek-R1-32B y Claude 3 Haiku/3.5 Haiku. Publica F1, BLEU-1, ROUGE-2/L, METEOR y similitud SBERT. A-MEM mejora especialmente Multi-Hop y Temporal, y en DialSim alcanza F1 3,45 frente a 2,55 de contexto completo y 1,18 de MemGPT. Sin embargo, no domina todas las celdas de LoCoMo: el contexto completo o MemGPT obtienen mejores resultados en varias combinaciones Single-Hop, Open Domain o Adversarial. Por tanto, la afirmación general de superioridad frente a todos los baselines excede sus propias tablas. La ablación con GPT-4o-mini apoya que Link Generation y Memory Evolution aportan rendimiento, pero no hay repeticiones, intervalos, tests de significación ni evaluación humana. El checklist confirma expresamente esa ausencia por coste de API. El k se ajusta por modelo y categoría hasta 50 sin documentar un conjunto de validación separado, lo que introduce riesgo de selección sobre el benchmark. Las figuras t-SNE son visualizaciones sin semilla, parámetros ni medida cuantitativa de estructura, y no prueban que la memoria sea semánticamente mejor. Las cifras de eficiencia tampoco equivalen a liderazgo absoluto: A-MEM usa unos 1.200–2.500 tokens frente a unos 16.900 de contexto completo/MemGPT, pero ReadAgent y MemoryBank usan aún menos en la Tabla 1; a un millón de entradas A-MEM reporta 3,70 µs de recuperación frente a 1,91 µs de MemoryBank. El coste inferior a 0,0003 dólares por operación carece de modelo tarifario, fecha y cálculo reproducible. El código oficial de evaluación auditado en el commit 0c8039f28fdcc08189a23c07a3437d9d2482f9c2 es posterior al paper y no contiene resultados publicados, DialSim, lockfile ni CI. Incluye solo diez conversaciones LoCoMo con 1.986 QA, mientras el artículo describe 7.512 pares; ordena aleatoriamente las opciones adversariales sin semilla, usa temperaturas no nulas y scripts completos con rutas absolutas del servidor de los autores. El repositorio de sistema auditado en f303dfc71e07bdc787f4bc135d4cea328ae30e99 aporta licencia MIT, empaquetado y tests, pero depende de modelos/APIs externos, no congela versiones y no reproduce las tablas. Su inicialización reinicia una colección ChromaDB llamada “memories” y varios backends convierten errores de LLM en JSON vacío, decisiones delicadas para un uso de producción. A-MEM es relevante para la continuidad contextual que podría sostener una persona de agente, pero el paper no induce, mide, valida ni demuestra rasgos de personalidad, identidad persistente o coherencia conductual.

Research question

Can an agentic memory system inspired by Zettelkasten organize, link, and update memories dynamically to improve question answering about long conversations compared to existing context and memory systems?

Method

Each turn is converted by an LLM into an atomic note with context, keywords, tags, and time. Embeddings all-MiniLM-L6-v2 retrieve neighbors; LLM prompts decide links and context/tag updates, and top-k supplies the memory used to answer. Long-term QA is evaluated on LoCoMo and DialSim against LoCoMo/full context, ReadAgent, MemoryBank, and MemGPT. Six main models are tested, k between 10 and 50 depending on model/category, two module ablations, scaling up to one million entries, and t-SNE visualizations.

Sample: The article describes LoCoMo with 7,512 question-answer pairs, conversations of about 9,000 words on average and up to 35 sessions, and DialSim with more than 1,300 sessions simulating five years of conversations from Friends, The Big Bang Theory, and The Office. It does not publish the exact number of questions actually scored per table or exclusion lists. The current evaluation repository contains locomo10.json with 10 conversations and 1,986 QA items when loaded with its own parser, not the 7,512 described; it does not contain DialSim.

Findings

  • On DialSim, A-MEM reports F1 3.45, BLEU-1 3.37, ROUGE-L 3.54, ROUGE-2 3.60, METEOR 2.05, and SBERT similarity 19.51, above LoCoMo/full context and MemGPT in all six columns of Table 2.
  • On LoCoMo, A-MEM usually improves Multi-Hop and Temporal, but does not win all combinations: for example, with GPT-4o-mini LoCoMo/full context obtains adversarial F1 69.23 versus 50.03 for A-MEM; with GPT-4o it obtains Single-Hop 61.56 versus 48.43 and Adversarial 52.61 versus 36.35.
  • The ablation with GPT-4o-mini reports that removing Link Generation and Memory Evolution reduces Multi-Hop F1 from 27.02 to 9.65 and Temporal F1 from 45.85 to 24.55; removing only Memory Evolution leaves intermediate results.
  • The optimal published k varies between 10 and 50 by model and category. Figure 3 shows plateaus and declines at high k, but no separate validation is documented to select these values.
  • Table 1 shows about 1,126 to 2,600 tokens for A-MEM depending on model versus about 16,900 for full context and MemGPT, although ReadAgent and MemoryBank can use fewer tokens than A-MEM.
  • In the one million memories test, A-MEM reports 1,464.84 MB and 3.70 plus or minus 0.74 us of retrieval; MemoryBank reports the same memory and 1.91 plus or minus 0.31 us, so A-MEM is not the fastest of the three.
  • The ten t-SNE visualizations show apparently more clustered distributions for A-MEM, but the work does not provide a clustering metric, seeds, hyperparameters, or a stability test.

Limitations

  • The work does not study personality. The only substantive mention of personality in the body describes the user portrait of the MemoryBank baseline; A-MEM is evaluated only as memory for conversational QA.
  • The claim of general superiority does not match the LoCoMo tables, where baselines outperform A-MEM in several combinations of model, category, and metric.
  • There are no independent repetitions, error bars on QA metrics, confidence intervals, statistical tests, or correction for multiple comparisons. The checklist answers No due to the cost of repeating API calls.
  • Generation temperatures are not fixed in the paper, seeds are not published, and closed APIs and local models are mixed without reproducible snapshots.
  • k is tuned per model and category using values from 10 to 50, but no validation split or nested protocol is defined; the result may benefit from selection on the test set.
  • It is not reported how many effective questions make up each table, what API or parsing errors are excluded, how many answers are invalid, or how retries and failures were handled.
  • F1, BLEU, ROUGE, METEOR, and embedding similarity are automatic proxies of overlap or resemblance. There is no human evaluation of correctness, attribution to retrieved memory, contradictions, hallucinations, or link quality.
  • The ablation only uses GPT-4o-mini and does not separate the effect of more LLM calls, more stored text, links, neighbor updating, and k selection.
  • The t-SNE figures are descriptive: they do not publish seed, perplexity, learning rate, normalization, selection criterion, or a quantitative cluster metric. A clustered appearance does not validate semantic quality or causality.
  • The cost below 0.0003 dollars per operation does not specify provider/model, current prices, input/output tokens, calls per note, retries, or calculation date.
  • The token reduction is compared with full context and MemGPT, not with all baselines; ReadAgent and MemoryBank use fewer tokens in Table 1. The scaling test also does not document hardware, index construction, warm-up, number of repetitions, or the nature of the million memories.
  • DialSim derives from television series and questions from fan sites; LoCoMo is a synthetic/curated benchmark. Real longitudinal conversations, multimodal memory, privacy, deletion, consent, or sensitive data are not evaluated.
  • The limitations section of the paper only acknowledges dependence on LLM capabilities and restriction to text; it does not address hyperparameter selection, uncertainty, privacy, cost, automatic evaluation, or reproducibility.
  • The current evaluation repository is later than the paper version, does not tag an experimental release, and does not include results, DialSim, lockfile, or CI. Its locomo10 corpus contains 1,986 QA, while the paper describes 7,512.
  • The robust scripts shuffle without a seed the order of the two adversarial options and generate with temperatures 0.5/0.7; ratio takes the first conversations, not a random sample. The code F1 uses token sets and does not count repetitions.
  • run_all_experiments.sh and run_k_sweep.sh contain paths /common/users/wx139, environments, and GPU assignments specific to the authors; furthermore, the former runs seven models and does not exactly match the six in the main table.
  • The system repository uses dependencies with minimum bounds, without lockfile or CI; part of the tests requires OpenAI and embedding downloads, so it is not hermetic. Syntactic compilation passes, but the suite could not be executed without installing services and external dependencies.
  • AgenticMemorySystem restarts when constructing a global ChromaDB collection called memories, which can erase shared state. The Ollama, SGLang, and OpenRouter controllers convert exceptions into empty JSON responses, hiding failures as if they were valid metadata.

What the study does not establish

  • It does not demonstrate that A-MEM creates, induces, measures, or maintains a synthetic personality; it only evaluates retrieval and question answering about conversational memory.
  • It does not establish persistent identity, behavioral consistency, traits, values, preferences, emotions, or an agent persona over time.
  • It does not demonstrate universal superiority over all baselines, models, categories, and metrics; its own tables contain important exceptions.
  • It does not prove that the links or updates generated by the LLM are true, stable, explainable, or better than non-agentic alternatives under the same budget.
  • It does not validate generalization to real human conversations, agents with tools, multimodal memory, other languages, multi-user deployments, or sensitive information.
  • It does not allow reproducing end-to-end all tables of the paper from a frozen release and the current official artifacts.
  • It does not justify by itself including A-MEM as central evidence about personality; its value in this review is contextual, as continuity infrastructure for agents.

Traceability

Scope: Full text

Version: arXiv:2502.12110v11, submitted 17 February 2025, revised 8 October 2025, 28 pages; NeurIPS 2025

Consulted source: https://arxiv.org/pdf/2502.12110v11

Review: Codex full-text, bilingual-fidelity, 28-page visual, arXiv-v11, NeurIPS-2025, dual-official-repository, scope-fit, baseline-exception, hyperparameter-selection, statistical, metric-validity, t-SNE, cost, scaling, privacy, production-safety and reproducibility audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o-mini
  • GPT-4o
  • Qwen2.5-1.5B
  • Qwen2.5-3B
  • Llama-3.2-1B
  • Llama-3.2-3B
  • DeepSeek-R1-32B appendix experiment
  • Claude 3 Haiku appendix experiment
  • Claude 3.5 Haiku appendix experiment
  • all-MiniLM-L6-v2 sentence embeddings

Instruments and metrics

  • Zettelkasten-style structured memory notes
  • LLM-based note construction, link generation and memory evolution prompts
  • Top-k embedding retrieval
  • F1 and BLEU-1
  • ROUGE-2 and ROUGE-L
  • METEOR
  • SBERT cosine similarity
  • Module ablation
  • Retrieval-k sweep
  • Token, memory-use and retrieval-time comparison
  • t-SNE visualization

Data used

  • LoCoMo long-term conversational memory benchmark
  • DialSim long-term multi-party dialogue benchmark
  • locomo10.json code snapshot, 10 conversations and 1,986 QA items

Evidence and location

  • Scope, architecture, and contributions: Paper, pp. 1 to 5, Abstract, Introduction, Figure 1 and Sections 3.1 to 3.4
  • Datasets, models, metrics, and implementation: Paper, pp. 5 to 6, Sections 4.1 to 4.2
  • LoCoMo results and exceptions to superiority: Paper, pp. 6 and 16 to 17, Tables 1, 5, 6 and 7
  • DialSim, ablation, and cost: Paper, p. 7, Tables 2 to 3 and Cost-Efficiency Analysis
  • k selection and scaling: Paper, pp. 7 to 8 and 18, Figure 3 and Tables 4 and 8
  • t-SNE and conclusions: Paper, pp. 9 and 18, Figures 4 to 5 and Section 5
  • Acknowledged limitations: Paper, p. 9, Section 6 Limitations
  • Declared absence of statistical significance: Paper, p. 25, NeurIPS checklist item 7
  • Evaluation code and drift relative to the paper: Official AgenticMemory repository commit 0c8039f28fdcc08189a23c07a3437d9d2482f9c2, README, requirements, locomo10, evaluation and shell scripts
  • System code and operational risks: Official A-mem-sys repository commit f303dfc71e07bdc787f4bc135d4cea328ae30e99, pyproject, llm_controller, memory_system and tests
  • Comprehensive visual inspection: Paper, all 28 rendered pages, including all tables, figures, prompt templates, examples and NeurIPS checklist