A Heterogeneous Temporal Memory Governance Framework for Long-Term LLM Persona Consistency

Personas, identity, and agents2026arXivApproved editorial review

Authors: Zhao Yang, Wang Huan, Li Yingshuo, Tu Haomiao, Lin Hujite

Keywords: Persona consistency, Role-playing agents, External memory, Long-term dialogue, Temporal ranking, Retrieval-augmented generation, Reproducibility

Source: Open primary source (opens in a new tab)

5
Authors
15
Findings
19
Limitations
4
Evidence

Editorial summary

English

This preprint presents ARPM, a prototype external-memory system for factual and temporal continuity in long dialogue. Its strongest contribution is not a new psychological personality but an inspectable architecture: stable knowledge and dialogue experience are separated, retrieved through different paths, fused with vector ranking and a BM25-style variant through RRF, decayed over round distance and physical time, unfolded chronologically, and checked through an analysis-before-response protocol. This moves continuity into auditable state, retrieval, prompts, and logs rather than leaving it entirely in weights or context.

In two structured 50-round tests, an automatic rule marks 27/50 answers correct at signal-to-noise 1:5 and 22/50 at 1:200+, while manual review raises them to 50/50 and 40/50. The audit exactly reproduces the arithmetic from released CSVs: 23 and 18 changes, all 0-to-1 and none reversed. The final label is not conventional recall; it jointly judges prompt evidence, support, and answer correctness. Without a second annotator, condition blinding, agreement, or reverse audit, the gap establishes disagreement with the automatic rule, not an unbiased advantage over another system.

The ablation workbook also reconstructs the headlines. History disabled is six questions with four normal labels and two errors, yielding 66.7%. Without hybrid retrieval there are 20 rows: 16 correct, three barely correct, and one wrong; 80% counts only strict correct. H and A2 are 20/20 and anomaly-free. Groups differ in size, date, prompt, preset, and label vocabulary, with no repeats, randomization, seeds, intervals, or inference. These are descriptive log ablations, not precise causal effects.

The cross-model test hands one evolving conversation across models after context clearing against a claimed 5.1-million-Chinese-character novel-noise substrate. The paper states 183 rounds and two excluded timeouts, but the public aggregate and three logs span indexes 1-172. Phase II is a 154-record memory dump, not an eleven-round scoring table that reconciles 183. The denominator is not reproducible. Model, round, topic, accumulated contamination, and difficulty also change together; Qwen2.5 decline, Qwen3/LongCat recovery, and MiniMAX drift are trajectory observations, not causal model comparisons.

“Persona consistency” combines tone, occupation, address form, task and sensory continuity, factual memory, time, and tag compliance. Its A-E scale has no psychometric validation, factor structure, reliability, or human ground truth, and the protocol acknowledges mixed human/model-assisted scoring. The faithful construct is externally supported continuity of facts, references, and style in one trace, not personality, identity, or subjective experience.

The repository is useful and inspectable, but execution reveals defects. Temporal weight exceeds 1 for future timestamps and silently remains 1 for timezone-aware past timestamps; direct tests produced 1.15356 at +24h and 1.0 at timezone-aware -24h. Knowledge uses unnormalized inner product while history uses normalized cosine although both share a documented 0-1 threshold. “BM25+” is a custom shifted-IDF variant. Missing package initializers allowed another storage package to block imports. Paper-era tests pass 9/10 configuration, 9/11 role-validator, and 3/5 isolated retriever tests; failures persist on current main, with no CI or paper tag. Main has drifted to v4.1.

A public experiment note also contains a plaintext API credential, not reproduced here; it should be revoked and purged. Flask lacks authentication, binds to 0.0.0.0, defaults to debug outside Docker, and exposes destructive endpoints; the UI stores the provider key in localStorage. This is a local prototype, not a network-ready service.

Overall, ARPM supports feasible, inspectable factual-temporal continuity. CSVs support manual counts and the workbook supports descriptive labels. The artifact does not establish 183 rounds, causal cross-model comparisons, superiority over other memory/long-context systems, or psychological personality. “Not applicable” availability declarations conflict with the repository, and current main is not the exact submission snapshot.

Español

Este preprint presenta ARPM, un prototipo de memoria externa para conservar continuidad factual y temporal en diálogos largos. Su aportación más sólida no es una personalidad psicológica nueva, sino una arquitectura inspeccionable: separa conocimiento estable de experiencias conversacionales, recupera ambos por rutas distintas, fusiona vector y una variante BM25 mediante RRF, aplica decaimiento por distancia de ronda y tiempo físico, ordena cronológicamente las evidencias y exige una fase de análisis antes de responder. Así convierte parte de la continuidad en estado, retrieval, prompt y logs auditables, en vez de dejarla enteramente en pesos o contexto.

En dos pruebas estructuradas de 50 rondas, una regla automática marca 27/50 respuestas correctas con señal:ruido 1:5 y 22/50 con 1:200+; la revisión manual las eleva a 50/50 y 40/50. La auditoría reproduce exactamente esa aritmética desde los CSV: hay 23 y 18 cambios, todos de 0 a 1, y ninguno inverso. La etiqueta final no es recall clásico, sino un juicio conjunto de evidencia presente, apoyo y respuesta correcta. Sin segundo anotador, cegamiento de condición, acuerdo o auditoría inversa, la diferencia prueba desacuerdo con la regla automática, no una estimación imparcial de ventaja frente a otro sistema.

La hoja de ablación permite reconstruir los titulares con otra frontera. Desactivar historial corresponde a seis preguntas con cuatro etiquetas normales y dos errores: 66,7 %. Sin retrieval híbrido hay 20 filas: 16 correctas, tres «apenas correctas» y una errónea; el 80 % cuenta solo las 16 estrictas. H y A2 tienen 20/20 y ninguna anomalía. Pero los grupos cambian en tamaño, fecha, prompt, preset y vocabulario; no hay réplicas, aleatorización, seeds, intervalos ni inferencia. Son ablations descriptivas de un log, no efectos causales precisos.

La prueba cross-model encadena modelos a medida que avanza una conversación con contexto limpiado y un sustrato descrito como 5,1 millones de caracteres chinos de novela-ruido. El paper afirma 183 rondas y dos timeouts excluidos, pero el agregado y los tres logs públicos cubren índices 1-172. Phase II contiene 154 bloques de memoria, no una tabla de once rondas que reconcilie 183. El denominador no es reproducible. Además, modelo, ronda, tema, contaminación acumulada y dificultad cambian juntos; la caída de Qwen2.5, recuperación de Qwen3/LongCat y deriva de MiniMAX son observaciones de una trayectoria, no un leaderboard causal.

«Consistencia de persona» mezcla tono, profesión, forma de dirigirse al usuario, tarea, referencias sensoriales, recuerdo factual, tiempo y cumplimiento de etiquetas. La escala A-E carece de validez psicométrica, estructura factorial, fiabilidad o ground truth humano. El protocolo admite mezcla de puntuación humana y model-assisted. El constructo fiel es continuidad externa de hechos, referencias y estilo en un caso, no personalidad, identidad o experiencia subjetiva.

El repositorio es valioso e inspeccionable, pero la auditoría ejecutada encuentra defectos. El peso temporal supera 1 con timestamps futuros y se queda silenciosamente en 1 con timestamps timezone-aware; una prueba produjo 1,15356 para +24 h y 1,0 para -24 h con zona. Knowledge usa inner product no normalizado mientras historial usa coseno normalizado aunque comparten umbral 0-1. El «BM25+» es una variante propia con IDF desplazado. Faltan init de paquetes y otro package storage bloqueó imports. Los tests del snapshot pasan 9/10 configuración, 9/11 role validator y 3/5 retriever; fallos persisten en main, sin CI ni tag del paper. Main ya es v4.1.

Un documento público contiene además una credencial API en claro, no reproducida aquí; debe revocarse y purgarse. Flask carece de autenticación, enlaza 0.0.0.0, activa debug por defecto fuera de Docker y expone borrado/vaciado; la UI guarda la clave en localStorage. Es prototipo local, no servicio listo para red.

En conjunto, ARPM demuestra factibilidad e inspeccionabilidad de continuidad factual-temporal. Los CSV sostienen los recuentos manuales y la hoja las etiquetas descriptivas. No demuestra 183 rondas, comparación causal entre modelos, superioridad frente a otras memorias/long context ni personalidad psicológica. Las declaraciones de disponibilidad «not applicable» contradicen el repo y el artefacto actual no es el snapshot exacto del envío.

Research question

Can an external memory that separates knowledge and experience, combines hybrid retrieval, temporal decay, chronological reading, and prior verification maintain factual, temporal, and stylistic continuity under noise, context cleaning, and model relay?

Method

ARPM maintains global knowledge and per-session history. In knowledge it fuses vectorial inner product and a BM25 variant via RRF; in history it uses normalized cosine. It adds role boosts, decay by rounds and hours, and chronological evidence to the prompt. It evaluates two QA runs of 50 rounds, nine configurations in a sheet of 141 rows and a sequential cross-model conversation. The audit reviewed 23 pages, TeX, git history, CSV, XLSX, logs, tests, and temporal execution.

Sample: Two QA runs of 50 rounds. Ablation: A=6, B=6, C=6 and D/E/F/G/H/A2=20, total 138 rows. The paper claims 183 cross-model rounds and two excluded timeouts, but the public aggregate covers 172. There are not enough users, characters, replicas, or independent annotators to estimate generality.

Findings

  • 27/50 are reproduced exactly at 50/50 and 22/50 at 40/50.
  • All manual changes are 0 to 1; none 1 to 0.
  • 66.7% is 4/6 with history disabled.
  • 80% is 16/20 strict; another three are barely correct.
  • H and A2 record 20/20 and zero anomalies.
  • The groups are neither equivalent nor repeated.
  • The public cross-model aggregate has 172 rounds, not 183.
  • The sequence confuses model with time, topic, and contamination.
  • Persona consistency mixes constructs without psychometric validation.
  • The external design and logs are inspectable.
  • The temporal calculation fails with futures and time zones.
  • The snapshot suite is not green.
  • Main is v4.1 and there is no immutable tag from the paper.
  • A public credential requires revocation and purge.
  • The app is a local prototype without network controls.

Limitations

  • Preprint without verified peer review.
  • One system and one main trajectory.
  • No equivalent baseline from another memory or long context.
  • Manual review without second annotator or agreement.
  • Manual changes exclusively favorable.
  • Ablations with different sizes and criteria.
  • No seeds, repetitions, intervals, or inference.
  • 183 rounds not reconciled with 172 public.
  • Cross-model sequential and not counterbalanced.
  • 5.1M noise not published.
  • Raw ratings and annotators not published.
  • Human/model-assisted scoring mixed.
  • Persona construct not validated.
  • Defects in temporal weight and scores.
  • Tests fail and no CI.
  • No exact tag or environment from the paper.
  • Contradictory availability statements.
  • Public credential and destructive APIs without authentication.
  • Provider key persisted in localStorage.

What the study does not establish

  • Stable psychological personality or human traits.
  • Identity, subjective experience, or genuine understanding.
  • Psychometric validity of persona consistency.
  • Superiority over other memories or long context.
  • Independent causal effect of each component.
  • Causal leaderboard among models.
  • Generalization to other users, characters, languages, or tasks.
  • Robustness under independent replicas.
  • Reproduction of 183 rounds or of the full noise.
  • Secure or production-ready service.
  • Verified suite free of regressions.

Traceability

Scope: Full text

Version: arXiv:2605.14802v1, submitted 2026-05-14, 23 pages, complete TeX; public ARPM repository audited at paper-era c4bf1f2 and current df1425a

Consulted source: https://arxiv.org/abs/2605.14802v1

Review: Codex 23-page visual full-text, complete TeX, repository history, CSV/XLSX reproduction, construct, code, tests, temporal-weight, artifact-security and reproducibility audit, 2026-07-17

Approval: Codex fidelity pass, 2026-07-17

English translation: approved, 2026-07-18

Models evaluated

  • DeepSeek-v4-pro
  • GPT-5.5
  • Claude Opus 4.6
  • Gemini-3-pro
  • GLM-5.1
  • Kimi-K2.6X
  • Qwen2.5-7B
  • Qwen3-8B
  • LongCat-Flash
  • MiniMAX-M2-Her
  • shibing624/text2vec-base-chinese

Instruments and metrics

  • ARPM dual-source external memory
  • Vector inner-product retrieval
  • Custom positive-IDF BM25-style scorer
  • Reciprocal Rank Fusion
  • Dual round and physical-time exponential weighting
  • Chronological evidence unfolding
  • Analysis/response protocol and repair
  • Two 50-round structured QA audits
  • 141-row ablation workbook
  • A-E reasonableness, tone and persona-continuity scale
  • Five-dimensional persona-consistency heatmap
  • Independent code, artifact and claim audit

Data used

  • Released 1:5 manual-review CSV, 50 rows
  • Released 1:200+ manual-review CSV, 50 rows
  • ARPM ablation workbook, 138 experimental rows across nine groups
  • Released 172-round Phase III dialogue, recall and analysis logs
  • Phase II JSON memory dump with 154 records
  • Claimed but unreleased complete 5.1-million-character noise substrate
  • Unreleased complete runtime database and engineering logs

Evidence and location

  • Text, method, results, figures, and statements: arXiv:2605.14802v1; PDF sha256 676d2a160baaf8e340bbcd7b1709e52aedccf572b29974a845de0cbb901a780b; TeX sha256 e210349865709f4e6577268a795e292e98b5ec102171cc8e1b5ba0459638689a
  • Associated code and derivation: ARPM commits c4bf1f2b7a3333932f0b52a6e0ef515c11894606 and df1425af51aa51ef1f8509cc4479ce4ea621f1d5
  • CSV/XLSX reproduction: CSV sha256 970ae238... and 8adc57de...; XLSX sha256 a2852c99...; artifact-tool visual and arithmetic audit
  • Independent audit: reports/verification/article-331-arpm-temporal-memory-persona-construct-experiment-code-repository-security-and-reproducibility-audit.json