Personality-Driven Decision-Making in LLM-Based Autonomous Agents

Personas, identity, and agents2025arXivApproved editorial review

Authors: Lewis Newsham, Daniel Prince

Keywords: Artificial Intelligence, Multiagent Systems, Large Language Models, Autonomous Agents, OCEAN model, Cyber Defense

Source: Open primary source (opens in a new tab)

2
Authors
15
Findings
57
Limitations
17
Evidence

Editorial summary

English

The paper studies whether an OCEAN personality description changes the order in which an LLM selects tasks from a daily schedule. It starts with 500 workday schedules previously generated by an unidentified LLM. Each activity receives a UID computed with SHA-512 from its name and assigned time. At every cycle, the target model sees a personality statement first, the current time, remaining JSON tasks, completed tasks, and an instruction to return only the next UID. It must complete every activity and cannot skip any, so the observed outcome is a permutation of a fixed list rather than open-ended decision-making or action performance in an environment. GPT-4o, GPT-4o-mini, and GPT-3.5-Turbo are tested under high and low variants of each OCEAN trait plus a no-personality baseline. Exact model snapshots and much of the API configuration are not reported.

The analysis uses each task's positional movement and five normalized sequence-comparison measures: longest common substring, longest common prefix, Levenshtein distance, a longest-common-subsequence-based ratio, and Hamming distance. For each model, the ten conditions are compared with baseline through 50 independent Welch t-tests with a Bonferroni threshold of p≤.001. At temperature 1.0, all 50 comparisons are declared significant for GPT-4o and GPT-4o-mini, and 41 of 50 for GPT-3.5-Turbo. The dominant pattern, however, is that persona conditions reorder schedules much more than baseline. The paper itself notes that GPT-3.5 absolute differences are small despite statistical significance. In selected task-level plots, high-conscientiousness GPT-4o moves work activities earlier and low conscientiousness moves personal activities earlier; high extraversion advances social tasks and low extraversion favors solitary tasks. This plausibility analysis is limited to conscientiousness and extraversion and relies on task categories assigned by the authors.

The evidence does not separate personality from literal instruction following. Prompts contain transparent descriptors such as “outgoing, energetic, public,” while tasks have semantically obvious names such as Work, Social Media, and Team Collaboration. There is no equal-length nonpsychological control, descriptor ablation, blinded human evaluation, psychometric inventory, or test that the pattern persists beyond this task list. Statistical difference from baseline does not by itself show trait alignment; for openness, agreeableness, and neuroticism the paper mainly establishes the amount of reordering. The same 500 schedules appear to be reused across conditions, yet independent tests are used instead of a paired or repeated-measures analysis. Each schedule-condition prompt is apparently sampled once, so stability under the nondeterminism invoked by the paper is not estimated.

The audit identifies a material inconsistency in the SR metric. The method defines it as an LCS-derived Similarity Ratio and states that 1 means perfect alignment. In Table 1, however, SR rises with temperature, from .321 to .446 for GPT-4o and .393 to .530 for GPT-4o-mini, while the prose says it decreases. In Table 2, baseline SR is .35/.45 and personality conditions range from .57 to .94, the opposite pattern to LCSS and LCP and one that behaves more like a distance. The temperature narrative also quotes zero-temperature values that do not match the table: it gives .652/.615 for GPT-4o and .528/.489 for GPT-4o-mini, whereas the printed values are .635/.604 and .522/.476. These contradictions prevent SR from serving as independent validation and weaken the temperature interpretation.

The connection to SANDMAN and cyber defense remains prospective. No honeypot, adversary, observer-rated realism, attacker retention, safety property, or consequential decision is tested. The ethics section recognizes risks of misinformation and tailored persuasion and recommends oversight, bias mitigation, and alignment in general terms, but implements no safeguards and grounds defensive use in “rightful deception.” No code, schedules, complete prompts, results, or data are linked, and targeted search did not locate an official repository. The defensible contribution is a procedure for quantifying how strongly an LLM reorders a closed list under OCEAN descriptions and an exploratory demonstration of predictable semantic associations in GPT-4o; it is not evidence of psychological personality, operational autonomy, or cyber-defense effectiveness.

Español

El artículo estudia si una descripción de personalidad OCEAN cambia el orden en que un LLM elige tareas de una agenda diaria. A partir de 500 horarios laborales generados previamente por un LLM no identificado, cada actividad recibe un UID calculado con SHA-512 a partir del nombre y la hora asignada. En cada ciclo, el modelo ve primero una frase de personalidad, la hora actual, las tareas restantes en JSON, la lista de tareas completadas y la instrucción de devolver únicamente el UID siguiente. Debe completar todas las actividades y no puede omitir ninguna: la variable observada es, por tanto, una permutación de la misma lista, no una decisión abierta ni el desempeño de una acción en un entorno. Se prueban GPT-4o, GPT-4o-mini y GPT-3.5-Turbo con los cinco rasgos OCEAN en dirección alta y baja, más una condición sin personalidad. Los checkpoints exactos y gran parte de la configuración de API no se publican.

El análisis usa el desplazamiento de posición de cada tarea y cinco medidas normalizadas de comparación de secuencias: longest common substring, longest common prefix, distancia de Levenshtein, un ratio basado en longest common subsequence y distancia de Hamming. Para cada modelo se comparan las diez condiciones con el control mediante 50 t-tests de Welch independientes y corrección de Bonferroni hasta p≤0,001. Con temperatura 1,0, las 50 comparaciones de GPT-4o y las 50 de GPT-4o-mini se declaran significativas; para GPT-3.5-Turbo son 41 de 50. Sin embargo, la diferencia principal es que las condiciones con persona reordenan mucho más la agenda que el control. El propio artículo reconoce que en GPT-3.5 las diferencias absolutas son pequeñas pese a la significación. Al inspeccionar tareas concretas, GPT-4o con alta responsabilidad adelanta actividades de trabajo y con baja responsabilidad adelanta actividades personales; la alta extraversión adelanta tareas sociales y la baja extraversión favorece tareas solitarias. Ese análisis de plausibilidad se limita a responsabilidad y extraversión y utiliza categorías de tareas asignadas por los autores.

La evidencia no separa personalidad de seguimiento literal de instrucciones. Los prompts contienen descriptores transparentes como «outgoing, energetic, public» y las tareas tienen nombres semánticamente obvios como Work, Social Media o Team Collaboration. No hay control de igual longitud sin significado psicológico, ablación de los descriptores, evaluación humana ciega, inventario psicométrico ni prueba de que el patrón se mantenga fuera de esa lista. La significación frente a un control no demuestra que el cambio sea coherente con un rasgo; para apertura, amabilidad y neuroticismo el artículo ofrece sobre todo magnitud de reordenación. Además, los mismos 500 horarios parecen reutilizarse entre condiciones, pero se aplican pruebas independientes en vez de un análisis emparejado o de medidas repetidas. Cada prompt se ejecuta una sola vez por horario y condición, por lo que no se estima la estabilidad ante la no determinación que el propio trabajo invoca.

La auditoría detecta una inconsistencia sustancial en el ratio SR. El método lo define como Similarity Ratio derivado de LCS y afirma que 1 representa alineación perfecta. No obstante, en la Tabla 1 SR aumenta con la temperatura, de 0,321 a 0,446 para GPT-4o y de 0,393 a 0,530 para GPT-4o-mini, mientras el texto sostiene que disminuye. En la Tabla 2, el control obtiene SR de 0,35/0,45 y las condiciones con personalidad valores de 0,57 a 0,94, patrón opuesto al de LCSS y LCP y más compatible con una distancia. La narración también cita valores de temperatura cero que no coinciden con la tabla: informa 0,652/0,615 para GPT-4o y 0,528/0,489 para GPT-4o-mini, frente a 0,635/0,604 y 0,522/0,476 impresos. Estas contradicciones impiden interpretar SR como validación adicional y reducen la confianza en la lectura de temperatura.

La relación con SANDMAN y la ciberdefensa es prospectiva. No se prueba un honeypot, un adversario, el realismo para observadores, la retención de atacantes, la seguridad ni una decisión con consecuencias. La sección ética reconoce riesgos de desinformación y persuasión y propone supervisión, mitigación de sesgos y alineación de forma general, pero no implementa controles y basa el uso defensivo en la idea de «rightful deception». No se enlazan código, horarios, prompts completos, resultados o datos, y una búsqueda dirigida no localizó un repositorio oficial. La contribución defendible es un procedimiento para cuantificar cuánto reordena un LLM una lista cerrada bajo descripciones OCEAN y una demostración exploratoria de asociaciones semánticas previsibles en GPT-4o; no evidencia personalidad psicológica, autonomía operativa ni eficacia de ciberdefensa.

Research question

To what extent do high or low OCEAN descriptions alter the selection and ordering of tasks of GPT-controlled agents, and do some shifts correspond to expected priorities of conscientiousness and extraversion?

Method

Sequence permutation experiment over 500 daily schedules generated by an LLM. GPT-4o, GPT-4o-mini, and GPT-3.5-Turbo receive, in each cycle, an OCEAN persona, the time, the remaining and completed tasks, and must return the SHA-512 UID of the next task until all are exhausted. Ten trait conditions and a control are compared through per-task shifts, five sequence metrics, and 50 Welch t-tests per model with Bonferroni. An additional test varies the temperature from 0.0 to 1.6 in GPT-4o and GPT-4o-mini without a persona. The editorial audit read and rendered the ten pages, cross-checked tables, formulas, narrative, ethics, metadata, license, and artifact availability.

Sample: Five hundred synthetic schedules with variable frequency, duration, order, activity type, and start time. Each schedule is run under ten personality conditions and a control for each of three models; the text does not clarify independent repetitions of the same combination. For each condition, 500 values per metric are obtained. The temperature sweep uses the same 500 schedules in the control for GPT-4o and GPT-4o-mini. No distribution of the number of tasks, schedule length, seed, error rate, or count of discarded responses is reported.

Findings

  • Personality descriptions markedly alter the ordering of the same tasks compared to the no-persona control.
  • GPT-4o with high conscientiousness advances, on average, tasks classified as work-related.
  • GPT-4o with low conscientiousness advances, on average, activities classified as personal.
  • GPT-4o with high extraversion advances social tasks such as collaboration, meetings, and calls.
  • GPT-4o with low extraversion favors tasks presented as solitary.
  • The content-based plausibility analysis is restricted to conscientiousness and extraversion in GPT-4o.
  • At temperature 1.0, the ten GPT-4o conditions differ from the control across all five metrics according to the Bonferroni threshold.
  • GPT-4o-mini also shows 50 of 50 reported comparisons as significant.
  • GPT-3.5-Turbo shows 41 of 50 comparisons significant and nine non-significant.
  • The absolute differences of GPT-3.5-Turbo relative to the control are much smaller than those of GPT-4o and GPT-4o-mini.
  • The control preserves the initial order much more than most personality conditions in GPT-4o and GPT-4o-mini.
  • LCSS and LCP tend to decrease and Levenshtein tends to increase as temperature rises in the control.
  • Table 1 shows that SR increases, not decreases, between temperature 0.0 and 1.6 in both models.
  • The temperature narrative cites four initial values different from those printed in Table 1.
  • The work acknowledges risks of disinformation and persuasion arising from agents with induced personality.

Limitations

  • The observed task is solely reordering a closed list; there is no open-ended decision, environment, or consequence of actions.
  • All tasks are mandatory, so there is no selection under scarcity, renunciation, or opportunity cost.
  • No measurement of planning quality, deadline compliance, utility, efficiency, or schedule outcome is performed.
  • The program makes successive calls to the LLM, but autonomy in an operational environment is not demonstrated.
  • The personality descriptors directly contain the semantics expected to be observed in the tasks.
  • Task names, such as Work or Social Media, make the sought association transparent.
  • There is no control prompt of equal length and valence without personality meaning.
  • No ablations are performed between the trait label, its adjectives, and the initial position of the persona.
  • The persona always appears first, without counterbalancing that would allow measuring position bias.
  • High and low descriptions may differ in length, valence, and semantic strength.
  • The complete prompts for the ten traits are not published in the article or as a supplement.
  • The plausibility of task categories is assigned by the authors without blind coding.
  • No human validation, inter-rater agreement, or preregistration of the task-trait associations is reported.
  • Only conscientiousness and extraversion receive a specific per-task plausibility analysis.
  • For openness, agreeableness, and neuroticism, a sequence difference does not demonstrate coherence with the trait.
  • Differing from the control demonstrates reordering, not personality or psychological adequacy.
  • No psychometric inventory is administered, nor are reliability or construct validity assessed.
  • No human ratings of personality, realism, or similarity to human behavior are included.
  • Simple rules or semantic heuristics that could reproduce the same patterns are not compared.
  • The 500 schedules were generated by an unidentified LLM and its prompt or configuration is not published.
  • The distribution of the number of tasks, durations, categories, or structure of the schedules is not reported.
  • The composition of the synthetic schedules may encode biases and regularities favorable to the prompts.
  • The same schedules appear to be reused between condition and control, but independent rather than paired t-tests are applied.
  • A repeated-measures or mixed model representing shared schedules and multiple conditions is not used.
  • Repetitions of an identical prompt are not documented, so variability across executions is not estimated.
  • A single sample per schedule and condition does not support the claim of a stable effect under non-determinism.
  • Seeds, retries, invalid responses, UID failures, and recovery rules are not published.
  • Exact checkpoint identifiers, API dates, and provider versions are missing.
  • Top-p, seed, and other parameters necessary to reproduce the main conditions are not reported.
  • The five sequence metrics are correlated and partially redundant; they do not constitute independent evidence.
  • It is unclear whether Levenshtein and LCS are computed over UID tokens or over characters of concatenated hashes.
  • If computed at the character level, the distance between SHA-512 hashes does not represent a semantic difference between tasks.
  • LCSS and LCP measure preservation of the initial order, not quality or plausibility of the decision.
  • Hamming weights any positional disagreement equally and does not incorporate time, duration, or cost.
  • The Similarity Ratio is defined as similarity, but its values in the tables behave as distance.
  • The text states that SR decreases with temperature, while Table 1 shows a net increase in both models.
  • The LCSS and LCP values at temperature zero cited in the text do not match Table 1.
  • The drop in Hamming with temperature contradicts a simple reading of greater misalignment and is not tested statistically.
  • No trend or monotonicity tests are applied to the temperature sweep.
  • The significance tables omit exact p-values, effect sizes, and confidence intervals.
  • With n=500, small absolute differences can reach significance, as shown by GPT-3.5-Turbo.
  • The Bonferroni correction is applied to 50 tests per model, not to all analytical families in the article.
  • There is no formal model-by-trait interaction test supporting capability comparisons between models.
  • Attributing the larger effects of GPT-4o to superior reasoning does not rule out differences in instruction following.
  • The expression "accepting the null hypothesis" for nine tests is incorrect; one can only fail to reject it.
  • The shift error bars are standard deviations, not confidence intervals over the mean.
  • The mean shift may conceal heterogeneous patterns and cancellation across schedules.
  • In any permutation, advancing some tasks forces delaying others; per-task effects are compositional.
  • Other task lists, domains, languages, cultures, or open models are not tested.
  • No code, schedules, data, outputs, complete prompts, or analysis scripts are published.
  • A targeted search did not locate an official repository or reproducible artifacts linked to the work.
  • Cost, latency, token consumption, or temporal dependence on closed APIs is not reported.
  • No honeypot, attacker, SANDMAN operation, or cyberdefense outcome is evaluated.
  • Plausibility for deceiving an observer or retaining an adversary is not measured.
  • The ethics section proposes generic safeguards, but does not implement oversight, access control, or mitigation.
  • The idea of "rightful deception" does not analyze false positives, legal limits, or concrete accountability mechanisms.
  • The article does not contain a formal section delineating its limitations.

What the study does not establish

  • It does not demonstrate that GPT-4o, GPT-4o-mini, or GPT-3.5-Turbo possess personality.
  • It does not demonstrate a stable, reliable, or psychometrically valid OCEAN construct.
  • It does not separate induced personality from direct semantic following of the prompt.
  • It does not prove that reordering improves a schedule or a decision.
  • It does not demonstrate autonomous planning in a real environment.
  • It does not test consistency across repeated executions of the same prompt.
  • It does not establish generalization to other tasks, languages, domains, or model families.
  • It does not demonstrate plausibility for human observers or attackers.
  • It does not validate all OCEAN traits through trait-specific behavior.
  • It does not demonstrate that five correlated metrics provide five independent validations.
  • It does not establish a reliable monotonic relationship between temperature and all metrics.
  • It does not prove that GPT-4o has greater reasoning capacity by showing more reordering.
  • It does not demonstrate the efficacy of SANDMAN, honeypots, or proactive cyberdefense.
  • It does not evaluate safety, legality, fairness, or harms of the proposed system.
  • It does not offer a complete replication with the available public artifacts.

Traceability

Scope: Full text

Version: arXiv:2504.00727v1 (1 Apr 2025); published in AAMAS 2025 proceedings; CC BY 4.0

Consulted source: https://arxiv.org/pdf/2504.00727v1

Review: Codex full-text, visual, statistical, metric-consistency, ethics and artifact audit, 2026-07-15

Approval: Codex fidelity pass, 2026-07-15

English translation: approved, 2026-07-18

Models evaluated

  • GPT-4o (exact snapshot not reported)
  • GPT-4o-mini (exact snapshot not reported)
  • GPT-3.5-Turbo (exact snapshot not reported)
  • Unidentified LLM used to generate the 500 schedules
  • SANDMAN deceptive-agent architecture as prior/proposed application context

Instruments and metrics

  • High and low prompt descriptions for the five OCEAN traits
  • Task-position movement delta
  • Longest Common Substring (LCSS)
  • Longest Common Prefix (LCP)
  • Levenshtein Distance (LEV)
  • LCS-derived Similarity Ratio (SR)
  • Hamming Distance (HAM)
  • Independent two-sample Welch t-tests
  • Bonferroni family-wise correction to p≤.001
  • Temperature sweep from 0.0 to 1.6 in increments of 0.2

Data used

  • 500 synthetic work-oriented daily schedules generated by an unidentified LLM
  • Ten OCEAN persona conditions plus one no-personality baseline
  • Task categories manually treated as work, personal, social or solitary
  • Per-condition transformed UID sequences for three closed OpenAI model families

Evidence and location

  • Objective, scope, and declared contributions: arXiv v1, abstract and section 1, pp. 1–2
  • Generation of 500 schedules and SHA-512 UID: arXiv v1, section 3.1, pp. 2–3
  • Decision cycle, prompt order, and obligation to complete tasks: arXiv v1, section 3.1 and Figure 1, p. 3
  • Construction of the ten OCEAN personas and control: arXiv v1, section 3.2, p. 3
  • Shift and sequence comparison metrics: arXiv v1, section 3.3, pp. 3–4
  • Per-task results for conscientiousness: arXiv v1, sections 4.1.1–4.1.2 and Figures 2–4, pp. 4–5
  • Per-task results for extraversion: arXiv v1, section 4.1.3 and Figures 5–7, p. 5
  • Normalized definition of similarity and distance: arXiv v1, section 4.2, pp. 5–6
  • Temperature sweep values and SR inconsistency: arXiv v1, Table 1 and section 4.2.1, p. 6
  • Contradiction between cited temperature values and table: arXiv v1, Table 1 and section 4.2.1, p. 6
  • Welch, 50 comparisons, and Bonferroni: arXiv v1, section 4.2.2, p. 7
  • Complete results for GPT-4o and GPT-4o-mini: arXiv v1, Table 2 and section 4.2.3, p. 7
  • Results and small absolute magnitude of GPT-3.5-Turbo: arXiv v1, Table 3 and section 4.2.4, p. 7
  • Interpretation, proposed application, and limits of the result: arXiv v1, section 5, pp. 7–8
  • Declared ethical risks and safeguards: arXiv v1, section 5.1, p. 8
  • Version, publication, and license: arXiv:2504.00727v1 metadata and paper front matter; AAMAS 2025; CC BY 4.0
  • Absence of linked public artifacts: arXiv v1 paper and metadata plus targeted repository search; audited 15 Jul 2026