When artificial minds negotiate: Dark personality and the Ultimatum Game in large language models

Society, culture, and collective behavior2026ElsevierApproved editorial review

Authors: Vinícius Ferraz, Tamas Olah, Ratin Sazedul, Robert Schmidt, Christiane Schwieren

Keywords: Personality, Persona conditioning, Safety and bias

Source: Open primary source (opens in a new tab)

5
Authors
12
Findings
39
Limitations
5
Evidence

Editorial summary

English

The paper tests whether graded Dark Factor of Personality descriptions (D1-D5) change language-model choices in a binary, one-shot Ultimatum Game. Seventeen Ollama model labels separately generate proposers choosing either a fair EUR20/EUR20 split or a selfish EUR32/EUR8 split, and responders accepting EUR8 or rejecting so both receive zero. The released main data contain 339,956 D-conditioned completions rather than independent agents: 169,981 proposer and 169,975 responder rows across five levels, two temperatures and 17 models. They are descriptively compared with 4,166 reused human decisions from Hilbig and Thielmann (2025), with additional strong-prompt analyses and 800 generations attributed to GPT-4.1 and GPT-5.1.

In the released D1-D5 data, aggregate fair offers fall from 0.912 to 0.168, and all 17 models have a negative association between D and fair proposing. Responding does not follow an equivalent gradient: aggregate acceptance is 0.689, 0.781, 0.916, 0.859 and 0.754, with five models always accepting and others showing positive, negative or flat patterns. The defensible contribution is therefore evidence that explicit personality instructions can create strong regularities in a forced proposer choice, while responder behavior is highly model-dependent and does not reproduce a uniform human gradient. Justification text also shifts lexically, but terms such as fair, unfair, accept and reject are generated with the decision and predict it in the same fitting sample; this supports observable prompt echoing and rationalization, not internal processing or motivation.

Review of the PDF, all five official CSV files and the repository identifies material problems. Figure 1 reports 0.526 for AI proposers, whereas Table 1, the prose and the D-only release give 0.491; the announced neutral observations are not released. Human H2 reverses the sign shown by OR=0.397 and the human data. The human-likeness score appears with incompatible formulas and values, handles constant models inconsistently and depends on notebooks with unrecorded interactive state. Strong-prompt data contain 50,000 proposer but only 32,083 responder rows, omit cells, publish a qwen2.5 responder result absent from the raw data and use the strong responder descriptions in every proposer row. The reproducible paired test uses only three models and yields p=0.1231 even though another cell states p<0.05. The claimed causal decomposition is an order-dependent sequence of R-squared increments with pseudoreplication, incorrect pairings, hard-coded fallbacks and a bootstrap routine that pools groups before resampling.

The findings should be interpreted as prompt compliance in a narrow hypothetical task, not evidence of negotiating minds, latent dark personality or human psychological similarity. There is no negotiation, interaction, real incentive, persistence or learning. Human D is continuously measured in a natural sample; AI D is a balanced linguistic treatment that sometimes directly prescribes behavior, so the scales are not psychometrically equivalent. The current repository postdates publication, has no tag, retains broken entry points and does not freeze models, dependencies or neutral data. The package also redistributes human microdata containing demographics and 98 psychometric responses while stating that no participants or personal data were used; this creates a governance and privacy concern without, by itself, determining a legal conclusion.

Español

El artículo estudia si descripciones graduadas del Dark Factor of Personality (D1-D5) cambian las elecciones de modelos de lenguaje en una versión binaria y de una sola ronda del juego del ultimátum. Diecisiete etiquetas de modelos ejecutadas con Ollama debían actuar, en generaciones separadas, como proponentes que elegían un reparto justo de 20/20 euros o uno egoísta de 32/8, y como respondedores que aceptaban 8 euros o rechazaban para dejar a ambos con cero. Los datos principales liberados contienen 339.956 completions condicionadas por D, no agentes independientes: 169.981 como proponente y 169.975 como respondedor, repartidas entre cinco niveles, dos temperaturas y 17 modelos. Se comparan descriptivamente con 4.166 decisiones humanas reutilizadas de Hilbig y Thielmann (2025), y se añaden análisis con prompts más explícitos y 800 generaciones atribuidas a GPT-4.1 y GPT-5.1.

En los datos D1-D5 publicados, la proporción agregada de ofertas justas cae de 0,912 a 0,168, y los 17 modelos muestran una asociación negativa entre D y elección justa. La respuesta no sigue un gradiente equivalente: la aceptación agregada es 0,689, 0,781, 0,916, 0,859 y 0,754, con cinco modelos que aceptan siempre y otros con patrones positivos, negativos o planos. La contribución defendible es, por tanto, mostrar que instrucciones explícitas de personalidad pueden producir regularidades fuertes en una elección forzada como proponente, mientras la respuesta depende mucho de la etiqueta del modelo y no reproduce un gradiente humano uniforme. Los textos justificativos también cambian léxicamente, pero palabras como fair, unfair, accept y reject se generan junto con la decisión y permiten predecirla dentro de la misma muestra; esto evidencia eco lingüístico y racionalización observable, no procesamiento interno ni motivación.

La revisión del PDF, los cinco CSV oficiales y el repositorio descubre problemas materiales. La Figura 1 da 0,526 para proponentes de IA, mientras la Tabla 1, el texto y los datos D-only dan 0,491; las observaciones neutrales anunciadas no están publicadas. La hipótesis humana H2 invierte el signo que muestran el OR=0,397 y los propios datos. El score de semejanza humana aparece con fórmulas y valores incompatibles, maneja de forma inconsistente modelos constantes y deriva de notebooks con estado no registrado. Los datos de prompt fuerte tienen 50.000 proponentes pero solo 32.083 respondedores, faltan celdas, publican un resultado de qwen2.5 respondedor ausente del raw y todas las filas de proponente contienen por error las descripciones fuertes de respondedor. El test pareado reproducible usa solo tres modelos y produce p=0,1231, aunque otra celda afirma p<0,05. La supuesta descomposición causal es una secuencia de incrementos R² dependientes del orden, con pseudorreplicación, emparejamientos erróneos, valores de respaldo codificados y un procedimiento de bootstrap que mezcla los grupos antes de remuestrear.

El resultado debe interpretarse como cumplimiento de prompts en una tarea hipotética estrecha, no como evidencia de mentes que negocian, personalidad oscura latente o parecido psicológico con humanos. No hay negociación, interacción, incentivos reales, persistencia ni aprendizaje. La D humana es una medida continua en una muestra natural; la D de IA es un tratamiento lingüístico equilibrado y a veces prescribe directamente la conducta, por lo que ambas escalas no son psicométricamente equivalentes. El repositorio actual es posterior a la publicación, no tiene tag, conserva puntos de entrada rotos y no congela modelos, dependencias ni datos neutrales. Además, el paquete redistribuye microdatos humanos con demografía y 98 respuestas psicométricas pese a declarar que no emplea participantes ni datos personales; esto exige cautela de gobernanza y privacidad, aunque no permite inferir por sí solo una conclusión jurídica.

Research question

To what extent do graded descriptions of the dark factor D modify the choices and justifications of different LLMs in proposer and responder roles of a binary ultimatum game, and do those patterns resemble associations observed in human data?

Method

Each prompt assigns one of five hand-written D profiles and an independent role. The proposer chooses between 20/20 and 32/8 euros; the responder accepts 8 euros or rejects. 1,000 completions per cell are requested for 17 Ollama labels, two roles, five levels, and temperatures 0.2/0.8; a neutral baseline that does not appear in aidata.csv is also declared. Decisions are parsed from free text and compared with 4,166 human decisions with measured D. Logistic and OLS regressions, per-model correlations, a composite human-likeness score, lexical analysis with the 100 most frequent unigrams, PCA/t-SNE, strong prompts on five models, and a GPT-4.1/GPT-5.1 comparison are fitted. The notebooks call causal decomposition the sequential increments of R² and the effect/mediation analysis that does not identify causality.

Sample: The released main sample has 339,956 D-only generations: 169,981 from proposers and 169,975 from responders, across 340 model×role×D×temperature cells; five cells fall in 975-997 observations. They are repetitions of 17 deployments and fixed prompts, not 339,956 independent agents. The human sample has 4,166 rows: 2,079 proposers and 2,087 responders. The frontier file contains exactly 800 balanced rows. The strong prompt raw file contains 82,083, not the 100,000 declared: 50,000 proposers and 32,083 responders; all qwen2.5 responders are missing, one llama3.2 temperature, and part of dolphin3. The neutral baseline of the design is not published.

Findings

  • Aggregate fair offers fall from 0.912 at D1 to 0.168 at D5 in the D-only data.
  • The 17 models show negative correlation between D and fair choice as proposer, with very heterogeneous magnitudes.
  • Acceptance as responder is non-monotonic: 0.689, 0.781, 0.916, 0.859, and 0.754 between D1 and D5.
  • Five models accept 100% of offers, so their responder correlation and odds ratio are undefined.
  • Aggregate D-only rates are 0.491 for AI fair offer and 0.800 for AI acceptance, versus 0.808 and 0.810 in humans.
  • Aggregate differences between temperatures 0.2 and 0.8 are small, although some models change appreciably.
  • The 800 frontier rows reproduce binary switching in proposers and constant acceptance of GPT-5.1 as responder.
  • The lexical regression obtains in-sample R² of 0.855 in proposers and 0.539 in responders, with direct leakage of words that express the decision.
  • Figure 1 does not reproduce the proposer rate of Table 1 or of aidata.csv.
  • The reproducible paired strong prompt test for responders uses three models and is not significant: t=2.579, p=0.1231.
  • The human microdata reproduce negative associations between D and behavior coded as prosocial in both roles, not the positive direction formulated in H2 for responders.
  • The strongest evidence is differential compliance with explicit instructions, not latent personality or human equivalence.

Limitations

  • The title speaks of negotiation, but there is no exchange, counteroffer, dialogue, or interaction between agents.
  • The task only offers two decisions, a fixed amount, and a hypothetical round without real incentives.
  • The D profiles are explicit instructions and in the strong versions they directly prescribe which option to take.
  • Measured human D and manipulated AI D are not psychometrically equivalent scales.
  • Calling repeated completions of the same models and prompts independent agents incurs pseudoreplication.
  • Of 340 main cells, 219 have a constant binary decision.
  • The observations from the announced neutral baseline, as well as parsing logs, failures, or retries, are not published.
  • Figure 1 gives 0.526 for AI proposers, versus 0.491 in Table 1, text, and D-only data.
  • H2 and the interpretation of the human responder OR invert the real sign of the result.
  • The human binning rules change between figures, text, and notebooks and leave some extremes with two or three cases.
  • The human-likeness score combines non-equivalent magnitudes through an arbitrary normalization dependent on the set of models.
  • The score has incompatible formulas and figures between Figure 4, sections 4.3/4.5, discussion, and notebooks.
  • Undefined correlations and ORs of constant models are omitted when averaging, instead of receiving the described treatment.
  • The pre-publication notebook depended on unregistered interactive state and the post-publication output changes rankings.
  • The supposed interval equivalence uses five hand-written texts; the curve in Figure 8 falls at D5 and is not monotonic.
  • The code for MiniLM and for the GPT-4.1/GPT-5.1 collection is not released.
  • The paper says temperature 0.7 for frontier, the CSV uses 0.2/0.8, and the README says 0.0/0.7/1.4.
  • The strong prompt raw file has 17,917 responders fewer than declared and several cells absent or incomplete.
  • strong_prompt_results.csv contains a qwen2.5 responder result impossible to derive from the published raw file.
  • The 50,000 strong proposer rows contain by error descriptions designed for responders.
  • The matched strong comparison only has three complete models, but the table declares N=5.
  • One cell claims p<0.05 although the stored and reproducible test gives p=0.1231.
  • The lexical regression fits and evaluates on the same rows, with no holdout, CV, or split by model.
  • Words such as accept, reject, fair, and unfair directly reveal the decision and produce target leakage.
  • The t-SNE uses binary presence of unigrams, not semantic embeddings of justifications.
  • The justifications are outputs posterior or simultaneous to the choice, not measures of internal processing.
  • The language-behavior correlations are computed over only five D averages and do not include uncertainty.
  • The variance decomposition uses order-dependent Type-I OLS increments on a binary outcome and clustered observations.
  • The effects code matches models by position, includes hardcoded fallbacks, and presents percentiles between models as intervals.
  • The Cohen d bootstrap mixes the groups before resampling and generates a null distribution, not an interval of the observed effect.
  • The mediation uses language generated with the decision as mediator and does not identify a causal mechanism.
  • The repository is posterior to publication, has no release/tag, and retains notebooks with obsolete outputs.
  • The experimental notebooks point to nonexistent prompts and the wrapper invokes an absent script.
  • Required dependencies, lockfile, seeds, environment, Ollama digests, quantization, and exact templates are missing.
  • Transfer of the D4-D5 prompts to harm, bias, rejection, or safety failures is not evaluated.
  • Accepting an unfair offer and proposing a fair split are both coded as prosocial although they express different norms.
  • The article declares no use of participants or personal data, but it analyzes and redistributes detailed human microdata.
  • hudata.csv is not accompanied by a specific license, dictionary, privacy assessment, or documentary basis for consent/reuse.
  • Attributing deviations to absence of social desirability is speculative and is not tested against alternative explanations.

What the study does not establish

  • It does not establish that the models possess a Dark Factor, personality, mind, or stable motivation.
  • It does not establish negotiation capacity, because there is no negotiation or interaction.
  • It does not establish psychometric equivalence between human D levels and AI D prompts.
  • It does not establish human likeness through an internally inconsistent score and without external validation.
  • It does not establish that justifications reveal internal processing or psychologically mediate the choice.
  • It does not causally establish effects of architecture, task, personality, or internal mechanisms.
  • It does not establish generalization to other games, stakes, conversations, populations, or deployments.
  • It does not establish validity for population simulation or prediction of human behavior.
  • It does not establish that the responder pattern is flat or uniform in each model.
  • It does not establish robust significance of the strong prompt with five complete models.
  • It does not allow reproducing the 0.526 rate, the neutral baseline, the complete strong table, or a single likeness formula.
  • It does not allow an exact replication without frontier/MiniLM code, frozen models, dependencies, and functional inputs.
  • It does not demonstrate that all data are synthetic or that no privacy risks exist in the human microdata.
  • It does not demonstrate safety of profiles that normalize manipulation, cruelty, or indifference to harm.

Traceability

Scope: Full text

Version: Computers in Human Behavior: Artificial Humans 7 (2026) 100281, Version of Record available online 2026-02-24; OSF rvph8 and GitHub commit c214cb551ffd30e09ddeaa7c1040fdd5cf3d33d0 inspected

Consulted source: https://doi.org/10.1016/j.chbah.2026.100281

Review: Codex 13-page visual full-text, OSF five-file data reproduction, full Git history, notebook, sample-accounting, measurement, statistical, causal-claim, safety, privacy and reproducibility audit, 2026-07-18

Approval: Codex fidelity pass, 2026-07-18

English translation: approved, 2026-07-18

Models evaluated

  • dolphin3
  • deepseek_1.5b
  • mistral
  • deepseek_7b
  • granite3.3_8b
  • gptoss_20b
  • gemma3n_e2b
  • qwen3
  • gemma3_4b
  • phi4
  • llama3.2
  • llama3.1
  • qwen2.5
  • gemma3_1b
  • gemma3_12b
  • gemma3n_e4b
  • llama2uncensored_7b
  • GPT-4.1, snapshot de Azure no especificado
  • GPT-5.1, snapshot de Azure no especificado

Instruments and metrics

  • Cinco descripciones graduadas D1-D5 escritas por los autores
  • Juego del ultimátum binario de una sola ronda y sin interacción
  • Dark Factor of Personality de 70 ítems para la muestra humana
  • Short Dark Triad de 28 ítems incluida en los microdatos humanos
  • Regresiones logísticas, correlaciones y odds ratios por rol/modelo
  • Score compuesto y normalizado de semejanza humana
  • CountVectorizer binario de 100 unigramas, OLS, PCA y t-SNE
  • Prompts conductuales fuertes específicos por rol
  • Incrementos secuenciales de R² presentados como descomposición causal
  • MiniLM y distancia coseno descritos para comparar intervalos semánticos, sin código liberado

Data used

  • aidata.csv: 339.956 completions D-conditioned de 17 etiquetas de modelo
  • hudata.csv: 4.166 decisiones humanas con demografía, 70 ítems D y 28 SD4
  • frontier_models_results.csv: 800 filas de GPT-4.1 y GPT-5.1
  • strong_prompt_raw_data.csv: 82.083 filas, con muestra de respondedor incompleta y prompts de rol contaminados
  • strong_prompt_results.csv: 10 resúmenes, incluido un qwen2.5 respondedor no trazable al raw
  • Repositorio dfactor-llm-ultimatum-game, commit postpublicación c214cb551ffd30e09ddeaa7c1040fdd5cf3d33d0

Evidence and location

  • Design, hypotheses, results, figures, prompts, appendices, ethics, and claims: Computers in Human Behavior: Artificial Humans 7 (2026) 100281, Version of Record; all 13/13 PDF pages rendered and individually inspected
  • Counts, rates, constant models, temperatures, microdata, strong prompts, and frontier results: Official OSF project rvph8: aidata.csv, hudata.csv, frontier_models_results.csv, strong_prompt_raw_data.csv and strong_prompt_results.csv downloaded and hash-validated
  • Likeness formulas, NLP, t-SNE, decomposition, mediation, tests, and post-publication drift: Official GitHub repository vferraz/dfactor-llm-ultimatum-game, full history and current commit c214cb551ffd30e09ddeaa7c1040fdd5cf3d33d0; all analysis notebooks inspected
  • Publication metadata, license, dates, and authorship: Official KIT record 1000191347, Crossref DOI metadata and Version of Record inspected 2026-07-18
  • Independent reproduction and consistency audit, sample, code, statistics, privacy, and limits: reports/verification/article-398-ultimatum-dfactor-sample-accounting-figure-notebook-data-artifact-and-claim-audit.json