This paper asks whether Big Five-prompted LLM agents reproduce the personality-behavior relationships observed in people during conflict. It compares a reported subset of 248 human-human KODIS dialogues with 1,000 same-model LLM negotiations: 500 GPT-4o-mini and 250 each for Claude 3.7 Sonnet and Gemini 2.0 Flash. All negotiate the same buyer-seller jersey dispute across five refund, review, and apology decisions. Final outcomes are payoff, acceptance, and not walking away; strategies are classified under the Interests-Rights-Power framework as cooperative, neutral, competitive, or residual, with additional reciprocity, escalation, and de-escalation metrics.
Agents receive 15 adjectives, three per trait, selected from 70 bipolar pairs and modified to encode six levels. Their marginal trait distributions imitate the human sample, but measurement is not equivalent: people answer ten BFI items whereas LLMs receive discretized labels in a prompt. The design also inserts a human association in advance: apology importance is made dependent on agreeableness using a KODIS regression (B=2.13, p=.02), while other issue weights are random. Any agreeableness effect on outcomes therefore combines the proposed psychological mechanism with an author-programmed utility preference.
In KODIS, score has no personality effects; acceptance is negatively associated with self-neuroticism (B=-.26, p=.026) and positively with partner neuroticism (B=.27, p=.025); not walking away has none. LLMs produce different sets of significant coefficients across models and outcomes. Humans rely heavily on facts and change strategies over dialogue stages; LLMs favor proposals and concessions, follow flatter trajectories, and display model-specific styles. Claude is descriptively closest to humans, Gemini is most skewed toward proposal, power, and residual behavior, and GPT-4o-mini uses more power and rights. Cooperative reciprocity is 88.5–98.2% for LLMs versus 73.7% for humans; GPT-4o-mini also reaches 48.8% competitive reciprocity versus 13.1%. These contrasts support the cautious conclusion that personality prompts do not make these systems reliable human behavioral proxies.
“Alignment,” however, is assessed mainly by comparing which coefficients cross p<.05 in separate regressions. Shared nonsignificance is not evidence of equivalence, and significance in one group but not another does not establish an effect difference. There is no pooled system-by-trait interaction, coefficient-difference interval, equivalence test, or multiplicity correction across hundreds of comparisons. Dyadic clustering is not documented even though each conversation contributes two related observations. Unequal sample sizes of 500/250/250/248 yield unequal power, and temperature 1 without seeds or repeated runs leaves simulation variance unknown.
IRP annotation uses GPT-4o-2024-08-06 at temperature 1. Three annotators, including an author, abandon direct classification after low agreement and instead judge GPT predictions as correct or incorrect on 25 human conversations. The final system reports 81% accuracy, .79 macro F1, and .81 weighted F1, with Positive Expectations at .69. This partial validation covers human text only and may not transfer equally to GPT-4o-mini, Claude, and Gemini styles; using an annotator from the same model family as one generator also permits style bias.
The official repository at commit 811840284ae06b689655bfa71a2da72bf00403e1 cannot reproduce the tables. It omits the 1,000 simulations, annotations, results, seeds, and raw human CSV. Its only processed CSV has 440 observations, 220 dyads rather than the reported 248, and 58 missing acceptance values. The required src/llm_api/base.py module, statsmodels dependency, and google-genai package used by the code are absent; there are no tests, CI, lockfile, or license despite an MIT claim. Several documented paths and pipeline steps do not exist, arguments and defaults conflict, and the appendix mistakenly describes the Gemini and GPT datasets as generated by two Claude models. The artifact exposes part of the design but is not an end-to-end executable or a verification package for the published results.